In this case study, we evaluate the consistency of the cell type annotations provided by the authors of a white adipose tissue atlas.
2 Dataset description
Emont et al., 2022 generated a single-cell atlas of human and mouse white adipose tissue across multiple depots, donors, and adipose cell compartments. The atlas captures the major adipose, stromal/vascular, and immune populations, including adipocytes, adipocyte progenitors, endothelial cells, fibroblasts/pericytes, and resident leukocytes. For this case study, we use the human white adipose tissue object distributed through SCP1376, which contains 166,149 cells and the accompanying metadata used for downstream annotation consistency analyses.
3 Download data
Data can be found at SCP1376. We will download lite objects that only contain RNA data and thus are easier to download.
library(dplyr)library(tidyr)library(ggplot2)library(scTypeEval)library(Seurat)theme_set(theme_bw(base_size =12))# Blacklisted genes that are inherently variable across samples# and should be excluded from ISC (scTypeEval) computationsdata(black_list)blacklist_genes <-c(black_list$TCR, black_list$Immunoglobulins, black_list$Ygenes)# These are the local and global ISC thresholds empirically described in the manuscript (Fig. 5)q_local <-0.4467538q_global <-0.7443436
5 Load Human WAT atlas dataset
# this function load Seurat, SCE, or AnnData object and returns# a list with 1) counts and 2) metadata if `split = TRUE`hwat <-load_single_cell_object(dataset_file,split = T)# split into metadata and countsmetadata <- hwat$metadatacounts <- hwat$countsdim(counts)
[1] 31533 166149
Dataset consists of >166k cells.
5.1 Explore metadata
For the scTypeEval pipeline we need to identify: 1. the individual of origin: individual (22 patients in total) 2. the author’s annotation or clustering, or another cell classification derived from gene expression: - cell_type2: Broad annotation - cell_type: Fine annotation
Visually explore the different metadata information by looking at the UMAPs.
vars <-c("technology","tissue","depot","individual","cell_type2")umap_expl <-lapply(vars, function(v){ lab <-ifelse(n_distinct(seu@meta.data[[v]]) <20, T, F) pl <-DimPlot( seu,group.by = v,label = lab,repel =TRUE,raster =TRUE ) +NoAxes() +theme(aspect.ratio =1)# do not show legend if too many valuesif(!lab){ pl <- pl +NoLegend() }return(pl)}) patchwork::wrap_plots(umap_expl[1:4],ncol =2)
umap_expl[[5]]
5.2 Dataset filtering
Because tissue of origin, depot, and technology are not balanced, we keep only Chromium-v3, the most abundant technology in the dataset. This protocol uses single-nucleus RNA-seq (snRNA-seq), which, unlike whole-cell approaches, captures adipocytes and mesothelial cells more effectively. The authors also used these snRNA-seq data to build the atlas. See the original paper for more details.
6 Inter-Sample Consistency (ISC) metrics on Broad annotation
We will start by evaluating the ISC of broad annotation provided by authors (metadata column cell_type2)
For that we will perform the full ISC analysis pipeline: 1.Create a scTypeEval object from counts and metadata. 2.Process the data and define samples/cell type annotations. 3.Select highly variable genes (HVGs). 4.Compute a PCA embedding. 5.Calculate local and global dissimilarities. 6.Compute consistency metrics. 7.Build the final Integrated ISC table.
6.1 Process data and obtain Highly Variable Genes
ident <-"cell_type2"# broad annotation# use filtered Seurat object with only Chromimum v3 snNuc-seqsceval <-create_scTypeEval(seu_chromv3) sceval_broad <-run_processing_data(scTypeEval = sceval,ident = ident, # in this case broad annotationsample ="individual",min_samples =5,min_cells =10,verbose =FALSE)sceval_broad <-run_hvg(scTypeEval = sceval_broad,ngenes =2000,sample =TRUE,aggregation ="single-cell",black_list = blacklist_genes,ncores =4,verbose =FALSE)
6.2 Add embedding (run_pca)
# remove data from single-cell grouping as we don't need it# to compute ISC other than WasserSteinsceval_broad@data$`single-cell`<-NULLset.seed(22)sceval_broad <-run_pca(scTypeEval = sceval_broad,ndim =30,verbose =FALSE)# Visualize PCA by cell type and samples groupsplot_pca(sceval_broad)
We can observe the individual-wise cell type variability, clearly separating the immune and non-immune compartment
6.3 Compute dissimilarities (run_dissimilarity)
Build pairwise distance matrices between all cell types & sample pairs
sceval_broad <-run_dissimilarity( sceval_broad,method ="recip_classif:Match",reduction =FALSE,ncores =4,verbose =FALSE)sceval_broad <-run_dissimilarity( sceval_broad,method ="Pseudobulk:Cosine",reduction =TRUE,verbose =FALSE)# Visualize reciprocal classification match (RCM) plot# This inform us of the cross-sample reproducbility of each annotationplot_heatmap(sceval_broad,dissimilarity_slot ="recip_classif:Match", sort_consistency ="silhouette",sort_similarity ="Pseudobulk:Cosine")+ggtitle("RCM plot - Broad Annotation")
RCM plot is already telling us that authors broad annotation (cell_type2) is highly reproducible across samples, showing clear on-diagonal blocks.
6.4 Compute consistency (get_consistency)
We then compute the consistency metrics from the previously generated distance matrices. We use global and local ISC, defined here as 2-label silhouette + pseudobulk cosine and silhouette + recip_classif:Match, respectively. For additional details, see Benchmarking of ISC metrics.
The final object (cons_broad) contains, for each cell type: - Local ISC: silhouette + recip_classif:Match - Global ISC: 2-label silhouette + pseudobulk cosine - Integrated ISC (local × global)
All broad annotation labels, except neutrophils, are above the empirical thresholds reported for these metrics, suggesting that this annotation framework is highly reproducible. Neutrophils are usually reported to have lower quality in single-cell RNA-seq datasets due to their intrinsic fragility during tissue dissociation and sample processing. Their sensitivity to handling conditions can lead to increased RNA degradation, reduced transcript complexity, and greater technical variability, which may negatively impact consistency-based evaluation metrics. Some immune cell types, including dendritic cells, mast cells, B cells, and NK cells, show global ISC below 0.9, which suggests some room for improvement through finer subdivision.
7 Inter-Sample Consistency (ISC) metrics on fine annotation
We next repeat the same pipeline for the authors’ finer annotation (cell_type).
7.1 Process data and obtain Highly Variable Genes
ident <-"cell_type"# author's fine annotationsceval_fine <-run_processing_data(scTypeEval = sceval,ident = ident, # in this case broad annotationsample ="individual",min_samples =5,min_cells =10,verbose =FALSE)sceval_fine <-run_hvg(scTypeEval = sceval_fine,ngenes =2000,sample =TRUE,aggregation ="single-cell",black_list = blacklist_genes,ncores =4,verbose =FALSE)
7.2 Add embedding (run_pca)
# remove data from single-cell grouping as we don't need itsceval_fine@data$`single-cell`<-NULLsceval_fine <-run_pca(scTypeEval = sceval_fine,ndim =30,verbose =FALSE)# Visualize PCA by cell type and samples groupsplot_pca(sceval_fine)
Based on the RCM plot, some cell types show high reproducibility across samples (for example, SMC1, Peri, Tcell1, and Mac1), whereas others, particularly adipocyte subtypes, show a much lower rate of reproducibility. This suggests that some of these adipocyte subtypes are more similar to other subtypes from the same sample than to the same subtype across samples.
The local versus global ISC scatter plot confirms the pattern suggested by the RCM plot. Most adipocyte subtypes (hAd*) fall below the empirical threshold for local consistency, suggesting that these labels reflect over-partitioning of adipocyte populations that are not reproducibly separated across samples.
This interpretation is also consistent with the paper, where the molecular distinction between adipocyte subpopulations appears limited based on the reported marker specificity, as shown in Fig. 3A and Extended Data Fig. 9:
Characterization of human adipocyte subtypes. Source: Emont et al., 2022. Extended Data Fig. 9
8 Reannotation of adipocytes
After finding that the adipocyte subtypes in the authors’ fine annotation do not show good cross-sample reproducibility, we can test alternative annotations guided by this criterion. We therefore explore sample-level metadata to identify factors that may drive the marked inter-sample variability across adipocyte subtypes.
8.1 Explore sample level variability
First build an scTypeEval object with only adipocytes to explore inter-sample variability.
ident <-"cell_type"# fine annotation# Create scTypeEva object using only adipocytessceval_adipo <-create_scTypeEval(seu_chromv3[, seu_chromv3$cell_type2 =="adipocyte"]) sceval_adipo <-run_processing_data(scTypeEval = sceval_adipo,ident = ident,sample ="sample",min_samples =5,min_cells =10,verbose =FALSE)sceval_adipo <-run_hvg(scTypeEval = sceval_adipo,ngenes =2000,sample =TRUE,aggregation ="pseudobulk",black_list = blacklist_genes,ncores =4,verbose =FALSE)# remove data from single-cell grouping as we don't need it# to compute ISC other than WasserSteinsceval_adipo@data$`single-cell`<-NULLset.seed(22)sceval_adipo <-run_pca(scTypeEval = sceval_adipo,ndim =30,verbose =FALSE)# Visualize PCA by cell type and samples groupsplot_pca(sceval_adipo)
We can now assess how relevant sample metadata behave in PCA space.
sample_vars <-c("depot", "source", "bmi.range")# render sample/cell type level metadata dataframecol_annot <-data.frame(id = sceval_adipo@reductions$pseudobulk@group) %>%# this includes <sample_label>_<celltype_label>separate(id, sep ="_", into =c("sample", "celltype"), remove = F) %>%left_join(., sceval_adipo@metadata %>%distinct(sample, .keep_all = T) %>%mutate(sample = scTypeEval:::purge_label(sample)) %>%select(sample, depot, source, bmi.range),by ="sample") %>% tibble::column_to_rownames("id")for(a in sample_vars){ sceval_adipo@reductions$pseudobulk@ident$cell_type <- col_annot[[a]] pl <-plot_pca(sceval_adipo) +ggtitle(a)print(pl)}
We can see that depot and source separate quite well the samples on the PCA space. We confirm this by measure the silhouette score on the pseudobulk space.
# Visualize MDS of cosine distancessceval_adipo@dissimilarity$`Pseudobulk:Cosine`@ident$cell_type <- col_annot$depotplot_mds(sceval_adipo)
Overall, these findings suggest that: 1) the adipocyte subpopulations proposed by the authors do not represent cell types that are reproducibly recovered across patients; and 2) adipocytes from VAT and SAT show marked gene expression differences.
8.2 Reannotate adipocytes subpopulations
Based on this, we next classify adipocytes into SAT and VAT.
Let’s explore if categorizing adipocytes based on their depot of origin is reproducible across samples using scTypeEval pipeline. Consider that most individuals contain samples from both SAT and VAT, hence the similarity between them is still present.
sceval_depot <-create_scTypeEval(seu_chromv3)
ident <-"fine_annotation_new"# author's fine annotationsceval_depot <-run_processing_data(scTypeEval = sceval_depot,ident = ident, # in this case broad annotationsample ="individual",min_samples =5,min_cells =10,verbose =FALSE)sceval_depot <-run_hvg(scTypeEval = sceval_depot,ngenes =2000,sample =TRUE,aggregation ="single-cell",black_list = blacklist_genes,ncores =4,verbose =FALSE)# remove data from single-cell grouping as we don't need itsceval_depot@data$`single-cell`<-NULLsceval_depot <-run_pca(scTypeEval = sceval_depot,ndim =30,verbose =FALSE)
We can see on the RCM plot that the new adipocytes subtypes, split into SAT- and VAT derived (adipocyte.SAT and adipocyte.VAT, respectively) show a much larger degree of reproducibility across individuals.
We confirm these findings. New annotations of adipocytes split by the depot of origin increase their ISC.
9 Interpretation
At the broad annotation level, the atlas is highly reproducible across individuals, which suggests that the major adipose, stromal, vascular, and immune compartments are robustly defined in this dataset. In contrast, the fine annotation reveals a clear local inconsistency concentrated in the adipocyte subtypes, consistent with the ISC failure mode described in the paper in which closely related labels cannot be reliably distinguished across samples.
In practice, this means that the original adipocyte fine labels are likely too granular to serve as stable targets for cross-sample comparison or supervised classification. The exploratory analyses further suggest that depot of origin is a stronger and more reproducible source of adipocyte variation than the original subtype labels. Overall, this case study supports using the atlas confidently at the broad level while treating the original adipocyte subtypes with caution and testing simpler, biologically grounded relabeling strategies such as VAT versus SAT.
