FAQ

What is scTypeEval measuring?

scTypeEval quantifies how reproducible cell type annotations are across samples, without requiring external ground-truth labels. In practice, it evaluates whether cells assigned to the same annotation form coherent, sample-consistent groups in dissimilarity space.

Which ISC metrics should I use by default?

For most datasets, we recommend the pair used across these tutorials and case studies:

  • Local ISC: silhouette on recip_classif:Match
  • Global ISC: 2label_silhouette on Pseudobulk:Cosine

This combination captures complementary failure modes: local over-partitioning versus global within-label heterogeneity. See tutorials/benchmarking-of-isc-metrics and tutorials/defining-isc-metrics.

What are the minimum required inputs?

At minimum, you need:

  • A count matrix (genes x cells)
  • Metadata with one annotation column (ident) and one sample column (sample)
  • Matching cell IDs between count matrix columns and metadata row names

You can start from a matrix + metadata, a Seurat object, or a SingleCellExperiment object.

Why do many workflows use min_samples = 5 and min_cells = 10?

These defaults help ensure each retained cell type is represented across enough biological replicates and has enough cells per sample-celltype group for robust pseudobulk and dissimilarity estimation. You can relax them for small datasets, but interpret ISC more cautiously when support is sparse.

Should I run dissimilarities on PCA (reduction = TRUE) or on expression (reduction = FALSE)?

Use both, depending on the method:

  • For Pseudobulk:* dissimilarities, PCA space (reduction = TRUE) is often much faster and usually gives similar behavior.
  • recip_classif:Match-based dissimilarities rely on gene expression.

This is also the pattern used in tutorials/scTypeEval-workflow and both case studies.

Why remove blacklisted genes (TCR, immunoglobulins, Y genes)?

These genes can dominate inter-sample variability for technical or lineage-composition reasons that may obscure annotation consistency. Removing them from HVG-based feature sets often improves ISC interpretability.

Typical usage:

data(black_list)
bl <- c(black_list$TCR, black_list$Immunoglobulins, black_list$Ygenes)

How do I interpret low local ISC versus low global ISC?

  • Low local ISC often indicates overlap between labels or unsupported splitting (possible over-partitioning).
  • Low global ISC often indicates heterogeneous states forced into one label (possible under-partitioning).

See tutorials/diagnostics-low-consistent-annotations and tutorials/isc-upon-gene-expression-perturbations.

How can ISC guide annotation refinement in practice?

A practical strategy is:

  1. Compute local and global ISC on the baseline annotation.
  2. Identify cell types with low consistency.
  3. Use metadata and/or expression structure, as well as your domain knowledge, to propose targeted splits or merges.
  4. Recompute ISC and verify that edited labels improve the expected metric while leaving stable labels largely unchanged.

See worked examples in case-studies/human-lung-cell-atlas and case-studies/atlas-wat.

Should I use wrapper_scTypeEval() or step-by-step functions?

  • Use wrapper_scTypeEval() for fast iteration and standardized setup.
  • Use the explicit step-by-step workflow when you need fine control over filtering, feature sets, dissimilarity choices, and diagnostics.

Internally, wrapper_scTypeEval() runs a compact end-to-end pipeline in this order:

  1. Create (or reuse) an scTypeEval object.
  2. Run run_processing_data() with your ident, sample, filtering, and normalization settings.
  3. Define features:
    • if gene_list is provided, it is added and used;
    • otherwise HVGs are computed via run_hvg().
  4. If reduction = TRUE, run run_pca().
  5. Run run_dissimilarity() for each method listed in dissimilarity_method.

In other words, the wrapper prepares the object and computes dissimilarity assays. One still needs to call get_consistency() (and usually plotting functions) afterward to evaluate local/global ISC and decide on annotation refinement.

How should I summarize ISC into one score?

Many analyses report both metrics separately and also an integrated summary (for example, local x global). The key is consistency: use the same definition across all compared annotations and report local/global values alongside any combined score.

What are common reasons ISC fails or gives unstable results?

Frequent causes include:

  • Mismatch between count matrix columns and metadata row names
  • Too few samples per label after filtering
  • Too few cells per sample-celltype group
  • Running run_hvg() or run_dissimilarity() before run_processing_data()
  • Missing required dependencies for selected methods

How can I speed up analyses on large datasets?

Common approaches used in this project:

  • Downsample cells per sample-celltype group for exploratory runs
  • Use PCA-based dissimilarities when appropriate
  • Start with a subset of candidate low-ISC labels, then scale up
  • Increase ncores where supported

Where should I start if I am new to scTypeEval?

Recommended order:

  1. tutorials/scTypeEval-workflow
  2. tutorials/defining-isc-metrics
  3. tutorials/diagnostics-low-consistent-annotations
  4. case-studies/human-lung-cell-atlas
  5. case-studies/atlas-wat