📊 statistics

Conformal inference for cell type annotation with graph-structured constraints

This paper introduces a novel conformal inference framework implemented in the R package `scConform` that leverages graph-structured cell ontologies and risk control to enhance the interpretability and coherence of cell-type annotations in single-cell transcriptomics, while also addressing distribution shifts between training and test data.

Original authors: Daniela Corbetta, Livio Finos, Ludwig Geistlinger, Davide Risso

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniela Corbetta, Livio Finos, Ludwig Geistlinger, Davide Risso

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Guessing with a Safety Net

Imagine you are a detective trying to identify a suspect from a lineup of 15 different people. A standard computer model acts like a detective who points to one person and says, "That's the one!" But sometimes, the detective isn't 100% sure. Maybe the suspect looks a bit like Person A, but also a little like Person B.

If the detective just picks Person A, they might be wrong. If they pick both A and B, they are safer, but if A and B are total strangers to each other (like a baker and a pilot), the list doesn't make much sense.

This paper introduces a new way for computers to make these guesses. Instead of just pointing to one person, it gives a list of possibilities that are guaranteed to include the real answer most of the time. Even better, it makes sure that the people on the list are related to each other, like a family tree, so the list makes logical sense.

The Problem: The "Family Tree" of Cells

In biology, scientists study tiny building blocks called cells. There are many types of cells (like T-cells, B-cells, muscle cells), and they are related to each other in a specific hierarchy, much like a family tree.

The Tree: All "T-cells" are children of "Lymphocytes." "CD4+ T-cells" and "CD8+ T-cells" are cousins.
The Issue: Old computer methods might say a cell is either a "CD4+ T-cell" OR a "Muscle Cell." These two are very far apart on the family tree. If the computer is confused, giving you a list with these two unrelated options is confusing and not very helpful.

The Solution: A "Smart Safety Net"

The authors developed a method called Conformal Inference. Think of this as a "safety net" for predictions.

The Guarantee: The method promises, "If you use my list of guesses, I guarantee that the real answer is inside that list 90% of the time." It doesn't matter how good or bad the original computer model is; the safety net adjusts itself to keep that promise.
The Graph Constraint: This is the paper's special twist. Instead of just grabbing random guesses, the method looks at the "Family Tree" (the graph).
- If the computer is very sure, it gives you one specific leaf on the tree (e.g., "CD4+ T-cell").
- If the computer is unsure, instead of giving you a random mix of distant relatives, it moves up the family tree to a common ancestor.
- Analogy: If you aren't sure if the suspect is a "CD4+ T-cell" or a "CD8+ T-cell," the method doesn't list both. Instead, it says, "It's definitely a T-cell." This is a single, clear answer that covers both possibilities without being confusing.

Handling the "Different Rooms" Problem (Distribution Shift)

Imagine you trained your detective using photos of people from New York, but now you are trying to identify people from Tokyo. The lighting, clothing, and average features might be different. This is called "distribution shift."

The Problem: If the computer was trained on a dataset with mostly "Muscle Cells" and you test it on a dataset with mostly "Blood Cells," the safety net might break because the computer is confused by the new mix.
The Fix: The authors created a "resampling" trick. Before making the final safety net, they pretend to shuffle the training photos so that the mix of people in the training room looks exactly like the mix in the testing room. This helps the safety net work correctly even when the data comes from different sources (like different hospitals or patients).

Real-World Tests

The authors tested this idea in two ways:

The Mouse Gut Test: They used data from mouse intestines. They found that their new method produced lists that were more logical. When the computer was unsure, it grouped related cells together (like saying "T-cell" instead of "CD4" and "B-cell" mixed together). They also showed that when the computer was truly confused (like with a cell type it had never seen before), their method would say, "I don't know, it could be any of these," which is a honest and useful answer.
The COVID-19 Test: They looked at blood cells from COVID-19 patients. They simulated a scenario where they had to predict cell types for a new patient based on old data. Their method successfully handled the differences in cell counts between the old and new data, providing reliable groups of guesses that respected the biological family tree.

The Bottom Line

This paper gives scientists a tool to make cell predictions that are:

Honest: They admit when they aren't sure by giving a broader, safer list.
Logical: The lists respect the biological family tree, so the answers make sense.
Reliable: They have a mathematical guarantee that the right answer is usually in the list.
Adaptable: They can handle situations where the new data looks different from the old training data.

The authors have made this tool available as a free software package called scConform so other scientists can use it to make their own cell predictions more trustworthy.

Technical Summary: Conformal Inference for Cell Type Annotation with Graph-Structured Constraints

Problem Statement
Single-cell transcriptomics faces significant challenges in cell type prediction, particularly regarding uncertainty quantification. Traditional reference-based approaches often rely on point predictions (assigning a single label with the highest probability), which fail to capture the inherent uncertainty of the model, especially when multiple cell types have comparable probabilities. While conformal prediction offers a framework for generating valid prediction sets with finite-sample guarantees, its application in single-cell genomics remains underexplored. Furthermore, standard conformal methods often produce prediction sets containing labels that are semantically distant or unrelated within the biological hierarchy (e.g., predicting both a specific T-cell and a stromal cell), complicating interpretation. Additionally, real-world applications frequently suffer from distributional shifts, specifically label shifts where the cell type composition differs between the reference (training/calibration) and test datasets, violating the exchangeability assumption required for standard conformal inference.

Methodology
The authors propose a novel framework that integrates conformal risk control with the graph-structured constraints of the Cell Ontology.

Graph-Structured Conformal Risk Control:
- Ontology Integration: The method utilizes the Cell Ontology as a Directed Acyclic Graph (DAG), where leaf nodes represent specific cell types and internal nodes represent broader categories.
- Score Construction: For a predicted class $\hat{y}(x)$ , the algorithm calculates scores $g(v, x)$ for ancestor nodes $v$ , defined as the sum of predicted probabilities of all leaf descendants of $v$ .
- Prediction Set Construction: Instead of returning scattered leaf nodes, the algorithm identifies the most specific ancestor $v$ of the predicted class whose cumulative score exceeds a threshold $\lambda$ . The prediction set $C_\lambda(x)$ consists of all leaf descendants of this ancestor, plus the leaf descendants of any other ancestors with scores strictly below $\lambda$ (to ensure the nesting property required for risk control).
- Loss Function: The method employs a miscoverage loss function, ensuring that the probability of the true label falling outside the prediction set is bounded by a user-specified error rate $\alpha$ .
Addressing Distributional Shift (Label Shift):
- Recognizing that cell type compositions often differ between reference and test datasets, the authors propose a two-step approach. First, they assume technical and biological covariate shifts are mitigated via existing integration methods (e.g., Harmony, LIGER).
- Second, to address label shift (differences in cell type frequencies), they introduce a resampling strategy. The test data is split into two subsets ( $S_1, S_2$ ). A model is trained on the reference data, and probabilities are estimated on $S_1$ . The calibration set is then resampled according to these estimated probabilities to match the test distribution. This resampled set is used to calibrate prediction sets for $S_2$ . The process is repeated with swapped roles. This strategy approximates the importance-weighted expectation required for valid coverage under label shift without requiring prior knowledge of the test distribution.

Key Contributions

Graph-Constrained Prediction Sets: The paper introduces a conformal algorithm that leverages the hierarchical structure of cell ontologies to generate prediction sets that are semantically coherent. Instead of disjoint labels, the method outputs sets corresponding to specific ancestors in the ontology, providing a single, interpretable label (or its descendants) that summarizes the uncertainty.
Label Shift Correction: The authors provide a practical resampling technique to correct for label shift in single-cell annotation, restoring valid coverage guarantees when reference and test datasets have different cell type compositions.
Implementation: The method is implemented in the open-source R package scConform (available via Bioconductor), making the approach accessible to the genomics community.

Results
The authors evaluated their method using two datasets:

Merfish Mouse Ileum Dataset:
- Compared against standard split conformal inference, the graph-structured method produced prediction sets with lower intra-set distances (average distance 1.148 vs. 1.616), indicating higher semantic coherence.
- While the graph-based sets were slightly larger on average, they frequently produced single-label predictions when confidence was high.
- Crucially, for cells belonging to categories not present in the reference (e.g., "Myenteric Plexus"), the graph-based method correctly produced "inconclusive" sets (containing all 15 labels), whereas standard conformal inference incorrectly assigned specific reference labels, failing to flag the out-of-distribution nature of the data.
COVID-19 Patient Dataset:
- In a realistic clinical setting with unbalanced cell type distributions, the proposed resampling strategy successfully corrected under-coverage issues.
- Without resampling, empirical coverage exceeded the nominal level (over-coverage). With the resampling correction (both oracle and estimated), coverage converged closer to the target 0.9.
- The method successfully consolidated fine-grained predictions into biologically meaningful ancestors (e.g., grouping plasma cell subtypes under "antibody secreting cell"), demonstrating improved interpretability.

Significance and Claims
The paper claims that incorporating graph-structured constraints into conformal inference significantly enhances the interpretability of cell type predictions in single-cell data. By aligning prediction sets with the biological hierarchy, the method avoids the ambiguity of disjoint labels and provides a more intuitive summary of model uncertainty.

The authors emphasize that their approach provides finite-sample validity guarantees regardless of the underlying classifier's accuracy. Unlike hierarchical classification methods that propagate errors down the tree without coverage guarantees, this method ensures that the reported set contains the true cell type with a prescribed probability ( $1-\alpha$ ). Furthermore, the proposed resampling strategy offers a principled solution to the common problem of label shift in genomics, allowing for reliable uncertainty quantification even when reference and test datasets differ in composition. The work positions conformal inference as a robust tool for downstream analysis, allowing researchers to explicitly incorporate model uncertainty rather than relying on potentially misleading point estimates.