Classification with Missing Data - A NIFty Pipeline for Single-Cell Proteomics

The paper introduces NIFty, a robust single-cell proteomics classification pipeline that utilizes a top-scoring pairs feature selection method to accurately classify unlabeled cells without requiring data imputation, avoiding circular analysis, or applying explicit batch correction.

The Gist

The Big Picture: Identifying Cells Without a Name Tag

Imagine you are at a massive, chaotic party where thousands of guests are mingling. In the world of biology, these guests are cells. Scientists want to know exactly who is who (e.g., "That's a heart cell," "That's a skin cell") to understand how the body works.

In the past, scientists had to put a literal name tag on every guest before the party started. But in Single-Cell Proteomics (SCP), the technology is so advanced it can take a "photo" of the proteins inside a cell, but it doesn't come with a name tag. The cell is just a mystery guest.

To solve this, scientists use Machine Learning (a computer program) to look at the photos and guess the identity of the cells. However, the old ways of doing this had three huge problems:

1. The "Fill-in-the-Blanks" Problem: The photos were often blurry or had missing spots (missing data). Computers hated this, so scientists had to guess what was missing and "fill in the blanks" (imputation) before the computer could work. This often led to wrong guesses.
2. The "Cheat Sheet" Problem: To teach the computer, scientists would look at the data to find clues, then use those same clues to test the computer. It's like studying for a test using the answer key, then taking the test with the answer key in your pocket. The computer gets a perfect score, but it doesn't actually know the material.
3. The "Different Cameras" Problem: If you took photos of the party with a cheap camera and then with a professional camera, the colors would look different. In science, different labs use different machines, creating "batch effects" that make it hard to compare data.

The Solution: Enter "NIFty"

The authors of this paper created a new tool called NIFty (which stands for Never Impute Features, thank you). Think of NIFty as a super-smart detective that solves the mystery of cell identity without needing to fill in missing blanks, cheat, or worry about different cameras.

Here is how NIFty works, using a simple analogy:

1. The "Within-Sample" Rule (Solving the Missing Data & Camera Problem)

Most old methods tried to compare Protein A in Cell #1 against Protein A in Cell #2.

• The Problem: If Cell #1 was measured with a bright light and Cell #2 with a dim light, the numbers look different even if the cells are the same. Also, if the light was too dim to see Protein A in Cell #2, you have a "missing value."

NIFty's Trick: Instead of comparing Cell #1 to Cell #2, NIFty looks inside a single cell and asks: "Is Protein A bigger than Protein B?"

• The Analogy: Imagine you are trying to identify a person by their height.
  • Old Way: You measure Person A's height in inches, then Person B's height in centimeters. If the rulers are different, you get confused.
  • NIFty's Way: You just ask, "Is Person A taller than Person B?"
  • Why it works: It doesn't matter if the light is bright or dim, or if the camera is different. As long as you can see both proteins in the same cell, you can compare them. Even if one protein is missing (invisible), NIFty has a rule: "If Protein A is there and Protein B is invisible, then Protein A is 'bigger'." This means no need to guess (impute) missing data.

2. The "No Cheat Sheet" Rule (Solving the Double Dipping)

Old methods would look at the whole dataset to find the "best" proteins to use, then use those same proteins to train the computer.

• The Analogy: It's like a teacher showing a student the test questions before the exam, then giving them the same test. The student passes, but they didn't learn anything.

NIFty's Trick: NIFty generates millions of tiny rules (e.g., "Is Protein 1 > Protein 2?") and scores them based on how well they separate the groups without peeking at the final answer key. It selects the best rules based on pure logic, ensuring the computer is learning the pattern, not memorizing the data. This keeps the results honest and scientifically valid.

3. The "Teamwork" Approach (Solving Batch Effects)

Because NIFty compares things inside a cell rather than between cells, it ignores the "noise" caused by different labs or machines.

• The Analogy: If you are trying to identify a song, you don't need to know if it was played on a piano in New York or a guitar in London. You just need to know that the melody (the relationship between the notes) is the same. NIFty listens to the melody inside the cell, ignoring the instrument it was played on.

The Results: Does it Work?

The authors tested NIFty on a bunch of real-world data:

• Missing Data: They fed it data with holes in it (unimputed) and data where someone tried to fill the holes (imputed). NIFty did just as well, or better, with the messy, hole-filled data.
• Different Labs: They tested it on data from different machines and labs with huge differences. NIFty didn't get confused; it still identified the cells correctly.
• Many Types: They tested it on a party with many different types of guests (not just two), and it figured them all out.

The Bottom Line

NIFty is a new, smarter way to label cells in single-cell proteomics.

• It doesn't need you to clean up messy data first.
• It doesn't cheat by using the answer key to study.
• It doesn't care if the data came from different machines.

This makes it much easier for scientists to build massive "Cell Atlases" (maps of every cell type in the body) because they can combine data from many different labs without worrying about the data being incompatible. It's a more honest, robust, and efficient way to understand the building blocks of life.

Technical Summary

1. Problem Statement

Single-cell proteomics (SCP) is a powerful tool for cell-type characterization, trajectory inference, and microenvironment mapping. However, annotating unlabeled cells in SCP datasets using machine learning faces three critical statistical and computational challenges:

• Double Dipping (Circular Analysis): Traditional annotation methods (clustering or classifiers) use protein abundance measurements to group or label cells. If these same measurements are subsequently used for downstream analyses (e.g., differential expression), it results in artificially inflated significance and invalid biological conclusions because the data was used twice.
• Missing Value Imputation: SCP data is inherently sparse, with a high proportion of missing values due to low protein abundance and stochasticity in LC-MS acquisition. Most machine learning algorithms require complete data, forcing researchers to impute missing values. This imputation can obscure true biological variation and introduce bias, especially since the mechanisms behind missingness in SCP are complex.
• Batch Effects: Protein measurements are often not directly comparable across different experiments, labs, or instruments due to batch effects. Classifiers trained on reference data often fail when applied to new experimental data if the data has not been rigorously normalized or batch-corrected, limiting the utility of large-scale cell atlases.

2. Methodology: The NIFty Pipeline

The authors present NIFty (Never Impute Features, thank you), a classification pipeline designed specifically for single-cell proteomics. Its core innovation lies in its feature generation strategy, based on an enhanced implementation of Top-Scoring Pairs (TSP).

Core Feature Generation

Instead of using absolute protein abundance values as features (which requires cross-sample comparability), NIFty generates features based on pairwise protein comparisons within a single sample.

• Rule Definition: A feature is a binary rule comparing two proteins (e.g., "Protein A > Protein B").
• Binary Matrix: For each sample, if the rule is true, the feature value is 1; if false, it is 0.
• Handling Missing Data: NIFty reimagines rules to handle missing values without imputation. A rule can be satisfied if "Protein A > Protein B" OR if "Protein A is present and Protein B is absent." This allows the generation of a complete binary feature matrix directly from incomplete quantitative data.
• Batch Effect Resistance: Since comparisons are confined within a sample, differences in absolute abundance between samples (batch effects) do not affect the binary outcome of the rule.

Feature Selection Process

To manage the combinatorial explosion of rules (from $N$ proteins, there are $N(N-1)/2$ possible pairs), NIFty employs a robust, non-circular feature selection workflow:

1. Pre-filtering: Proteins with excessive missingness (>50% by default) are removed.
2. Scoring: Rules are scored based on their ability to distinguish one class from another (using the Geman et al. scoring function: $|P(\text{True}|Class A) - P(\text{True}|Class B)|$).
3. Significance Testing: Instead of traditional permutation tests (which are computationally expensive), NIFty bins rules by their proportion of "True/False" values. It randomizes labels once to generate null distributions for these bins, calculating p-values efficiently.
4. Redundancy Filtering: Selected rules are filtered using Mutual Information to ensure the final set of features provides unique, non-redundant information.
5. Model Training: The selected top-$k$ rules are used to train a classifier (Support Vector Machine or Random Forest) using stratified k-fold cross-validation.

Multiclass Extension

For multiclass problems, NIFty uses a "One-vs-Rest" strategy. It generates a unique set of features for each class against all others, aggregates these features, and trains a multiclass model (using Scikit-learn).

3. Key Contributions

• Elimination of Imputation: NIFty is the first proteomics classification tool that natively handles missing data without requiring pre-imputation, preserving the integrity of the raw quantitative data.
• Prevention of Double Dipping: By converting abundance data into within-sample binary rules, the features used for classification are distinct from the raw abundance values used in downstream differential expression analyses, breaking the cycle of circular analysis.
• Batch Effect Immunity: The within-sample comparison logic renders the classifier robust to batch effects, allowing models trained on diverse reference datasets (e.g., cell atlases) to be applied to new experimental data without complex normalization.
• Open-Source Implementation: The tool is fully open-source with documentation for both running the pipeline and reproducing the manuscript's results.

4. Results

The authors validated NIFty across multiple datasets and scenarios:

• Imputed vs. Unimputed Data:

  • Tested on datasets from Leduc et al. and Montalvo et al., NIFty performed comparably or slightly better on unimputed data compared to imputed data.
  • Across 10 diverse datasets, the median difference in validation accuracy between imputed and unimputed data was negligible (0.6% for 50 samples/class; 0.3% for 100 samples/class).
  • Crucially, the top 15 rules selected in trials almost never contained proteins with zero missing values, proving the method relies on incomplete data effectively.

• Large Batch Effects:

  • Using data from the HUPO Single Cell Initiative (8 different batches), NIFty was tested on normalized vs. non-normalized data.
  • When training on 3 or fewer batches, NIFty performed better on non-normalized (uncorrected) data.
  • With more batches, performance on non-normalized data was indistinguishable from normalized data, demonstrating that the method inherently overcomes batch effects without explicit correction.

• Multiclass Classification:

  • Tested on a developmental time-course dataset (iPSC to cardiomyocytes, 5 timepoints), NIFty achieved high accuracy (diagonal accuracy >86% for most stages).
  • The most difficult distinction was between Day 10 and Day 21 (both cardiomyocyte stages), which is biologically expected, but the overall framework successfully handled multiclass scenarios.

5. Significance and Future Implications

The NIFty pipeline addresses the primary bottlenecks preventing the widespread adoption of single-cell proteomics atlases.

• Atlas Integration: As the field moves toward creating large-scale single-cell proteome atlases (aggregating data from multiple labs), NIFty provides the necessary statistical framework to integrate these heterogeneous datasets without the need for complex batch correction or data imputation.
• Robustness: The authors argue that for an atlas to be robust, it should include data from at least three different labs to generalize features across technical variations. NIFty facilitates this by making the model invariant to the specific batch or instrument used.
• Exclusion of TMT Data: The paper notes that while NIFty is powerful, it is not compatible with TMT (isobaric) multiplexing data for atlas building, as TMT normalization destroys the "within-sample" ratio logic required for the TSP rules.
• Workflow Standardization: By removing the need for imputation and preventing double dipping, NIFty enables a more transparent, statistically sound, and reproducible workflow for cell annotation and downstream biological discovery in single-cell proteomics.