Towards Label-Free Single-Cell Phenotyping Using Multi-Task Learning

Technical Summary: Towards Label-Free Single-Cell Phenotyping Using Multi-Task Learning

Problem Statement
Accurate characterization of White Blood Cells (WBCs) is critical for diagnosing immune dysregulation, infections, and hematologic malignancies. Conventional workflows rely on fluorescence-based flow cytometry or manually stained smears, which suffer from limitations including the need for specialized instruments, fluorophore labeling, labor intensity, and subjectivity. While label-free optical modalities like Differential Phase Contrast (DPC) imaging offer a scalable, non-invasive alternative, inferring molecular phenotypes directly from bright-field morphology remains challenging. Existing deep learning approaches predominantly treat WBC analysis as a discrete classification problem, overlooking the continuous spectrum of protein-expression levels. Furthermore, the "black-box" nature of deep neural networks hinders clinical adoption by lacking interpretable biological reasoning. This work addresses the difficulty of predicting both discrete cell types and continuous protein-expression levels directly from single-cell label-free DPC images, a task complicated by subtle, sparsely distributed morphological cues that are easily attenuated by standard convolutional pooling.

Methodology
The authors propose a unified Deep Learning (DL) framework that jointly performs WBC classification and continuous protein-expression regression from multi-channel DPC images. The architecture consists of three primary components:

1. Hybrid Dual-Branch Encoder: The model fuses local and global representations to capture complementary morphological cues.

   • CNN Branch: Utilizes a stem followed by Inception-style modules with residual connections to extract fine-grained local texture features (e.g., membrane contours, nuclear texture, granules). It processes the input into spatial tokens.
   • ViT Branch: Employs a compact Vision Transformer (ViT) to model global spatial dependencies via self-attention, capturing long-range structural cues like overall cell shape or polarization that convolutional kernels may miss.
   • Cross-Modal Fusion: A learnable fusion mechanism unifies the heterogeneous representations from the CNN and ViT branches. It extracts global features from both branches, projects them to a shared space, and combines them using learnable weights normalized by softmax to ensure interpretable contribution analysis.

2. Task-Adaptive Gating Mechanism: To mitigate negative transfer between tasks with conflicting gradient dynamics, the framework employs a gating mechanism. This module dynamically modulates feature sharing between the classification and regression pathways. It generates soft gates via sigmoid-activated linear layers to determine the balance between preserving task-specific features and incorporating mixed information from the other task, allowing the model to learn inherent dependencies (e.g., T-cell classification correlating with CD3 expression) without explicit supervision.

3. Multi-Task Prediction Heads: The shared fused representation is refined through task-specific pathways (MLPs with residual connections, batch normalization, and dropout) before being passed to separate heads:

   • A classification head outputs WBC class probabilities using a three-layer MLP with high dropout (0.4) for regularization.
   • A regression head outputs continuous values for four protein markers (CD45, CD3, CD19, CD14) without a final activation function to allow unrestricted continuous predictions.

4. LLM-Guided Interpretability: To enhance transparency, a Large Language Model (Gemini 1.5 Pro) is integrated post-hoc. It converts quantitative predictions into concise, biologically grounded textual summaries. The generation is constrained via template-based prompting to prevent hallucinations, ensuring outputs are strictly factual and derived solely from model predictions.

Loss Function
The model is trained using a weighted multi-task objective:

• Classification Loss ($L_{cls}$): Focal loss with class weights to handle imbalance and suppress well-classified samples.
• Regression Loss ($L_{reg}$): A combination of Smooth L1 loss (for stability against outliers) and a Pearson correlation alignment term (to preserve the relative ordering of marker intensities).
• Auxiliary Loss ($L_{aux}$): Regularizes intermediate fused representations to enforce feature-level consistency between shared and task-specific pathways.

Experimental Results
The framework was evaluated on the Berkeley Single Cell Computational Microscopy (BSCCM) dataset and the Blood Cells Image (BCCD) dataset.

• Classification Performance: On the BSCCM test split, the proposed model achieved 91.3% accuracy, outperforming strong baselines including VGG (88.3%), AttentionCNN (89.6%), and ResNet (86.2%). On the BCCD dataset (conventional brightfield), it achieved a macro-averaged F1-score of 0.933, demonstrating successful generalization.
• Regression Performance: For protein expression regression on BSCCM, the model achieved a Pearson correlation of 0.7263, with an RMSE of 0.6801 and MAE of 0.4416. This outperformed all single-task and single-backbone baselines (e.g., DenseNet achieved r=0.7246).
• Per-Marker Analysis: Performance varied by marker. Lineage-specific markers with strong morphological correlates (e.g., CD16, r=0.819; CD45, r=0.799) were predicted with high accuracy. In contrast, markers reflecting transient activation states (e.g., CD123/HLA-DR/CD14) showed lower correlation (r=0.339), indicating that label-free morphology cannot fully replace molecular assays for dynamic functional states.
• Ablation Studies: Removing the ViT branch reduced accuracy by 1.58% and regression correlation by 0.0335. Removing the CNN branch caused a larger degradation, confirming the necessity of local texture extraction. Single-task variants performed worse than the multi-task model, validating that joint optimization acts as a regularizer and sharpens feature representations.

Key Contributions

1. Unified Multi-Task Framework: A novel approach that jointly predicts WBC class and protein-expression levels from label-free DPC images, eliminating the need for chemical staining.
2. Hybrid Architecture: A dual-branch encoder combining multi-scale convolutional texture features with transformer-based global representations via a learnable fusion layer.
3. Task-Adaptive Gating: A mechanism that balances shared and task-specific information, improving both classification and regression accuracy by managing gradient dynamics.
4. Interpretability Integration: The incorporation of a constrained LLM module to generate biologically grounded summaries, advancing model transparency beyond standard attention visualization.

Significance and Claims
The paper claims to be one of the first works to jointly regress marker-level protein expression and classify WBC phenotype from label-free DPC microscopy. The results underscore the promise of label-free single-cell imaging for cost-effective hematological profiling, enabling simultaneous phenotype identification and quantitative biomarker estimation without fluorescent staining. The authors assert that their approach demonstrates the feasibility of recovering functional phenotypes from unstained morphology. However, they modestly note limitations: the model approaches a performance ceiling for cell types with high morphological ambiguity (e.g., monocytes vs. lymphocytes), and it cannot fully replace molecular assays for predicting dynamic activation states that lack stable morphological signatures. The work highlights the potential of pairing structured prediction with controlled language generation for interpretable computational hematology.