Technical Summary: Cell as Point (CAP)

Problem Statement

Conventional cell tracking methodologies typically rely on multi-stage pipelines that necessitate explicit cell detection or segmentation in each frame as a prerequisite for tracking. These approaches, often categorized as SegLnk, Seg&Lnk, or DetSegLnk, suffer from several critical limitations:

1. Resource Intensity: They require substantial computational resources to generate high-quality segmentation masks and produce high-quality ground truth annotations, which are labor-intensive and time-consuming.
2. Error Accumulation: Errors occurring in early stages (detection/segmentation) are accumulated and amplified in subsequent tracking stages, limiting overall framework performance.
3. Data Imbalance and Long Sequences: Existing methods struggle with the imbalanced occurrence of cell division events in datasets and face challenges in efficiently inferring long sequences, often leading to the omission of complete division events or requiring excessive computational time.

Methodology: Cell as Point (CAP)

The authors propose CAP (Cell as Point), a novel end-to-end one-stage framework that reimagines cell tracking by treating each cell as a point rather than relying on segmentation masks. CAP jointly tracks numerous cell points within a sequence by leveraging inherent correlations among their trajectories.

Core Representation

Instead of segmentation masks, CAP utilizes two key concepts to represent cell tracking data:

• Cell Point Trajectory: Defined by the centroid locations $L^i_t = (x^i_t, y^i_t)$ of $N$ cells over time $T$.
• Cell Point Visibility: A binary indicator $V^i_t \in \{0, 1\}$ denoting whether a cell exists at time $t$. This representation explicitly handles cell states including migration, mitosis (division into two daughters), and apoptosis (death).

Key Innovations

To address the challenges of data imbalance and long-sequence inference, CAP introduces two specific mechanisms:

1. Adaptive Event-Guided (AEG) Sampling:

   • Problem: Cell division events are rare compared to migration, leading to data imbalance where models fail to learn division dynamics effectively.
   • Solution: AEG is a temporal sampling strategy that prioritizes frames containing complete cell division events (from the death of the mother cell to the birth of daughter cells). It adaptively calculates the probability of sampling a sequence containing a division event based on the total number of divisions and their duration in the dataset. This ensures the model is exposed to balanced division events during training.

2. Rolling-as-Window (RAW) Inference:

   • Problem: Processing entire long sequences in a single stage is computationally expensive and difficult for capturing newly emerging cells.
   • Solution: RAW is an inference strategy that processes sequences using a rolling window. It sequentially extracts sub-sequences for inference. When a new cell is detected (via visibility changes), its predicted location is immediately concatenated to the query points for the subsequent window. This allows for continuous, stable tracking of new cells over extended sequences without requiring the entire sequence to be processed at once.

Architecture

• Backbone: The framework leverages a transformer-based neural network ($\Theta$) built upon the RAFT (Recurrent All-Pairs Field Transforms) architecture, specifically adapted from CoTracker.
• Feature Extraction: A 2D CNN extracts image features. Tracking features are distilled from these maps via bilinear interpolation at cell point locations.
• Correlation Volume: CAP employs a 4D correlation volume to compute similarities between tracking features and image features across scales, enabling the handling of large displacements.
• Iterative Updates: The transformer performs $M$ iterative updates, refining cell trajectories and tracking features using cross-trajectory and cross-time attention.
• Output: The model predicts updated locations and visibility scores. A sigmoid activation determines if a cell has divided (producing two daughter locations) or died.
• Loss Function: The model is supervised using a trajectory loss (location regression) and a visibility loss (cross-entropy for binary existence).

Key Contributions

1. One-Stage Framework: CAP is the first end-to-end framework to track cells without explicit detection or segmentation stages, relying solely on tracking ground truth (coarse masks and cell acyclic graphs).
2. AEG Sampling: A novel strategy to resolve the imbalance of cell division occurrences in training data, ensuring robust learning of division events.
3. RAW Inference: A unified inference approach that efficiently handles long sequences and captures new cell emergence without dataset-specific tailoring.
4. Efficiency and Performance: The framework achieves state-of-the-art (SOTA) performance while significantly reducing inference time and annotation requirements.

Experimental Results

The authors evaluated CAP on the DeepCell database (HeLa, PC-3, 3T3, RAW264) and ISBI CTC datasets (U373, GOWT1, Huh7).

• Performance: CAP achieved SOTA Tracking Accuracy (TRA) scores across most datasets. For instance, on HeLa, it achieved a TRA of 0.926, and on U373, it achieved 0.985 with zero structural errors (EdgeSplit, EdgeAdd, EdgeDelete, EdgeSemantic).
• Efficiency: CAP demonstrated a massive reduction in inference time, being 8 to 32 times faster than existing multi-stage methods. For example, on the HeLa dataset, CAP required only 1.3 seconds compared to 23.3s for GNN and 41.5s for EmbedTrack.
• Ablation Studies:
  • Disabling AEG sampling caused significant performance drops (e.g., U373 TRA dropped from 0.985 to 0.556), confirming its necessity for learning division events.
  • Removing cross-trajectory attention (using time-only attention) reduced performance, highlighting the importance of joint tracking.
  • Reducing the feature stride from 8 to 4 significantly improved accuracy, indicating the need for higher-resolution features for fine-grained dynamics.

Significance and Claims

The paper claims that CAP represents a significant shift in cell tracking by eliminating the dependency on segmentation-based preprocessing. Its primary significance lies in:

• Practicality: By reducing the need for high-quality segmentation masks, it lowers the barrier for data annotation, making it more viable for clinical applications where expert labeling is scarce.
• Scalability: The 8–32x speedup in inference makes real-time or large-scale analysis of cell behaviors feasible.
• Robustness: The framework effectively handles complex biological events like mitosis and apoptosis within a unified one-stage model, addressing the limitations of error accumulation found in multi-stage pipelines.

The authors acknowledge limitations regarding generalization across different imaging devices, cell densities, and annotation styles, noting that variations in expert annotation can affect performance. Future work is proposed to address domain shifts and diverse cell populations.