Modeling and Tracking of Heterogeneous Cell Populations via Open Multi-Agent Systems

This paper presents an enhanced cell-tracking algorithm that utilizes open multi-agent systems and an Extended Kalman Filter to model, parameterize, and predict the complex dynamics, interactions, and lineage of heterogeneous cell populations, such as osteosarcoma and mesenchymal stromal cells, in co-culture microscopy videos.

The Gist

Imagine you are watching a busy, chaotic dance floor through a security camera. But this isn't just any dance floor; it's a microscopic one filled with two very different groups of dancers: normal, calm dancers (healthy cells) and hyperactive, aggressive dancers (cancer cells).

Your goal is to keep track of every single dancer: where they are, who they are, who they are dancing with, and when they suddenly split into two new dancers (a process called mitosis).

This is exactly what the researchers in this paper tried to solve, but with a twist: instead of just watching the video, they built a mathematical "rulebook" to predict how the dancers move.

Here is the breakdown of their solution in simple terms:

1. The Problem: Why is this so hard?

Usually, scientists use "black box" AI (like deep learning) to track cells. Think of this like hiring a super-smart but mute security guard who can point at a dancer and say, "That's Person A," but has no idea why Person A is moving or what they will do next.

• The Issue: These AI guards need thousands of hours of video to learn. In biology, we rarely have that much data. Also, they don't understand the rules of the dance (like how cancer cells push away from healthy ones).
• The Challenge: In this specific video, the number of dancers keeps changing. New ones enter the room, some leave, and occasionally, one dancer splits into two. Most tracking software gets confused when the crowd size changes.

2. The Solution: The "Open Multi-Agent" Rulebook

Instead of a black box, the authors created a transparent, rule-based system inspired by how flocks of birds or schools of fish move. They call this an "Open Multi-Agent System."

Imagine every cell is an autonomous robot with its own personality. The researchers wrote a set of instructions (a mathematical model) for these robots:

• The Cohesion Rule: "Stay close to your friends."
• The Alignment Rule: "Move in the same direction as your neighbors."
• The Repulsion Rule: "Don't bump into people you don't like."
• The Shape Rule: "Try to keep your specific body shape (round vs. long)."

The Magic Trick:
They didn't just guess these rules. They watched a small amount of real video, measured how the cells actually moved, and used math to tune the rules until they perfectly matched reality. It's like tuning a radio until the static clears and you hear the music perfectly.

3. The Engine: The "Crystal Ball" (Extended Kalman Filter)

Once they had the rulebook, they plugged it into a mathematical engine called an Extended Kalman Filter (EKF).

Think of the EKF as a super-powered crystal ball:

1. Prediction: Based on the rulebook, the crystal ball predicts where every cell should be in the next second.
2. Observation: The camera takes a picture and sees where the cells actually are.
3. Correction: The crystal ball compares its prediction with the reality and says, "Ah, I was slightly off. Let me adjust my guess."

The "Open" Part:
Most crystal balls break if the number of people in the room changes. This one is special. It can handle:

• Mitosis: If a dancer splits, the system instantly knows, "Okay, one person became two. I need to add a new ID to my list."
• Entrances/Exits: If a dancer walks out of the camera frame, the system quietly removes them from the list. If a new one walks in, it adds them.

4. The Result: A Family Tree

Because the system tracks every split and every movement so carefully, it can draw a Family Tree (Lineage Tree) at the end of the video.

• It knows exactly which cancer cell divided, which daughter cell it produced, and how that new cell interacted with the healthy cells.
• This is crucial for doctors because it helps them understand how cancer grows and spreads.

5. Why is this better than the competition?

The researchers tested their method against two other popular tracking tools (SORT and DeepSORT).

• SORT is like a basic calculator: fast, but it doesn't understand the dance rules well.
• DeepSORT is like a super-computer AI: very smart, but it needs a massive library of videos to learn. Since the researchers only had 4 short videos, the AI got confused and performed poorly.
• The New Method (HEOM-EKF): It was the Goldilocks. It didn't need thousands of videos because it used the "rulebook" approach. It was faster than the super-computer AI and much more accurate than the basic calculator.

The Big Picture

This paper is about teaching computers to understand biology not just by "memorizing" pictures, but by understanding the physics and rules of life. By treating cells like intelligent agents with specific behaviors, the researchers created a tool that can track complex, changing crowds of cells with high precision, even when data is scarce. This could help scientists figure out how to stop cancer cells from "dancing" their way into spreading.

Technical Summary

1. Problem Statement

The paper addresses the challenge of tracking heterogeneous cell populations in in vitro time-lapse microscopy videos. Unlike standard tracking tasks which often focus on monocultures (single cell types), this work targets co-culture scenarios where multiple cell types (e.g., tumor cells and normal stromal cells) interact dynamically.

Key challenges identified include:

• Heterogeneity: Different cell types exhibit distinct morphological and behavioral dynamics.
• Dynamic Population Changes: The number of cells changes over time due to biological processes (mitosis) and observation constraints (cells entering or exiting the Field of View).
• Data Scarcity: High-quality, annotated co-culture datasets are rare, making data-hungry deep learning approaches difficult to apply effectively.
• Interpretability: Many state-of-the-art methods rely on "black-box" deep learning models that lack insight into the underlying biological mechanisms driving cell motion.

2. Methodology

The authors propose a novel algorithm called HEOM-EKF (Heterogeneous Open Multi-Cell EKF-based Tracking). The methodology is structured into three main phases:

A. System Modeling: Open Multi-Agent Systems

The cell population is modeled as an Open Multi-Agent System, where each cell is an agent.

• State Representation: Cells are represented by a state vector including position, velocity, area, aspect ratio, orientation, and their rates of change. Two formats are supported: a detailed Minimum Enclosing Rectangle (MER) for orientation-aware tracking and a simplified axis-aligned bounding box for standard benchmarking.
• Interaction Dynamics: The motion of a cell is governed by a parametric function dependent on its own state, the states of its neighbors (within a sensing radius), and its specific cell type parameters.
  • Behavioral Terms: Modeled after swarm intelligence (cohesion, alignment, repulsion) and chemotaxis.
  • Morphological Terms: Novel terms modeling the tendency to maintain a target area and aspect ratio specific to the cell type.
• Open System Dynamics: The framework explicitly handles "open" system events:
  • Mitosis: A parent cell splits into two daughters (parent removed, two new agents added).
  • Entry/Exit: Cells entering or leaving the camera frame.

B. Parameter Identification

Instead of training a neural network, the authors use System Identification to learn the model parameters ($\theta$) from real data.

• Approach: The dynamics are formulated as a linear-in-parameters system. A Least Squares (LS) problem is solved to find the optimal parameters that minimize the error between predicted and observed trajectories.
• Robustness: To handle noise and outliers, the authors employ Kalman-based differentiation for velocity estimation and Cook's distance metrics to remove influential outliers before solving the LS problem.
• Data Augmentation: Due to the small dataset size, rotations and flips are applied to the training data to improve generalization.

C. Tracking Algorithm (HEOM-EKF)

The tracking loop follows a Track-by-Detection paradigm but integrates the learned multi-agent model into an Extended Kalman Filter (EKF).

1. Object Detection: Cells are localized in each frame (using semi-automatic segmentation).
2. Prediction (EKF Step): The state of existing cells is predicted using the learned non-linear multi-agent dynamics.
3. Data Association: Detected cells are matched to predicted tracks using the Hungarian algorithm based on Intersection over Union (IoU) similarity.
4. Population Change Handling:
   • Mitosis Detection: If a predicted track splits and new detections appear nearby, the algorithm updates the lineage tree (parent $\to$ two daughters).
   • Entry/Exit Detection: Unmatched detections near image boundaries are flagged as entries; unmatched predictions persisting near boundaries are flagged as exits.
5. Dimensionality Adaptation: The EKF matrices (state vector, covariance, gain) are dynamically resized to accommodate the changing number of agents.

3. Key Contributions

1. Novel Algorithm (HEOM-EKF): A tracking framework specifically designed for heterogeneous, time-varying cell populations using open multi-agent systems.
2. Interpretable Modeling: Unlike deep learning approaches, the motion prediction model is a structured dynamical system with identifiable parameters that offer biological insights (e.g., quantifying cohesion vs. repulsion).
3. Handling Population Dynamics: The method explicitly models and reconstructs the cell lineage tree, distinguishing mitosis events from cells entering/exiting the field of view.
4. New Dataset: The authors created and annotated a unique dataset involving Osteosarcoma (143b) and Mesenchymal Stromal Cells (MSC) co-cultures, a scenario not covered by standard benchmarks like the Cell Tracking Challenge.
5. Data Efficiency: The approach achieves high performance with very limited data (4 videos) by leveraging physical/biological priors rather than massive datasets.

4. Experimental Results

The algorithm was tested on the co-culture dataset and compared against SORT and DeepSORT baselines using cross-validation.

• Performance Metrics:
  • MOTA (Multiple Object Tracking Accuracy): HEOM-EKF achieved significantly higher scores (e.g., 83.56% to 96.17% across wells) compared to SORT (40–80%) and DeepSORT (52–55%).
  • MOTP (Precision): The proposed method provided more precise bounding box predictions (95–97%) compared to baselines.
  • IDF1 (Identity F1): Superior identity preservation, crucial for lineage tracking.
• Biological Insights: The identified parameters successfully distinguished between MSCs and Osteosarcoma cells.
  • MSCs showed higher cohesion and alignment (organized behavior).
  • Osteosarcoma cells showed lower cohesion, higher repulsion, and greater morphological variability (aggressive, disorganized behavior).
• Efficiency: While slower than the geometric SORT (due to EKF complexity), it is significantly faster than DeepSORT (which requires heavy neural network inference) and operates within tens of milliseconds per frame, suitable for real-time microscopy intervals.
• Lineage Reconstruction: The algorithm successfully reconstructed cell lineage trees, correctly identifying the higher mitosis rate of cancer cells compared to the quiescent stromal cells.

5. Significance

This work represents a significant shift in biomedical cell tracking from purely data-driven "black box" methods to physics-informed, interpretable modeling.

• Clinical Relevance: It provides a tool to study tumor aggressiveness and drug resistance by analyzing how cancer cells interact with and evolve within a stromal environment.
• Data Scarcity Solution: It demonstrates that for specialized biological problems where large datasets are unavailable, tailored system identification and multi-agent modeling can outperform deep learning.
• Framework for Systems Biology: The ability to extract quantitative parameters (cohesion, repulsion, target shape) from video data offers a new way to quantify phenotypic differences and cellular interactions in complex co-culture systems.