Technical Summary: TopoAgent

Problem Statement

Topological Data Analysis (TDA), specifically Persistent Homology (PH), captures geometric structural properties in medical images (e.g., connected components, loops, shape characteristics) that conventional pixel-level deep learning often neglects. To utilize these insights, researchers rely on topological descriptors to convert persistence diagrams (PDs) or raw images into fixed-length feature vectors. However, no single descriptor is universally effective across all datasets; each encodes distinct geometric characteristics with specific mathematical properties, hyperparameters, and failure modes.

Determining the optimal descriptor for a given image dataset is currently a manual, expertise-dependent process involving trial-and-error. This is exacerbated by the lack of standardized benchmarks for comparing descriptors across diverse medical morphologies. Furthermore, while Large Language Model (LLM)-based agents have shown promise in medical AI, no existing framework can automatically compute PH, reason about topological structures, and produce suitable topological feature vectors for downstream tasks without task-specific training.

Methodology: TopoAgent Framework

The authors propose TopoAgent, an LLM-based agentic framework designed to automate topology learning. It operates through a Perception–Reasoning–Action–Reflection (PRAR) loop, supported by a distilled skill set and dual memory, without requiring task-specific training.

1. Core Components

• Toolset (21 Tools):
  • Perception Tools (6): Handle image loading, quality analysis, denoising, PH computation (via GUDHI with cubical complex filtration), PH profile extraction, and Betti ratio computation.
  • Descriptor Tools (15): A library of 15 topological descriptors, including 10 PH-based (e.g., Persistence Images, Persistence Landscapes) and 5 image-based (e.g., Minkowski functionals, Euler characteristic transform).
• Skill Set ($S$): A static knowledge base distilled offline from systematic evaluations, comprising:
  • $S_{prop}$: Qualitative properties of descriptors (definitions, strengths, weaknesses).
  • $S_{rank}$: Empirical evidence in the form of tiered rankings and reasoning chains mapping PH patterns to descriptor suitability.
  • $S_{param}$: Validated parameter configurations for specific object types.
• Dual Memory:
  • Short-term Memory ($M_s$): Tracks tool invocations within a single run to prevent repeated failures.
  • Long-term Memory ($M_l$): Accumulates diagnostic entries (failed descriptors, causes, corrections) across runs within a dataset to enable runtime adaptation.

2. The PRAR Pipeline

1. Perception: The agent deterministically computes the PH profile ($h(I)$) and visual statistics ($v(I)$). An LLM interprets these alongside the raw image to identify the object type ($o$) (e.g., cells, glands, vessels). This step ensures the agent understands the topological and visual context before reasoning.
2. Reasoning: This phase is split into two steps to mitigate anchoring bias:
   • Proposal: The agent proposes a candidate descriptor and parameters based on $S_{prop}$ and observed characteristics, without seeing the full empirical rankings.
   • Determination: The agent integrates the full $S_{rank}$ and $M_l$ to weigh the proposal against empirical evidence, potentially overriding the proposal or adopting a correction from memory.
3. Action: The agent executes the determined descriptor tool with validated parameters to produce a topological feature vector ($f$).
4. Reflection: The agent assesses the quality of $f$ using summary statistics (sparsity, variance, etc.) and reference ranges from $S$. If the output is unsuitable (e.g., excessive sparsity), the agent records a diagnosis in $M_l$ and retries from the determination step. If two retries fail, it falls back to the top-ranked descriptor for the object type.

Key Contributions

1. TopoAgent: The first agentic framework bridging LLM-based reasoning and TDA, enabling automated topology learning from raw medical images with full reasoning traces.
2. TopoBenchmark: The first standardized benchmark for evaluating topological descriptors across diverse medical morphologies. It comprises 26 datasets (113,182 samples) spanning five object types: cells, glands/lumens, organ shapes, vessel trees, and surface lesions. It uses convergence-based sampling to ensure consistent evaluation.
3. Empirical Validation: Demonstration that adaptive, per-image descriptor determination significantly outperforms fixed-descriptor baselines and general-purpose LLMs.

Results

Experiments were conducted on the TopoBenchmark using six classifiers (including TabPFN, XGBoost, and Random Forest).

• Performance: TopoAgent achieved an average balanced accuracy of 68.21%.
• Comparisons:
  • Outperformed the strongest baseline (an "object-type oracle" that selects the best descriptor per category without per-image reasoning) by 9.32%.
  • Outperformed general-purpose LLMs (GPT-4o, Gemini, Claude) equipped with the same tools but lacking the structured PRAR pipeline and skill set by over 21%.
  • Outperformed the medical imaging agent MedRAX by nearly 18%.
• Ablation Studies:
  • The Skill Set ($S$) was the most critical component; removing it dropped performance to near the level of the object-type oracle.
  • The Proposal step was essential for per-image adaptation; removing it reduced performance to the oracle level, confirming the value of the anti-anchoring design.
  • Reflection was crucial for correcting parameter mismatches, particularly for organ shapes and lesions.
• Downstream Integration: When integrated into ResNet-152 and SwinV2-B backbones, TopoAgent's adaptively determined descriptors matched or exceeded the performance of PHG-Net (a learnable topological layer) on external datasets, providing interpretable geometric signatures without additional training.
• Efficiency: The pipeline completes in a single pass for ~87% of images, averaging 4.2 LLM calls and ~32 seconds per image.

Significance and Claims

The paper claims that TopoAgent addresses a critical bottleneck in medical image analysis: the reliance on extensive domain expertise and trial-and-error to select topological descriptors. By automating this process, TopoAgent makes topological insights actionable for downstream tasks without requiring task-specific training or differentiable topological layers.

The authors emphasize that their approach provides interpretable feature vectors and adapts to the specific topological characteristics of each image, rather than relying on a one-size-fits-all descriptor. They acknowledge limitations, noting that the skill set is distilled from 2D medical data and may not generalize to 3D volumetric data or non-medical domains without further extension. Future work is proposed to extend the benchmark to 3D and explore multi-descriptor fusion.