🤖 AI

NeuroRefiner: Morphology-Aware Multi-Agent Refinement for 3D Fluorescence Microscopy Neuron Segmentation

NeuroRefiner is a morphology-aware multi-agent system that combines collaborative error diagnosis, instruction generation, and validation with a dedicated TopoRefineNet tool to iteratively refine 3D neuron segmentation, achieving state-of-the-art accuracy and topological integrity on challenging fluorescence microscopy datasets.

Original authors: Haiyang Yan, Jinyue Guo, Yanchao Zhang, Bingqing Wang, Zhenchen Li, Jing Liu, Jiazheng Liu, Linlin Li, Hua Han

Published 2026-08-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Haiyang Yan, Jinyue Guo, Yanchao Zhang, Bingqing Wang, Zhenchen Li, Jing Liu, Jiazheng Liu, Linlin Li, Hua Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to map the entire internet, but instead of fiber-optic cables, you are tracing billions of tiny, glowing threads floating in a dark, foggy room. This is the daily challenge for neuroscientists who study the brain. They use powerful microscopes to take 3D pictures of neurons, the brain's communication cells, which look like delicate, spindly trees. To understand how the brain thinks, they need to turn these fuzzy, noisy pictures into perfect, clean maps of every single branch. However, the "fog" (noise) and the "tangled wires" (complex shapes) often trick standard computer programs. These programs might snap a long branch into pieces or accidentally glue two different trees together, creating a map that looks right at first glance but falls apart when you look closely. Getting these maps right is crucial because if the map is broken, our understanding of how the brain works is broken too.

Enter NeuroRefiner, a clever new system that acts like a team of expert editors working together to fix these messy maps. Instead of relying on a single, super-fast computer program that tries to draw the whole picture in one go (and often makes mistakes), NeuroRefiner uses a "multi-agent" approach. Think of it as a high-stakes game of "Spot the Difference" played by three specialized AI characters: a Global Inspector, a Refinement Advisor, and a Change Validator.

The Global Inspector is the big-picture detective. It looks at the whole 3D image and asks, "Does this look like a complete tree, or are there broken branches?" It uses a mathematical trick called Betti numbers (basically a way to count how many separate pieces a shape has) to find spots where the map is fragmented or has extra noise. Once it finds a problem, it doesn't try to fix it itself. Instead, it calls in the Refinement Advisor. This agent is the creative editor; it looks at the specific broken spot and writes a clear, natural language instruction, like "Connect the two floating pieces in the upper-left corner" or "Remove this tiny speck of noise."

Finally, the Change Validator acts as the strict quality control manager. It takes the instruction and the proposed fix, checks to see if the edit actually follows the rules and improves the shape, and decides whether to keep the change or reject it. If the fix is bad, the system loops back, and the Advisor tries again with a new instruction. This cycle repeats until the map is perfect.

To make this teamwork possible, the researchers built a special tool called TopoRefineNet. You can think of this as a magical paintbrush that understands human instructions. When the Advisor says "connect these," the paintbrush knows exactly which pixels to color to make that connection happen, even in a complex 3D space.

The results of this approach are impressive. When tested on three different challenging datasets of brain images, NeuroRefiner consistently outperformed the best existing methods. On the toughest dataset, known as ZBFWB, which is full of low-contrast and tricky neuron shapes, the new system improved the accuracy score (F1 score) by 3.02% compared to the previous best method. It also significantly reduced the number of broken connections and extra noise. The authors found that this "teamwork" approach is far better than trying to fix errors with a single pass, proving that by breaking the problem down into inspection, instruction, and validation, we can create much more accurate maps of the brain's intricate wiring.

Technical Summary: NeuroRefiner

Problem Statement

Accurate 3D neuron segmentation in fluorescence microscopy is a critical bottleneck for neuroscience, particularly for deriving neuronal skeletons for quantitative analysis. Existing segmentation methods face significant challenges due to the sparse and elongated morphology of neurons, which are often corrupted by substantial noise and exhibit extreme signal sparsity in high-resolution 3D volumes.

Current state-of-the-art approaches typically rely on sliding-window strategies with limited receptive fields. This precludes the modeling of long-range dependencies essential for preserving global neuronal morphology, often resulting in fragmented segmentation and topological errors (e.g., broken connections, false positives/negatives). Furthermore, these end-to-end models operate as "black boxes," lacking interpretability and offering no mechanism to iteratively correct significant topological errors. While agent-based biomedical image segmentation methods exist, they often rely on foundation models (like MedSAM2) that process 3D volumes slice-by-slice using 2D encoders, which is inadequate for sparse neuronal structures, or they lack the flexibility for iterative, instruction-guided refinement.

Methodology: NeuroRefiner

To address these limitations, the authors propose NeuroRefiner, a novel LLM-based multi-agent system designed for iterative, topology-aware neuron segmentation. The system mimics the cognitive paradigm of human neuroscientists, alternating between global observation and local refinement.

1. Multi-Agent Collaborative Framework

NeuroRefiner operates through a closed-loop pipeline involving three specialized agents and a dedicated execution tool:

Global Inspector: This agent performs a holistic assessment of the initial segmentation mask ( $M^{(t)}$ ). It utilizes Maximum Intensity Projection (MIP) along the z-axis to create 2D sub-blocks and computes 0th-order Betti numbers ( $\beta_0$ ) to quantify topological irregularities (e.g., the number of connected components). It identifies sub-regions violating topological priors and categorizes error types (False Positives or False Negatives).
Refinement Advisor: Upon identifying anomalous regions, this agent generates structured, morphology-aware refinement instructions ( $t_i$ ). It analyzes orthogonal projections (xy and yz planes) to localize defects along the z-axis and formulates natural language instructions (e.g., "connect neurons in the upper-left-posterior region" or "remove isolated noise").
Change Validator: This agent acts as a gatekeeper, evaluating the output of the refinement tool against the original mask and the generated instruction. It accepts refinements only if they satisfy two criteria: instruction adherence (the edit matches the intent) and topological improvement (reduced fragmentation/noise without new artifacts). If validation fails, the loop retries with a new instruction.

2. TopoRefineNet: The Execution Tool

To bridge high-level agent reasoning with low-level voxel operations, the authors introduce TopoRefineNet, a dedicated 3D U-Net-based tool.

Architecture: It treats segmentation refinement as an instruction-guided image editing task. It takes the image ( $I_i$ ) and the noisy mask ( $M_i$ ) as input, encoded by a visual encoder. The natural language instruction ( $t_i$ ) is embedded via a frozen text encoder.
Mechanism: The system employs cross-attention between the deepest visual features and the text embeddings to dynamically recalibrate visual features based on the editing operations specified in the instruction.
Training Strategy: A two-stage training paradigm is employed:
1. Stage 1: Trained on synthetic data where ground-truth masks are perturbed (erosion for False Negatives, copy-paste/noise for False Positives) to learn cross-modal alignment.
2. Stage 2: Trained on real-world segmentation outputs from various networks containing representative topological errors, synthesized with instructions generated by a VLM (Qwen3-VL) to enhance generalization.

Key Contributions

First LLM-based Multi-Agent System for Neuron Segmentation: NeuroRefiner is the first framework to achieve topology-aware, iterative optimization of segmentation results using a collaborative agent system, moving beyond single-pass inference.
TopoRefineNet: The design of a dedicated segmentation editing tool coupled with a tailored two-stage training strategy that aligns model behavior with structural refinement instructions, effectively handling sparse neuronal structures.
Enhanced Interpretability and Robustness: By decoupling reasoning from voxel-level operations via natural language, the system offers transparent, auditable diagnostic decisions and achieves state-of-the-art performance across diverse datasets.

Experimental Results

The authors evaluated NeuroRefiner on three benchmarks: BigNeuron, CWMBS, and ZBFWB.

Performance: NeuroRefiner outperformed state-of-the-art methods, including 3D U-Net, SwinUNETR, nnUNet, and neuron-tailored architectures like GBP-Net and SGSNet.
- On the challenging ZBFWB dataset (characterized by low contrast and long-range axonal projections), NeuroRefiner achieved a 3.02% improvement in F1 score over the best-performing baseline (GBP-Net) and a 13.19% improvement over the initial 3D U-Net segmentation.
- On the CWMBS dataset, the method improved F1 scores by over 10% on the weak-signal subset compared to baseline segmentation masks.
Metrics: The method demonstrated superior performance in topological completeness (MES), spatial distance (SD), and structural similarity (SSD), indicating better preservation of global connectivity and reduced fragmentation.
Ablation Studies:
- Removing any of the three agents (Global Inspector, Refinement Advisor, Change Validator) resulted in significant performance drops, confirming the necessity of the synergistic collaboration.
- The system is robust to the choice of foundation model (VLM), with Qwen3-VL-8B showing optimal performance but weaker models still yielding significant gains.
- The two-stage training strategy for TopoRefineNet was proven essential, as a single-stage training approach led to higher rejection rates and lower F1 scores.

Significance and Claims

The paper claims that NeuroRefiner addresses the critical bottleneck of topological fidelity and interpretability in automated neuronal reconstruction. By formalizing the human expert workflow into a multi-agent loop, the system produces segmentations that are not only more accurate but also consistent with topological priors.

The authors emphasize that their framework is agnostic to the foundation model, ensuring it can evolve alongside advancements in Vision-Language Models (VLMs). They position NeuroRefiner as a step toward transparent, iterative, and topology-aware automation in biomedical image analysis, overcoming the limitations of "black box" end-to-end models that struggle with the complex, sparse nature of neuronal structures.

Limitations: The authors acknowledge that the current 2D MIP-based approach loses depth information in dense regions, potentially hindering precise 3D error localization. They suggest future work could incorporate depth-aware encoding to resolve occlusions.