🤖 AI

Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration

The paper introduces NIAgent, a multi-agent system that autonomously performs end-to-end neuroimaging analysis by collaboratively synthesizing and dynamically optimizing executable code with hierarchical verification, thereby outperforming static workflows in predictive performance and adaptability.

Original authors: Keqi Han, Songlin Zhao, Yao Su, Lifang He, Carl Yang

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Keqi Han, Songlin Zhao, Yao Su, Lifang He, Carl Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to solve a massive, complex puzzle where the pieces are brain scans, and the picture you need to reveal is a medical diagnosis. For decades, doing this has been like hiring a team of highly specialized but rigid robots. You give them a strict manual (a "workflow"), and they follow it step-by-step. If a piece is broken or doesn't fit, the robot stops, and a human expert has to step in, fix the manual, and restart the process. This is slow, expensive, and prone to human error.

The paper introduces NIAgent, a new system designed to be a "Virtual Neuroscientist." Instead of a rigid robot, think of NIAgent as a highly skilled, self-managing construction crew that can build its own blueprint while it works.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Rigid Factory" vs. The "Adaptive Crew"

Current tools (like fMRIPrep) are like a factory assembly line. They are great at doing the same thing over and over, but if a raw material (a brain scan) is slightly different or defective, the line jams. The factory can't "think" about why it stopped or decide to try a different machine. A human has to come in, figure out the problem, and manually tweak the settings.

NIAgent changes this by acting like a dynamic construction crew. It doesn't just follow a pre-written script; it looks at the materials, decides what tools to use, builds the plan, and if something goes wrong, it figures out a new way to fix it without asking a human for help.

2. The Team: A "Specialized Orchestra"

NIAgent isn't just one giant brain; it's a team of four specialized "agents" (AI assistants) working together, led by a Supervisor.

The Supervisor (The Conductor): This agent doesn't do the heavy lifting. It listens to the human's goal (e.g., "Find brain patterns related to ADHD"), breaks the big job into smaller tasks, and assigns them to the right specialist. It keeps the whole project on track.
The Data Awareness Agent (The Librarian): Before building anything, this agent walks into the "library" of raw data to understand what's there. It checks the file names, the folder structure, and the metadata so the team doesn't get lost.
The Processing Agent (The Builder): This agent actually does the heavy construction. It takes the raw brain scans and cleans them up (removing noise, aligning images). Crucially, instead of just clicking buttons, it writes its own computer code to build the workflow. This allows it to handle complex, long projects that would confuse simpler AI.
The Quality Control Agent (The Inspector): This is the most critical new feature. In old systems, quality control was just a report at the end saying "This looks bad." In NIAgent, the Inspector is active. It checks the work while it's being built. If it sees a blurry image, it doesn't just flag it; it tells the Builder, "Hey, this step failed, try a different tool or skip this person." It uses a "two-stage" check: first, it looks at numbers (metrics) to find suspicious scans, and then it uses a "smart eye" (a visual AI) to zoom in and look at the actual images to confirm if they are bad.

3. The Secret Sauce: "Just-in-Time" Tools

Imagine a carpenter who has a warehouse with 10,000 different tools. If you ask them to build a chair, and they have to look at a catalog of all 10,000 tools to find the right hammer, they will get confused and slow down.

NIAgent uses a trick called "Just-in-Time Context Injection."

Instead of showing the AI all possible tools at once, it only shows the few tools needed for the specific task at hand.
If the task is "skull stripping" (removing the skull from the brain scan), the AI only sees the tools for that. This keeps the AI focused, prevents it from getting confused by irrelevant options, and makes it much faster and smarter.

4. The Results: Building Better Bridges

The researchers tested NIAgent on two real-world datasets: one for ADHD (using functional MRI) and one for Alzheimer's (using structural MRI).

Better Predictions: NIAgent consistently built models that predicted diagnoses more accurately than the standard, rigid factory lines used today.
Self-Correction: When the AI tried a method that didn't work, it didn't crash. It realized the failure, switched to a different tool (like switching from one brand of software to another), and kept going.
Human-Level Judgment: The "Inspector" agent was surprisingly good at spotting bad data. Its decisions matched what human experts would have decided about 80-95% of the time, proving it can replace the tedious manual checking humans currently have to do.

The Bottom Line

The paper claims that NIAgent is a major step toward autonomous science. It moves neuroimaging analysis from a "follow the manual" process to a "think and adapt" process. By using a team of specialized AI agents that write their own code and check their own work, the system can turn messy, raw brain scans into reliable medical insights faster and more accurately than current methods, without needing a human to babysit every step.

What the paper does NOT claim:

It does not claim this system is currently ready to diagnose patients in a hospital tomorrow.
It does not claim it works for every type of brain scan or disease (it was tested specifically on ADHD and Alzheimer's datasets).
It does not claim it replaces human scientists entirely, but rather acts as an autonomous assistant that handles the heavy, repetitive lifting of data analysis.

Technical Summary: Towards a Virtual Neuroscientist (NIAgent)

Problem Statement

Transforming raw neuroimaging data (e.g., sMRI, fMRI) into clinically actionable biomarkers is a knowledge-intensive, labor-intensive process. While standardized workflows like fMRIPrep have improved robustness and reproducibility, they remain statically configured. They lack the ability to autonomously reason about downstream scientific objectives, explore alternative analytical strategies, or close the loop between intermediate evidence and subsequent decisions. Consequently, domain experts are often trapped in manual cycles of trial-and-error to tune parameters and remediate pipeline failures, severely constraining the scalability of clinical biomarker development.

The authors identify four key challenges in automating this process:

Ecosystem Heterogeneity: Neuroimaging relies on diverse software backends with incompatible interfaces and intermediate representations, making low-level operations unreliable for Large Language Models (LLMs) to recover at runtime.
Long-Horizon Workflow Complexity: End-to-end analysis requires tightly coupled dataflow, structured control logic, and robust error handling, which conventional flat tool-calling agents struggle to manage.
Quality-Control-Driven Decision Making: Quality control (QC) is not merely a reporting step but a judgment-intensive decision point that must be integrated into the workflow to support downstream adaptations.
Distributed Expertise Requirement: The process demands distinct expertise (data understanding, image processing, QC, statistical modeling) that can overload a single monolithic agent.

Methodology: NIAgent Framework

The authors introduce NIAgent, a multi-agent system designed for autonomous end-to-end neuroimaging analysis. The system reframes neuroimaging analysis as an adaptive sequential decision-making problem, moving away from fixed pipelines toward dynamic, objective-aware workflows.

System Architecture

NIAgent employs a centralized-planning, decentralized-execution architecture:

Supervisor Agent: Acts as the orchestrator, translating high-level natural language objectives into executable workflows, dispatching tasks, and coordinating inter-agent collaboration based on runtime feedback.
Domain-Specific Professional Agents: Specialized agents handle specific subtasks within bounded action spaces:
- Data Awareness Agent: Extracts dataset context (structure, metadata) to ground planning.
- Quality Control (QC) Agent: Performs hierarchical assessment and issues evidence-backed verdicts.
- Processing Agent: Executes preprocessing pipelines by composing tools and writing code.
- Downstream Analysis Agent: Conducts statistical analysis and predictive modeling.

Key Technical Innovations

1. Code-Centric Execution with Composable Primitives
Instead of exposing raw, heterogeneous software tools as flat callable actions, NIAgent encapsulates specialized neuroimaging routines into composable domain-specific primitives. These are atomic Python callables with standardized Input/Output schemas.

Just-in-Time (JIT) Context Injection: To prevent context inflation and reasoning degradation, the system dynamically selects only the task-relevant subset of primitives for each agent's prompt, rather than injecting the entire library.
Code Synthesis: Professional agents synthesize executable Python programs that orchestrate these primitives. This "code-as-act" paradigm enables explicit dataflow, structured control logic (e.g., loops, conditionals), parallel execution, and robust error handling, overcoming the brittleness of sequential tool calls.

2. Hierarchical Agent-as-a-Judge for Closed-Loop QC
The QC Agent operates in a closed loop to drive workflow remediation. It employs a two-stage hierarchical strategy:

Metric Screening: Performs cohort-level outlier detection using quantitative image quality metrics (IQMs) to flag suspicious subjects.
Agentic Visual Inspection: For flagged subjects, the agent invokes a Vision-Language Model (VLM) configured with scoring criteria and few-shot exemplars. The VLM engages in iterative, multi-turn diagnostic protocols, including program-augmented zooming and region-of-interest inspection, to generate a final PASS/FAIL verdict.
Closed-Loop Adaptation: Verdicts are fed back to the Supervisor, which dynamically adjusts the workflow (e.g., excluding subjects, switching preprocessing backends, or refining parameters) based on the evidence.

Experimental Results

The authors evaluated NIAgent on two benchmark datasets: ADHD-200 (functional MRI, $N=200$ ) and ADNI (structural MRI, $N=200$ ). The system was tasked with developing complete workflows from raw data to predictive models for diagnosis.

Performance Metrics

Predictive Performance: NIAgent outperformed standard predefined workflow baselines (fMRIPrep, AFNI, FSL, SPM, ANTs) on both datasets.
- On ADHD-200, NIAgent achieved an AUC of 65.55% compared to the best baseline (AFNI) at 62.60%.
- On ADNI, NIAgent achieved an AUC of 93.00% compared to the best baseline (fMRIPrep) at 90.60%.
Execution Stability: NIAgent achieved a 100% completion rate across five independent runs on both datasets, whereas several ablation variants failed completely.
Agentic Behavior: The system demonstrated sophisticated behaviors, including exploring multiple candidate preprocessing pipelines (e.g., switching from fMRIPrep to AFNI or FSL based on QC outcomes), performing adaptive modeling refinement, and successfully recovering from runtime errors.

Ablation Studies

Ablation studies confirmed the necessity of the architectural components:

Single-Agent: Collapsing all roles into one agent led to state drift, hallucinations, and a sharp drop in error recovery (31% on ADNI).
Without JIT Injection: Injecting the full primitive library caused tool misuse and instruction violations due to context overload.
Without Code-Centric Execution: Replacing program synthesis with flat tool calling resulted in catastrophic failure (0/5 completion) due to scalability and orchestration errors.
Without Primitive Abstraction: Removing encapsulated primitives forced agents to generate low-level commands, leading to severe command misuse and execution failures.

Quality Control Alignment

The proposed hierarchical QC module showed moderate to strong agreement with human raters (measured by Gwet's AC1), outperforming metric-only or non-agentic visual-only QC variants. Removing the QC filtering stage significantly degraded downstream predictive performance, particularly on the noisier ADHD-200 dataset, validating the necessity of automated, evidence-driven sample exclusion.

Significance and Claims

The paper claims that NIAgent represents a significant step toward a "virtual neuroscientist" by bridging the gap between static, predefined pipelines and the dynamic, reasoning-driven workflows of human researchers.

Autonomy and Adaptability: The system demonstrates that neuroimaging analysis can be framed as a long-horizon decision-making problem where the agent autonomously constructs, refines, and adapts workflows based on data characteristics and intermediate evidence.
Robustness: The code-centric execution over composable primitives provides a stable substrate for handling the heterogeneity of the neuroimaging ecosystem, enabling reliable long-horizon execution that flat tool-calling agents cannot achieve.
Closed-Loop QC: By integrating QC as a decision-making component rather than a post-hoc report, NIAgent effectively intercepts artifacts and poor-quality features, improving the robustness of final scientific deliverables.
Modularity: The framework is designed to be extensible, providing a foundation for future systems to support broader modalities, richer primitive libraries, and diverse downstream tasks.

The authors conclude that while the current study is bounded to specific modalities (sMRI/fMRI) and tasks, the proposed framework offers a modular and extensible foundation for advancing neuroimaging analysis from manual, static processes to autonomous, adaptive scientific workflows.