🔬 physics

Agentic Autoresearch for CT Reconstruction

This paper demonstrates that an autonomous LLM agent can independently implement and benchmark 26 CT reconstruction methods, revealing that rankings based on idealized, noiseless data fail to predict performance under realistic noise conditions and highlighting the necessity of multi-factor realistic benchmarks for evaluating robustness.

Original authors: Andreas Maier, Lucas Kachelriess, Siming Bayer, Yixing Huang, Yan Xia, Amber Simpson, Moritz Zaiss

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Andreas Maier, Lucas Kachelriess, Siming Bayer, Yixing Huang, Yan Xia, Amber Simpson, Moritz Zaiss

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, blurry jigsaw puzzle. In the world of medical imaging, this puzzle is a picture of the inside of a human body, created by X-rays. The goal is to take the raw, noisy data from the scanner and turn it into a crystal-clear image that doctors can trust. For a long time, scientists have been teaching computers to do this using "deep learning," a type of artificial intelligence that learns by looking at thousands of examples. But there's a catch: if the computer learns too much about the specific examples it saw, it might start "hallucinating"—inventing body parts that aren't actually there, like adding a fake bone just because it looks cool. To stop this, researchers make the AI play by the rules of physics, ensuring the final picture matches the actual X-ray measurements.

Now, imagine you have a robot assistant who is incredibly good at reading instructions and writing code, but it doesn't have its own ideas. This paper asks a big question: Can we let this robot assistant run the entire research lab on its own? Can it try out dozens of different ways to solve the puzzle, tweak the code, run the experiments, and tell us which method is the best, all without a human holding its hand every step of the way? The researchers wanted to find out if this "agentic" robot could do the hard work of scientific discovery, and more importantly, they wanted to see if the "best" method found on a perfect, clean puzzle would still be the best when the puzzle pieces got a little dirty and noisy, just like in real life.

The Robot Scientist and the Two Puzzles

The authors built a loop where a large language model (a very smart AI that reads and writes text) acts as a researcher. This robot was given a specific job: take a list of 26 different "solutions" (mathematical recipes for fixing CT scans), try to improve them, and see which one works best. The robot didn't just guess; it read the results of a previous attempt, figured out what went wrong, changed one tiny thing in the code, and ran a new test. It did this over and over again, like a student taking a practice test, studying the mistakes, and trying again.

They tested these methods on two very different types of puzzles:

The "Noisy" Puzzle (Mayo Low-Dose CT): This is like trying to see a face in a foggy window. The data is real patient scans, but they are taken with a low dose of radiation, which makes the image grainy and full of static. The goal here is to clean up the noise without blurring the details.
The "Missing Pieces" Puzzle (Sparse-View Breast CT): This is like trying to solve a puzzle where half the pieces are missing. The scanner only took a few pictures from different angles instead of a full rotation. The goal is to figure out what the missing parts look like without making things up.

The Robot's Big Discovery: The "Perfect" Solver is a Trick

The robot successfully implemented, tuned, and tested all 26 methods. It also helped create a new, super-efficient "compact solver" by mixing and matching the best parts of the other methods, though the idea to combine these pieces into one compact solver actually came from a human researcher. This new solver was tiny—using only 969 parameters (the brain cells of the AI) compared to the millions used by the champions. On the noisy puzzle, this tiny solver performed just as well as the giants, tying for the top spot. On the missing-pieces puzzle, it found a different, even smaller recipe that was highly competitive, though not perfect, matching the top tier's performance very closely.

However, the most surprising part of the story happened when the researchers added a little bit of "noise" to the missing-pieces puzzle. They took the models that had been trained on perfect, clean data and asked them to solve the same puzzle, but this time with a little bit of static added to the input.

The ranking completely flipped.

The method that was the undisputed champion on the clean data (a "supervised image-domain denoiser") crashed and burned. It went from being the best to being useless, with its score dropping to zero. It was like a world-class chef who can cook a perfect meal in a clean kitchen but immediately burns the food the moment a little bit of dirt gets in the pot.

Meanwhile, methods that were in the middle of the pack on the clean data suddenly became the heroes. One method that used "physics" (staying true to the X-ray rules) and another that used simple, hand-crafted smoothing rules rose to the top. The robot had found that the "best" method depends entirely on the conditions. A method that wins on a clean, idealized test is not necessarily the one you want to use in the messy, real world.

The Lesson: Don't Trust the Clean Room

The paper argues that we have been looking at the wrong leaderboard. For years, scientists have ranked AI methods based on how well they perform on perfect, noise-free data. This paper shows that this is a trap. A method that looks amazing on a clean test might be incredibly fragile, collapsing as soon as it encounters a tiny bit of real-world imperfection.

The researchers found that if you retrain the "champion" methods on data that includes the noise, they can recover and become good again. This suggests that the failure wasn't because the method was bad, but because it was trained on the wrong kind of data. However, the authors warn that noise is just the easiest problem to fix. In the real world, there are many other things that can go wrong—like the way X-rays bend through bone, or different types of diseases. If a method only works when you fix one specific problem (like noise), it might fail when you face a different one.

The Takeaway

The robot assistant proved it can do the heavy lifting of research: it can write code, run thousands of experiments, and compare results fairly. But the robot also taught us a vital lesson about how we judge success. Just because a method wins a race on a smooth, empty track doesn't mean it can handle a bumpy, rainy road. To build AI that is truly useful for doctors and patients, we need to stop testing on perfect, idealized data and start testing on messy, realistic data that includes all the things that can go wrong. The paper concludes that the future of medical imaging isn't about finding one single "winner," but about creating challenges that test how well these methods handle a whole spectrum of real-world problems at once.

Technical Summary: Agentic Autoresearch for CT Reconstruction

Problem Statement
Deep learning has generated a vast and diverse array of Computed Tomography (CT) reconstruction methods, yet fair comparison remains labor-intensive, largely manual, and often reliant on idealized data. Furthermore, it is unclear whether rankings derived from noiseless or low-noise benchmarks predict how methods perform under realistic physical perturbations. The authors ask two primary questions: (1) Can a Large Language Model (LLM) agent autonomously implement, tune, and benchmark reconstruction methods at scale? and (2) Does a leaderboard measured on ideal data predict robustness under realistic noise, or does the ranking invert under small perturbations?

Methodology
The authors constructed an "agentic loop" driven by a single LLM (Claude Opus 4.8) to automate the research cycle. The agent operates within a fixed budget and a closed loop:

Input: It reads the previous iteration's result, identifies a specific failure mode (e.g., stalled headroom score, residual streaks), and formulates a hypothesis.
Action: It edits a single solver component (hyperparameter or code) and launches a short cluster job.
Evaluation: It reads a single, frozen, calibrated metric: the headroom score ($hr$).
- $hr = \max(0, 1 - \frac{RMSE(\hat{x})}{RMSE(x_{FBP})})$ , where $\hat{x}$ is the reconstruction and $x_{FBP}$ is the Filtered Back-Projection baseline.
- This metric measures improvement over the FBP baseline within the field of view.
Constraint: All methods are grounded in the same differentiable fan-beam projector (PYRO-NN) to ensure fair comparison.

The agent benchmarked 26 established methods across two distinct CT problems:

Mayo Low-Dose CT: A noise-limited problem using real helical data re-binned to fan-beam geometry.
Breast CT Challenge: A 128-view sparse-view problem using synthetic, noiseless phantoms (incompleteness-limited).

The study included a noise-robustness experiment:

Transfer Test: Trained models (optimized on noiseless data) were re-scored on test inputs with mild Poisson noise ( $I_0 = 10^5$ photons) without retraining.
Retraining Test: Models were retrained from scratch on matched-noise data to see if rankings could be restored.

Additionally, the agent attempted to recombine components of the 26 methods into a compact solver with an ultra-low parameter count, guided by a human suggestion to minimize parameters while maintaining performance.

Key Contributions

Agentic Loop Implementation: A fully autonomous loop that implemented, tuned, and benchmarked 26 methods with full provenance, requiring only high-level human guidance for strategy and auditing.
Compact Solver Discovery: The agent derived two problem-specific, ultra-compact solvers:
- Mayo (Noise-limited): A 969-parameter multi-scale bilateral denoiser.
- Breast (Sparse-view): A 195-parameter solver combining filtered data-consistency and primal-dual unrolling.
- These compact solvers achieved performance within the top tier (1% level) using ~0.4% of the champion's parameters.
Robustness Inversion: A demonstration that ideal-data rankings do not predict noise robustness. A small, realistic noise perturbation inverted the breast CT leaderboard.
Open Source Release: All 26 method implementations, the compact solvers, and the agentic loop code are released as open source.

Results

Benchmarking: On the Mayo dataset, the top methods formed a statistically indistinguishable tier (no single winner). On the noiseless Breast dataset, supervised dual-domain denoisers and unrolled networks dominated (headroom $hr \approx 0.89$ ).
The Inversion: When mild noise was added to the Breast test inputs without retraining, the ranking nearly inverted:
- The noiseless champion (supervised image-domain denoiser) collapsed to $hr = 0.00$.
- Physics-regularized methods (Learned Primal-Dual, hand-crafted smoothing) rose to the top. The Learned Primal-Dual method rose from rank 6 ($hr=0.72$) to rank 1 ($hr=0.93$).
- Supervised image-domain denoisers failed because they were trained on a clean distribution and could not handle the shift.
Retraining: Retraining the models on matched-noise data partially restored the original ranking (Spearman $\rho$ increased from 0.04 to 0.61), indicating the collapse was a distribution mismatch rather than a fundamental architectural deficit.
Agent Efficiency: The agent onboarded 13 solvers in two days and ran ~1,100 iterations. However, it required human intervention to suggest "untried directions" to escape local optima in the compact solver search.

Significance and Claims
The paper claims that agentic autoresearch is a viable enabler for scaling experiments and ensuring fair comparisons, as the agent can handle the labor of implementation and tuning under fixed metrics. However, the agent did not invent new reconstruction principles; it recombined existing ones.

The primary scientific finding is that ideal-data leaderboards are poor predictors of robustness. A method's performance on clean data does not guarantee performance under realistic noise; in fact, the ranking can invert completely. The authors argue that this inversion is a "transfer effect" caused by train/test distribution mismatch, which can be mitigated by retraining on matched noise.

Crucially, the paper concludes that no single-factor challenge can certify generality. While retraining fixed the noise issue, real-world CT reconstruction faces an open-ended, shifting set of confounders (beam hardening, scatter, anatomy, disease). Therefore, benchmarks should not vary factors one at a time but should model a broad spectrum of realistic, co-occurring factors simultaneously. Agentic autoresearch is presented as the tool that makes such complex, multi-factor benchmarking affordable and feasible.

The Robot Scientist and the Two Puzzles

The Robot's Big Discovery: The "Perfect" Solver is a Trick

The Lesson: Don't Trust the Clean Room

The Takeaway

Technical Summary: Agentic Autoresearch for CT Reconstruction

More like this