🔭 astrophysics

Beyond AI as Assistants: Toward Autonomous Discovery in Cosmology

This paper introduces two complementary AI agent systems, \texttt{CMBEvolve} and \texttt{CosmoEvolve}, that advance autonomous scientific discovery in cosmology by respectively optimizing quantitative tasks through code evolution and managing open-ended research workflows, as demonstrated by their successful application to weak-lensing map analysis and ACT DR6 data processing.

Original authors: Licong Xu, Thomas Borrett

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Licong Xu, Thomas Borrett

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of brilliant, tireless research assistants who don't just follow orders but can actually figure out how to solve problems on their own. That is the core idea behind this paper by Licong Xu and Thomas Borrett. They are testing two different types of "AI scientists" to see if computers can do more than just crunch numbers—they want to see if AI can actually discover new things in cosmology (the study of the universe).

Here is a simple breakdown of their two approaches, using everyday analogies:

1. The "Tournament of Code" (CMBEvolve)

Best for: Tasks where you know exactly what "winning" looks like (like a math problem with a single correct answer).

Think of CMBEvolve as a massive, automated coding tournament.

The Setup: You give the AI a specific goal, like "Find the best way to spot fake data in these maps of the universe."
The Process: The AI doesn't just write one program and hope for the best. Instead, it acts like a coach running a tree-search tournament.
- It starts with a basic "warm-up" idea.
- It creates many variations (mutations) of that code, like a chef trying slightly different recipes.
- It runs each version, scores them, and keeps the winners.
- It then takes the best winners, mixes them together, and tries again.
The Result: In their test, the AI was asked to find "out-of-distribution" errors in weak-lensing maps (basically, finding data that doesn't fit the rules). Through thousands of tiny, automated tweaks, the AI slowly improved its score, evolving a better solution than a human might have found quickly on their own.

The Analogy: Imagine trying to find the perfect key to open a lock. Instead of guessing one key, you have a robot that generates 1,000 keys, tries them all, keeps the ones that turn the lock a little bit, and then uses those "almost-right" keys to forge 1,000 new, slightly better keys. Eventually, it finds the perfect key.

2. The "Virtual Research Lab" (CosmoEvolve)

Best for: Open-ended mysteries where you don't know the answer yet (like exploring a new continent).

Think of CosmoEvolve as a simulation of a real university research lab.

The Setup: You have a "Principal Investigator" (PI) agent (the boss) and a team of "Student" agents (the workers).
The Process:
- The PI looks at the big picture, holds "meetings," and assigns tasks.
- The Students have their own tools, skills, and memories. They don't just follow a script; they can explore data, write code, and even break tasks down into smaller sub-tasks.
- They all share a "blackboard" where they post their findings, reviews, and ideas.
The Result: The team was given real data from the Atacama Cosmology Telescope (ACT DR6) with no specific instructions on what to look for. The AI team explored the data on its own. They discovered that different parts of the telescope data behaved differently depending on the frequency used. They figured out that you can't use the same "rules" for all the data; some parts need to be analyzed more carefully than others.

The Analogy: Imagine a detective (the PI) drops a pile of unsorted evidence on a table and says, "Figure out what's going on." The detective assigns different specialists (the students) to look at the evidence. One checks the fingerprints, another checks the timeline, and another checks the weather reports. They talk to each other, realize that the weather report changes how the fingerprints look, and together they write a report explaining the whole mystery.

Why This Matters

The paper argues that science today is getting too complex and expensive for humans to do alone.

CMBEvolve shows that AI can be a "super-optimizer" for problems with clear rules.
CosmoEvolve shows that AI can be a "super-explorer" for problems where the rules aren't clear yet.

The authors aren't claiming these systems have solved the universe yet. Instead, they are showing that these two different "AI scientist" styles work well for different types of cosmic puzzles, paving the way for a future where AI helps humans discover new things about the cosmos.

Technical Summary: Beyond AI as Assistants: Toward Autonomous Discovery in Cosmology

Problem Statement

Modern scientific discovery faces significant hurdles, including high time and monetary costs, potential cognitive biases, and increasing data complexity. While recent advances in Large Language Models (LLMs) and AI agents suggest a shift from passive tools to active contributors in scientific workflows, the field lacks robust frameworks for autonomous scientific discovery in cosmology. Specifically, there is a need for agentic systems capable of handling two distinct types of scientific tasks:

Quantitative tasks with explicit evaluation metrics (e.g., optimizing a specific score).
Open-ended scientific workflows where goals are only partially specified, requiring iterative exploration and hypothesis generation.

Methodology

The authors propose two complementary agentic systems designed to address these distinct problem spaces:

1. CMBEvolve: Automated Algorithm Discovery via Tree Search

Designed for tasks with explicit quantitative objectives, CMBEvolve utilizes an LLM-guided tree search to evolve scientific code.

Architecture: The system models the search process as a rooted tree $T = (V, E)$ . Nodes represent tasks, idea generation/selection, code generation, and code mutation.
Process:
- Idea Sampling & Selection: The system generates ideas and selects branches based on search statistics (visit counts, best scores).
- Mutation & Execution: Targeted code mutations are executed, and the resulting programs are scored.
- Backpropagation: Scores are backpropagated from the leaf nodes to the root, updating the statistics of ancestor nodes to guide future search directions.
Goal: To iteratively improve a benchmark score by evolving candidate solutions until an optimal algorithm is found.

2. CosmoEvolve: Virtual Multi-Agent Research Laboratory

Designed for open-ended scientific discovery, CosmoEvolve simulates a virtual research environment.

Architecture: The system comprises a Principal Investigator (PI) agent and a community of student scientist agents.
Roles & Coordination:
- The PI agent operates at a supervisory level, observing the shared lab state (blackboard) and selecting actions from a discrete space (e.g., group meetings, individual meetings, task assignments).
- Student agents execute scientific work independently based on PI directives. They can dispatch subtasks to specialized subagents for data exploration, planning, and code implementation.
Context Management: Agents are equipped with tools, skills, and persistent memory. Skills are loaded on-demand, and tool access is controlled via allowlists.
Goal: To autonomously explore data, identify patterns, and develop analysis paths without predefined objectives.

Key Results

The paper presents preliminary demonstrations of both systems on cosmological datasets:

CMBEvolve: Out-of-Distribution (OoD) Detection

Task: Applied to the FAIR Universe Weak Lensing ML Uncertainty Challenge, specifically for detecting OoD in weak-lensing maps.
Performance: The system successfully improved the benchmark score through iterative code evolution. The score trajectory demonstrated that CMBEvolve could refine candidate solutions over successive iterations, with the "winning lineage" showing specific code-level modifications that led to performance gains.

CosmoEvolve: ACT DR6 Data Analysis

Task: Autonomous analysis of public Atacama Cosmology Telescope (ACT) DR6 data products with no predefined scientific objective.
Findings:
- Stability Analysis: The system conducted a beam-aware split-cross pseudo- $C_\ell$ study, identifying percent-level within-channel stability.
- Frequency Dependence: It detected tighter same-band cross-array agreement at 90 GHz compared to 150 GHz, and identified cross-frequency residuals at the few-percent level.
- Scale Cuts: Through multi-frequency coherence analysis, the system determined that no single multipole cut is appropriate for all ACT DR6 channel pairs. It identified that 150 GHz arrays show strong internal consistency, while pairs involving 220 GHz are better interpreted as foreground diagnostics requiring more conservative scale cuts.
- Output: The system produced analysis-grade diagnostics and recommended stability windows for different pair types.

Significance and Claims

The paper claims that these examples demonstrate how cosmology can serve as a dual-purpose testbed for AI scientist systems:

Controlled Benchmarking: It provides tasks with explicit metrics (like OoD detection) to rigorously evaluate algorithmic evolution.
Realistic Open-Ended Research: It offers complex, realistic research problems (like ACT DR6 analysis) to test the ability of AI to navigate undefined scientific workflows.

The authors position these systems as preliminary steps toward autonomous scientific discovery, showing that AI agents can move beyond being mere tools to actively contributing to workflows, generating diagnostics, and identifying non-trivial scientific patterns. The work suggests a future where AI agents can handle both the optimization of specific algorithms and the exploration of broad, open-ended scientific questions.