AI CFD Scientist: Toward Open-Ended Computational Fluid Dynamics Discovery with Physics-Aware AI Agents

The paper introduces AI CFD Scientist, an open-source framework that leverages vision-language models to autonomously execute, validate, and refine computational fluid dynamics simulations on OpenFOAM, successfully discovering a Spalart-Allmaras correction that reduces error by 7.89% while detecting silent failures that traditional solver checks miss.

The Gist

Imagine a team of highly intelligent, tireless research assistants working together to solve complex physics puzzles. This paper introduces AI CFD Scientist, a new open-source system designed to act as an autonomous scientist specifically for Computational Fluid Dynamics (CFD)—the field of using supercomputers to simulate how air and water flow around things like airplane wings, car bodies, or even blood vessels.

Here is how the system works, explained through simple analogies:

The Problem: The "Silent Failure" Trap

In many scientific fields, if a computer program finishes running without crashing, you assume the result is good. But in fluid dynamics, this is dangerous.

• The Analogy: Imagine a chef who follows a recipe perfectly, but accidentally uses salt instead of sugar. The cake bakes, rises, and looks perfect. The "log" (the recipe steps) says everything is fine. But if you taste it, it's inedible.
• The Reality: A CFD simulation can finish running without errors, yet still produce a physically impossible result (like air flowing backward through a solid wall) because of a subtle mistake in the setup or the math. Traditional AI tools often miss these "silent failures" because they only check the computer logs, not the actual picture of the flow.

The Solution: A Team of Specialized Agents

The authors built a system that doesn't just run code; it acts like a full research lab. It uses a "brain" (a large language model) to coordinate several specialized "agents" (software tools) that handle different parts of the job:

1. The Idea Generator: Instead of just guessing, this agent reads scientific papers to find gaps in knowledge and proposes new experiments.
2. The Code Builder: If the standard tools can't solve a specific problem, this agent writes and compiles new C++ code (the "engine" of the simulation) to create custom physics models.
3. The Mesh Inspector: Before running a simulation, it checks if the digital grid (the "mesh") is detailed enough to catch small details, ensuring the results aren't just a blurry guess.
4. The "Vision" Gatekeeper (The Star Feature): This is the most important innovation. After a simulation runs, the system doesn't just look at numbers. It takes a picture of the flow and shows it to a Vision-Language Model (an AI that can "see" images).
   • The Analogy: Think of this as a human art critic looking at a painting. Even if the artist says, "I followed the rules," the critic can spot that the perspective is wrong or the colors don't match reality.
   • The Result: If the AI "sees" that the flow looks weird (like a vortex in the wrong place), it rejects the result and tells the system to try again, even if the computer log said "Success."

What They Actually Achieved

The team tested this system on five different tasks. Here are the highlights:

• Routine Checks: It successfully ran standard tests, like checking how different turbulence models behave around a backward-facing step (a common test case in aerodynamics).
• Custom Engineering: It wrote its own code to simulate non-Newtonian fluids (fluids that change thickness when you stir them, like ketchup or blood) and proved it worked.
• The Big Discovery (Open-Ended Discovery): This is the most impressive part. The system was given a goal: "Find a way to make the standard turbulence model match real-world data better."
  • Without human help, the AI ran 44 iterations of experiments.
  • It tried different mathematical tweaks, wrote new code, ran simulations, and checked the results.
  • The Result: It discovered a new mathematical correction (a "runtime tweak") that reduced the error in predicting skin friction (how much air rubs against a surface) by 7.89% compared to the best available reference data.

The Safety Net: Catching Mistakes

To prove the system works, the researchers intentionally broke the simulations in 16 different ways (e.g., deleting the output file, making the numbers wrong, or stopping the simulation too early).

• The system's "Vision Gate" caught 14 out of 16 of these silent failures.
• Standard AI tools (without the vision check) would have accepted these broken results as valid.

Comparison to Other AI Scientists

The authors compared their system to two other general-purpose AI scientist tools (ARIS and DeepScientist).

• The Difference: The other tools could run the simulations and write a report, but they often missed the physics checks. They might claim a result was valid when it wasn't.
• The Advantage: AI CFD Scientist is "conservative." If the evidence isn't perfect (e.g., the mesh isn't fine enough or the picture looks weird), it admits it doesn't know the answer yet, rather than making up a false claim.

Summary

AI CFD Scientist is a new, open-source tool that automates the entire process of fluid dynamics research. It doesn't just run numbers; it reads papers, writes code, checks if the physics looks right by "seeing" the flow, and only publishes results that pass a strict visual and mathematical inspection. It successfully found a new way to improve a standard physics model on its own, proving that AI can now handle the complex, high-stakes world of physical simulation, not just software coding.

Technical Summary

Technical Summary: AI CFD Scientist

Problem Statement

Extending Large Language Model (LLM) agents to physical sciences, specifically Computational Fluid Dynamics (CFD), presents unique challenges not found in software-only machine learning or biological research. The primary difficulty lies in the fact that solver completion does not imply physical validity. A CFD simulation may converge cleanly without errors in the solver logs while still producing physically degenerate results due to incorrect geometry, missing flow features, or flawed closure models. These failure modes are often invisible to text-based log analysis. Furthermore, in CFD, the closure model is a research variable requiring source-code modification (C++ level) rather than simple configuration changes, and validity gates (such as mesh independence and reference-data alignment) are scientific objects that must be explicitly confirmed rather than assumed. Existing AI-scientist frameworks lack these domain-specific validity gates, while existing CFD agents often stop short of the full discovery loop, failing to integrate literature-grounded ideation, source-code modification, and vision-based verification.

Methodology

The authors present AI CFD Scientist, an open-source framework designed to close the scientific discovery loop for CFD. The system operates on OpenFOAM via the Foam-Agent interface and is orchestrated by a shared LLM backbone (GPT-5.5 in the experiments). The architecture is built on five operational design principles distilled from CFD practice:

1. Physical Validity is Not Log-Readable: Image-level inspection is mandatory.
2. Source Code Modification: Editing C++ source is a research object, not a configuration option.
3. Mesh Independence: A required convergence gate.
4. No Hallucination of Experiments: Agents cannot swap variables or relax criteria to make failing cases run.
5. Traceability: Every claim in a manuscript must trace back to specific, validated figures and numerical values.

The framework executes three coupled pathways:

• Regular Experimentation: Literature-aware ideation, novelty filtering, requirement validation, mesh-independence gating, and execution via Foam-Agent.
• Code Modification: Generation of case-local C++ source files and dictionaries for new physical models, compilation, and smoke testing.
• Open-Ended Discovery: An outer hypothesis loop that autonomously generates, compiles, and tests candidate model modifications against a reference comparator without further human input.

Core Innovation: The VLM Physics-Verification Gate
Central to the framework is a Vision-Language Model (VLM) physics-verification gate. Before any result is accepted, rerun, or written into a manuscript, the system renders flow fields (e.g., velocity contours, skin friction plots) and submits them to a VLM. The VLM performs two checks:

1. Quality Filter: Ensures figures are readable and not degenerate.
2. Physics Check: Inspects the visualized flow for expected features (e.g., recirculation zones, separation points) and judges consistency with the experiment requirement.
   This gate is designed to catch "silent failures" that pass solver logs but violate physical intuition.

The workflow also includes a Mesh-Independence Gate (comparing baseline and refined meshes) and a Rerun/Writer Loop that revises requirements upon failure and drafts LaTeX manuscripts grounded in the validated artifacts.

Key Contributions

1. First End-to-End Open-Source CFD Scientist: To the authors' knowledge, this is the first system to span literature-grounded ideation, validated execution, vision-based physics verification, source-code modification, and figure-grounded writing in a single inspectable workflow.
2. VLM-Based Physics Gate: The introduction of a vision-language subsystem that detects physical inconsistencies invisible to solver logs, addressing a critical gap in current AI-scientist frameworks.
3. Source-Level Discovery: The ability to autonomously edit C++ source code to modify turbulence models (e.g., Spalart–Allmaras) and compile them as case-local libraries, treating code modification as part of the hypothesis space.
4. Rigorous Evaluation: A controlled ablation study demonstrating the efficacy of the VLM gate and a head-to-head comparison against strong general AI-scientist baselines (ARIS, DeepScientist).

Results

The system was evaluated on five tasks using a GPT-5.5 backbone:

• T1 (BFS Turbulence Sensitivity): Successfully ran four RANS closures. The VLM gate flagged a sign-convention error in a reattachment-length extractor and triaged a $k$-$\varepsilon$ output as inconsistent with separated-flow physics, preventing the system from issuing a flawed ranking.
• T2 (Jet/Plume Re-sweep): Executed a 7-case Reynolds number sweep, recovering expected velocity scaling. The system flagged an anomalous case where the centerline mean collapsed.
• T3 (Custom Viscosity): Autonomously wrote, compiled, and validated a custom power-law viscosity library, reproducing the Newtonian limit ($n=1$) within 0.5% error.
• T4 (Custom SA Modifier): Compiled a Spalart–Allmaras variant with an adverse-pressure-gradient (APG) correction. Validated the custom code path against a control case ($APG=0$) and generated DNS-aligned $C_f$ overlays.
• T5 (Open-Ended Discovery): In a 44-iteration autonomous loop, the system discovered a quadrupolar runtime correction for the Spalart–Allmaras model on the periodic hill ($Re_h=5600$). This modification reduced the lower-wall skin-friction ($C_f$) RMSE against DNS by 7.89% (from 0.004297 to 0.003958).

Ablation Study:
In a planted-failure ablation involving 16 injected errors across four flow types, the VLM gate detected 14 out of 16 failures (87.5% recall). It achieved 100% recall on missing deliverables, wrong magnitudes, and broken post-processing, but only 50% recall on truncated convergence runs (where the simulation time was edited to match the truncated state, making it visually complete).

Comparison:
Compared to ARIS and DeepScientist under matched LLM costs, the baselines executed partial workflows but lacked domain-specific validity gates. They often issued closure rankings or correlations without mesh independence or DNS validation. AI CFD Scientist was more conservative, withholding claims when evidence was incomplete and ensuring all claims were backed by validated figures.

Significance and Claims

The paper claims that AI CFD Scientist represents a significant step toward open-ended computational fluid dynamics discovery. Its significance lies in:

• Closing the Loop: Moving beyond simple case setup to a full cycle of ideation, execution, verification, and writing.
• Physical Grounding: Demonstrating that integrating a vision-language gate is necessary to convert runnable simulations into defensible scientific claims, as solver logs alone are insufficient for physical validity.
• Autonomous Discovery: Showing that an AI agent can autonomously modify source code to discover novel turbulence model corrections that improve agreement with high-fidelity DNS data.
• Community Baseline: Providing an open-source, inspectable framework for CFD-specific scientific automation, contrasting with closed-source or partial-coverage alternatives.

The authors remain modest, noting that the system is currently a "supervised scientific assistance" tool rather than an unattended publication engine, and that evaluation relies on manual expert artifact reading due to the lack of automated CFD-paper rubrics. The results are bounded by the single backbone used (GPT-5.5) and the specific tasks tested.