PDE-Agents: An LLM-Orchestrated Multi-Agent Framework for Automated Finite Element Simulations with Knowledge Graph-Augmented Reasoning

PDE-Agents is an LLM-orchestrated multi-agent framework that automates finite element simulations through natural language, demonstrating that GraphRAG-augmented reasoning significantly enhances task success and material property fidelity compared to non-augmented baselines.

The Gist

Imagine you are a master architect who wants to build a complex bridge. You know exactly what you want it to look like, but you don't speak the language of the construction crew, and you don't have the blueprints handy. Usually, you'd have to hire a translator, draw the plans yourself, double-check the math, and hope the crew doesn't make a mistake.

PDE-Agents is a new system that acts like a team of super-smart, specialized robots that do all that work for you, just by listening to your voice.

Here is how the paper explains this system, broken down into simple concepts:

1. The Team of Robots (The Multi-Agent System)

Instead of one giant robot trying to do everything, the system uses a "supervisor" (like a project manager) who delegates tasks to three specialized workers:

• The Simulation Agent: This is the builder. It takes your idea (e.g., "Build a heat shield for a rocket") and writes the code to run the physics simulation.
• The Analytics Agent: This is the inspector. It looks at the results, checks if the numbers make sense, and compares them to previous builds.
• The Database Agent: This is the librarian. It remembers every project the team has ever done, storing the materials used and what went right or wrong.

All of this runs on powerful computers right in the lab (using local graphics cards), so no data leaves the building, keeping everything private and secure.

2. The "Brain" vs. The "Library" (The Knowledge Graph)

This is the most important part of the paper.

• The Brain (LLM): The robots use advanced AI models (like a very smart brain) that have read millions of books. They are great at general tasks.
• The Library (Knowledge Graph): However, the brain sometimes forgets specific details or makes up facts (hallucinates). To fix this, the team built a digital library (a Knowledge Graph) that contains exact, verified facts about materials (like how much heat steel conducts) and a log of every past simulation.

The Big Discovery: The paper tested three ways to use this library:

1. No Library (KG Off): The robot guesses the material properties. It finishes the job fast, but if the material is new or rare, it guesses wrong, leading to a physically impossible result (like a bridge that melts instantly).
2. Always Ask the Library (KG On): The robot stops to ask the library for every single detail before starting. It gets the facts right, but it gets so bogged down in asking questions that it often runs out of time or gets confused and gives up.
3. The "Smart" Mix (KG Smart): This is the paper's winning strategy.
   • Warm-Start: Before the robot even starts working, the system quietly looks up the 3 most similar past projects and hands those notes to the robot as a "cheat sheet."
   • Lazy Retrieval: The robot only asks the library for help if it hits a snag or encounters a material it truly doesn't know.

The Result: The "Smart" mix was the winner. It finished 100% of the tasks (unlike the "Always Ask" method) and got the physics 100% correct (unlike the "No Library" method).

3. The "Fictional Material" Test

To prove the system works, the researchers invented three fake materials (Novidium, Cryonite, and Pyrathane) that exist only in their digital library and nowhere in the AI's training data.

• Without the library: The AI made up random numbers for these fake materials. The simulation "ran," but the results were garbage.
• With the "Smart" library: The system looked up the exact, made-up properties of these fake materials from the library and used them perfectly.

The Lesson: The system isn't just a "random number generator." It becomes a reliable engineering tool only when it knows when to look up facts and how to use them without getting stuck.

4. Real-World Performance

The team ran over 1,300 simulations.

• Success Rate: 97.8% of the time, the system produced a working, verified simulation.
• First Try: About 57% of the time, it got it right on the first attempt. If it made a mistake, the "Analytics" and "Database" agents helped it debug and fix it automatically, much like a human engineer iterating on a design.
• Learning: As the system ran more simulations, it got better at the "hard" tasks. It learned from its own history to solve complex problems faster, though simple tasks were already easy for it.

Summary

The paper concludes that how you connect the AI to the library matters more than the library itself.

• If you force the AI to check the library constantly, it gets slow and fails.
• If you don't use the library, it makes dangerous mistakes.
• If you give it a "cheat sheet" of past successes upfront and let it ask for help only when needed, it becomes a highly reliable, autonomous engineer that can solve complex physics problems just by listening to your voice.

Technical Summary

Technical Summary: PDE-Agents

Problem Statement

Finite Element Method (FEM) simulations are critical for engineering analysis but remain labor-intensive, requiring a sequence of error-prone manual steps: geometry creation, boundary condition specification, solver parameter selection, and iterative debugging. While Large Language Models (LLMs) have shown promise in "AI for scientific computing," prior work has largely focused on surrogate modeling, operator learning, or dataset-specific fine-tuning. There is a significant gap in procedural automation—using LLMs to autonomously set up, manage, and debug solvers rather than merely replacing them. Furthermore, standard LLMs often hallucinate physical properties (e.g., material constants), leading to simulations that are computationally successful but physically incorrect.

Methodology

The authors present PDE-Agents, a containerized, multi-agent ecosystem designed to automate the full lifecycle of Partial Differential Equation (PDE) simulations via natural language.

System Architecture

The system is built on four tiers:

1. User Interface: Natural language task input.
2. Multi-Agent Orchestration: A LangGraph supervisor routes tasks to three specialist agents:
   • Simulation Agent: Executes a ReAct loop with up to 25 reasoning steps, utilizing tools for configuration validation, simulation execution (via FEniCSx/DOLFINx), and debugging.
   • Analytics Agent: Performs statistical comparisons and sensitivity analysis.
   • Database Agent: Manages history queries and run lineage in PostgreSQL.
3. Execution & Knowledge Layer:
   • Solver: A Dockerized FEniCSx runner (DOLFINx 0.10.0.post2) supporting 2D/3D heat equations with P1 elements and various boundary conditions.
   • Knowledge Graph (KG): A Neo4j graph augmented with GraphRAG. It stores material properties, known failure patterns (e.g., CFL violations), and run lineage. Nodes are embedded using nomic-embed-text (768-dim) and indexed via HNSW for vector similarity search.
4. LLM Backend: A locally deployed stack running on dual NVIDIA RTX PRO 6000 GPUs (≈196 GB VRAM), utilizing open-source models (Qwen3-Coder-Next, Llama 4 Scout) via Ollama.

The "KG Smart" Integration Pattern

The core methodological novelty is the KG Smart integration strategy, which contrasts with "KG Off" (no retrieval) and "KG On" (mandatory retrieval). KG Smart employs a two-pronged approach:

1. Warm-Start Injection: Before the agent reasoning loop begins, the task description is embedded, and the top-3 most similar past successful runs are retrieved from the HNSW index. Their configurations are injected directly into the system prompt as few-shot examples.
2. Lazy Conditional Retrieval: KG tools remain available but are only invoked after a simulation failure or when material properties are genuinely unknown, avoiding mandatory pre-simulation queries that consume the agent's iteration budget.

Key Contributions and Results

1. Verification and Validation (V&V)

The authors conducted a formal V&V study against closed-form analytical solutions for three benchmark cases (Steady-State Linear, Transient Fourier, Steady-State Poisson).

• Result: The solver demonstrated second-order spatial convergence ($O(h^2)$) for P1 elements, confirming the numerical kernel's correctness.

2. Ablation Study (50 Tasks)

A controlled ablation study compared KG Off, KG On, and KG Smart across 50 benchmark tasks, including 10 "novel" tasks involving fictional materials (Novidium, Cryonite, Pyrathane) whose properties exist only in the KG.

• Success Rate: KG Off and KG Smart achieved 100% success. KG On achieved 94%, with failures attributed to iteration budget exhaustion and timeouts caused by mandatory pre-simulation queries.
• Output Quality:
  • Material Property Fidelity (MPF): KG Smart achieved MPF = 1.00 on novel tasks. KG Off fabricated properties, resulting in MPF = 0.34.
  • Physics Score: KG Smart scored 0.933 (overall) vs. 0.853 for KG Off.
  • Error Propagation: Fabricated properties in KG Off led to physically impossible results (e.g., temperature overshoots of 949 K in transient tasks due to incorrect conductivity).

3. Failure Analysis

The study identified that warm-start injection is the dominant factor in KG Smart's reliability.

• KG On Failures: All 3 systematic failures in KG On were caused by the agent exhausting its iteration budget on mandatory KG queries before reaching the simulation step.
• KG Smart Success: By injecting context before the loop, KG Smart reduced average iterations from 7.5 (KG On) to 4.2, eliminating budget exhaustion risks while maintaining high fidelity.

4. Production Metrics and Learning Curve

• Scale: Analysis of 1,369 real simulation runs showed a 97.8% overall success rate and a 57.6% first-try success rate.
• KG Growth: A controlled experiment showed that as the KG accumulated run history, hard tasks (ambiguous descriptions, mixed BCs) saw an 8.8% improvement in MPF and a 5.8% improvement in physics score from Pass 1 to Pass 2. Easy tasks remained at ceiling performance, indicating the KG's value is difficulty-dependent.

Significance and Claims

The paper argues that integration pattern, rather than knowledge content alone, determines whether GraphRAG augmentation helps or hinders LLM agents.

• Reliability vs. Fidelity Trade-off: The authors demonstrate that mandatory retrieval (KG On) introduces latency and budget exhaustion, while no retrieval (KG Off) leads to hallucinations. The KG Smart pattern (warm-start + lazy retrieval) resolves this trade-off, achieving the reliability of a KG-free system with the fidelity of a KG-enabled one.
• Domain Applicability: The system transforms LLMs from "expensive random number generators" into reliable engineering tools, particularly for domains with proprietary or novel materials where LLM training data is insufficient.
• Design Principles: The authors propose three design principles for autonomous simulation assistants:
  1. Never make KG access mandatory in the agent's critical path.
  2. Front-load context via embedding similarity rather than forcing the agent to query.
  3. Allow the agent to decide when it needs more knowledge (lazy retrieval).

The work establishes a rigorous empirical baseline for LLM-driven autonomous simulation, proving that structured knowledge integration combined with iterative self-correction can automate complex scientific workflows without requiring domain-specific model fine-tuning.