A Novel Multi-Agent Architecture to Reduce Hallucinations of Large Language Models in Multi-Step Structural Modeling

This paper proposes a novel multi-agent architecture that automates structural modeling and analysis using OpenSeesPy by decomposing complex tasks into specialized agents to effectively reduce hallucinations and error accumulation, achieving high accuracy and scalability across benchmark frame problems.

The Gist

Imagine you are trying to build a complex LEGO castle, but instead of using your hands, you are asking a very smart, yet occasionally daydreaming, robot to do it for you. You describe the castle in words: "Make a tower with three levels, put a red flag on top, and add a bridge."

In the world of engineering, this "castle" is a structural model (like a building or a bridge), and the "robot" is a Large Language Model (LLM) like the ones powering chatbots today.

The problem? If you ask a single robot to build a huge, complex castle in one long list of instructions, it often gets confused. It might forget to add a wall, place a window in the wrong spot, or invent a door that doesn't exist. In engineering terms, this is called "hallucination." If a real building has these mistakes, it could be dangerous.

This paper introduces a new way to organize a team of robots to fix this problem. Instead of one robot trying to do everything, they created a specialized factory line where different robots handle specific jobs.

Here is how their "Multi-Agent Architecture" works, explained simply:

1. The Problem: The "Overworked Single Robot"

Previously, engineers tried to use one robot to do everything: understand the plan, draw the lines, place the beams, and write the computer code.

• The Analogy: Imagine asking one person to be the architect, the bricklayer, the electrician, and the software programmer all at once. As the building gets bigger, that person gets tired, makes mistakes, and eventually gives up because the instructions are too long.
• The Result: The robot would often fail on complex buildings, taking a long time and producing broken models.

2. The Solution: The "Specialized Factory Team"

The authors built a system where the work is split into four distinct teams (Agents), each with a specific job. If one team makes a mistake, the system catches it before it ruins the whole project.

Team A: The Planners (Analysis & Planning)

• Role: These robots read your messy, human description (e.g., "Make a 3-story building with a 6-meter wide room") and turn it into a strict, organized checklist (a JSON file).
• The Analogy: Think of them as the Project Managers. They don't build anything; they just make sure everyone agrees on what needs to be built and how many bricks are needed. They double-check the math before anyone starts working.

Team B: The Builders (Geometry Assembly)

• Role: Two robots work side-by-side here. One places the nodes (the corners/joints), and the other places the elements (the beams and columns).
• The Analogy: These are the Construction Crew.
  • Robot 1 says, "I'll put a corner at point X."
  • Robot 2 says, "I'll connect a beam between point X and point Y."
  • Because they work in parallel (at the same time) instead of waiting for each other, the building goes up much faster.
  • The Safety Net: A "Quality Inspector" (a checkpoint) checks their work immediately. Did they put two corners in the same spot? Did they forget a beam? If yes, they restart that specific step instantly.

Team C: The Loaders (Load Integration)

• Role: This robot figures out where the weight goes. Where is the wind pushing? Where is the snow sitting?
• The Analogy: This is the Logistics Manager. They take the "heavy stuff" (loads) and assign it to the correct parts of the building. They make sure the 50-ton truck is parked on the strong foundation, not on a weak balcony.

Team D: The Coders (Code Translation)

• Role: Finally, these robots translate the physical plan into computer code (OpenSeesPy) that the computer can actually run to test the building.
• The Analogy: These are the Translators. They take the blueprint and turn it into the "secret language" the computer speaks. They break this down into two steps: first, they write the code for the shape, then they write the code for the forces. This prevents the translator from getting overwhelmed by a huge block of text.

3. Why This is a Big Deal

The paper tested this new "Factory Team" against the old "Single Robot" method and even against the smartest general-purpose AI bots available.

• Accuracy: The new team got it right 99% of the time. The old method only got it right about 60-80% of the time, especially on big buildings.
• Speed: The new team was 5 to 6 times faster. Because they split the work, they didn't have to wait for one robot to finish a long list of tasks.
• Scalability: The old robot would crash (time out) when asked to build a 10-story building. The new team handled it easily, like a well-oiled assembly line.
• Flexibility: They even tested it with architecture students who described buildings in very different, casual ways. The "Planner" robot was smart enough to understand all the different styles and still build the correct model.

The Bottom Line

This paper shows that to build complex things with AI, you shouldn't just ask one super-smart robot to do everything. Instead, you should create a team of specialized robots that check each other's work.

It's the difference between asking one person to write, edit, and publish a novel in an hour (they will make mistakes) versus having an editor, a fact-checker, a typesetter, and a publisher working together (the result is professional and reliable).

This new system makes it possible to use AI to design and test real-world buildings safely, quickly, and without the AI "daydreaming" about things that don't exist.

Technical Summary

1. Problem Statement

Large Language Models (LLMs) have shown promise in automating engineering tasks, but they face significant hurdles when applied to multi-step structural modeling (e.g., creating Finite Element Models in OpenSees).

• Hallucination & Error Accumulation: Existing LLMs struggle with long-sequence operations. As the number of modeling steps increases (defining nodes, elements, loads, boundary conditions), the probability of hallucinations and error accumulation rises, leading to invalid structural models.
• Limitations of Sequential Agents: Previous attempts using sequential multi-agent architectures (where agents pass data linearly) suffer from:
  • Low Accuracy: Accuracy drops significantly (to ~60%) as structural complexity increases (e.g., from 3-bay to 5-bay frames).
  • Inefficiency: Sequential processing creates bottlenecks, resulting in long inference times (269–949 seconds).
  • Poor Scalability: These architectures often fail on larger structures due to API time limits and context window constraints.
• Need: A robust architecture is required to automate structural modeling while maintaining numerical accuracy, consistency, and efficiency against hallucinations.

2. Methodology: Novel Multi-Agent Architecture

The authors propose a robust, modular multi-agent architecture designed to automate the generation of executable OpenSeesPy scripts from natural language descriptions. The system decomposes the workflow into four coordinated modules and utilizes a hybrid LLM backbone strategy.

A. Architecture Modules

1. Analysis and Planning:
   • Problem Analysis Agent: Extracts key parameters (geometry, loads, materials, boundary conditions) from user text and organizes them into a structured JSON format.
   • Construction Planning Agent: Formulates a step-by-step assembly plan (bay-by-bay, story-by-story). It assigns specific "Step Types" (1–5) based on structural logic (e.g., base frame, adding stories, expanding bays).
   • Checkpoint: A consistency check verifies that the planned steps match the total bays and stories before proceeding.
2. Geometry Assembly (Parallel Processing):
   • Node Agent & Element Agent: These agents operate in parallel (not sequentially) to generate node coordinates/boundary conditions and element connectivity/types, respectively.
   • Connectivity Mapping: A deterministic Python function converts coordinate pairs into node ID connectivity compatible with OpenSeesPy.
   • Checkpoint: Validates for duplicate nodes/elements and ensures all nodes are referenced. If errors are found, the agents regenerate.
3. Load Integration:
   • Load Assignment Agent: Maps abstract load descriptions (e.g., "uniform load on girders") to specific structural components (nodes/elements) based on the geometry generated.
   • JSON Compiler: Integrates load data with geometry and material data into a single structured JSON file.
4. Code Translation:
   • Geometry Code Translator: Converts geometric JSON data into OpenSeesPy syntax (nodes, elements, boundary conditions).
   • Complete Code Generator: Assembles the geometric code with load definitions and model configuration commands (e.g., time series, patterns) to produce a final, executable script.

B. Hybrid LLM Backbone Strategy

To optimize performance and cost, the architecture assigns different LLMs based on task complexity:

• GPT-OSS 120B: Assigned to agents requiring complex reasoning (Problem Analysis, Construction Planning, Node/Element Agents).
• Llama-3.3 70B Instruct Turbo: Assigned to agents performing information mapping and translation (Load Assignment, Code Translators). This leverages Llama's strength in instruction following and syntax adherence.

3. Key Contributions

1. Novel Multi-Agent Framework: Introduced a modular architecture that decomposes structural modeling into specialized, parallelizable tasks, effectively mitigating the "long-context" hallucination problem.
2. Hybrid LLM Integration: Demonstrated that combining a high-reasoning model (GPT-OSS) with a high-precision instruction-following model (Llama) yields superior robustness compared to single-LLM systems.
3. Checkpoint Mechanism: Embedded deterministic consistency checks (JSON validation, duplicate detection) that trigger regeneration, significantly reducing error propagation.
4. Scalable Workflow: Successfully demonstrated the ability to model large-scale structures (up to 10 bays and 10 stories) without the API timeout failures common in sequential designs.
5. Open Implementation: The system is deployed as a public web application (civilbot.netlify.app) for community use.

4. Results

The architecture was evaluated on a benchmark of 20 representative 2D frame problems over ten repeated trials.

• Accuracy:
  • Achieved 100% accuracy in 18 cases and 90% in the remaining 2.
  • Outperformed the previous sequential architecture (which dropped to 60% on complex frames) and state-of-the-art general LLMs (Gemini-2.5 Pro: 37%, GPT-4o: 0%).
• Efficiency:
  • Reduced average inference time from 269–949 seconds (sequential) to 75.4–194.2 seconds (proposed).
  • Achieved an 85% reduction in runtime for complex cases (e.g., 3-4-5-4-3 frame).
• Scalability:
  • Successfully handled larger structures (7-bay and 10-bay frames) with accuracies of 90%, 100%, and 80%, and runtimes under 550 seconds.
• Robustness to Linguistic Styles:
  • Pilot tests with non-expert users (architecture students) using diverse, unstructured input styles resulted in 100% accuracy, proving the system's adaptability to natural language variations.
• Cost:
  • The economic cost per problem is minimal, ranging from $0.013 to $0.023 (2026 USD), demonstrating economic viability.

5. Significance and Impact

• Automation of Engineering Workflows: This research bridges the gap between natural language intent and rigorous numerical structural analysis, potentially democratizing structural engineering for non-experts and accelerating workflows for professionals.
• Mitigation of AI Hallucinations: By decomposing tasks and using deterministic checkpoints, the paper offers a blueprint for reducing LLM hallucinations in high-stakes, quantitative domains.
• Scalability: The architecture proves that multi-agent systems can scale to industrial-sized problems, overcoming the limitations of current single-agent or linear pipelines.
• Future Directions: The authors identify limitations in handling non-linear dynamics, seismic loads, and complex geometries (diagonal bracing), setting the stage for future research in advanced structural typologies and dynamic analysis.

In conclusion, this paper presents a significant advancement in AI-driven engineering, providing a reliable, efficient, and scalable framework for automating structural modeling while effectively managing the inherent risks of LLM hallucinations.