💻 computer science

Verication-driven closed-loop multi-agent large language modelframework for code-compliant structural design

This paper introduces a verification-driven closed-loop multi-agent LLM framework that integrates an external physics-based verifier with a dual-node repair mechanism to transform code violations into actionable constraints, achieving a significant increase in code compliance and safety for structural design while reducing material usage.

Original authors: Jianbin Luo, Weibin Lin, Yiran Lin, Qing Wei, Wei Guo

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Jianbin Luo, Weibin Lin, Yiran Lin, Qing Wei, Wei Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Digital Architect's Safety Net

Imagine you are trying to teach a super-smart robot how to build a bridge. You give the robot a language model—a brain trained on millions of books, articles, and blueprints—so it can understand your request and "dream" up a design. This is the world of Large Language Models (LLMs). They are incredible at writing stories, coding software, and even explaining complex physics in simple words. But here is the catch: an LLM is a master of language, not a master of physics. It can describe a bridge that sounds perfect, using all the right technical words, but if you actually built it, it might collapse because the math doesn't add up. This is known as "hallucination"—the robot is confident, but wrong.

In the real world, especially when building things like skyscrapers or bridges, being "mostly right" isn't good enough. You need code compliance, which means the design must strictly follow the rulebook (the building codes) to ensure safety. For decades, engineers have used computers to check these rules, but the process of translating a human idea into a computer model and then checking it has been slow and manual. The big question researchers are asking is: Can we make an AI that doesn't just guess a design, but actually checks its own work and fixes its mistakes before it ever touches a piece of steel? This paper dives into that exact challenge, trying to turn a chatty AI into a reliable, safety-conscious engineer.

The "Self-Correction" Trap and the Magic Loop

The researchers started with a common idea: maybe if we just ask the AI to "check its own work" and try again, it will get better. But they found something surprising. When they let the AI try to fix its own mistakes without outside help, it often made things worse or just spun its wheels. It's like asking a student who failed a math test to just "try again" without a teacher or a calculator; they might just write the same wrong answer with more confidence. The paper argues that self-correction alone is unreliable for safety-critical tasks.

Instead, the team built a verification-driven closed-loop framework. Think of this not as a single student, but as a construction crew with a very strict, unblinking safety inspector. The system works like this:

The Dreamer (The AI): A team of AI agents works together to design a structure, like a beam or a truss.
The Inspector (The Physics Verifier): Before the design is ever approved, it gets sent to a "hard" computer program (a finite-element solver) that acts like a physics simulator. This isn't an AI guessing; it's a math engine that calculates exactly how the structure would behave under weight and stress.
The Rulebook (The Code Checker): A second tool checks the design against the actual building codes (like GB 50010 for concrete and GB 50017 for steel).
The Loop: If the design fails, the system doesn't just say "try again." It sends a specific note back to the AI: "Your beam is too thin by 21%," or "The stress is too high." The AI then uses this hard data to adjust the design and tries again. This cycle repeats until the design passes every single test.

What They Found: From "Maybe" to "Definitely"

The team tested this system on 44 different design cases involving five types of structures: simple beams, cantilever beams, continuous beams, trusses, and frames. The results were dramatic.

When they let the AI work without this strict loop (the "open-loop" method), it only got the design right 56.8% of the time. That means nearly half the time, the AI would hand over a design that violated safety rules. But once they turned on the closed-loop system with the external physics inspector, the success rate skyrocketed to 98.6%.

It's not just about passing; it's about being efficient, too. The AI designs that went through the loop used about 5.8% less material than the ones that didn't. This is a big deal because it means the AI learned to stop over-designing (making things unnecessarily huge just to be safe) and found the "sweet spot" where the structure is safe but uses fewer resources. The overall quality score of the designs jumped from 63.8 to 71.4.

How the "Two-Node" Repair Works

The paper breaks down the repair process into two distinct "nodes" or stages, which act like different tools in a toolbox:

Node 1 (The Hard Fix): If the design breaks a rule (like a beam bending too much), this node treats it as a hard constraint. It tells the AI, "You must make this bigger." It's like a strict coach saying, "Run until you can do it right." This step fixed the big, dangerous errors.
Node 2 (The Soft Polish): Once the design is safe, this node looks at a four-dimensional score (safety, cost, efficiency, and sustainability). If the score is a bit low, it gently nudges the AI to tweak the design to make it even better, without risking safety.

The researchers found that Node 1 did the heavy lifting for designs that started out broken, while Node 2 was responsible for polishing designs that were already safe. If they removed either node, the system's performance dropped, proving that both the "hard fix" and the "soft polish" are necessary.

The "Black Box" Problem is Solved

One of the biggest worries with AI is that it's a "black box"—you get an answer, but you don't know why. This system solves that. Because it uses a Retrieval-Augmented Generation (RAG) system, every time the AI makes a mistake, it doesn't just say "wrong." It pulls up the exact page and paragraph of the building code that was violated. It's like a student who, when they get a question wrong, can point to the specific sentence in the textbook that explains the rule. This makes the AI's decisions auditable and traceable.

The Limits and the Future

The paper is very honest about what this system can't do yet. The AI is great at adjusting the size of beams or the amount of steel (parameters), but it can't redesign the entire shape of the building (topology) if the original shape is fundamentally flawed. For example, if the AI tries to fix a bridge that is too short for the gap it needs to cross, no amount of tweaking the beam size will save it. In those cases, the system "fails loudly"—it stops and tells the human engineer, "I can't fix this; you need to change the design." It refuses to give a dangerous answer.

The researchers also tested if this worked with different AI brains (switching from one large model to another). They found that the success rate stayed the same regardless of which AI model they used. This suggests that the safety comes from the external physics verifier, not from the AI being "smarter." It's the safety net, not the trapeze artist, that keeps the show safe.

The Bottom Line

This paper shows that we can't just trust AI to "think" its way to a safe building. Instead, we need to build a system where the AI does the creative work, but a strict, math-based inspector checks every single step. By closing the loop between "making" and "checking," the researchers turned a system that was only right 56.8% of the time into one that is right 98.6% of the time, using less material and following the rules perfectly. It's a step toward a future where AI doesn't just talk about engineering, but actually helps build safer, smarter, and more efficient structures for everyone.

Technical Summary: Verification-Driven Closed-Loop Multi-Agent LLM Framework for Code-Compliant Structural Design

1. Problem Statement

Structural design requires iteratively balancing safety and economy under strict code constraints. While tools like CAD, BIM, and finite-element software (e.g., SAP2000, ANSYS) are mature for modeling and analysis, the critical steps relying on professional judgment—translating intent to parameters, interpreting code clauses, and reconciling results—remain manual bottlenecks.

Direct application of Large Language Models (LLMs) to these tasks faces a fundamental reliability issue: LLMs are language models, not mechanics solvers. They often produce coherent but physically unsound or code-violating designs ("hallucinations"). Recent studies indicate that LLM self-correction without external feedback is unreliable and can degrade performance. Consequently, safety-critical engineering design cannot rely on unverified LLM outputs or unguided self-reflection. There is a lack of frameworks that integrate an external, physics-based verifier into a closed-loop repair mechanism to drive automatic, directional improvements in structural design.

2. Methodology

The authors propose a verification-driven closed-loop multi-agent framework that replaces LLM self-correction with an external arbitration mechanism. The system is built on a three-tier architecture (presentation, business, tool) and utilizes a five-stage state machine driven by a task orchestrator (PlanningFlow).

Core Components

Multi-Agent System:
- StructuralDesignAgent: Interprets requirements and parameters.
- FEAnalysisAgent: Performs finite-element analysis (using OpenSeesPy) and code checking.
- EvaluationAgent: Conducts four-dimensional assessment and RAG-based code tracing.
- CADDrawingAgent & ReportGenerationAgent: Handle output generation.
- Agents communicate via structured JSON contracts, ensuring data integrity between stages.
Three-Layer Finite-Element Trustworthiness-Verification System:
To prevent errors from propagating, the framework implements a defense-in-depth strategy:
- Layer 1 (Engine Accuracy): Regression tests against analytical solutions or independent ANSYS models to ensure the finite-element wrapper is accurate (tolerance <1–3%).
- Layer 2 (Model Construction): Pre-analysis validation of node connectivity, boundary conditions, and load application, coupled with a geometric preview for human confirmation to catch parameter misreadings.
- Layer 3 (Result Compliance): Static code checking against limits (deflection, stress, slenderness) to generate a structured list of violations.
Dual-Node Closed-Loop Feedback Mechanism:
Instead of open-loop generation, the system employs two sequential nodes to drive repair:
- Node 1 (Hard-Constraint Repair): If code violations exist, they are converted into a repair prompt with specific constraints (e.g., "increase section height by X%"). The LLM iterates (up to $k_{max}=10$ ) until violations are resolved. This transforms blind retries into directional repair.
- Node 2 (Soft-Constraint Refinement): Once compliant, a four-dimensional score (Safety, Economy, Efficiency, Sustainability) is calculated. If the score falls below a threshold, the system generates multiple candidate schemes within safety-first constraints, selects the best via independent verification, and refines the design.
Traceable RAG Code Base:
A Retrieval-Augmented Generation (RAG) system using ChromaDB indexes Chinese building codes (GB 50010, GB 50017). Violations are traced to specific clauses, and citations are embedded in the report, ensuring auditability.
Four-Dimensional Assessment:
Designs are scored on Safety, Economy, Structural Efficiency, and Sustainability. Weights are determined via a transparent, auditable two-factor procedure based on failure consequence severity and economic optimization potential.

3. Key Contributions

The paper addresses three specific gaps in existing literature:

Closing the Generation-Verification Loop: Unlike open-loop multi-agent systems that rely on manual intervention or unguided retries, this framework injects external physics-based feedback into an automatic, directional repair loop.
Three-Layer Verification: It introduces a general verification template for LLM-driven engineering that validates the engine, the model construction, and the final result, rather than relying on post-hoc comparisons.
Interpretability and Auditability: It ensures every violation is traceable to a specific code clause via RAG and uses transparent weighting for quality assessment, making the AI's decisions auditable.

Additionally, the authors release the complete framework, a 44-case benchmark, and all experiment scripts as open source.

4. Experimental Results

The framework was evaluated on 44 parametric cases across five structure types (simply supported beams, cantilever beams, continuous beams, trusses, and frames).

Compliance Improvement: The closed-loop framework (B4) achieved a 98.6% code-compliance rate, a significant increase from the 56.8% observed in open-loop baselines (B1–B3). The difference is statistically significant ( $p < 10^{-6}$ ).
Design Quality: The composite score improved from 63.8 (open-loop) to 71.4 (closed-loop).
Economy: The framework reduced material usage by approximately 5.8% while maintaining compliance, suggesting it avoids the over-sizing often seen in open-loop attempts to ensure safety.
Node Contributions:
- Node 1 was responsible for the majority of the gain in non-compliant scenarios (repairing feasibility), contributing ~8.2 points to the score.
- Node 2 provided quality refinement in already compliant scenarios, contributing ~1.8 points.
- Ablation studies confirmed that removing either node significantly degraded performance (Node 1 removal dropped compliance to ~71%; Node 2 removal lowered quality scores).
Model Independence: Replacing the backbone LLM (DeepSeek V4) with Claude Sonnet 4.6 resulted in no statistically significant difference in compliance or scores, indicating that the verification loop, not the specific LLM, drives the reliability.
Limitations Identified: The framework cannot fix defects originating from the structural topology (e.g., frame configurations with poor force flow); it optimizes parameters under a fixed topology. In cases where topology is the issue, the system fails "loudly" with explicit warnings rather than silently delivering a non-compliant design.

5. Significance and Claims

The paper claims that trustworthiness in LLM-driven structural design is achieved not by more capable models, but by closing the generation–verification loop with an external verifier.

Shift in Paradigm: The work demonstrates that adding agents or using ReAct patterns alone does not improve reliability; the critical factor is the integration of an external, physics-based arbitration mechanism.
Practical Utility: The system acts as a "verifiable assistant" rather than an autonomous designer. It is most effective when the structural topology is reasonable, and the challenge lies in member sizing and code compliance.
Safety First: By failing loudly (explicitly flagging non-compliance due to topological limits) rather than silently passing errors, the framework aligns with safety-critical engineering requirements, leaving final judgment to human engineers.
Efficiency: The automated process reduces design time from hours to minutes (approx. 5–12 minutes per task) with negligible cost (approx. CNY 0.05 per task), making it a viable tool for engineering workflows.

The authors conclude that while the current scope is limited to planar linear-elastic systems and parameter adjustment, the proposed three-layer verification system and closed-loop mechanism provide a generalizable template for trustworthy AI in other safety-critical engineering domains.