🤖 AI

Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation

This paper proposes a framework for generating physically consistent and simulation-executable structural models by introducing a domain-specific dataset (CivilInstruct), a two-stage fine-tuning strategy for constraint satisfaction, and a verification-driven benchmark (MBEval) to ensure engineering accuracy.

Original authors: Yongqing Jiang, Jianze Wang, Zhiqi Shen, Zhenghong Lin, Jiayuan Wang, Yijian Yang, Kaoshan Dai, Haoran Luo

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yongqing Jiang, Jianze Wang, Zhiqi Shen, Zhenghong Lin, Jiayuan Wang, Yijian Yang, Kaoshan Dai, Haoran Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Smart but Clumsy" Architect

Imagine you hire a brilliant, super-fast architect who has read every book ever written about buildings. You tell them, "Build me a 5-story apartment complex that can survive a massive earthquake."

The architect (the Large Language Model or LLM) starts typing away at lightning speed. They produce thousands of lines of complex blueprints and construction instructions (the Code). At first glance, it looks perfect. The lines are straight, the math looks fancy, and the instructions are written in perfect professional language.

But then, you try to actually build it.

Suddenly, you realize the architect made some "hallucinations." They designed a beautiful staircase that leads into a solid wall. They calculated the weight of the roof so poorly that the building would collapse under its own weight. Or, they used a specific type of bolt that doesn't actually exist in the real world.

In the world of engineering, this is a disaster. A "pretty" blueprint that violates the laws of physics is worse than useless—it’s dangerous. Current AI is great at "looking" like an engineer, but it often fails at "thinking" like one.

The Solution: The "AutoBM" Framework

The researchers at Sichuan University and Nanyang Technological University created a system called AutoBM. Think of this not just as a "smart architect," but as an "Architect + Master Builder + Safety Inspector" all working in a perfect loop.

Here is how they fixed the "clumsy architect" problem using three main ingredients:

1. The "CivilInstruct" Textbook (Better Schooling)

Instead of letting the AI learn from the entire messy internet, they gave it a specialized, high-quality textbook called CivilInstruct.

The API Manual: It teaches the AI exactly how to use the specific tools (like a digital hammer called OpenSeesPy).
The "What Not To Do" Guide: They intentionally showed the AI examples of broken code and taught it how to fix them (this is the Bug-CoT part). It’s like teaching a student not just how to solve an equation, but how to spot their own mistakes.

2. The "Two-Stage Training" (The Apprenticeship)

They don't just teach the AI once; they train it in two distinct phases:

Phase 1 (The Basics): The AI learns the language. It learns how to write code that doesn't crash and follows the right grammar.
Phase 2 (The Reality Check): This is the secret sauce called RLA-SPC. They put the AI in a "virtual construction site" (a sandbox). If the AI generates a building that is physically impossible (e.g., it wobbles too much in a simulated earthquake), the system gives it a "bad grade." If the building is sturdy and follows all the safety laws, it gets a "gold star." Over time, the AI learns to prioritize physics over just looking good.

3. The "BMEval" Inspector (The Final Exam)

To make sure the AI is actually getting smarter, they created a rigorous final exam called BMEval. It doesn't just check if the code "runs"; it checks:

Does it work? (Can we actually execute the code?)
Is it accurate? (Does the building vibrate at the correct frequency?)
Is it safe? (Does it follow the strict legal codes for earthquake safety?)

The Result: From "Hallucinating" to "High-Precision"

Before this research, if you asked an AI to model a building, it might give you a "hallucinated" design that looked right but was physically broken.

After applying the AutoBM method, the AI becomes a professional-grade tool. It doesn't just write code; it writes simulation-ready, physics-consistent, and engineering-compliant instructions. It has moved from being a fast-talking dreamer to a reliable, digital engineer.

Technical Summary: Rethinking Scientific Modeling

Title: Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation
Authors: Yongqing Jiang, Jianze Wang, et al. (Sichuan University & Nanyang Technological University)

1. Problem Statement

In computational engineering, structural modeling requires extreme precision. While Large Language Models (LLMs) have shown promise in generating code, they face two critical failures in scientific domains like structural engineering:

Non-executability: Models often produce syntactically incorrect code or violate API invariants (e.g., duplicate element tags in OpenSeesPy), leading to runtime crashes.
Physical Inconsistency ("Physical Illusion"): Even when code executes successfully, the resulting models often violate fundamental laws of physics or engineering codes (e.g., unrealistic structural periods or excessive lateral drift), rendering the simulation scientifically invalid.

2. Methodology

The authors propose AutoBM (Automatic Building Modeling), a physics-aware framework designed to bridge the gap between natural language specifications and simulation-ready code. The methodology is built on three pillars:

A. Data Construction (CivilInstruct Dataset):
To move beyond general-purpose coding, the authors developed a four-part domain-specific dataset:

Fine-grained Supervised Data: API-learning samples derived from OpenSeesPy documentation.
Parameterized Long Code Data: Large-scale structural instances covering diverse building functions and seismic conditions.
Bug-CoT (Chain-of-Thought) Data: A specialized dataset of execution error logs and debugging trajectories to teach the model how to fix common structural modeling errors.
Physics-Informed Expert Data: High-quality, verified code samples paired with ground-truth physical metrics (e.g., fundamental periods).

B. Training Strategy (RLA-SPC):
The authors employ a two-stage alignment process:

Stage I: Domain Instruction Fine-Tuning (SFT): Initializes the model to master the syntax and API patterns of OpenSeesPy.
Stage II: Structural Physics-Constrained Reinforcement Learning (SPC-GRPO): Uses a variant of Group-Relative Policy Optimization. Instead of a standard reward, it uses a Multi-Granularity Hybrid Reward (MGHR) that penalizes:
- Format errors (Markdown/tag compliance).
- Logical incompleteness (Missing API tiers like topology, boundary, or solver).
- Physical inconsistency (Relative error in structural periods and violation of engineering drift limits).

C. Evaluation (BMBench/BMEval):
A new benchmark that evaluates models not just on code correctness, but on Engineering-Level Output, measuring:

Pass@k_period: Accuracy of structural dynamic properties.
Pass@k_compliance: Adherence to seismic design codes.
Pass@k_strict: A joint metric requiring successful execution, physical accuracy, and code compliance.

3. Key Contributions

Formalization of the AutoBM Task: Defines the mathematical objective of generating optimal structural models $\hat{M}$ conditioned on engineering specifications $D$ .
CivilInstruct & BMBench: Provides the community with a large-scale, high-quality dataset and a rigorous, physics-aware evaluation protocol.
RLA-SPC Framework: Introduces a novel reinforcement learning approach that integrates hierarchical AST (Abstract Syntax Tree) rewards and sandbox-based physical validation.

4. Experimental Results

Superiority over Baselines: The proposed framework significantly outperforms state-of-the-art models (including DeepSeek-Coder, Qwen, and Claude-Sonnet) on the Pass@k_strict metric.
Effectiveness of RLA-SPC: Ablation studies show that while SFT improves basic executability, the reinforcement learning stage is essential for eliminating "physical illusions" and ensuring engineering compliance.
Impact of Debugging Data: The inclusion of Bug-CoT data was shown to substantially increase code robustness and the model's ability to handle complex, multi-step analysis pipelines.

5. Significance

This work represents a paradigm shift from "Code-Generation LLMs" to "Scientific-Modeling LLMs." By integrating domain knowledge and physical constraints directly into the training loop, the authors demonstrate that LLMs can move beyond being mere "text predictors" to becoming reliable "computational engineers" capable of producing simulation-ready, physically valid scientific models.