💬 NLP

BIM Information Extraction Through LLM-based Adaptive Exploration

This paper introduces an adaptive exploration paradigm using LLM-based agents that iteratively execute code to dynamically discover BIM model structures at runtime, demonstrating significant performance improvements over static query generation methods on the newly proposed ifc-bench v2 benchmark.

Original authors: Sylvain Hellin, Suhyung Jang, Stefan Fuchs, Stavros Nousias, André Borrmann

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sylvain Hellin, Suhyung Jang, Stefan Fuchs, Stavros Nousias, André Borrmann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Universal Translator" That Gets Lost

Imagine you have a massive, incredibly detailed 3D blueprint of a building (called a BIM model). This blueprint holds every single piece of information: how many doors there are, how wide they are, what material the walls are made of, and even the geometry of the roof.

However, asking a computer to find a specific piece of info in this blueprint is like asking a librarian to find a book in a library where:

The catalog is broken: One librarian calls a door "Door," another calls it "Tür" (German), and a third calls it "Rough Width."
The books are hidden: Sometimes the width of a door is written on a sticky note; other times, you have to measure the drawing yourself to figure it out.
The rules change: Every time you walk into a new building (a new BIM model), the librarian changes the rules of how they organize the shelves.

Currently, most computer programs try to ask for this information by guessing the rules beforehand. They say, "I bet the door width is in the 'Width' folder." If they are wrong, the program crashes or gives up. This is called the Static Approach.

The Solution: The "Detective Agent"

The authors of this paper propose a new way to do this called Adaptive Exploration.

Instead of guessing the rules, they built an AI "Detective Agent." Here is how it works:

It doesn't guess; it investigates. When you ask, "How wide is Door 1?", the agent doesn't just look for a folder named "Width."
It writes its own instructions. The agent writes a small piece of computer code (like a note to itself) to go look at the door.
It learns from mistakes. If the agent looks in the "Width" folder and finds nothing, it doesn't quit. It sees the error, thinks, "Oh, maybe it's called 'Breite' in this model," and writes a new note to look there.
It keeps going until it finds the answer. It repeats this loop—look, check, adjust, look again—until it finds the answer or runs out of time.

The paper calls this Adaptive Exploration because the agent adapts its strategy based on what it actually finds in the model, rather than assuming the model is organized a certain way.

The Experiment: The "Gym" for AI

To test if this detective agent was better than the old "guessing" programs, the researchers built a giant test called ifc-bench v2.

Think of this as a gym with 1,027 different workout challenges.
These challenges were based on 37 different building blueprints from real projects, made by different software tools (like Revit and ArchiCAD).
The questions ranged from simple ("How many doors are there?") to complex ("Calculate the total volume of concrete in the walls," which requires doing math because the number isn't written down).

They tested two types of AI "brains":

The Super Brain: A very powerful Large Language Model (LLM).
The Smaller Brain: A less powerful version of the same type of model.

They tested both brains using two methods:

Static: The brain guesses the answer in one go.
Adaptive: The brain uses the detective loop (write code, check, adjust, repeat).

The Results: Why "Trying Again" Wins

The results were clear and surprising:

1. The Detective Agent crushed the Guessers.
The "Adaptive" approach was vastly better than the "Static" approach.

The Gap: The detective agent was about 37% more accurate than the static guesser.
The "Give Up" Rate: The static guesser gave up (said "I don't know") on about 50% of the questions. The detective agent only gave up on 6%.
The "Super Brain" vs. "Smaller Brain": Even the "Smaller Brain" using the detective method performed better than the "Super Brain" using the guessing method. This proves that how you ask the question (the method) matters more than how smart the brain is.

2. The "Cheat Sheets" (Augmentation) didn't help the Smart Brain.
The researchers tried giving the agents "cheat sheets" (documentation about how to read the blueprints) and "pre-made tools" (shortcuts to common tasks).

For the Super Brain: The cheat sheets and tools did nothing. The Super Brain was smart enough to figure out the rules on its own by exploring.
For the Smaller Brain: The cheat sheets helped a little bit. But the pre-made tools actually hurt the Smaller Brain. It got confused by the tools and got stuck in loops, giving up more often.

The Main Takeaway

The paper concludes that the biggest mistake in trying to read building blueprints is assuming the blueprint is organized perfectly.

Old Way: "I know exactly where the data is, so I will write one command to get it." (Fails often because real-world data is messy).
New Way: "I don't know where the data is, so I will write code to look around, check my mistakes, and keep searching until I find it." (Works much better).

The authors say that for the future of reading building models, we shouldn't just build smarter AI brains or write more cheat sheets. Instead, we need to build better explorers that can handle the messiness of real-world data by adapting on the fly.

What the Paper Does NOT Say

It does not claim this system is ready to be used in hospitals or construction sites today (the accuracy is around 56%, which isn't high enough for safety-critical tasks yet).
It does not claim this works for every type of building software, only for the specific format (IFC) they tested.
It does not claim that "tools" are useless forever, but rather that they didn't help in this specific, messy environment.

Technical Summary: BIM Information Extraction Through LLM-based Adaptive Exploration

1. Problem Statement

The construction industry increasingly relies on Building Information Modeling (BIM) for data-rich representations of building geometry, semantics, and topology. However, extracting specific information from these models remains a critical bottleneck due to two interrelated challenges:

Query Expertise Gap: Domain experts (architects, facility operators) typically lack the programming skills or knowledge of specific query languages (SQL, SPARQL, API calls) required to retrieve non-trivial data, such as cross-element aggregations or derived quantities.
BIM Heterogeneity: Even within standardized formats like Industry Foundation Classes (IFC), data organization varies significantly. Properties may be stored as explicit attributes, derived from geometry, or named differently depending on the authoring tool (e.g., Revit vs. ArchiCAD), modeling conventions, and language.

Existing approaches generally fail to address both challenges simultaneously. Query translation methods assume a fixed data structure and require manual preprocessing, breaking down when faced with heterogeneous data. Static LLM-based approaches (e.g., single-pass code generation) introduce semantic understanding but retain static architectures that cannot adapt to runtime data discoveries. Prior agentic approaches using pre-defined toolsets are limited by the scope of those tools, which often fail to cover the vast diversity of real-world BIM schemas.

2. Methodology

The authors propose a new paradigm: Adaptive Exploration. This approach utilizes an LLM-based agent that iteratively generates and executes Python code against live BIM models to discover structure and extract information at runtime, rather than assuming a known schema.

2.1. Agent Architecture

The system employs a CodeAct architecture where the agent operates in a loop:

Input: Natural language question, BIM model path, execution history, and optional augmentation (tools/documentation).
Action: The agent generates Python code to query the model (using the IfcOpenShell library).
Feedback: The code is executed in a sandboxed environment. The agent receives execution outputs (return values, errors, print statements) and uses this feedback to refine its strategy in subsequent iterations.
Termination: The loop continues until a final answer is produced or a maximum iteration limit is reached.

2.2. Augmentation Strategies

The study evaluates the interaction between the adaptive paradigm and two augmentation strategies:

Documentation Retrieval: A hybrid retrieval pipeline (combining dense vector search, BM25, and reverse-question matching) injects relevant API documentation (specifically for IfcOpenShell) into the agent's context.
Domain-Specific Tools:
- Manual Tools: Hand-crafted Python functions for common extraction patterns.
- Automated Tools: A multi-agent system pipeline that analyzes successful and failed extraction histories to automatically generate, debug, and validate new tools.

2.3. Experimental Design

The evaluation uses a 3×4 factorial design:

Rows (Model/Paradigm):
- Adaptive-4.7 (Iterative exploration with a high-capability model, GLM 4.7).
- Adaptive-4.5 Air (Iterative exploration with a reduced-capability model, GLM 4.5 Air).
- Static-4.7 (Single-pass code generation with the high-capability model).
Columns (Augmentation): None, Documentation, Manual Tools, Auto-generated Tools.

3. Key Contributions

Adaptive Exploration Paradigm: A validated framework where an LLM agent iteratively executes code to discover BIM data structures at runtime, significantly outperforming static query generation.
ifc-bench v2: The first large-scale, open, permissively licensed BIM question-answering benchmark. It comprises 1,027 tasks across 37 IFC models from 21 projects. Unlike prior datasets, it includes open-ended questions requiring judgment and estimation, reflecting real-world information needs.
Systematic Ablation Study: A comprehensive evaluation demonstrating that augmentation effectiveness is strongly dependent on model capability. The study challenges the assumption that more augmentation always yields better results, showing that high-capability models are invariant to augmentation while weaker models are sensitive to it.

4. Results

4.1. Paradigm Dominance

Adaptive exploration significantly outperforms static code generation across all configurations.

Accuracy Gap: The adaptive paradigm (GLM 4.7) achieved 55.4%–56.6% accuracy, while the static baseline (GLM 4.7) achieved only 16.9%–20.6%. This represents a +36.8 to +38.5 percentage point (pp) advantage.
Capability Compensation: Even the weaker model with adaptive exploration (GLM 4.5 Air, ~2.7x fewer active parameters) outperformed the stronger model with static generation (25.7% vs. 19.1%), indicating that the iterative paradigm compensates for reduced model capability.
Abstention Rates: Static models abstained on ~50% of questions, whereas adaptive models abstained on <7%.

4.2. Category-Specific Performance

Geometric/Spatial Computation (Category 3): The largest performance gap was observed here. Static models abstained on >75% of these tasks, while adaptive models achieved 66.7% accuracy by iteratively deriving geometric quantities.
Direct Retrieval (Category 1): Adaptive models outperformed static ones, though the gap was smaller than in computation tasks.
Inversion of Difficulty: Contrary to intuition, the adaptive system performed better on complex geometric computations (66.7%) than on direct property retrieval (55.6%). This is attributed to the difficulty of navigating deeply nested IFC material structures for retrieval, whereas the iterative loop excels at step-by-step geometric derivation.

4.3. Augmentation Effects

Model Capability Dependence:
- High-Capability (GLM 4.7): Augmentation strategies (documentation, manual tools, auto tools) had no measurable effect (accuracy range 55.4%–56.6%, all $p > 0.8$ ). The iterative loop provided sufficient information discovery, rendering external scaffolds redundant.
- Reduced-Capability (GLM 4.5 Air): Augmentation effects were significant and varied:
  - Documentation: Improved accuracy by +4.9pp ( $p < 0.05$ ).
  - Manual Tools: Harmful, reducing accuracy by -10.2pp ( $p < 0.001$ ). The weaker model got trapped in unproductive tool invocation loops.
  - Auto Tools: Marginally negative but non-significant compared to the baseline.

5. Significance and Claims

The paper claims that BIM heterogeneity is best addressed at the paradigm level rather than by optimizing static approaches or adding augmentation layers.

Paradigm Shift: The choice of an adaptive, iterative exploration paradigm is the primary determinant of extraction accuracy, outweighing both model capability and augmentation strategies.
Limitations of Static Approaches: Static approaches, which assume known schemas, systematically fail on heterogeneous real-world data. The "bitter lesson" of AI applies here: leveraging general computation (iterative execution) outperforms encoding human knowledge (tools/docs) as model capabilities increase.
Practical Implications: For high-capability models, investing in augmentation infrastructure (documentation retrieval, tool generation) yields no measurable benefit and adds cost. For weaker models, documentation helps, but pre-built tools may hinder performance if they constrain the agent's exploration.
Future Directions: The authors suggest that standardized information requirements (e.g., IDS, mvdXML) could reduce representational heterogeneity, potentially making tool-based approaches more viable in the future. They also call for cross-family replication of these findings across diverse LLM providers.

The study concludes that adaptive exploration is the most effective current strategy for BIM information extraction, enabling agents to handle the "norm" of data variation in practice rather than just edge cases.