An Industrial Dataset for Scene Acquisitions and Functional Schematics Alignment

This paper introduces IRIS-v2, a comprehensive industrial dataset containing multimodal data such as images, point clouds, and P&IDs to address the challenge of aligning functional schematics with 3D scene acquisitions for digital twin creation, demonstrating its utility through a case study that combines segmentation and graph matching to reduce manual alignment time.

The Gist

Imagine you are trying to renovate a massive, ancient factory. You have two very different maps of this place:

1. The "Real World" Map: This is a super-detailed 3D scan (like a digital ghost) of the factory floor, showing every pipe, valve, and pump exactly where they sit, even the ones covered in dust or hidden behind walls.
2. The "Blueprint" Map: This is an old, flat drawing (called a P&ID) that shows how the machines should be connected. It's like a subway map, showing which station connects to which, but it doesn't tell you exactly where the stations are located in the room.

The Problem:
For years, engineers have had to manually match these two maps. They'd look at a drawing, walk around the factory, find the pump, and say, "Okay, this drawing symbol matches that real pump." It's like trying to solve a giant, 3D jigsaw puzzle while wearing blinders. It takes forever, it's boring, and if the factory has changed over the years (pipes moved, new pumps added), the old drawings don't match reality at all.

The Solution (IRIS-v2):
This paper introduces a new tool called IRIS-v2. Think of it as a "training gym" for computers. The authors took a huge industrial room (over 500 square meters) and created a massive dataset that includes:

• 3D Point Clouds: Millions of dots creating a 3D shape of the room.
• 360° Photos: High-resolution spherical images taken from every angle.
• The Blueprints: The actual functional schematics (P&ID).
• Annotations: Humans have already labeled thousands of objects in these images and 3D models to teach the computer what a "valve" or a "pipe" looks like.

How the Computer Solves the Puzzle:
The paper doesn't just give you the data; it shows a clever three-step method to automatically align the "Real World" map with the "Blueprint" map:

1. The Detective (Segmentation): First, the computer looks at the 3D scan and the photos. It acts like a detective, using AI to find and outline every single object. It says, "That's a pump," "That's a pipe," "That's a valve." It's like the computer putting sticky notes on every item in the room.

   • Analogy: Imagine a robot walking through the factory with a highlighter, circling every piece of equipment it sees.

2. The Social Network (Graph Construction): Next, the computer turns these objects into a "social network."

   • In the Blueprint, the network shows who is friends with whom (e.g., "Pump A is connected to Valve B").
   • In the Real World, the computer builds a similar network based on who is physically touching or close to whom.
   • Analogy: It's like taking a photo of a crowded party and drawing lines between people who are standing next to each other.

3. The Matchmaker (Graph Alignment): Finally, the computer tries to match the "Blueprint Network" with the "Real World Network." It looks for patterns. "In the drawing, the Pump is connected to a Filter. In the real scan, I see a Pump connected to a Filter. Therefore, this Pump in the scan must be the Pump in the drawing."

   • The Twist: Sometimes the real world is messy. Maybe a pipe is hidden, or a valve was removed. The computer uses a special math trick (called "optimal transport") to say, "Okay, even though I can't see the Filter, I know the Pump is connected to it, so I can guess where the Filter should be."

The Human Touch:
The system isn't perfect yet. If the computer gets confused (e.g., "Wait, this pipe connects to two pumps, but the drawing says only one"), it flags the error and asks a human expert to fix it. Once the human corrects it, the computer tries again. It's a loop of "Computer guesses -> Human corrects -> Computer learns."

Why This Matters:
This is a huge step toward creating "Digital Twins." A Digital Twin is a perfect, living digital copy of a real factory. If you can automatically align the old blueprints with the current 3D reality, you can:

• Predict when machines will break before they do.
• Train new workers in Virtual Reality using the exact layout of the real factory.
• Save thousands of hours of manual work for engineers.

In a Nutshell:
This paper gives us a new, massive dataset and a smart method to teach computers how to match old 2D factory drawings with new 3D scans of the actual factory, turning a tedious, manual headache into an automated, intelligent process.

Technical Summary

1. Problem Statement

The paper addresses the critical challenge of aligning functional schematics (specifically Piping and Instrumentation Diagrams, or P&IDs) with 3D scene acquisitions (point clouds and images) to create digital twins of industrial facilities.

• Context: Old industrial facilities often lack native digital models. Creating them requires manually aligning 2D/3D data with functional diagrams, a process that is tedious, time-consuming, and requires expert human intervention.
• Challenges:
  • Scale & Complexity: Facilities contain thousands of equipment pieces and hundreds of meters of piping.
  • Inconsistencies: Discrepancies exist between the "as-built" reality (scene) and the design schematics (e.g., missing objects, occlusions, or unrecorded modifications).
  • Data Scarcity: There is a lack of public datasets containing both real-world industrial acquisitions and their corresponding functional schematics, hindering the development of automated solutions.
  • Missing Metrics: Schematics often lack distance information, making matching based solely on geometry difficult; structural relationships must be used instead.

2. Key Contributions

The primary contribution of the paper is the release of the IRIS-v2 dataset, which extends the previous IRIS dataset.

• Dataset Composition:
  • Scene Data: A dense LiDAR point cloud (150 points/cm²) of a 530m² industrial room, reconstructed from 67 acquisition stations.
  • Imagery: 300 high-resolution spherical images (16384x8192) paired with the point cloud.
  • Annotations:
    • 6,000+ annotated 2D bounding boxes across 171 classes.
    • 47,000+ 2D segmentation masks projected from a CAD model.
    • 3D pipe routing information (extracted semi-automatically).
  • Functional Data: The corresponding P&ID (PDF format) containing ~500 pieces of equipment and pipe routing logic.
• Use Case Demonstration: The authors propose and validate a pipeline for Scene-Functional Alignment using this dataset, demonstrating how to bridge the gap between raw data and functional logic.

3. Methodology

The proposed alignment framework (Algorithm 1) consists of three main stages:

A. 3D Segmentation

The goal is to locate objects in the 3D environment.

• Equipment: Uses a 2D-to-3D projection approach.
  • Detection: Uses Grounding DINO (fine-tuned on the dataset) to detect objects via text prompts.
  • Segmentation: Uses SAM (Segment Anything Model) to generate 2D masks within bounding boxes.
  • Projection: Masks are projected onto the point cloud. A hidden point removal operator is applied to prevent points from occluded objects behind the target from being included.
  • Fusion: 3D masks representing the same object are fused based on minimum common points.
• Pipelines: Due to the complexity of pipe shapes and junctions, fully automatic segmentation is currently unreliable. The authors use PipeRunner (a semi-automatic tool in Trimble RealWorks) to reconstruct pipe lines, elbows, and junctions, achieving high efficiency (>200 m/h).

B. Graph Construction

Both the scene and the schematics are converted into a unified graph representation:

• Nodes: Represent both equipment and pipe segments (cut at T/Y-junctions). This is crucial because pipes often connect to more than two pieces of equipment and must be matched explicitly.
• Edges: Represent physical connections (objects touching or connected).
• Attributes: Each node is tagged with object type attributes.
• Preprocessing: Pipe nodes with a degree lower than 2 (open ends) are removed to ensure topological consistency at boundaries.

C. Robust Graph Matching & Human-in-the-Loop

• Matching Algorithm: The authors employ SLOTAlign, a graph matching method based on optimal transport. It is chosen for its ability to handle node attributes and its robustness to structural perturbations (differences between the scene graph $S$ and schematic graph $F$).
• Inconsistency Handling:
  • The schematic ($F$) is treated as the "target" (more reliable), and the scene ($S$) as the "source."
  • The system automatically detects inconsistencies:
    1. Multiple scene nodes mapped to one schematic node.
    2. Schematic nodes with no preimage in the scene (e.g., hidden objects).
    3. Broken edges (topology mismatch).
  • Human Resolution: Detected inconsistencies are presented to a human operator for correction. The graph matching is then re-run. This loop continues until no inconsistencies remain.

4. Results

The method was validated on the specific industrial room provided in the IRIS-v2 dataset:

• Segmentation Performance:
  • Equipment: High accuracy for most objects (valves, manometers). The pump was difficult to detect automatically and required manual segmentation, highlighting the need for further fine-tuning data.
  • Pipes: The semi-automatic PipeRunner tool provided accurate routing data, though fully automatic separation of individual pipe elements (e.g., distinguishing a T-junction from a straight pipe) remains a challenge.
• Alignment Success:
  • The graph matching successfully aligned the scene and the P&ID.
  • Robustness Test: In a scenario where a filter was hidden in the 3D scene (occluded by insulation) but present in the P&ID, the algorithm successfully aligned the surrounding pipes. This allowed the system to infer the approximate location of the hidden filter, demonstrating robustness to occlusions and structural perturbations.
  • The human-in-the-loop process effectively resolved remaining mismatches, confirming the feasibility of a semi-automated workflow.

5. Significance and Future Work

• Bridging the Gap: IRIS-v2 fills a critical void in the literature by providing the first comprehensive dataset pairing real industrial 3D acquisitions with functional schematics.
• Digital Twin Enabling: The proposed workflow offers a scalable path to creating digital twins for legacy infrastructure, enabling predictive maintenance, operator training, and safety analysis.
• Future Directions:
  • Expanding the dataset to larger, more complex facilities.
  • Developing fully automatic pipe tracing algorithms to replace semi-automatic tools.
  • Automating the inconsistency correction step to reduce human reliance.

In summary, the paper presents a robust, semi-automated framework for aligning industrial 3D scenes with functional diagrams, underpinned by a new, rich dataset (IRIS-v2) that enables research into a previously underexplored domain.