Three-dimensional reconstruction and segmentation of an aggregate stockpile for size and shape analyses

This paper presents an innovative, mobile-device-based 3D imaging approach using Structure-from-Motion and segmentation algorithms to reconstruct and analyze aggregate size and shape from stockpiles, offering a convenient and affordable solution for onsite quality assurance in road construction and geotechnics.

The Gist

Imagine you are a construction engineer standing in front of a massive mountain of rocks (an aggregate stockpile) at a quarry. You need to know: How big are these rocks? Are they round like marbles or flat like pancakes? And how many of each size do we have?

Traditionally, answering these questions was a nightmare. Engineers had to either:

1. Guess: Look at the pile with their eyes (which is unreliable and subjective).
2. Lift: Use heavy cranes to pick up individual rocks, measure them with calipers, and weigh them (which is slow, expensive, and dangerous).

This paper introduces a "Magic Camera" solution that turns a smartphone into a high-tech rock scanner. Here is how it works, broken down into simple steps with some creative analogies.

1. The "Paparazzi" Method (3D Reconstruction)

Instead of trying to measure one rock at a time, the engineers walk around the entire pile of rocks, taking a video or a series of photos from every angle—high up, low down, left, and right.

• The Analogy: Think of this like a swarm of paparazzi trying to take a picture of a celebrity. If you only have one photo, you only see their face. But if 50 people take photos from every angle, you can piece together exactly what they look like in 3D.
• The Tech: The computer uses a technique called Structure-from-Motion (SfM). It looks at all the photos, finds the same "dots" (features) on the rocks in different pictures, and mathematically calculates where those dots exist in 3D space.
• The Result: The computer builds a "Digital Twin" of the rock pile. It's not just a flat photo; it's a cloud of millions of tiny 3D dots (a point cloud) that perfectly mimics the shape of the pile, complete with colors and textures.

2. The "Clay Sculptor" (Turning Dots into Mesh)

Right now, the computer has a cloud of dots. It's like having a bucket of loose sand. To understand the shape of the rocks, we need to turn that sand into a solid object.

• The Analogy: Imagine the dots are loose clay particles. The computer uses a technique called Poisson Surface Reconstruction to smooth those particles together, turning the "loose sand" into a smooth, continuous "clay sculpture" of the pile. Now, instead of dots, we have a digital mesh (like a wireframe model) where every rock is a connected surface.

3. The "Smart Cookie Cutter" (3D Segmentation)

Now the computer has one giant digital sculpture of the whole pile. But the engineers need to know about individual rocks. They need to separate the pile into distinct pieces.

• The Analogy: Imagine the rock pile is a giant, fused block of cookie dough. You need to cut out individual cookies without squishing them.
• The Trick: The computer looks for the "valleys" between the rocks.
  • The top of a rock is usually curved outward (convex).
  • The gap between two rocks is usually a sharp, deep dip (concave).
• The Algorithm: The computer runs a "search" (like a flood filling a bucket) across the surface. When it hits a sharp "valley" or a steep drop in curvature, it stops. It says, "Okay, this is the edge of Rock A; everything past this line belongs to Rock B."
• The Result: The computer successfully "cuts" the digital pile into 10 individual rocks, coloring each one differently so you can see them clearly.

Why This Matters (The "So What?")

This isn't just a cool tech demo; it's a game-changer for construction quality control.

• Speed & Safety: Engineers can walk around a pile with an iPhone, record a 30-second video, and get a full analysis in minutes. No cranes, no heavy lifting, no climbing on dangerous piles.
• Better Roads: The size and shape of rocks determine how strong a road base or a railway track will be. If the rocks are too flat, the road might crack. If they are too round, the track might shift. This system gives precise data to ensure the materials are perfect.
• The Future: The authors admit this is a "proof of concept." Right now, it only sees the top of the rocks (like looking at a pile of oranges; you can't see the bottom ones). Future versions will use AI to "guess" the hidden parts of the rocks based on what they see, making the analysis even more accurate.

In a nutshell: This paper teaches us how to turn a smartphone video into a 3D map, then use a digital "cookie cutter" to separate individual rocks, allowing engineers to measure the quality of a rock pile instantly, safely, and cheaply.

Technical Summary

Here is a detailed technical summary of the paper "Three-dimensional reconstruction and segmentation of an aggregate stockpile for size and shape analyses" presented at the 20th International Conference on Soil Mechanics and Geotechnical Engineering (Sydney 2021).

1. Problem Statement

Aggregate size and shape are critical morphological properties that influence the performance of materials in road construction, railway ballast, and hydraulic applications (e.g., riprap). However, current characterization methods face significant limitations:

• Subjectivity and Labor: Traditional field practices rely on visual inspection or manual measurements, which are subjective, time-consuming, and require heavy machinery to handle individual large rocks.
• Limitations of 2D Imaging: Existing computer vision techniques typically analyze 2D images of manually separated particles in a lab. These methods lose significant spatial information when projecting 3D scenes onto 2D planes and are not scalable for large, in-place stockpiles.
• Lack of Field-Ready 3D Solutions: While 3D scanning (laser, CT) exists for individual particles, there is no convenient, affordable system to acquire 3D data of entire aggregate stockpiles directly in the field for Quality Assurance/Quality Control (QA/QC).

2. Methodology

The authors propose a novel, field-deployable 3D imaging approach using mobile devices (smartphones) and computer vision techniques. The methodology consists of two primary stages:

A. 3D Reconstruction via Structure-from-Motion (SfM)

• Data Acquisition: Engineers capture a sequence of images or a video by moving around the stockpile at various heights and angles.
• SfM Process: The system uses SfM techniques to recover the 3D stationary structure from 2D images. This involves:
  1. Extracting and matching local features across multiple views.
  2. Estimating camera motion (translation and rotation).
  3. Bundle Adjustment: Jointly estimating camera parameters and 3D point coordinates to minimize the total reprojection error (the squared pixel distance between observed and projected features).
• Output: The result is a dense 3D point cloud containing spatial coordinates, RGB color values, and surface normal directions.

B. 3D Segmentation of Individual Aggregates

To extract individual particle data from the raw point cloud, the authors developed a specific pipeline:

1. Mesh Generation: The unordered point cloud is converted into a triangular mesh using Poisson surface reconstruction. This creates a continuous surface where vertices are points and faces indicate connectivity.
2. Curvature-Based Segmentation: A customized Breadth-First Search (BFS) algorithm is applied to the mesh to separate individual rocks.
   • Logic: The algorithm assumes that boundaries between adjacent aggregates exhibit high concave curvature, while the surface of a single aggregate is convex or slightly concave.
   • Criterion: During the BFS traversal, the algorithm calculates the dot product between the normal vector of the current face and the sum of the neighboring face's normal vector and center difference vector.
   • Thresholding: If the calculated value exceeds a specific threshold (set to 0.7, approx. 45°), the faces are considered part of the same aggregate. If the value is below the threshold, the algorithm identifies a boundary and stops the traversal for that segment.
   • Iteration: The process restarts automatically from unvisited faces until the entire mesh is segmented.

3. Key Contributions

• Field-Deployable Workflow: Demonstrated a proof-of-concept for using standard smartphone cameras to perform 3D analysis of large stockpiles, eliminating the need for expensive laboratory setups or heavy machinery.
• Novel Segmentation Algorithm: Developed a curvature-constrained BFS algorithm specifically designed to segment individual aggregates from a reconstructed stockpile mesh, addressing the challenge of separating touching particles in 3D space.
• Integration of SfM and Segmentation: Successfully linked multi-view 3D reconstruction with automated particle extraction, moving beyond 2D image analysis to true 3D morphological characterization.

4. Results

The approach was tested on a small stockpile consisting of 10 riprap rocks (ranging from 3 to 10 inches in size).

• Data Collection: 46 multi-view images were captured using a smartphone (2400x3000 resolution).
• Reconstruction Quality: The SfM process generated a high-fidelity 3D point cloud. Visual comparison with the original scene confirmed that surface textures, colors, and marker signs were accurately recovered.
• Segmentation Success: The curvature-based BFS algorithm successfully segmented all 10 individual rocks from the stockpile. The boundaries identified by the algorithm aligned with human perception, effectively distinguishing adjacent aggregates.
• Limitations Observed:
  • The reconstruction is currently dimensionless (lacks a real-world scale) without a calibration object.
  • The algorithm uses fixed parameters; adaptive or AI-based segmentation could improve robustness.
  • Only the visible surface of the aggregates is captured; the "unseen" sides require shape prediction algorithms for full morphological analysis.

5. Significance and Future Outlook

• QA/QC Transformation: This technology offers a pathway to automate and standardize onsite QA/QC for aggregate stockpiles, reducing labor costs, transportation needs, and reliance on subjective visual inspections.
• Design Optimization: Accurate 3D size and shape data can lead to better material selection, improved layer stiffness modeling, and optimized designs for transportation infrastructure.
• Future Directions: The authors identify several areas for development:
  • Integrating calibration objects for scale determination.
  • Replacing fixed-parameter segmentation with Deep Learning/AI techniques to handle complex geometries and varying lighting conditions.
  • Developing shape prediction algorithms to estimate the full 3D geometry of partially visible aggregates.
  • Exploring alternative sensors like LiDAR for potentially higher precision and calibration-free mapping.

In conclusion, the paper establishes a foundational framework for transitioning aggregate characterization from 2D/manual methods to efficient, 3D, field-based digital inspection.