MetroGS: Efficient and Stable Reconstruction of Geometrically Accurate High-Fidelity Large-Scale Scenes

MetroGS is a novel Gaussian Splatting framework that achieves efficient, stable, and geometrically accurate reconstruction of large-scale urban scenes by integrating distributed 2D Gaussian representation, structured dense enhancement with SfM priors, progressive hybrid geometric optimization, and depth-guided appearance modeling.

The Gist

Imagine you are trying to build a perfect, life-sized 3D model of a massive city using only thousands of 2D photographs taken from different angles. This is the challenge of Large-Scale Scene Reconstruction.

For a long time, computer scientists have been good at making these models look pretty (great colors and lighting), but they often struggled to make them accurate (the buildings might look wobbly, roads might have holes, or trees might float in mid-air).

Enter MetroGS. Think of MetroGS as a new, super-smart construction crew that doesn't just paint the model; it builds the actual skeleton first, ensuring the geometry is rock-solid before adding the paint.

Here is how MetroGS works, explained through simple analogies:

1. The Problem: The "Sparse" Blueprint

Imagine you are trying to draw a map of a city, but you only have photos of the main streets. The parks, alleyways, and the backs of buildings are missing. If you try to build a 3D model from just those few photos, you end up with a "sparse" model—lots of gaps and floating pieces.

• Old methods tried to fill these gaps by guessing, which often led to messy, inaccurate results.
• MetroGS says: "Let's get a better blueprint first."

2. The Solution: The "Smart Assistant" (Structured Dense Enhancement)

MetroGS brings in a "Smart Assistant" (a pre-trained AI model called a pointmap model) to help fill in the blanks before the construction even starts.

• The Analogy: Imagine you are assembling a giant puzzle, but you're missing 30% of the pieces. Instead of staring at the empty spots, you ask a friend who knows the picture perfectly to sketch in the missing pieces for you.
• MetroGS uses this "sketch" to create a dense, complete starting point. It also has a "Spot Checker" that looks for any remaining tiny holes and fills them in specifically, ensuring no part of the city is left empty.

3. The Construction: The "Two-Stage Refinement" (Progressive Hybrid Geometric Refinement)

Building a city is hard. If you try to fix the whole city at once, you might get overwhelmed. MetroGS builds it in two smart phases:

• Phase 1 (The Rough Draft): It uses a single camera's view to get a quick, rough idea of where the walls and roads are. It's like sketching the outline of a building with a pencil.
• Phase 2 (The Team Review): Then, it brings in the whole team. It looks at the same building from many different angles at once (like a group of architects standing around a model). They compare notes to fix any errors.
• The Magic Trick: Sometimes, looking at it from many angles creates "holes" because the team disagrees on a specific spot. MetroGS is smart enough to say, "Okay, the team is confused here, let's go back to the single-camera sketch to help us decide." This back-and-forth ensures the final shape is perfect.

4. The Paint Job: Separating Shape from Color (Depth-Guided Appearance)

One of the biggest headaches in 3D modeling is lighting. A building might look blue in the morning photo and orange in the evening photo. Old methods often got confused, thinking the building changed color or that the shape was wrong.

• The Analogy: Imagine you are painting a statue. If you try to paint the shape and the color at the same time, you might accidentally paint the shadow on the statue instead of just on the wall behind it.
• MetroGS separates the two jobs. It first builds the Shape (the geometry) perfectly. Once the shape is locked in, it uses that shape as a guide to figure out the Color and Lighting. This means the building looks consistent and realistic, no matter how the sun moved between photos.

5. The Result: Fast, Strong, and Beautiful

The paper highlights that MetroGS isn't just accurate; it's also incredibly fast.

• The Analogy: If other methods are like a single master carpenter trying to build a whole city alone (taking weeks), MetroGS is like a well-organized construction site with a fleet of trucks and workers all working in sync.
• The Stats: The authors claim their method achieves better results in less than 25% of the time it takes for the previous best methods. It's like finishing a marathon in a quarter of the time while still running a perfect race.

Summary

MetroGS is a new way to turn 2D photos into 3D worlds. It does this by:

1. Filling in the blanks before starting (using a smart assistant).
2. Building in stages, checking work from many angles (progressive refinement).
3. Separating the shape from the lighting so the model doesn't get confused by shadows.

The result is a digital city that is not only beautiful to look at but is also geometrically perfect, making it ready for real-world uses like self-driving cars, virtual reality, and aerial surveys.

Technical Summary

1. Problem Statement

While 3D Gaussian Splatting (3DGS) and its derivatives have revolutionized large-scale scene reconstruction by offering real-time rendering and high visual fidelity, significant challenges remain in achieving geometric accuracy and training stability in complex urban environments.

• Geometric Imbalance: Current methods often prioritize rendering quality at the expense of geometric precision, leading to structural inconsistencies.
• Sparse Initialization: In large-scale scenes, regions with weak textures or sparse views result in overly sparse initial point clouds (from SfM), causing surface holes and structural artifacts.
• Appearance Inconsistency: Large-scale datasets often suffer from varying illumination and exposure conditions. Existing methods struggle to decouple geometry from appearance, leading to optimization instability.
• Scalability & Efficiency: Existing surface reconstruction methods (e.g., 2DGS, PGSR) are often optimized for object-level scenes and struggle to scale efficiently to city-level reconstructions without compromising convergence speed or geometric completeness.

2. Methodology: MetroGS

MetroGS is a novel Gaussian Splatting framework built upon a distributed 2D Gaussian Splatting (2DGS) backbone. It introduces four key modules to address the challenges above:

A. Scalable Parallel Training Strategy

To handle massive datasets, MetroGS adopts a Gaussian-wise distributed training paradigm:

• Initialization: The initial point cloud is uniformly distributed across multiple GPUs.
• Batching: Multi-view batched training assigns images evenly among devices.
• Communication: Workers fetch only the required Gaussian subsets based on spatial locality.
• Load Balancing: Periodic redistribution of Gaussians during dynamic densification ensures balanced computational load.

B. Structured Dense Enhancement Scheme

To overcome sparse initialization in weak-texture regions, the authors propose a two-stage enhancement:

1. Pointmap-Assisted Initialization:
   • An undirected image graph is constructed based on SfM feature matches and partitioned into clusters matching the GPU count.
   • A pre-trained Pointmap model is used to predict dense 3D structures for each cluster.
   • These dense predictions are aligned with the SfM coordinate frame via a similarity transformation and merged to create a high-quality, dense auxiliary point cloud for initialization.
2. Sparsity Compensation Densification:
   • During the densification phase, a specific mechanism targets regions that remain sparse.
   • It identifies Gaussians with large contribution areas but low local voxel density and splits them. This prevents the formation of large, coarse primitives in sparse regions, improving geometric coverage without over-densification.

C. Progressive Hybrid Geometric Refinement

A two-stage optimization strategy balances efficiency and accuracy:

1. Single-View Optimization (Early Stage):
   • Uses a monocular depth prior (from an off-the-shelf estimator) to guide depth supervision.
   • Includes a scale regularization term to prevent large Gaussians from blurring details and consuming excessive memory.
2. Hybrid Multi-View Refinement (Later Stage):
   • Utilizes a PatchMatch-based algorithm to refine rendered depths using neighboring views, ensuring multi-view geometric consistency.
   • Alignment & Restoration: Since strict multi-view filtering can create holes in depth maps, the method reintroduces the monocular depth prior. It aligns the monocular depth with the filtered multi-view depth via least-squares estimation to recover valid regions that were mistakenly filtered out.

D. Depth-Guided Appearance Modeling

To handle illumination variations and decouple geometry from appearance:

• Tri-Mip Structure: Uses a Tri-Mip representation to store scale-adaptive, multi-resolution 3D features that maintain cross-view consistency.
• Geometry-Aligned Querying: The module queries 3D features using the high-quality optimized depth maps, ensuring the appearance features are geometrically consistent.
• Learnable Embeddings: Each training image is assigned a learnable appearance embedding to capture global illumination and exposure, allowing the model to focus on color/lighting variations independent of geometry.

3. Key Contributions

1. Structured Dense Enhancement: A novel scheme combining Pointmap models and sparsity compensation to solve initialization deficiencies in sparse urban regions.
2. Progressive Hybrid Refinement: A strategy integrating monocular priors with PatchMatch-based multi-view optimization to achieve both speed and high geometric accuracy.
3. Depth-Guided Appearance: A module that leverages optimized depth to decouple geometry and appearance, significantly enhancing stability in varying lighting conditions.
4. Scalable Framework: A distributed 2DGS training pipeline that supports large-scale reconstruction with high efficiency.

4. Experimental Results

The method was evaluated on GauU-Scene (real-world urban) and MatrixCity (synthetic) datasets, comparing against SOTA methods like CityGSV2, CityGS-X, 2DGS, and NeuS.

• Geometric Accuracy: MetroGS achieved the highest F1-scores and Precision/Recall metrics across all datasets. On the GauU-Scene "Modern Building" dataset, it improved the F1-score by 0.033 over CityGSV2.
• Rendering Quality: It consistently outperformed baselines in PSNR, SSIM, and LPIPS. For example, on the "Russian Building" scene, it achieved a PSNR of 24.94 (vs. 24.12 for CityGSV2).
• Efficiency:
  • MetroGS reached better performance in less than 25% of the training time required by CityGSV2.
  • On the MatrixCity-Aerial dataset, it achieved superior geometric fidelity (F1: 0.677 vs. 0.582) with a 1.7x reduction in training time compared to CityGS-X.
• Qualitative Results: Visual comparisons showed MetroGS produces cleaner meshes with fewer floating artifacts, better preservation of fine details (vegetation, building edges), and more consistent depth maps compared to competitors.

5. Significance

MetroGS represents a significant advancement in large-scale 3D scene reconstruction. By effectively addressing the trade-off between geometric accuracy and computational efficiency, it provides a unified solution for high-fidelity urban modeling.

• Practical Impact: The ability to reconstruct complex city scenes with high geometric fidelity and rapid convergence makes it highly suitable for applications in autonomous driving, aerial surveying, and immersive AR/VR.
• Methodological Innovation: The integration of dense priors (Pointmap), progressive hybrid optimization, and depth-guided appearance modeling sets a new standard for handling the specific challenges of large-scale, unstructured environments.

In conclusion, MetroGS demonstrates that with a well-designed training framework and specific geometric priors, Gaussian Splatting can be scaled to produce industrial-grade, geometrically accurate 3D models of entire cities efficiently.