Fast and Generalizable NeRF Architecture Selection for Satellite Scene Reconstruction

1. Problem Statement

Neural Radiance Fields (NeRF) have revolutionized 3D reconstruction from multi-view images but face significant barriers when applied to satellite imagery:

• Computational Cost: Traditional NeRF requires scene-specific training. Optimizing the architecture via Neural Architecture Search (NAS) for each scene demands hours to days of GPU time, making it infeasible for large-scale pipelines processing thousands of scenes.
• Data Complexity: Satellite data presents unique challenges, including varying geometries, illumination shifts, long capture intervals, and non-pinhole RPC camera models, which introduce view inconsistencies.
• Misguided Assumption: Existing approaches assume that architectural optimization is the primary driver of reconstruction quality. The authors argue this overlooks the fundamental reality that input view consistency is often more critical than model architecture for satellite scenes.

2. Core Insight

Through extensive SHAP (SHapley Additive exPlanations) analysis on diverse satellite scenes (suburban Jacksonville and industrial Omaha), the authors discovered a counter-intuitive finding:

• Multi-view consistency (e.g., photometric variance, view angle alignment, geometric coverage) is the dominant factor determining NeRF reconstruction quality.
• Architectural parameters (network depth, width, sampling density) have a minimal impact compared to scene-level descriptors.
• Conclusion: It is possible to predict reconstruction quality before training by analyzing scene characteristics, thereby eliminating the need for expensive per-scene NAS.

3. Methodology: PreSCAN Framework

The authors propose PreSCAN, a predictive framework that estimates NeRF reconstruction quality in seconds without training the actual NeRF model.

A. Feature Engineering

PreSCAN utilizes two types of lightweight descriptors:

1. Architectural Configurations:
   • Number of MLP layers ($L$).
   • Hidden layer dimensions ($f$).
   • Number of ray samples ($N_{sample}$).
   • Noise standard deviation ($\sigma_{std}$).
2. Scene-Specific Descriptors (Geometric & Photometric):
   • Weighted Inverse PSNR ($w'$): Penalizes dissimilar image pairs based on viewing angles.
   • Pair Density ($p(\phi)$): Fraction of valid image pairs relative to total views.
   • Coverage Score ($C$): Proportion of images participating in valid pairs.
   • Cosine Similarity: Angular alignment of view direction vectors.
   • Photometric Variance ($Var(I_i)$): Pixel-level variance capturing shadows and occlusions.
   • View Angle Statistics: Mean, max, and std of camera configurations.

B. Prediction Model

• Architecture: A lightweight parametric linear regression model (Ordinary Least Squares).
• Rationale: SHAP analysis revealed approximately linear relationships between scene descriptors and quality. A simple linear model ensures interpretability, prevents overfitting on limited data, and generalizes well.
• Formula: $PSNR_{pred} = \beta_{arch}^T X_{arch} + \beta_{scene}^T X_{scene} + \beta_0$.

C. Workflow

1. Input: Satellite scene images and a candidate set of architectural hyperparameters.
2. Feature Extraction: Compute geometric/photometric descriptors from the raw images.
3. Prediction: The model outputs a predicted PSNR for each architecture.
4. Selection: The architecture with the highest predicted PSNR (or best cost/quality trade-off) is selected for actual training.

4. Key Contributions

1. Scientific Finding: Quantitative validation via SHAP analysis that multi-view consistency, not model architecture, dictates NeRF performance in satellite imagery.
2. PreSCAN Framework: A predictive system that selects suitable NeRF architectures in < 30 seconds with < 1 dB prediction error, achieving a 1000× speedup over traditional NAS.
3. Generalizability: The model generalizes across diverse scene types (suburban vs. industrial) and different NeRF variants (S-NeRF, SatNeRF) without retraining.
4. Hardware-Aware Deployment: A case study demonstrating how PreSCAN predictions can be combined with offline cost profiling to optimize for edge devices (e.g., Jetson Orin).

5. Experimental Results

• Datasets: Evaluated on 73 scenes from the DFC2019 dataset (Jacksonville and Omaha).
• Speed:
  • NAS: ~9.7 hours per scene (on NVIDIA A6000).
  • PreSCAN: ~28.3 seconds per scene (on Tesla T4).
• Accuracy:
  • Mean Absolute Error (MAE) in PSNR prediction: 0.06 – 0.89 dB.
  • Most predictions fall within 0.5 dB of ground truth.
  • Robust performance even with sparse training data (45 scenes).
• Model Invariance: The framework works effectively across different NeRF variants (S-NeRF and SatNeRF), confirming it captures scene-driven difficulty rather than model-specific biases.

6. Case Study: Hardware-Aware Deployment

The authors demonstrated a practical deployment strategy on Jetson Orin edge platforms:

• Strategy: Combine PreSCAN's quality predictions with an offline database of hardware costs (power/latency) for various architectures.
• Optimization: Select the architecture that meets a minimum quality threshold ($PSNR_{min}$) while minimizing energy and latency.
• Results:
  • Power Reduction: 26% reduction in rendering power.
  • Latency Reduction: 43% reduction in rendering time.
  • Quality Loss: Minimal, only 0.79 dB PSNR difference compared to the unconstrained "best accuracy" model.

7. Significance

This work fundamentally shifts the paradigm for NeRF deployment in remote sensing:

• Scalability: By removing the need for per-scene architecture search, it makes large-scale satellite 3D mapping pipelines feasible.
• Edge Viability: It bridges the gap between high-fidelity 3D reconstruction and the strict power/latency constraints of satellite onboard systems and UAVs.
• Efficiency: It proves that understanding the data (scene consistency) is more valuable than brute-force model optimization for this specific domain.