💻 computer science

DA-NBV: A Direction-Aware Next-Best-View Planner for Efficient 3D Reconstruction of Ships at Sea

This paper proposes DA-NBV, a direction-aware next-best-view planner that integrates directional observation statistics and a learnable Position Advantage Field to overcome the challenges of ship self-occlusion and sea-state dynamics, thereby significantly improving 3D reconstruction completeness and efficiency for maritime applications.

Original authors: Jiaming Chen, Juntao Yang, Zhentao Zou, Qi Ming, Yi Yu, Zhihang Zhong, Xue Yang, Xue Jiang, Yue Zhou

Published 2026-08-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiaming Chen, Juntao Yang, Zhentao Zou, Qi Ming, Yi Yu, Zhihang Zhong, Xue Yang, Xue Jiang, Yue Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a perfect 3D model of a giant, floating castle using only a drone camera. This isn't just a video game; it's a real challenge in the world of robotics and computer vision, where machines try to "see" and map the physical world. Usually, when robots scan objects, they use a simple rule: "If I can see this part of the object, I'm done with it." They treat the object like a flat painting, checking off boxes as they spot different spots. But real objects, especially ships at sea, are tricky. They have towers, decks, and hidden corners that block each other. If you only look at a ship from the front, you miss the back of the smokestack. If the ship is rocking on waves, the view changes every second. To get a perfect 3D map, a robot needs to realize that seeing a spot once isn't enough; it needs to see that same spot from many different angles to understand its true shape. This is the core puzzle: how do you teach a drone to be a smart, curious explorer that knows exactly where to fly next to fill in the missing pieces, even when the target is moving and the weather is messy?

This is the problem tackled by a new system called DA-NBV (Direction-Aware Next-Best-View). Think of the old way of scanning a ship like a person walking around a statue with their eyes closed, only opening them when they think they've seen enough. They might walk in circles, looking at the same side of the statue over and over, while the back remains a mystery. The paper argues that existing methods make this mistake by only tracking if a spot has been seen, ignoring from which direction it was seen.

The authors propose that for complex objects like ships, you need a "directional memory." Imagine a ship's hull covered in thousands of tiny sensors. Instead of just asking, "Has this sensor been touched?" the new system asks, "Has this sensor been touched from the left? From the right? From above?" If a sensor has only been hit by a beam of light from the front, the system knows it's still "hungry" for a view from the side. To solve this, they built a special "Position Advantage Field" (PAF). You can think of this as a magical 3D map that glows brighter in spots where the drone needs to fly to get a new angle on a tricky part of the ship. It's like a GPS that doesn't just say "go here," but whispers, "go there to see the part you've been missing."

The paper introduces a few clever tricks to make this work. First, the drone doesn't just pick a random spot far away; it uses a "locally constrained" strategy. Instead of teleporting across the ocean, it takes small, smart steps, checking its surroundings and adjusting its camera angle step-by-step. This is like a hiker carefully stepping over rocks rather than trying to jump the whole mountain at once. Second, the system is trained in a super-realistic simulator where the ocean waves actually rock the ship, and the wind pushes the drone around. This is crucial because, in the real world, ships don't sit still; they bob, roll, and pitch. The researchers created a dataset called SeaShip-3D with 300 different ship models to train their AI in these wobbly conditions.

The results of their experiments are quite promising, though they are based on simulations and not yet tested on a real drone in the middle of the Atlantic. In their digital ocean, the DA-NBV system managed to capture about 98.49% of the ship's surface, compared to roughly 95.53% for the next-best existing method. More importantly, the 3D models it created were much sharper. The "Chamfer distance" (a fancy math way of saying "how far off the model is from the real shape") dropped significantly, from 6.49 cm down to 3.68 cm. This means the digital twin was nearly twice as accurate. The system also flew more efficiently, covering more ground with less wasted movement.

The authors explicitly argue against the idea that simply seeing a spot once is enough for a good reconstruction. They show that ignoring the direction of the view leads to "blind spots" in the 3D model, especially on complex structures like ship superstructures. Their method suggests that by focusing on missing directions rather than just missing spots, the drone can fill in the gaps much faster. While the paper doesn't claim this is a solved problem for the real world yet, the simulations suggest that this "direction-aware" approach is a major step forward. It turns a drone from a passive camera into an active detective, constantly asking, "What haven't I seen yet, and from where should I look to find it?" This could eventually help in inspecting ships for damage or managing maritime traffic without needing humans to climb high, dangerous ladders.

Technical Summary: DA-NBV

Problem Statement

Accurate 3D reconstruction of ships at sea is critical for maritime supervision, damage assessment, and autonomous operations. However, current data acquisition relies heavily on manual trajectories or skilled operators, leading to high costs and limited scalability. While Next-Best-View (NBV) planning automates viewpoint selection, existing policies suffer from two primary limitations in maritime contexts:

Lack of Directional Awareness: Conventional occupancy-based methods track whether a voxel has been observed but ignore the diversity of viewing directions. For ships with complex superstructures and severe self-occlusions, a single observation is often insufficient; repeated views from similar angles yield redundant data and incomplete reconstructions.
Dynamic Environmental Challenges: Maritime environments introduce continuous ship motion (heave, roll, pitch) due to waves, along with wind disturbances and limited onboard power. These factors alter surface visibility and pose strict requirements for scanning efficiency and trajectory robustness.

Existing methods often fail to distinguish between redundant observations and complementary views from new directions, leading to inefficient trajectories and persistent blind spots in dynamic scenes.

Methodology

The authors propose DA-NBV, a direction-aware NBV policy formulated as a Markov Decision Process (MDP). The framework integrates three coupled components to address the limitations of occupancy-only representations:

1. Direction-Aware Reconstruction State

Instead of simple occupancy grids, DA-NBV maintains a state representation $s_t$ that includes:

Occupancy Grid ( $G_t$ ): Encodes occupied, free, and unknown space.
Directional Grid ( $D_t$ ): Accumulates unit viewing-direction vectors for each voxel. The unit sphere is discretized into directional bins to record which directions have been observed. This distinguishes between redundant views and complementary views from unexplored directions.
Geometric Complexity ( $C_t$ ): An online estimate of local geometric complexity (linearity, scattering, curvature) derived from accumulated depth observations to prioritize complex regions.
Motion Compensation: To handle ship motion, the system maintains a unified ship-centered map. Iterative Closest Point (ICP) estimates the relative transformation between the current point cloud and the historical map, fusing aligned observations to ensure spatial consistency.

2. Learnable Position Advantage Field (PAF)

The PAF guides viewpoint selection by evaluating candidate positions based on:

Directional Incompleteness: It calculates the alignment between a candidate's viewing direction and the unobserved directional bins of visible voxels.
Visibility: It estimates the transmittance of rays to candidate positions using an axis-aligned bounding box approximation of accumulated point clouds, avoiding expensive ray tracing.
Geometric Importance: It weights candidates based on the online geometric complexity of the target voxels.
A Learnable Position Advantage Scorer (LPAS) aggregates these utilities (distance, visibility, missing-direction alignment, and complexity) to generate a structured 3D tensor ( $U_t$ ) that provides state-dependent guidance for the policy.

3. Locally Constrained Autoregressive Action Policy

To improve path efficiency and motion continuity, the action space is defined as a locally constrained discrete tuple: position displacement ( $\Delta x, \Delta y, \Delta z$ ), orientation ( $\psi, \phi$ ), and a binary termination decision.

Autoregressive Structure: The policy first predicts the relative position and termination decision. Once the position is determined, the orientation is predicted conditioned on the shared state and the selected position.
Benefits: This factorization reduces the single-step search space, discourages large jumps and backtracking, and enables progressive exploration of the workspace through successive local movements.

4. Reward Design

The policy is trained using Proximal Policy Optimization (PPO) with a reward function designed to prioritize geometrically complex regions and difficult-to-observe directions:

Complexity-Weighted Directional Coverage: The primary reward is based on the gain in directional coverage, weighted by geometric complexity.
Nonlinear Shaping: A monotonically increasing convex function is applied to the directional coverage increment. This strengthens the learning signal in later scanning stages, encouraging the agent to seek difficult-to-observe directions even when overall coverage is high.
Efficiency Penalties: The reward includes penalties for path length, step costs, and collisions, while rewarding proper termination based on final reconstruction quality.

Key Contributions

Direction-Aware State Representation: The introduction of a voxel-level directional observation history and a Position Advantage Field (PAF) that explicitly models missing viewing directions, addressing the "redundant observation" problem in complex structures.
Locally Constrained Autoregressive Policy: A novel action space design that decouples position and orientation prediction, enabling efficient local search and smoother trajectories suitable for UAVs in dynamic environments.
Nonlinear Coverage-Shaping Reward: A reward mechanism that dynamically adjusts incentives to prioritize difficult regions and directions as scanning progresses.
SeaShip-3D Dataset and Simulation: The construction of a ship-oriented dataset with 300 diverse models and a physics-based maritime simulation environment (NVIDIA Isaac Gym) that incorporates wave-induced motion (heave, roll, pitch) and wind disturbances.

Experimental Results

Experiments were conducted on the SeaShip-3D dataset under varying sea states (calm to severe) and compared against baselines including Random, ActiveRMAP, Scan-RL, GenNBV, and Hestia.

Reconstruction Quality: DA-NBV achieved a 98.49% coverage rate (CR) and reduced the Chamfer distance (CD) to 3.68 cm on SeaShip-3D. Compared to the best baseline (Hestia), this represents an improvement of approximately 3 percentage points in completeness and a 43% reduction in Chamfer distance.
Path Efficiency: DA-NBV demonstrated higher path efficiency, achieving a coverage-path-length area under the curve ( $A_p$ ) of 94.35%, significantly outperforming baselines.
Robustness: Under severe maritime conditions (Sea State 9), DA-NBV showed minimal performance degradation (CR decreased by only 0.89 percentage points, CD increased by 0.88 cm), outperforming the strongest baselines (Hestia and GenNBV) in stability.
Generalization: The method maintained competitive performance on static datasets (Houses3K and OmniObject3D), indicating generalizability beyond maritime scenarios.
Computational Efficiency: The decision-making pipeline achieved an average latency of 8.697 ms, demonstrating real-time feasibility on standard hardware (NVIDIA RTX 4090).

Significance

The paper claims that DA-NBV addresses a critical gap in 3D reconstruction for maritime applications by moving beyond simple spatial occupancy to model directional observation history. By explicitly accounting for the diversity of viewing angles required for complex superstructures and integrating motion compensation, DA-NBV enables efficient, high-quality reconstruction of ships at sea. The proposed framework offers a robust solution for autonomous UAV operations in dynamic environments, reducing reliance on manual intervention and improving the geometric fidelity of digital twins for vessel maintenance and management.