🤖 AI

DarkVesselNet: Multi-Modal Remote Sensing and Trajectory Reasoning for Dark Vessel Detection

This paper introduces DarkVesselNet, a multi-modal remote sensing system that fuses Sentinel-1 SAR, Sentinel-2 optical imagery, and AIS trajectory data using geospatial foundation models and anomaly detection techniques to identify dark vessels, with its implementation and validation provided as an open-source Python package and Hugging Face Space.

Original authors: Arun Sharma

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Arun Sharma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Silent Ship" Detective

Imagine the ocean is a giant, busy highway. Most cars (ships) have a GPS tracker that broadcasts their location to everyone. But some "dark vehicles" turn off their GPS, drive in the fog, or pretend to be somewhere else. Finding these silent ships is hard because you can't just ask them where they are.

DarkVesselNet is a new digital detective system designed to find these silent ships. Instead of relying on just one clue, it acts like a team of investigators, each with a different superpower, working together to spot something suspicious.

The Three Investigators (The Sensors)

The system combines three different ways of looking at the ocean, much like a detective using a flashlight, a map, and a witness statement:

The Night-Vision Goggles (SAR):
- What it is: Radar satellites (Sentinel-1) that can "see" through clouds and darkness.
- The Analogy: Imagine a security guard with night-vision goggles. Even if it's pitch black or storming outside, this guard can spot a shiny metal object (a ship) on the water.
- The Catch: Sometimes the goggles get confused by waves, rocks, or docks, thinking they are ships.
The Human Eye (Optical Imagery):
- What it is: Standard satellite photos (Sentinel-2) that look like regular pictures.
- The Analogy: This is like a human looking out a window on a sunny day. You can clearly see the shape of the ship and what it looks like.
- The Catch: If it's night or cloudy, the human eye is useless.
The Witness Log (AIS Tracks):
- What it is: The digital log of where ships say they are.
- The Analogy: Imagine a guest list at a party. If a guest says, "I'm in the kitchen," but the night-vision guard sees a person in the garden, the system knows something is wrong. If a guest is missing from the list entirely, that's also a red flag.

How the Detective Works (The "Fusion" Engine)

The paper describes DarkVesselNet not as a single camera, but as a "fusion" system. It takes the clues from the Night-Vision, the Human Eye, and the Guest List and mixes them together.

The "Missingness" Trick: The system is smart about what is missing. If the "Human Eye" is covered by clouds, the system doesn't panic; it just knows, "Okay, we don't have a photo right now, but we still have the radar and the guest list." It uses a special "mask" to remember which clues are available and which are missing, so it doesn't get confused.
The "Reasoning" Step: The system doesn't just look at one picture. It looks at the history. Did a ship suddenly vanish from the guest list? Did two ships meet in the middle of the ocean (a "rendezvous")? This is like checking if a guest left the party early or if two guests met in a back alley.

The "Black Box" vs. The "Notebook"

Many AI systems are "black boxes"—they give you a score (e.g., "85% chance this is a bad ship") but don't tell you why.

DarkVesselNet is different. It produces a Reasoning Trace.

The Analogy: Instead of just handing you a verdict, the detective hands you a notebook.
The Notebook Says: "I flagged this ship because:
1. The Radar saw a bright object (Check).
2. The Guest List says no one is there (Check).
3. The History shows the ship stopped moving for 2 hours (Check).
4. We couldn't get a photo because of clouds (Note)."

This allows a human analyst to read the notes and decide if it's actually a problem, rather than just trusting a computer's guess.

What the Paper Actually Claims (The "Test Drive")

The paper is very careful not to overpromise. It is not a live system that is currently arresting pirates or stopping illegal fishing.

It's a Prototype: Think of it as a fully built car engine that has been tested in a garage. The author has written code that proves all the parts work together.
The Tests: The paper lists 15 specific "smoke tests" that passed. These tests checked things like:
- Does the radar filter out static noise correctly?
- Does the math for measuring distance on a round Earth work?
- Can the system handle missing data without crashing?
The Limitation: The paper admits it hasn't been tested on the massive, real-world "xView3" dataset yet to prove it can find real illegal ships better than existing methods. It is currently a "proof of concept" that the architecture works.

The Golden Rule: Evidence, Not Guilt

The most important takeaway from the paper is a distinction in language:

What the system does: It finds "unmatched evidence." It says, "Hey, I see a ship here, but the guest list doesn't match."
What the system does NOT do: It does not say, "This ship is illegal."

The Analogy: The system is a triage nurse, not a judge. It sorts patients (ships) into a "needs to be looked at" pile based on symptoms (missing GPS, weird movement). It does not decide if the patient is guilty of a crime. That decision requires a human to look at the evidence trace and make a judgment.

Summary

DarkVesselNet is a new software framework that combines radar, photos, and ship logs to find silent vessels. It is designed to be transparent, showing its work in a "notebook" so humans can verify the clues. Currently, the code works and passes its internal tests, but it is a research tool waiting for real-world data to prove it can catch the bad guys.

Technical Summary: DarkVesselNet

Problem Definition
Dark vessel detection addresses the challenge of identifying maritime vessels that cease broadcasting Automatic Identification System (AIS) signals, spoof their locations, or operate in regions with unreliable self-reported data. The core difficulty lies in the "disagreement problem": a system must reconcile cooperative self-reports (AIS) with physical observations (Synthetic Aperture Radar [SAR] and optical imagery) and temporal context (trajectory history). Each modality has distinct limitations: SAR provides all-weather observation but suffers from speckle and coastal clutter; optical imagery offers interpretable visual context but is unavailable at night or under clouds; and AIS is semantically rich but incomplete and cooperative. The paper frames the task not as a single image classification problem, but as an evidence-fusion problem under missingness, where the goal is to produce a triage-ready alert with a traceable reasoning path rather than a black-box probability of "illegal activity."

Methodology and Architecture
DarkVesselNet is a modular, multi-modal remote sensing stack designed to ingest, encode, and fuse heterogeneous evidence streams. The pipeline consists of four primary stages:

Sensor Preprocessing:
- SAR: Implements Lee filtering for speckle reduction and local statistics estimation.
- Optical: Applies cloud masking and calculates interpretable band ratios (NDVI, NDWI) to distinguish water, land, and vessel-like structures.
- Coregistration: Handles the misalignment between SAR and optical grids caused by incidence angles and terrain, utilizing identity or implemented coregistration paths.
Geospatial Foundation Backbone Adapter:
- The system introduces a GeoBackbone adapter that standardizes outputs from various Earth-observation foundation models (including Prithvi-2, Clay, SatMAE++, DOFA, SatlasNet, and RemoteCLIP-style encoders).
- This adapter converts diverse model outputs into a common token interface $(B, N, D)$ , allowing the downstream fusion head to remain independent of the specific encoder used. It supports lightweight fallback projections for testing without downloading full model weights.
Trajectory Reasoning:
- AIS Processing: AIS tracks are represented as sequences of time, position, speed, and course.
- Gap Detection: A TGARD-style component identifies long or infeasible gaps in AIS transmission by calculating geodesic distances (Haversine) and checking required movement velocities.
- Anomaly Scoring: A Pi-DPM-inspired anomaly head takes scene tokens and AIS segments. It pools scene features and processes AIS points through an MLP to predict both a logit score and a reconstructed AIS segment. This lightweight head is designed to be extensible to full diffusion samplers.
Fusion and Alert Decoding:
- The fusion architecture preserves modality-specific uncertainty using availability masks ( $m_s, m_o, m_a$ ) to prevent the model from confusing missing data (e.g., cloudy optical imagery) with empty scenes.
- The decoder produces a calibrated alert probability and, crucially, an evidence trace. This trace explicitly exposes sensor availability, AIS matching status, anomaly evidence, and uncertainty, allowing human analysts to audit the decision.

Key Contributions
The paper outlines four primary contributions:

Unified Architecture: A reference architecture spanning SAR, optical, AIS, and geospatial foundation-model tokens, treating dark-vessel detection as a multi-modal evidence fusion problem.
Common Backbone Adapter: A GeoBackbone interface enabling the swapping of foundation models (Prithvi, Clay, etc.) without rewriting downstream fusion logic.
Trajectory Reasoning Path: A combination of TGARD-style gap detection and a Pi-DPM-inspired reconstruction head to score temporal anomalies and physical plausibility.
Reproducible Implementation: A tested Python package and a public Hugging Face Space that validates core operators (SAR filtering, band ratios, Haversine distance, tensor shapes) and demonstrates a CPU-safe workflow.

Results and Validation
The paper explicitly states that current results are software-grounded rather than benchmark-driven.

Implementation Validation: The repository includes a test suite where 15 tests pass. These verify the correctness of SAR speckle filtering, optical band-ratio utilities, Haversine distance calculations, gap emission logic, sensor coregistration, backbone token shapes, and the backward pass of the anomaly head.
Benchmark Status: The paper does not claim xView3 leaderboard performance or live data ingestion. Instead, it proposes a rigorous evaluation protocol (ablation studies, stress tests, calibration plots) as the path forward.
Expected Patterns: Theoretical ablation tables suggest that while SAR-only detection may yield high recall, the addition of AIS matching and trajectory evidence is expected to significantly improve alert precision and reduce false positives (particularly near shore), rather than just increasing raw object recall.

Significance and Claims
The paper positions DarkVesselNet as a triage and analyst support system, not an autonomous enforcement tool.

Evidence vs. Intent: The system is designed to detect "unmatched evidence" (e.g., a SAR object without a plausible AIS track) rather than infer legal status or intent. It distinguishes between a "SAR object," an "AIS-unmatched vessel," and "confirmed illegal activity," ensuring the model does not overclaim.
Traceability: A core claim is that the system's value lies in its evidence trace. By decomposing the alert score into contributions from SAR, AIS, trajectory, and context, the system allows analysts to understand why an alert was generated (e.g., driven by a SAR detection with no AIS match vs. a suspicious rendezvous pattern).
Modesty: The authors emphasize that the current artifact is a "portfolio-scale implementation" with tested operators. The paper avoids claiming automatic attribution of illegal fishing, instead framing the output as a prioritized review queue with calibrated uncertainty and explicit missingness handling.

In summary, DarkVesselNet provides a structured, auditable framework for fusing remote sensing and AIS data, prioritizing interpretability and uncertainty quantification over raw detection scores, with a current focus on validating the software infrastructure required for such a system.