A Composite Evaluation Framework for Unsupervised Maritime Anomaly Detection in AIS Data with Applications to Ocean Monitoring

Technical Summary: A Composite Evaluation Framework for Unsupervised Maritime Anomaly Detection in AIS Data

Problem Statement
Maritime transportation accounts for over 80% of global trade, necessitating robust monitoring for safety, security, and environmental protection. While the Automatic Identification System (AIS) provides high-frequency spatio-temporal data for vessel tracking, identifying anomalous vessel behavior remains a significant challenge. The primary obstacle is the lack of labelled anomaly data, which renders supervised learning approaches impractical and limits the ability to objectively evaluate unsupervised models. Existing unsupervised techniques, such as Isolation Forest, are widely used for their efficiency but suffer from a critical gap: the absence of a standardized, interpretable, and quantitative benchmark to assess their performance. Current evaluation methods often rely on subjective visual inspection or expert judgment, which are difficult to reproduce and compare across different models.

Methodology
The study proposes a comprehensive framework centered on the Maritime Anomaly Detection Quality Index (MADQI), a composite metric designed to evaluate unsupervised anomaly detection without ground truth labels. The methodology consists of three main pillars:

1. Feature Engineering and Anomaly Taxonomy:
   The framework processes raw AIS data (latitude, longitude, Speed Over Ground [SOG], Course Over Ground [COG], and timestamps) by grouping observations by vessel identifier (MMSI) and sorting them chronologically. It derives a multi-dimensional feature set including:

   • Spatial: Haversine distance between consecutive positions to detect unrealistic spatial jumps.
   • Temporal: Time differences between messages to identify transmission gaps.
   • Kinematic: Implied speed (derived from distance/time) compared against reported SOG.
   • Behavioral: Speed differences and Turn Rates (TR) to capture abrupt maneuvers.

   Based on these features, the study defines five specific anomaly types: Speed Anomaly, Position Jump Anomaly, Temporal Gap Anomaly, Turn Rate Anomaly, and Composite Behavioural Anomaly.

2. Unsupervised Detection Model:
   The Isolation Forest algorithm is employed as the core detection engine. It operates by recursively partitioning the feature space to isolate anomalies, which typically require fewer splits than normal observations. The model outputs an anomaly score for each observation, where lower scores indicate higher degrees of abnormality.

3. The MADQI Evaluation Framework:
   To address the lack of evaluation metrics, the authors introduce MADQI, a weighted composite index comprising four normalized components:

   • Anomaly Rate Consistency (ARC): Measures the alignment between the observed contamination level and the expected contamination level set in the model.
   • Physical Plausibility Score (PPS): Evaluates whether detected anomalies exhibit meaningful physical deviations (in speed, distance, and time) compared to normal behavior.
   • Score Distribution Separation (SDS): Quantifies the statistical separation between the anomaly scores of normal and anomalous observations.
   • Extreme Case Evidence (ECE): Assesses the presence of extreme behavioral deviations, particularly in spatial displacement.

   Normalization Strategy: A novel multi-chunk evaluation strategy is used to ensure robustness. The dataset is partitioned into non-overlapping subsets to estimate empirical distributions. Scaling parameters are derived automatically using the lower quartile (25th percentile) of these distributions. Finally, an exponential saturation function is applied to normalize components to a [0, 1] scale, avoiding the centralization bias common in soft rational normalization. The final MADQI score is the arithmetic mean of these four normalized components.

Key Contributions
The paper outlines four primary contributions:

1. Unified Unsupervised Approach: A robust framework utilizing Isolation Forest with spatial (Haversine) and behavioral features to detect anomalies in unlabelled AIS data.
2. Structured Anomaly Taxonomy: A classification system categorizing anomalies into five distinct types based on spatial and behavioral dynamics.
3. MADQI Metric: A novel, domain-specific, model-agnostic evaluation index that provides a numeric, scalable, and interpretable measure for unsupervised anomaly detection, solving the problem of missing benchmark metrics.
4. Practical Deployment Tool: An implementation that integrates geospatial visualization (using the Folium library) to project detected anomalies onto interactive maps, bridging the gap between detection and actionable intelligence.

Experimental Results
The framework was tested on a dataset of 1,048,576 AIS observations from the United States on March 31, 2022. Using a subset of 500,000 observations for testing with an Isolation Forest configured with 100 trees and a contamination rate of 0.001:

• The model identified 498 anomalies (approx. 0.1% of the data) and 10 extreme anomalies after secondary filtering.
• The MADQI score achieved was 80.37%.
• Component analysis revealed the model performed strongest in Extreme Case Evidence (ECE) (normalized score of 1.000) and Anomaly Rate Consistency (ARC) (0.907), with moderate performance in Physical Plausibility (PPS) and Score Distribution Separation (SDS).
• The multi-chunk evaluation demonstrated that single-segment testing could lead to biased results, validating the need for the proposed robust estimation strategy.

Significance and Claims
The authors claim that this work establishes a robust and domain-specific standard for evaluating unsupervised AIS-based anomaly detection. By enhancing the transparency, interpretability, and reliability of detection models, the MADQI framework enables more trustworthy maritime monitoring. The study posits that this contribution provides a practical foundation for data-driven ocean governance, supporting efforts to combat illegal activities, improve navigational safety, and protect marine ecosystems in alignment with UN Sustainable Development Goal 14. The framework is presented as a step toward moving from subjective analysis to objective, quantitative assessment in maritime security.