SeaAlert: Critical Information Extraction From Maritime Distress Communications with Large Language Models

1. Problem Statement

Maritime distress communications via Very High Frequency (VHF) radio are critical for search-and-rescue operations under the Global Maritime Distress and Safety System (GMDSS). However, automatic analysis of these messages faces significant challenges:

• Data Scarcity: Real-world labeled distress data is rare due to privacy and safety concerns.
• Environmental Degradation: Messages are often brief, noisy, and spoken under stress, leading to high error rates in Automatic Speech Recognition (ASR).
• Protocol Variability: Distress messages may deviate from standardized formats, omit codewords (e.g., "MAYDAY"), or contain ambiguous phrasing.
• Limitations of Current Methods: Traditional Keyword Spotting (KWS) and Bag-of-Words (BoW) approaches often fail when codewords are missing, distorted by noise, or when context is required to distinguish drills from real emergencies.

The core problem is the lack of robust systems capable of accurately classifying distress severity and extracting structured operational data (e.g., position, vessel ID) from noisy, non-standardized maritime transcripts.

2. Methodology

The authors propose SeaAlert, an end-to-end framework that integrates synthetic data generation, noise simulation, and comparative model evaluation.

A. Synthetic Data Generation Pipeline

To overcome data scarcity, the authors developed a pipeline using GPT-4 to generate 1,872 synthetic maritime messages.

• Diversity: Messages cover four severity levels (Distress, Urgency, Safety, Routine), three communication styles (formal, informal, third-party), and 12 scenario types.
• Metadata: Each message includes structured fields (Vessel Name, Call Sign, MMSI, Position, POB, Nature of Incident) serving as ground truth.
• Adversarial Variants: The dataset includes "masked" versions where standard GMDSS codewords are replaced with placeholders to test model reliance on lexical anchors.

B. Audio Simulation and ASR Processing

To simulate real-world VHF conditions:

1. Text-to-Speech: Clean text is converted to audio using Coqui TTS (VITS) with 109 diverse speakers.
2. Noise Injection: Audio is degraded with simulated VHF channel noise at two levels: Medium (12dB SNR) and High (6dB SNR).
3. ASR Transcription: The noisy audio is transcribed using Whisper ASR, introducing realistic recognition errors (Word Error Rates of ~29.6% at Medium and ~36.2% at High noise).

C. Model Architectures and Evaluation

The study compares two modeling paradigms:

1. Classical Baseline: Bag-of-Words (BoW) features with Logistic Regression.
2. Transformer Model: RoBERTa-base.

Experimental Design:

• Baseline & ASR Robustness: Testing performance on clean text vs. noisy ASR transcripts.
• Keyword Ablation: Evaluating performance when GMDSS codewords are masked during training and/or testing.
• Adversarial Testing: Assessing robustness against negations, false alarms, drills, and ambiguous phrasing.
• Information Extraction: Comparing Regex-based extraction vs. GPT-4 (Zero-shot) for extracting structured fields from noisy transcripts.

3. Key Contributions

1. End-to-End Evaluation Framework: A novel pipeline combining synthetic message generation, realistic VHF noise simulation, ASR transcription, severity classification, and structured extraction.
2. Robustness Analysis: A comprehensive comparison showing that while BoW and Transformers perform similarly on clean text, RoBERTa is substantially more robust under ASR corruption and adversarial conditions.
3. Codeword Dependence Study: Evidence that while both models rely on explicit codewords, RoBERTa generalizes better when these keywords are omitted or obscured, whereas BoW models fail significantly.
4. Extraction Superiority: Demonstration that GPT-4 outperforms Regex for structured information extraction from noisy ASR transcripts, particularly for alphanumeric fields (MMSI, Call Sign) and semantically variable fields.
5. Open Resources: Public release of data, code, and implementation resources to support reproducibility.

4. Key Results

Classification Performance

• Clean Text: RoBERTa (Macro-F1: 0.679) and Logistic Regression (Macro-F1: 0.674) perform comparably.
• ASR Noise (High): RoBERTa maintains a Macro-F1 of 0.608 (only 10% drop), whereas Logistic Regression drops to 0.423 (37% drop).
• Class Sensitivity: "Distress" is the most reliably recognized class. "Safety" and "Urgency" are the most challenging; BoW models suffer severe degradation in the "Safety" class under noise, while RoBERTa remains stable.
• Adversarial Traps: Both models struggle with "Drill" messages (0% accuracy), highlighting a critical safety gap. However, RoBERTa outperforms BoW in "Ambiguous" and "Real_no_codeword" scenarios.

Information Extraction

• Regex vs. LLM: Regex-based extraction degrades sharply under noise, failing almost completely on alphanumeric fields like MMSI and Call Sign due to ASR transcription errors.
• GPT-4 Performance: GPT-4 significantly outperforms Regex across all fields in high-noise conditions, successfully recovering critical data (Position, POB, Incident) even when surface forms are corrupted.

5. Significance and Implications

• Safety-Critical Resilience: The study proves that transformer-based models are essential for maritime safety systems because they can handle the "graceful degradation" required when channel quality is poor.
• Beyond Keywords: The results indicate that relying solely on keyword spotting is insufficient; systems must understand context to handle non-standard phrasing and missing codewords.
• Operational Reality: The inability of current models to distinguish "Drills" from real emergencies suggests that human oversight remains essential. These models should be deployed as decision-support tools rather than fully autonomous detectors.
• Future Architecture: The findings advocate for a hybrid approach: Transformer-based severity assessment combined with LLM-based structured extraction, rather than traditional rule-based parsing, to ensure reliable data recovery in noisy environments.

In conclusion, SeaAlert demonstrates that while no current model is perfect for adversarial maritime scenarios, Large Language Models (specifically RoBERTa for classification and GPT-4 for extraction) offer a significantly more robust foundation for next-generation maritime safety intelligence than traditional methods.