Generating Realistic, Protocol-Compliant Maritime Radio Dialogues using Self-Instruct and Low-Rank Adaptation

This paper addresses the scarcity of high-quality maritime radio data by introducing a compliance-aware Self-Instruct framework enhanced with LoRA fine-tuning and a 26-filter verification pipeline to generate realistic, SMCP-compliant VHF dialogues for AI-assisted safety systems.

The Gist

Imagine the ocean is a giant, chaotic office building where thousands of ships are constantly talking to each other and to the shore using walkie-talkies. This is the world of maritime radio.

The problem? It's a noisy, stressful place. Sometimes the signal is fuzzy, sometimes people speak different languages or accents, and sometimes, in a panic, a captain might forget the exact "script" they are supposed to use. In this high-stakes environment, a misunderstanding isn't just an annoyance; it can lead to collisions, fires, or even sinking ships.

The authors of this paper wanted to build a smart AI assistant to help captains communicate clearly and follow the rules. But to teach an AI how to do this, you need a massive library of "perfect" examples of radio calls. The catch? Real radio recordings are secret, private, and hard to get. You can't just ask a captain for a recording of their emergency call.

So, the researchers decided to build a "fake" library that looks and feels exactly like the real thing. Here is how they did it, explained through a simple story:

1. The "Robot Apprentice" (The Base AI)

They started with a smart robot brain (a Large Language Model called Llama 3.1). Think of this robot as a very talented actor who has read every book in the world but has never been on a ship.

• The Problem: If you ask this actor to play a captain in distress, they might say, "Help me, I'm sinking!" which sounds dramatic but breaks the strict rules of maritime radio. They might invent fake ship names or forget to say "Mayday" three times.
• The Goal: We need to train this actor to be a realistic maritime professional who never breaks the rules.

2. The "Scriptwriter" (Self-Instruct)

Instead of hiring a human to write thousands of scripts (which is impossible due to privacy), they taught the robot to write its own scripts.

• They gave the robot a few "seed" examples (like a starter pack of 100 perfect scripts).
• Then, they told the robot: "Here are some examples. Now, imagine a new emergency (like a fire or a collision), invent a ship name, pick a location, and write a new script."
• The robot generated thousands of new radio calls.

3. The "Strict Inspector" (The 26-Filter Pipeline)

This is the most important part. The robot is creative, but it's also prone to lying (hallucinating) and making mistakes.

• Imagine a 26-point security checkpoint at an airport. Every single script the robot wrote had to pass through this checkpoint.
• The Filters checked things like:
  • Did you say "Mayday" three times? (If no -> Reject)
  • Did you invent a fake ship ID number? (If yes -> Reject)
  • Is the ship actually on the ocean, or did you put it in a desert? (If desert -> Reject)
  • Did you repeat the same sentence 50 times? (If yes -> Reject)
• Only the scripts that passed all 26 checks were kept. The bad ones were thrown in the trash. This ensured the "fake" library was 100% compliant with international safety rules.

4. The "Specialized Training" (LoRA)

Now they had a perfect library of fake-but-realistic radio calls. They used this library to give the robot a specialized "boot camp."

• Instead of retraining the whole robot (which would take a supercomputer and years), they used a technique called LoRA (Low-Rank Adaptation).
• The Analogy: Imagine the robot is a general doctor. You don't need to retrain them to become a heart surgeon from scratch. You just give them a specialized pocket guide (the LoRA adapter) that teaches them the specific rules of maritime emergencies.
• This was fast, cheap, and efficient.

5. The Result: A "Digital Twin" of Reality

After training, they tested the robot.

• Before training: The robot sounded like a confused tourist. It failed almost every time.
• After training: The robot sounded like a seasoned captain. It followed the rules, used the right jargon, and created unique, logical stories about ship emergencies.

Why Does This Matter?

This "fake" library is now being released to the public. It allows scientists to build better Speech-to-Text systems that can understand noisy radio calls, and Decision Support systems that can warn captains if they are about to say something dangerous.

In short: The researchers built a "fake reality" so perfect that it can train AI to save lives in the real world, all without needing to steal any private data from real ships. They turned a robot actor into a maritime safety expert using a strict inspector and a specialized pocket guide.

Technical Summary

Here is a detailed technical summary of the paper "Generating Realistic, Protocol-Compliant Maritime Radio Dialogues using Self-Instruct and Low-Rank Adaptation."

1. Problem Statement

The maritime industry faces significant safety risks due to miscommunication over Very High Frequency (VHF) radio, with human factors accounting for over 58% of recorded incidents in Europe (2014–2023). While Artificial Intelligence (AI) systems (e.g., Automatic Speech Recognition and procedural guidance) could mitigate these risks, their development is hindered by a severe data scarcity problem.

• Data Constraints: Real-world maritime radio datasets are scarce due to operational sensitivity, regulatory privacy constraints, and the episodic nature of voyages.
• Limitations of Existing Synthetic Data:
  • Rule-based templates: Lack contextual fluency and semantic diversity.
  • General-purpose Large Language Models (LLMs): Often fail to adhere to strict regulatory standards (International Maritime Organization's Standard Marine Communication Phrases - SMCP) and frequently hallucinate operational details (e.g., fake vessel names, coordinates, or MMSI numbers).
• The Core Challenge: How to generate high-quality, procedurally compliant, and operationally realistic synthetic maritime dialogue data without access to large volumes of real-world labeled data.

2. Methodology

The authors propose a Compliance-Aware Self-Instruct methodology integrated with Low-Rank Adaptation (LoRA) to generate and fine-tune models for maritime distress calls.

A. Data Sources and Preparation

The study utilizes four public datasets to ensure entity realism:

• GSHHG & GeoNames: For shoreline and geographical entity data (names, coordinates, countries).
• Marine Cadastre & Danish Maritime Authority AIS: For real vessel data (names, types, MMSI, call signs).
• Preprocessing: The authors cleaned and merged AIS data to create a vessel dataset of ~58,855 unique vessels, removing over-represented categories (e.g., pleasure craft) and standardizing vessel types.

B. Compliance-Aware Self-Instruct Pipeline

The authors adapted the Self-Instruct framework (Wang et al.) to the maritime domain through an iterative bootstrapping process:

1. Seed Generation: 100 seed instances were manually created covering 10 distress categories (e.g., Fire, Collision, Sinking) based on SMCP guidelines.
2. Context Generation: Real-world context (coordinates, vessel names, nearby ports) is dynamically generated from the public datasets.
3. Iterative Generation: The LLM (Llama 3.1 8B) is prompted to generate new distress call dialogues using 5 random examples from the current instance pool and the generated context.
4. 26-Filter Verification Pipeline: This is the core innovation. Before a generated instance is added to the training pool, it passes through a rigorous 26-filter pipeline:
   • Entity Accuracy: Verifies vessel names, MMSI, call signs, and coordinates against the context.
   • Hallucination Detection: Rejects instances where the model invents MMSI/call signs when the context is null.
   • SMCP Compliance: Checks for correct structure (e.g., starting with "Mayday"), sequencing, and keyword usage specific to the distress category.
   • Logical Consistency: Ensures turn-taking between the vessel and Coast Guard and eliminates repetitive sentences.
   • Uniqueness: Uses Rouge-L scoring (threshold < 0.7) to prevent near-duplicate generation.
   • Result: Only instances passing all 26 filters are retained for training.

C. Parameter-Efficient Fine-Tuning (LoRA)

• Model: Llama 3.1 8B.
• Technique: Low-Rank Adaptation (LoRA) is used to fine-tune the model on the filtered synthetic dataset.
• Efficiency: Instead of updating all 8 billion parameters, only ~671 million parameters (8.35%) are trained. This allows for deployment on resource-constrained maritime systems.
• Training: Separate LoRA adapters are trained for each of the 10 distress categories.

D. Evaluation Framework

A four-metric framework assesses the quality of the generated data:

1. Format Accuracy: Weighted score based on structural filters (e.g., presence of "Mayday", correct punctuation).
2. Information Accuracy: Weighted score based on factual consistency (e.g., correct MMSI, coordinates).
3. Uniqueness: Measured via Rouge-L similarity against seed and training data.
4. Logical Coherence: Manually rated by maritime domain experts (1–5 scale) on a subset of dialogues to assess narrative flow and realism.

3. Key Results

The study compared the Vanilla LLM (base model) against the LoRA-adapted models across 100 generated dialogues per category.

• Vanilla LLM Performance:
  • 0% Valid Chatters: The base model failed to generate any dialogue that passed all 26 filters when prompted with context only.
  • Low Scores: Average Logical Coherence was ~0.1; Format Accuracy was ~0.61.
  • Failure Modes: Repetitive loops, missing critical SMCP phrases, and hallucinated entities.
• LoRA Adapter Performance:
  • High Success Rate: 87% of generated dialogues passed all filters.
  • Metric Improvements:
    • Format Accuracy: Increased from 0.613 to 0.986.
    • Information Accuracy: Increased from 0.740 to 0.947.
    • Logical Coherence: Increased from 0.104 to 0.83 (a 698% improvement).
  • Efficiency: Achieved these results by training on only 500 valid samples per category (derived from 10 seeds) and updating only ~8% of model parameters.
• Uniqueness: The generated dialogues showed low Rouge-L similarity to seed instances and training data, confirming high diversity.

4. Key Contributions

1. Compliance-Aware Self-Instruct: A novel methodology that embeds regulatory verification (26 filters) directly into the iterative generation loop, ensuring compliance is an intrinsic property of the model rather than a post-hoc filter.
2. High-Quality Synthetic Dataset: A publicly released dataset of procedurally compliant, logically coherent maritime distress calls with real-world vessel identifiers and geographical data.
3. Evaluation Framework: A robust, four-metric system (Format, Information, Uniqueness, Coherence) combining automated scoring with expert human judgment.
4. Resource Efficiency: Demonstrated that combining Self-Instruct with LoRA allows for high-quality domain adaptation with minimal manual annotation and low computational cost, making it viable for safety-critical edge deployment.

5. Significance and Future Work

• Safety Impact: This work provides the foundational data required to train robust ASR and decision-support systems for maritime safety, directly addressing the human-factor root cause of many accidents.
• Scalability: The approach offers a sustainable alternative to collecting expensive, privacy-restricted real-world data.
• Transferability: The compliance-aware framework is applicable to other safety-critical domains with strict regulatory standards, such as aviation and industrial control.
• Future Directions:
  • Expanding to other communication types (ship-to-ship, pilotage).
  • Converting text to synthetic speech (TTS) to create multimodal datasets for ASR training.
  • Developing hybrid models combining real and synthetic data for improved robustness.

The authors have open-sourced the code, datasets, and verification tools to ensure reproducibility and facilitate further research in AI-assisted maritime safety.