MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals

The paper introduces MERLIN, a novel training framework accompanied by the EM-100k dataset and EM-Bench benchmark, to overcome data scarcity, evaluation gaps, and low-SNR fragility in building robust Multimodal Large Language Models for electromagnetic signals.

The Gist

Imagine you have a super-smart robot (a Large Language Model, or LLM) that is amazing at reading books, writing stories, and chatting with people. But right now, this robot is completely deaf to the world of radio waves, radar, and Wi-Fi signals. It can't "hear" the electromagnetic (EM) signals that power our modern world.

The paper you shared, "MERLIN," is like a blueprint for teaching this robot to not only hear these invisible signals but to understand them, even when the signal is very weak and full of static (noise).

Here is the story of how they did it, broken down into three simple parts:

1. The Problem: The Robot is Deaf and the Signal is Faint

Currently, trying to make these robots understand radio signals is like trying to teach someone to read a book written in a language they don't know, while they are standing in a hurricane.

• No Dictionary: There are almost no books (datasets) that pair radio signals with human explanations. The robot has nothing to learn from.
• No Test: There is no standardized exam (benchmark) to see if the robot is actually getting smarter or just guessing.
• The Static: When the signal is weak (low "Signal-to-Noise Ratio"), it's like trying to hear a whisper in a rock concert. The robot gets confused and fails completely.

2. The Solution: Building the Library, the Exam, and the Training Camp

The team created three things to fix this:

A. The Library: EM-100k

They realized the robot needed a massive library to study. So, they built EM-100k, a dataset containing 100,000 pairs of radio signals and their descriptions.

• Analogy: Imagine they took 35 million raw radio "snippets" (like recording every possible sound a car engine makes) and hired experts to write down exactly what each sound means (e.g., "This is a radar pulse," "This is a jamming signal"). Now the robot has a massive textbook to study.

B. The Exam: EM-Bench

To know if the robot is actually learning, they built EM-Bench, a rigorous test.

• Analogy: Instead of just asking, "Do you know what a radio is?", they give the robot a multi-level exam.
  • Perception: "What kind of modulation is this?" (Like identifying the instrument playing a note).
  • Reasoning: "Someone is jamming this signal; what strategy should we use to fight back?" (Like a chess player figuring out a counter-move).
  • The exam covers everything from spotting simple signals to planning complex electronic warfare strategies.

C. The Training Camp: MERLIN

This is the most important part. They didn't just feed the robot more data; they invented a new way to train it called MERLIN.

The Two-Stage Training:

1. Stage 1 (The Basics): They teach the robot to match signals with text using the new library (EM-100k). It learns the "vocabulary" of radio waves.
2. Stage 2 (The Noise Challenge): This is the magic trick.
   • The Problem: When the signal is noisy (static), the robot's brain gets foggy. It can't tell the difference between a real signal and the noise.
   • The Fix: They use a technique called Knowledge Distillation.
   • The Analogy: Imagine a Teacher and a Student.
     • The Teacher is the robot looking at a perfect, crystal-clear signal.
     • The Student is the robot looking at the same signal but covered in static and noise.
     • The Teacher says, "Even though you see noise, I see a clear pattern. Here is what the pattern should look like."
     • The Student tries to mimic the Teacher's "clean" understanding, ignoring the noise.
   • The Secret Sauce: They added a special filter (called the Denoising Subspace Module) that acts like noise-canceling headphones for the robot's brain. It strips away the static before the robot tries to understand the signal, forcing it to learn the true shape of the message.

3. The Result: A Super-Listener

After this training, the MERLIN robot became a master of the electromagnetic world.

• It passed the exam: It scored higher than any other model (even huge, expensive ones from big tech companies) on the EM-Bench test.
• It's tough: Even when the signal is very weak and noisy (like trying to hear a whisper in a storm), MERLIN didn't crash. It kept working because it learned to ignore the static and focus on the core message.

Summary

In short, the paper says: "We built a giant library of radio signals, created a hard test to measure progress, and invented a special training method where a 'clean' teacher guides a 'noisy' student. This allows our AI to finally understand the invisible language of the electromagnetic spectrum, even when the connection is terrible."

This is a huge step forward for things like radar, secure communications, and autonomous vehicles that need to "hear" the world around them without getting confused by interference.

Technical Summary

Here is a detailed technical summary of the paper "MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals."

1. Problem Statement

The paper addresses the critical gap in applying Multimodal Large Language Models (MLLMs) to the Electromagnetic (EM) domain. While MLLMs have succeeded in vision and audio, their application to EM signals (e.g., radar, communications) faces three fundamental challenges:

• Data Scarcity: There is a severe lack of high-quality, large-scale datasets pairing raw EM signals (IQ data) with descriptive text annotations, which are essential for pre-training MLLMs.
• Lack of Standardized Benchmarks: Existing methods often use task-specific or pipelined architectures. There is no comprehensive benchmark to systematically evaluate and compare models across diverse EM tasks ranging from basic perception to complex reasoning.
• Low-SNR Fragility: Standard encoder-LLM architectures suffer catastrophic performance degradation in low Signal-to-Noise Ratio (SNR) environments (defined as SNR < 0 dB). The authors identify that noise corrupts low-level signal features, causing "feature collapse" where embeddings from different classes overlap, leading to semantic ambiguity and a loss of discriminative power.

2. Methodology

The authors propose a tripartite solution involving a new dataset, a new benchmark, and a novel training framework.

A. Data and Benchmark Construction

• EM-100K: A large-scale pre-training dataset containing 100,000 signal-text pairs. It is constructed from a foundational corpus of over 35 million IQ samples derived from:
  • Real-world collections (e.g., modulation types, protocols).
  • Professional simulations (covering radar jamming, communication jamming, anti-jamming strategies).
  • Open-source datasets.
  • Format: Instruction-tuning format (Signal + Question $\to$ Response) suitable for supervised fine-tuning.
• EM-Bench: The first comprehensive benchmark for EM MLLMs, featuring 4,200 expert-validated QA pairs. It evaluates models across two main capabilities:
  • Perception: 3 sub-categories (Signal Characterization, Jamming Identification, Fragment Detection) covering 14 fine-grained tasks (e.g., modulation classification, duty cycle estimation).
  • Reasoning: 1 sub-category (Strategy Generation) requiring open-ended generation of counter-measure strategies.

B. The MERLIN Framework

MERLIN (Multi-modal Electromagnetic Robust Learning) is a two-stage training framework designed to align signal representations with language while explicitly enhancing robustness to noise.

1. Stage 1: Foundational Pre-training

   • Architecture: A standard MLLM pipeline consisting of a Signal Encoder (pre-trained EMind), a Signal Projector (2-layer MLP), and a Large Language Model (Qwen3-4B).
   • Objective: Multi-task instruction tuning. The model learns to generate answers ($A$) given signals ($S$) and instructions ($Q$) using standard next-token prediction loss. This establishes the baseline cross-modal alignment.

2. Stage 2: Low-SNR Robustness Enhancement (Knowledge Distillation)

   • Motivation: Experiments showed that simply increasing low-SNR data volume does not solve the problem; the issue lies in feature collapse. Linear interpolation of noisy features toward clean features improved performance, suggesting a need for feature-level correction.
   • Mechanism: A Teacher-Student distillation framework.
     • Teacher: A frozen model initialized from Stage 1, processing High-SNR signals.
     • Student: A trainable model processing Low-SNR signals.
   • Loss Functions: The student is optimized using a composite loss:
     • Task Loss ($L_{task}$): Standard cross-entropy on the low-SNR input.
     • Feature-Level Distillation ($L_{feat}$): Aligns student embeddings with teacher embeddings. Crucially, it employs a Denoising Subspace Module (DSM). The DSM projects student embeddings into a signal subspace (filtering out noise-dominated components) before calculating the distance to the teacher's clean features.
     • Logit-Level Distillation ($L_{logit}$): Aligns the output probability distributions (logits) of the student and teacher LLMs using KL divergence.

3. Key Contributions

1. EM-100K Dataset: A massive, diverse dataset of 100k signal-text pairs covering 2 signal types, 14 modulations, and 8 protocols, addressing the data bottleneck.
2. EM-Bench Benchmark: A rigorous, multi-tiered evaluation framework (14 sub-tasks) covering perception and reasoning, enabling standardized comparison of EM MLLMs.
3. MERLIN Framework: A novel two-stage training paradigm that combines instruction tuning with feature-level knowledge distillation and a Denoising Subspace Module. This explicitly teaches the model to reconstruct high-SNR-like features from noisy inputs, solving the low-SNR performance collapse.

4. Experimental Results

The authors evaluated MERLIN on EM-Bench against leading proprietary (GPT-5, Claude-4, Gemini-2.5) and open-source (Qwen, DeepSeek) LLMs.

• Performance: MERLIN achieved State-of-the-Art (SOTA) results across both perception and reasoning tasks.
  • Perception: MERLIN scored 78.27% average accuracy, significantly outperforming the next best model (Qwen3-VL at 72.94%). It showed particular strength in complex parameter estimation and jamming identification.
  • Reasoning: MERLIN demonstrated superior strategic planning capabilities, achieving high Rouge/BLEU scores in strategy generation, whereas baseline LLMs (which process signals as raw text) scored near zero on complex reasoning tasks.
• Low-SNR Robustness: MERLIN maintained high accuracy in low-SNR environments where baseline models failed.
• Ablation Study:
  • Moving from Stage 1 to Stage 2 (without distillation) improved performance significantly.
  • Adding Feature Distillation further boosted results.
  • The DSM module provided additional gains by effectively filtering noise.
  • The full MERLIN framework (including Logit Distillation) yielded the highest performance, confirming the synergy of feature and output-level guidance.

5. Significance

This work establishes a foundational pathway for applying MLLMs to the electromagnetic domain. By moving away from task-specific pipelines to a unified, end-to-end generative paradigm, MERLIN demonstrates that:

• Generalization is possible: MLLMs can handle diverse EM tasks from modulation classification to strategic counter-measure generation.
• Robustness is achievable: The specific challenge of low-SNR degradation can be solved not just by more data, but by architectural innovations like feature-space denoising and knowledge distillation.
• Infrastructure is ready: The release of EM-100K and EM-Bench provides the necessary infrastructure for future research, shifting the field from isolated experiments to systematic, benchmark-driven development.

In summary, MERLIN represents a significant leap forward in electromagnetic intelligence, proving that robust, reasoning-capable AI can be built for noisy, real-world signal environments.