⚡ electrical engineering

Seeing Radio: From Zero RF Priors to Explainable Modulation Recognition with Vision Language Models

This paper demonstrates that lightweight fine-tuning of general-purpose vision-language models on RF-to-image visualizations enables zero-prior, explainable, and robust modulation recognition, achieving near-90% accuracy without requiring task-specific architectures or RF-specific inductive biases.

Original authors: Hang Zou, Bohao Wang, Yu Tian, Lina Bariah, Chongwen Huang, Samson Lasaulce, Mérouane Debbah

Published 2026-02-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hang Zou, Bohao Wang, Yu Tian, Lina Bariah, Chongwen Huang, Samson Lasaulce, Mérouane Debbah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Teaching a Text-Only Robot to "See" Radio Waves

Imagine you have a super-smart robot (a Vision-Language Model, or VLM) that is amazing at reading books and looking at photos of cats, cars, and landscapes. It can tell you what's in a picture and explain why it thinks that.

However, this robot has a major blind spot: Radio waves.

In the real world, radio signals (like Wi-Fi, 5G, or FM radio) are invisible. They are just complex mathematical numbers (called IQ data) that computers use to talk to each other. Traditional AI models are like specialized mechanics who can fix only one specific type of car engine. If you give them a different engine, they break. They need to be rebuilt from scratch for every new job, and they can't explain why they made a decision.

The authors of this paper asked a simple question:

"Can we take our super-smart, general-purpose robot and teach it to understand radio waves just by showing it pictures of the waves, without rebuilding its brain?"

The Solution: Turning Invisible Waves into Art

Since the robot can't "hear" or "feel" radio waves, the researchers had to translate them into a language the robot understands: Images.

They used a clever translation method to turn the invisible radio signals into two types of visual art:

The "Spectrogram" (The Heat Map): Imagine looking at a piano sheet music, but instead of notes, you see a colorful heat map. Bright colors show where the energy is loud, and the shape shows how the sound changes over time. This helps the robot see the "shape" of the signal's frequency.
The "IQ Trace" (The Wiggly Line): Imagine a seismograph recording an earthquake, or a heart monitor line. This shows the raw up-and-down movement of the signal over time.

The Magic Trick: The researchers took these two images and stitched them side-by-side. They fed this "Radio Art" into the robot and asked, "What kind of modulation (signal type) is this?"

The Results: From Clueless to Expert

Here is what happened when they tried this:

Before Training (Zero Priors): When they first showed the robot these radio pictures without any training, it was completely lost. It guessed randomly, getting about 10% of the answers right. It was like showing a picture of a radio wave to someone who has never seen a radio and asking, "Is this a dog or a cat?"
After Light Training (Fine-Tuning): They didn't rebuild the robot. They just gave it a "crash course" (called Fine-Tuning) using about 40,000 examples of these radio pictures.
- The Result: The robot's accuracy skyrocketed from 10% to nearly 90%.
- The Best Combo: The robot performed best when it saw both the heat map (spectrogram) and the wiggly line (IQ trace) together. It was like giving a detective both a fingerprint and a witness statement.

Why This is a Game-Changer

The paper highlights three superpowers this new approach gives us:

It Can Explain Itself:
Old AI models just say, "This is a 5G signal." They give no reason.
This new VLM says, "This is a 5G signal because I see a specific grid pattern in the heat map and a smooth wave in the time-line." It can write a paragraph explaining its reasoning in plain English. This is huge for trust and safety.
It's Tough Against Noise:
Radio signals often get messy (static, interference). The researchers tested the robot with "noisy" signals (like trying to hear a conversation in a loud bar). The robot that saw both images (heat map + wiggly line) was much better at ignoring the noise and finding the signal than the ones that only saw one type of image.
It Learns Fast (Data Efficiency):
Traditional AI models need thousands of hours of training to learn a new signal type. This VLM learned to recognize 57 different types of signals in just 3 epochs (a few passes through the data). It's like a student who can learn a new language in a week because they already know the grammar of other languages.

The "Out-of-Vocabulary" Test

The researchers also tested if the robot could recognize signals it had never seen before (like a secret code it wasn't trained on).

The Result: Even when the robot encountered a completely new type of signal, it didn't just crash. It used its general understanding of patterns to make a decent guess. This suggests it's learning the concept of radio waves, not just memorizing a list of answers.

The Bottom Line

This paper proves that we don't need to build a new, specialized AI for every single radio task. Instead, we can take a general-purpose AI (the kind that already understands images and text), show it a picture of a radio wave, and it can learn to identify, classify, and explain that signal almost instantly.

In a nutshell: They turned invisible radio static into a picture book, taught a smart robot to read it, and now the robot can act as a universal translator for the wireless world, helping us build smarter 6G networks that can "see" and "understand" the air around us.

1. Problem Statement

Current Radio Frequency (RF) machine learning pipelines for tasks like Automatic Modulation Classification (AMC) face several critical limitations:

Task-Specific Architecture: They rely on custom deep learning models (e.g., CNNs, Transformers) designed specifically for RF data, requiring significant engineering for each new task.
Data Dependency: They necessitate large, labeled datasets for every specific problem and struggle to generalize across varying Signal-to-Noise Ratios (SNRs) and channel conditions.
Lack of Interpretability: These models typically output a "hard prediction" (a class label) without providing human-readable reasoning or explanations.
Text-Centric AI Gap: While Large Language Models (LLMs) and Multimodal LLMs (MLLMs) have revolutionized text and image processing, they currently lack "spectrum intelligence." They cannot natively perceive or reason about RF signals, which are crucial for future 6G networks.

The core research question is: Can off-the-shelf Vision-Language Models (VLMs) be adapted to perceive and classify RF signals purely in the visual domain, without redesigning their architectures or injecting RF-specific inductive biases?

2. Methodology

The authors propose a pipeline that converts raw RF IQ (In-phase/Quadrature) data into visual formats interpretable by general-purpose VLMs.

A. RF-to-Image Conversion

The system transforms complex baseband signals into three distinct visual input modes:

Spectrogram (Spec): Generated via Short-Time Fourier Transform (STFT) using a Blackman window. This captures time-frequency patterns (e.g., bandwidth occupancy, sidebands).
IQ Time-Series (IQ): Visualizes the real and imaginary components of the signal as time-series traces. To handle varying symbol rates, the system extracts segments based on a fixed number of zero-crossing intervals rather than fixed time windows, normalizing the visual geometry.
Joint Representation: A concatenation of the Spectrogram and the IQ time-series panel, allowing the model to fuse spectral stability with temporal dynamics.

B. Benchmark Construction

Dataset: Built using TorchSig, covering 57 distinct modulation classes across 11 families (including OFDM, QAM, PSK, ASK, FSK, GFSK, MSK, GMSK, AM, Chirp/LFM, and Singleton signals like OOK).
Task Formulation: The problem is framed as a Visual Question Answering (VQA) task. The model is presented with an image and a list of candidate modulation classes, asked to select the correct one.
Evaluation Modes:
- Zero-shot: No examples provided.
- Few-shot: $s$ examples provided per class to aid in-context learning.
- Out-of-Vocabulary (OOV): Testing generalization by excluding specific classes during training.

C. Model Adaptation

Base Models: The study utilizes general-purpose VLMs (InternVL3-8B, Qwen2.5-VL-7B-Instruct, and GPT-5-Nano).
Fine-Tuning: Instead of pre-training from scratch, the authors apply Parameter-Efficient Fine-Tuning (PEFT) using QLoRA. This involves freezing the main model weights and training low-rank adapters ( $r=32, \alpha=64$ ) on the RF visual dataset.
Prompting: The system uses a chat-style schema where the user provides the image and a list of candidates, and the model outputs the exact class name.

3. Key Contributions

First RF-VLM Benchmark: Construction of a reproducible benchmark for RF modulation classification using visual representations (Spectrogram, IQ, and Joint) with 57 modulation classes.
Zero-Prior Adaptation: Demonstration that general-purpose VLMs, which possess zero prior knowledge of RF signals, can be adapted to achieve high accuracy (~90%) with only lightweight fine-tuning and no architectural changes.
Explainability: The ability of the fine-tuned VLMs to generate human-readable rationales for their decisions (e.g., explaining why a signal is FM based on observed patterns), a feature absent in traditional CNNs.
Efficiency: Showing that VLMs achieve competitive accuracy with significantly fewer training epochs compared to task-specific CNN baselines.

4. Key Results

Accuracy and Performance

Zero-Shot Baseline: Vanilla VLMs perform near-randomly (~10% accuracy) on RF tasks, confirming they lack inherent RF priors.
Fine-Tuned Performance: After PEFT, accuracy jumps dramatically:
- Qwen2.5-VL-7B-RF: Achieves 89.05% accuracy with the Joint representation, compared to ~77% for Spectrogram-only or IQ-only.
- InternVL3-8B-RF: Achieves 91.20% accuracy with the Joint representation.
Modality Fusion: The Joint representation consistently outperforms single modalities, particularly for difficult families like OFDM (improving from ~43% to ~88%) and AM (improving from ~50% to ~95%).

Robustness and Generalization

Noise Robustness: The Joint representation shows superior resilience to noise. At 10 dB SNR, accuracy is ~62%, rising to ~86% at 35 dB. IQ-only representations degrade significantly under heavy noise due to temporal fluctuations.
Out-of-Vocabulary (OOV): The model maintains strong performance even when 16 unseen classes are introduced during testing (dropping from ~89% to ~79%), indicating it learns visual-semantic features rather than memorizing a closed set.
Scalability: As the number of candidate classes increases (harder n-way classification), the Joint representation maintains a ~15% accuracy lead over single modalities.

Efficiency Comparison

Training Speed: A fine-tuned VLM reaches 56.4% accuracy in just 3 epochs. In contrast, CNN baselines (XCiT, ResNet) require 30 epochs to reach ~53-54% accuracy.
Sample Efficiency: VLMs leverage rich visual priors learned at scale, adapting to RF patterns with an order of magnitude fewer optimization steps.

Explainability

The fine-tuned models successfully generate textual explanations for their classifications (e.g., identifying FM signals by describing frequency modulation patterns in the IQ trace) without explicit supervision for explanation generation.

5. Significance and Future Outlook

This work establishes a scalable foundation for RF-aware AI in future 6G networks. By treating RF signals as visual data, the authors bridge the gap between general-purpose multimodal AI and specialized telecommunications tasks.

Practical Impact: It eliminates the need for custom RF architectures, allowing network operators to use a single, adaptable VLM backbone for diverse tasks (modulation recognition, interference detection, spectrum sensing).
Interpretability: The ability to generate "rationales" enhances trust in AI-driven network management and fault diagnosis.
Future Directions: The authors suggest extending this framework to wideband scenarios, multi-label classification, streaming RF data, and incorporating more realistic channel conditions and hardware impairments.

In summary, the paper proves that "Seeing Radio" is a viable paradigm: converting RF signals into images allows general-purpose VLMs to become powerful, explainable, and robust tools for next-generation wireless intelligence.