RF-GPT: Teaching AI to See the Wireless World

1. Problem Statement

Current Large Language Models (LLMs) and Multimodal Vision-Language Models (VLMs) excel at processing text, images, and audio but lack native support for Radio Frequency (RF) signals, which are the physical layer foundation of wireless communications, radar, and Integrated Sensing and Communications (ISAC).

Existing approaches suffer from a "modality gap":

• Traditional RF Models: Built as narrow, task-specific deep learning models (e.g., for modulation classification or channel estimation). They require separate architectures and datasets for each task, lack generalization across different SNR ranges or hardware scenarios, and provide only labels without explanations or natural language interfaces.
• Telecom LLMs: Current "TelecomGPT" style models focus on text-based data (logs, KPIs, tickets) and cannot process raw RF waveforms or spectrograms.
• Data Scarcity: High-quality, labeled RF datasets with expert textual annotations are non-existent due to the high cost of over-the-air measurement, privacy concerns, and the difficulty of obtaining ground truth for complex signal environments.

The core challenge is to create a Radio-Frequency Language Model (RFLM) that can perceive RF signals, reason about them, and interact via natural language, bridging the gap between low-level signal processing and high-level AI reasoning.

2. Methodology: RF-GPT Architecture

The authors propose RF-GPT, a framework that treats RF spectrograms as visual inputs to a multimodal LLM. The architecture consists of three main components:

A. RF Encoding via Spectrograms and Vision Encoders

Instead of processing raw complex IQ (In-phase/Quadrature) samples directly (which is computationally expensive and ignores time-frequency structure), the system converts RF signals into time-frequency spectrograms.

1. STFT Transformation: Raw IQ sequences are converted to Short-Time Fourier Transform (STFT) spectrograms.
2. Image Mapping: The complex spectrogram is converted to a magnitude (dB) spectrogram and mapped to a pseudo-RGB or grayscale image.
3. Vision Encoder: A pre-trained vision encoder (e.g., from Qwen2.5-VL) processes these images, treating them as visual patches.
4. Tokenization: The vision encoder outputs a sequence of RF tokens (latent representations) which are projected into the LLM's embedding space via a lightweight linear adapter.

B. Language Model Backbone

The core is a decoder-only Transformer LLM (based on Qwen2.5-VL).

• Conditioning: The RF tokens act as a prefix to the text sequence.
• Generation: The model generates RF-grounded text (answers, explanations, JSON) conditioned on the RF prefix and a natural language instruction.
• Architecture: Utilizes modern LLM features like Grouped-Query Attention (GQA), RMSNorm, and gated MLPs.

C. Synthetic Data Pipeline (The "RF Grounding" Strategy)

Since real-world labeled RF data is scarce, the authors constructed a massive synthetic dataset without manual labeling:

1. Waveform Generation: Used MATLAB toolboxes (5G, LTE, UMTS, WLAN, DVB-S2, Bluetooth) to generate standards-compliant waveforms with perfect ground truth metadata (modulation, SNR, bandwidth, user counts, overlaps).
2. Deterministic Captioning: A pipeline converts metadata into dense, structured natural language captions describing the signal scene.
3. Instruction Synthesis: A text-only LLM (GPT-OSS-120B) converts these captions into diverse Instruction-Answer pairs. These include:
   • Descriptive questions ("What signals are present?").
   • Quantitative queries ("How many users?").
   • Overlap analysis ("Do signals overlap in time/frequency?").
   • Structured outputs (JSON summaries).

• Scale: The pipeline generated ~12,000 unique RF scenes and 0.625 million instruction examples.

3. Key Contributions

1. Formulation of RFLM: Defined the Radio-Frequency Language Model as a conditional language model grounded on RF tokens, enabling unified instruction-following for diverse RF tasks.
2. RF-GPT Architecture: Successfully integrated RF spectrograms into a multimodal LLM using a vision encoder and adapter, allowing the model to "see" the wireless spectrum.
3. Synthetic Instruction Framework: Developed a fully automated pipeline to generate a large-scale, diverse, and standards-compliant RF instruction dataset, eliminating the need for expensive expert annotation.
4. Comprehensive Benchmarks: Established a new benchmark suite covering:
   • WBMC: Wide-band Modulation Classification (identifying overlapping signals).
   • WBOD: Wide-band Overlap Detection (time/frequency overlap reasoning).
   • WTR: Wireless Technology Recognition (identifying 5G, LTE, Wi-Fi, etc.).
   • WNUC: WLAN User Counting.
   • NRIE: 5G NR Information Extraction (SCS, SSB patterns, UE counts).

4. Experimental Results

The authors evaluated RF-GPT (3B and 7B parameter versions) against general-purpose VLMs (Qwen2.5-VL, GPT-5) and traditional CNN/Transformer baselines.

• General-Purpose VLMs Fail: Without RF grounding, general VLMs performed at chance levels (often <5-10% accuracy) on all RF tasks. They failed to identify the number of signals, modulation types, or overlaps, treating spectrograms as generic noise or unrelated images.
• RF-GPT Performance:
  • Modulation Classification (WBMC): Achieved 82.4% (Easy) and 47.8% (Hard) accuracy, compared to <2% for baselines. Crucially, it correctly identified the number of signals in ~98% of cases.
  • Overlap Detection (WBOD): Achieved 91.5% (Easy) and 71.7% (Hard) accuracy, demonstrating strong reasoning about time-frequency relationships.
  • Technology Recognition (WTR): Achieved 99.6% joint accuracy in identifying technology and link direction (DL/UL), whereas baselines were heavily biased and inaccurate.
  • User Counting (WNUC): Achieved 70% accuracy, significantly outperforming baselines (23%).
  • Information Extraction (NRIE): Achieved ~76% accuracy in extracting 5G NR parameters (SCS, SSB patterns), outperforming specialized CNN/Transformer baselines even with fewer training epochs.
• Robustness: The model showed robustness to impairments like Carrier Frequency Offset (CFO) and Power Amplifier nonlinearity, though performance degraded slightly with severe IQ imbalance.
• Efficiency: RF-GPT required only 3 epochs of fine-tuning to outperform baselines trained for 30 epochs, highlighting the efficiency of instruction tuning.

5. Significance and Impact

• Paradigm Shift: Moves RF intelligence from isolated, task-specific predictors to a unified, interactive, and explainable AI system. Engineers can now query the spectrum in natural language (e.g., "Find interference sources in this band").
• Scalability: The synthetic data pipeline proves that high-quality RF training data can be generated at scale without manual labeling, addressing a major bottleneck in the field.
• 6G Enabler: This work lays the foundation for AI-native networks (6G), where LLMs can directly reason about physical layer signals for autonomous network management, spectrum monitoring, and interference mitigation.
• Interpretability: Unlike black-box classifiers, RF-GPT provides textual explanations and structured JSON outputs, making its decisions auditable and integrable with higher-level network orchestration systems.

In conclusion, RF-GPT successfully bridges the gap between RF signal processing and large language models, demonstrating that with proper grounding and instruction tuning, LLMs can become powerful, general-purpose engines for wireless intelligence.