AIMC-Spec: A Benchmark Dataset for Automatic Intrapulse Modulation Classification under Variable Noise Conditions

1. Problem Statement

Automatic Intrapulse Modulation Classification (AIMC) is a critical task in electronic support systems and radar signal analysis. It involves identifying the specific modulation scheme embedded within a single radar pulse from its complex In-phase/Quadrature (I/Q) representation.

The field faces three primary challenges:

• Lack of Standardization: There is no publicly available, standardized benchmark dataset. Most existing studies rely on private, internally generated data with inconsistent signal definitions, modulation counts, and Signal-to-Noise Ratio (SNR) levels.
• Reproducibility Issues: The absence of common datasets makes fair comparison between different deep learning architectures difficult. Reported accuracies vary widely (88%–99.6%) due to differing evaluation conditions rather than model superiority.
• Noise Sensitivity: AIMC performance degrades significantly in low-SNR environments, particularly for Phase-Modulated (PM) signals which have subtle spectrogram patterns compared to Frequency-Modulated (FM) signals.

2. Methodology

A. The AIMC-Spec Dataset

The authors introduced AIMC-Spec, a comprehensive synthetic dataset designed to address the standardization gap.

• Scale & Composition: Contains 30 distinct intrapulse modulation types (15 Frequency Modulated and 15 Phase Modulated) across 5 SNR levels (+6, +3, 0, -3, -6 dB).
• Data Volume: Includes 1,000 testing signals and 100,000 training signals per modulation type.
• Signal Generation:
  • Parameters: Pulse widths (0.5–18 µs), carrier frequencies (centered at 9.5 GHz with 50 MHz bandwidth), and sampling rates (50 MHz).
  • Noise: Additive White Gaussian Noise (AWGN) is applied dynamically during data loading to the raw I/Q pulses, ensuring the SNR calculation is based strictly on pulse power.
  • Optimization: The dataset was redesigned to reduce storage footprint from >500 GB to <30 GB while preserving full signal fidelity.
• Format: Distributed as binary .h5 files containing complex I/Q samples and metadata.

B. Benchmarking Framework

To validate the dataset, the authors re-implemented and evaluated five representative deep learning algorithms from recent literature under a unified input format (FFT-based spectrograms with 50% window overlap) to isolate architectural performance from preprocessing biases.

1. LDC-Unet: A modified U-Net with local dense connections and a fine-tuned VGG-19 backbone, using Smooth Pseudo-Wigner Ville Distribution (SPWVD) and SSR-loss.
2. LPI-Net: A modular lightweight CNN using Continuous Wavelet Transform (CWD) and grayscale inputs.
3. CDAE-DCNN: A Convolutional Denoising Autoencoder followed by a Deep CNN, using Cohen's Class Distribution (CTFD).
4. STFT-CNN: A simple, lightweight 5-layer CNN using Short-Time Fourier Transform (STFT).
5. ViT (Vision Transformer): A transformer-based model operating on phase-spectrograms (carrier-aligned phase rows) to capture long-range dependencies.

C. Experimental Design

• Training: Models were trained on a combined dataset spanning all 5 SNR levels.
• Evaluation: Tested individually at each SNR level.
• Tasks:
  1. Full Classification: All 30 modulation types (FM + PM).
  2. FM-Only Classification: Restricted to the 15 FM types to isolate model robustness on visually distinct signals.

3. Key Contributions

1. Public Synthetic Benchmark: Introduction of AIMC-Spec, the first publicly available, standardized dataset specifically for spectrogram-based AIMC, covering 30 modulation types and variable noise conditions.
2. Optimized Storage: A redesigned data pipeline that reduces the dataset size by over 90% (from >500 GB to <30 GB) without losing signal fidelity, facilitating accessibility.
3. Comprehensive Benchmarking: A rigorous evaluation of five diverse architectures (CNNs, Denoising Autoencoders, U-Nets, and Transformers) on a consistent dataset, providing the first fair comparison of their performance in AIMC.
4. Open Source: Full release of data and code to ensure reproducibility and foster future research.

4. Results

Performance Rankings

Across all SNR levels and both experimental tasks, LDC-Unet consistently achieved the highest accuracy:

• Full Modulation Task: Mean accuracy of 66.91% (ranging from 57.37% at -6 dB to 74.29% at +6 dB).
• FM-Only Task: Mean accuracy of 87.95%.

ViT and LPI-Net followed as the second-best performers, while STFT-CNN (the simplest architecture) yielded the lowest results (32.62% mean on full task).

Key Observations

• FM vs. PM: All models performed significantly better on Frequency-Modulated (FM) signals than Phase-Modulated (PM) signals. PM signals exhibit subtle phase shifts that are harder to distinguish in noisy spectrograms, leading to higher inter-class ambiguity.
• SNR Impact: Accuracy improved steadily as SNR increased, with the most significant gains occurring between -6 dB and 0 dB.
• Architectural Resilience:
  • LDC-Unet demonstrated superior robustness due to its skip connections and residual blocks, which preserve multi-scale features and fine-grained details even under high noise.
  • Denoising Architectures (CDAE-DCNN) performed well at low SNRs but suffered steeper performance declines as noise increased, likely due to over-smoothing of high-frequency phase details.
  • Transformers (ViT) showed competitive results but were slightly less robust than top CNNs, suggesting a need for larger, more diverse training data to learn robust global attention patterns in noisy environments.
• Input Representation: Models utilizing RGB inputs and higher-resolution spectrograms (LDC-Unet, CDAE-DCNN) outperformed those using grayscale or low-resolution inputs (LPI-Net, STFT-CNN), highlighting the importance of preserving visual detail for complex modulations.

5. Significance

This paper establishes a reproducible baseline for the AIMC community. By providing a standardized dataset and benchmarking results, it:

• Eliminates Bias: Removes the variability caused by custom datasets, allowing for fair comparison of algorithmic innovations.
• Diagnoses Limitations: Reveals that current state-of-the-art models struggle significantly with Phase-Modulated signals in low-SNR conditions, identifying a clear area for future research.
• Guides Future Design: Suggests that future models should focus on architectures that preserve multi-scale features (like skip connections) and utilize rich input representations (RGB/high-res) to handle the complexity of real-world radar signals.
• Future Roadmap: The authors plan to expand the dataset to include channel effects (multipath, fading) and measured radar signals to further emulate operational environments.