Benchmarking Wireless Representations: High-Dimensional vs. Compressed Embeddings for Efficiency and Robustness

Technical Summary: Benchmarking Wireless Representations

Problem Statement
Recent advances in foundation models (FMs) for wireless communications, such as the Large Wireless Model (LWM), have demonstrated the potential of high-dimensional learned embeddings to serve as universal feature extractors. These models, pre-trained on synthetic data via masked channel modeling, aim to reduce reliance on extensive labeled datasets and improve robustness to out-of-distribution data. However, the practical trade-offs of deploying these high-dimensional embeddings in real-world systems remain under-explored. Specifically, there is a lack of systematic analysis regarding the cost-benefit ratio of high-dimensional learned representations compared to compact, low-dimensional alternatives and raw data baselines. Key concerns include the computational overhead, transmission latency, and noise robustness associated with expanding channel dimensionality.

Methodology
This work establishes a systematic benchmark comparing three distinct types of wireless channel representations across diverse downstream tasks:

1. High-Dimensional Learned Embeddings: Derived from the LWM, a transformer-based architecture pre-trained on Channel State Information (CSI). This includes both fine-grained "Channel embeddings" (patch-level) and a global "CLS embedding."
2. Compressed Latent Representations: Generated by compact autoencoders (AE) trained to reconstruct noisy CSI. Two compression ratios were tested (1:16 and 1:32), producing low-dimensional latent vectors.
3. Raw Data Baseline: The original complex CSI matrix, flattened for processing.

The evaluation was conducted across three downstream tasks designed to test complementary capabilities:

• LoS/NLoS Classification: A binary classification task to assess the preservation of physical propagation characteristics.
• Beam Selection: A multi-class classification task evaluating the ability to capture spatial geometric relationships for beamforming.
• Power Allocation: A novel regression task in a Multi-User MIMO (MU-MIMO) setting, aiming to maximize spectral efficiency under power constraints. This task utilized a hybrid training strategy involving a supervised warm-up (using Projected Gradient Descent labels) followed by unsupervised learning to maximize sum-rate.

Performance was measured using Data Efficiency (performance vs. training sample size), Noise Robustness (performance vs. Signal-to-Noise Ratio), and Computational Complexity (parameters, FLOPs, and inference time).

Key Contributions

• Systematic Benchmarking: The paper provides a comprehensive comparison of foundation model embeddings against compact autoencoder representations and raw data, moving beyond single-task evaluations to a multi-task framework.
• Introduction of Power Allocation: The authors introduce power allocation in a MU-MIMO system as a new downstream task to evaluate representations in a system-level resource management context, a task not previously used to benchmark these specific representations.
• Resource Overhead Characterization: The study explicitly quantifies the latency, parameter count, and transmission costs associated with high-dimensional embeddings, highlighting the practical implications of using foundation models in wireless systems.

Experimental Results

• Data Efficiency: In the LoS/NLoS classification task, high-dimensional LWM embeddings (specifically channel embeddings) demonstrated superior performance in few-shot regimes, saturating earlier than the raw CSI baseline. However, with sufficient data, all representations converged to similar performance levels.
• Noise Robustness: Compressed autoencoder representations consistently demonstrated superior noise robustness compared to LWM embeddings. In low SNR regimes, LWM embeddings suffered more pronounced performance degradation, often underperforming both the autoencoder and the raw baseline in beam selection tasks.
• Task-Specific Performance:
  • Beam Selection: Learned representations offered no clear advantage over the raw baseline or autoencoders. The global CLS embedding performed worse than other representations in both low and high-data regimes.
  • Power Allocation: The autoencoder-based approach showed higher data efficiency and training stability, outperforming both the baseline and LWM while achieving near-optimal spectral efficiency.
• Computational Complexity: The LWM incurred significant overhead. Despite having fewer FLOPs for embedding generation than some autoencoders, its transformer architecture resulted in longer inference times (approx. 10.98 ms vs. 0.73 ms for AEs) and a larger memory footprint. Furthermore, the high dimensionality of LWM embeddings increases the memory requirements for downstream models and transmission bandwidth.

Significance and Claims
The paper concludes that while high-dimensional embeddings from foundation models offer distinct advantages in low-data regimes for specific tasks like LoS/NLoS classification, they introduce substantial latency and parameter overhead that may hinder practical deployment. In contrast, compact latent representations learned via autoencoders provide a more balanced solution, demonstrating improved noise robustness, stable performance across diverse tasks, and significantly reduced computational and transmission costs.

The authors assert that representation design is central to determining the efficiency and robustness of learning-based wireless systems. They argue that the "richness" of high-dimensional embeddings must be carefully weighed against the resource constraints of real-world deployments. The findings suggest that for many practical applications, particularly those requiring robustness to noise and low latency, compact autoencoder-based representations may be a more viable alternative to large-scale foundation models.