NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models

Technical Summary: NeuroRVQ: Multi-Scale Biosignal Tokenization for Generative Foundation Models

Problem Statement
Biosignals such as electroencephalography (EEG), electrocardiography (ECG), and electromyography (EMG) encode physiological activity across multiple temporal and spectral scales. While foundation models trained via self-supervised masked token prediction have shown promise in learning generalizable representations for these signals, their performance is heavily dependent on the quality of the tokenizer. Existing tokenizers often struggle to preserve high-frequency dynamics and reconstruct signals with high fidelity, leading to entangled or redundant representations that lack the structural integrity required for robust generative modeling. The authors argue that high-fidelity tokenization, rather than model scale or architectural complexity, is the critical bottleneck for effective biosignal foundation models.

Methodology
The paper introduces NEURORVQ, a modality-adaptive biosignal tokenizer family designed for high-fidelity signal reconstruction. The methodology consists of three core components:

1. Multi-Scale Tokenizer Architecture:

   • Input Processing: Raw multivariate biosignals are segmented into temporal patches. To handle variable durations and channel counts, the tokenizer operates on patches of length $w$, which is tuned per modality (e.g., 1s for EEG, 200ms for ECG/EMG).
   • Multi-Scale Temporal Encoder: An inception-style module decomposes each patch into frequency-specific representations using $S$ distinct temporal branches with varying 1-D convolution kernel sizes. This allows the network to capture both slow and fast dynamics.
   • Transformer Encoder: Trainable temporal and spatial embeddings are added to the multi-scale features and processed through shared transformer layers to produce multi-scale patch representations.
   • Hierarchical RVQ Codebooks: Each temporal branch’s representation is discretized using Residual Vector Quantization (RVQ). By employing $N$ codebooks per branch, the tokenizer preserves fine-grained information through iterative residual quantization.

2. Phase-Aware Training Loss:
   Direct reconstruction of Fourier phase using Mean Squared Error (MSE) is suboptimal due to the periodic nature of phase angles ($[-\pi, \pi]$), causing discontinuities at boundaries. NEURORVQ employs a novel loss function operating in the Fourier domain:

   • Phase Loss: Instead of predicting raw phase $\phi$, the decoder predicts sine ($\hat{s}$) and cosine ($\hat{c}$) components. The loss uses cosine similarity between the predicted vector $(\hat{c}, \hat{s})$ and the target $(\cos \phi, \sin \phi)$, respecting the circular topology of phase and avoiding gradient vanishing issues.
   • Log-Amplitude Loss: Reconstructs log-amplitude $\log(1 + A)$ to compress the dynamic range and emphasize high-frequency components.
   • Temporal Loss: A time-domain MSE regularization term is added to provide stable gradients and guide reconstruction.

3. Modality-Adaptive Configuration:
   The architecture is modality-agnostic, but hyperparameters are tuned to the spectro-temporal characteristics of each signal:

   • EEG: Resampled to 200 Hz; 4 temporal branches; 8 RVQ codebooks per branch.
   • ECG: Resampled to 200 Hz; 4 temporal branches; 8 RVQ codebooks per branch.
   • EMG: Resampled to 1000 Hz (to capture broadband content up to 500 Hz); 4 temporal branches; 16 RVQ codebooks per branch (greater depth for higher representational capacity).

Key Contributions

1. NEURORVQ Tokenizer: A multi-scale, modality-adaptive tokenizer that decomposes biosignals into frequency-specific representations encoded via hierarchical RVQ, enabling high-fidelity reconstruction across EEG, ECG, and EMG.
2. Phase-Aware Loss: A training objective that combines log-amplitude reconstruction, temporal regularization, and a novel phase loss based on cosine similarity to respect the circular topology of Fourier phase.
3. NEURORVQ-FM: A simple masked-token foundation model for each modality that leverages NEURORVQ tokenizers. The authors demonstrate that this simple architecture achieves competitive or superior downstream performance compared to existing modality-specific foundation models, validating that tokenizer quality drives performance.

Results

• Reconstruction Fidelity: NEURORVQ achieves substantially lower reconstruction error than existing codebook-based tokenizers (LaBraM, BrainOmni) for EEG. For ECG and EMG, it is the first to demonstrate high-fidelity codebook-based reconstruction. Ablation studies show that the phase-aware loss provides the largest single-step improvement in reconstruction quality (e.g., 17x reduction in MSE for EEG compared to baseline).
• Downstream Performance:
  • EEG: NEURORVQ-EEG achieved the highest mean balanced accuracy across five classification tasks (Motor, ERP, Memory, Sleep, Eyes), outperforming the next-best model by +4.2 points. It used only 5.9M backbone parameters, significantly fewer than competitors like NeuroGPT (79.5M).
  • ECG: On the PTB-XL benchmark, NEURORVQ-ECG doubled the balanced accuracy of HuBERT-ECG on the fine-grained 43-class task (58.33% vs. 20.71%), demonstrating superior handling of class imbalance.
  • EMG: NEURORVQ-EMG outperformed PhysioWave and TinyMyo across four classification benchmarks (Discrete Gestures, EPN-612, NinaPro DB5, UCI-EMG) by wide margins.

Significance
The paper claims that high-fidelity tokenization is the critical ingredient for effective biosignal foundation models. By showing that a simple foundation model architecture paired with a high-fidelity tokenizer outperforms more complex existing models, the authors argue that the focus should shift toward improving tokenization quality. NEURORVQ provides a generalizable, modality-adaptive framework that preserves the multiscale structure of biosignals, paving the way for more efficient and effective large biosignal foundation models.