⚡ electrical engineering

Two-Dimensional Quantization for Geometry-Aware Audio Coding

This paper introduces Two-Dimensional Quantization (Q2D2), a novel scheme that projects feature pairs onto structured 2D grids to overcome the geometric limitations of traditional quantization methods, thereby achieving superior audio compression efficiency and state-of-the-art reconstruction quality across speech, audio, and music domains.

Original authors: Tal Shuster, Eliya Nachmani

Published 2026-05-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tal Shuster, Eliya Nachmani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a high-quality music file over a very slow internet connection. To make it fit, you have to compress it, turning the complex sound waves into a list of simple instructions (tokens) that a computer can understand and then rebuild.

For a long time, the best way to do this was like using a giant, messy dictionary.

The Old Way (Vector Quantization): Imagine you have a dictionary with thousands of words. To describe a sound, you look for the single word that matches best. But as the dictionary gets bigger, most of the words sit unused on the shelf, and the computer struggles to find the right one quickly. It's inefficient and often leads to "dead" words that never get used.
The "Simple" Way (Finite Scalar Quantization): To fix the mess, some researchers tried a simpler approach: they treated every single sound feature like a separate number line. It's like having 100 tiny, independent rulers. You just round each number to the nearest mark. This is very efficient and uses all the marks, but it's like trying to describe a 3D object by only measuring its height, width, and depth separately. You miss how those dimensions relate to each other (the "correlations"), so the sound loses some of its natural texture.

The New Idea: Q2D2 (The "Tile Floor" Approach)

The authors of this paper, Tal Shuster and Eliya Nachmani, introduced a new method called Q2D2 (Two-Dimensional Quantization).

Instead of using a messy dictionary or separate rulers, they decided to treat pairs of sound features like tiles on a floor.

Pairing Up: Instead of looking at sound features one by one, they take two features at a time and treat them as a single unit (a pair).
The Grid: They project these pairs onto a structured 2D grid, similar to how you might tile a floor. They tested different shapes for these tiles:
- Rectangles: Like a standard brick wall.
- Hexagons: Like a honeycomb.
- Rhombuses: Like diamonds or slanted squares.
Snapping to the Grid: When the computer needs to compress a sound, it looks at the pair of features and "snaps" them to the nearest point on this grid.

Why is this better?

Think of it like packing a suitcase:

Rectangles leave gaps in the corners.
Hexagons fit together perfectly but are hard to align with the shape of the items you are packing.
Rhombuses (Diamonds) turned out to be the "Goldilocks" solution in this paper. They fit together so tightly that they cover the space almost perfectly without gaps, and they align well with the natural shape of the sound data.

Because the grid is pre-built and structured (like a perfect honeycomb or diamond pattern), the computer doesn't need to learn a massive dictionary of words. It just needs to know the shape of the grid. This means:

No Wasted Space: Every single point on the grid gets used (high "codebook utilization").
Better Relationships: Because they look at pairs of features together on a 2D map, they capture how those features relate to each other, which the "separate ruler" method missed.
High Quality: The result is a sound file that sounds incredibly clear, even when compressed down to a tiny size.

The Results

The paper tested this on speech, music, and general audio. They found that Q2D2 could rebuild audio with state-of-the-art quality while using far fewer "tokens" (instructions) than previous methods.

The Analogy: If other methods needed 900 tiny instructions to describe a second of speech, Q2D2 could do it with just 166 or even 53, and it still sounded just as good (or better).
The Proof: In tests, Q2D2 beat the current best models (like DAC, Encodec, and WavTokenizer) in objective measurements (how well the math matches the original sound) and subjective tests (how good it sounds to human ears).

Summary

The paper claims that by changing how we "quantize" (compress) audio from a 1D list of numbers or a messy dictionary into a structured 2D grid of tiles, we can make audio compression much more efficient. The "Rhombic" (diamond-shaped) grid was the winner, offering a perfect balance of packing efficiency and capturing the natural relationships between sound features, resulting in crystal-clear audio at very low data rates.

Technical Summary: Two-Dimensional Quantization for Geometry-Aware Audio Coding (Q2D2)

1. Problem Statement

Recent neural audio codecs have achieved high reconstruction quality using quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ), and Finite Scalar Quantization (FSQ). However, these existing techniques face specific limitations regarding the geometric structure of the latent space:

RVQ and VQ: While expressive, they often suffer from codebook underutilization (where many codewords remain unused as codebook size increases) and require complex training tricks (e.g., commitment losses, codebook re-initialization) to stabilize. They also struggle to efficiently capture correlations between feature dimensions without reintroducing high-dimensional optimization instability.
FSQ: This method offers high codebook utilization and stability by quantizing each latent channel independently onto fixed scalar levels. However, its strictly one-dimensional nature fails to capture correlations between feature dimensions, limiting its representational capacity.

The core challenge addressed by this work is how to retain the simplicity and high utilization of FSQ while enriching the representational capacity of discrete audio codes to capture inter-channel correlations, without the instability of high-dimensional vector quantization.

2. Methodology: Q2D2

The authors propose Two-Dimensional Quantization (Q2D2), a geometry-aware quantization scheme that moves beyond simple 1D scalar grids.

Core Mechanism

Instead of quantizing each latent channel independently, Q2D2 groups latent feature channels into pairs and maps them onto structured two-dimensional grids.

Projection and Bounding: The encoder's output is projected to a latent dimension $d$ (where $d$ is even). A hyperbolic tangent nonlinearity bounds the features to $[-1, 1]$ .
Pairing: The $d$ dimensions are reshaped into $P = d/2$ pairs.
Grid Quantization: Each pair is jointly quantized to the nearest point on a fixed 2D grid. The grid can take one of three tiling forms:
- Hexagonal
- Rectangular
- Rhombic
Implicit Codebook: The overall codebook is defined implicitly by the product of all pairwise grids. The size is $|C| = \prod_{j=1}^{P} (l_{2j-1} \cdot l_{2j})$ , where $l_i$ is the number of quantization levels for dimension $i$ .
Training: The system uses Straight-Through Estimators (STE) for gradient propagation. It employs lightweight linear projections into and out of the quantization space but does not learn a codebook embedding matrix, similar to FSQ.

Geometric Advantage

By using 2D tilings, Q2D2 introduces geometric structure that captures correlations between feature pairs. The authors specifically highlight Rhombic grids in their ablation studies, noting they offer higher packing efficiency (denser tiling of the 2D embedding space) compared to rectangular or hexagonal grids, leading to more uniform latent-space coverage and lower quantization error.

3. Key Contributions

The paper outlines three primary contributions:

Conceptual: Introduction of Q2D2, the first quantization method to group channels into pairs and jointly quantize them using complex geometric structures (hexagonal, rhombic, rectangular). This enhances the semantic information of the codec and captures feature correlations without the instability of high-dimensional VQ.
Methodological: Design of a 2D single-quantizer space that tests various tiling geometries. The authors analyze the impact of grid shape, resolution levels, and projected dimension sizes on performance and codebook utilization.
Experimental: Demonstration that Q2D2 achieves competitive or superior speech reconstruction performance compared to State-of-the-Art (SOTA) models (e.g., WavTokenizer, DAC, Encodec) across speech, audio, and music domains. Crucially, it achieves this with low token rates (e.g., 53, 166, 333 tokens/sec) and high codebook utilization without relying on auxiliary tricks like commitment losses or codebook reseeding.

4. Experimental Results

The authors evaluated Q2D2 on extensive datasets including LibriTTS, LibriSpeech, LJSpeech, AudioSet, and Jamendo.

Reconstruction Quality:
- At 3.3 kbps (166 tokens/sec), Q2D2 with a rhombic grid outperformed all SOTA models in the 0.5–9 kbps range on UTMOS (a metric highly correlated with human perception), PESQ, and STOI.
- At 1 kbps (75 tokens/sec), Q2D2 outperformed DAC (100 tokens) and WavTokenizer (75 tokens) across most metrics on clean and noisy test sets.
- In comparisons against models trained on 150k hours of multilingual speech (Emilia/MLS), Q2D2 (333 tokens, 1 quantizer) achieved the highest UTMOS and STOI scores among models using >150 tokens.
Codebook Utilization: Unlike VQ-based methods that suffer from dead codewords, Q2D2 maintains near 100% pair utilization and high codebook utilization rates (e.g., 92–99%) across different bitrates without auxiliary stabilization losses.
Downstream Tasks: In text-to-speech (TTS) generation tasks, Q2D2 tokens produced audio with comparable quality to WavTokenizer and DAC, despite using a single quantizer.
Semantic Representation: On the ARCH benchmark (12 datasets covering speech, music, and audio events), Q2D2 with only 53 tokens/sec outperformed DAC and Encodec models using 100–900 tokens/sec in classification accuracy for several tasks, including music (MTT) and environmental audio (US8K).
Ablation Studies:
- Grid Type: Rhombic grids consistently outperformed rectangular and hexagonal grids, attributed to higher packing efficiency.
- Dimensions: A 6-dimensional configuration offered the optimal trade-off between compactness and representational capacity.
- Projections: Bounding features with a tanh nonlinearity before quantization was found crucial for stable training.

5. Significance and Claims

The paper claims that Q2D2 represents a simple yet powerful alternative to conventional scalar (FSQ) or vector (VQ/RVQ) quantization. Its significance lies in:

Efficiency: It unlocks richer discrete audio representations with significantly lower token rates and model parameter counts (no learned codebook embeddings).
Geometry-Awareness: It successfully captures correlations between feature dimensions through structured 2D grids, bridging the gap between the robustness of FSQ and the expressiveness of multi-dimensional grids.
Scalability: The method scales effectively from low bitrates (1 kbps) to higher bitrates while maintaining high reconstruction quality and utilization.

The authors conclude that Q2D2 preserves rich semantic information even under extreme compression, suggesting it can serve as a foundational component for future audio-based foundation models and multimodal systems. Future work is noted to explore extending these concepts to 3D geometric tilings.