The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling

1. Problem Statement

In the scaling of Vision-Language-Action (VLA) models, a prevailing assumption is that upgrading the vision encoder (e.g., from ResNet to SigLIP) will automatically improve downstream robotic manipulation performance, mirroring trends in vision-language modeling.

However, this paper challenges that assumption, specifically for models that represent actions as discrete tokens (e.g., Ordered Action Tokenization - OAT). The authors posit that the benefit of a richer visual representation is often blocked by the action representation method itself. They identify a phenomenon called the "Compression Gap," where the location of the information bottleneck in the visuomotor pipeline dictates whether encoder upgrades propagate to task success.

2. Theoretical Framework: The Compression Gap

The authors formalize the problem using the Information Bottleneck principle and the Data Processing Inequality.

• The Pipeline: $O \to Z \to A$ (Observation $\to$ Representation $\to$ Action).
• The Constraint: The mutual information $I(O; A)$ is bounded by the tightest bottleneck in the chain: $I(O; A) \leq \min(I(O; Z), I(Z; A))$.

The paper contrasts two pathways:

1. Continuous Pathway (e.g., Diffusion Policy):
   • Actions are predicted as continuous vectors via a denoising process.
   • Bottleneck: The vision encoder ($I(O; Z)$).
   • Result: Upgrading the encoder widens the bottleneck, allowing more information to flow to the action, directly improving performance.
2. Discrete Pathway (e.g., OAT):
   • Actions are compressed into a finite sequence of discrete tokens via a codebook (e.g., Finite Scalar Quantization - FSQ).
   • Bottleneck: The codebook capacity ($I(Z; T) \leq \log_2 |V|^{H_l}$).
   • Result: The codebook imposes a hard upper bound on information throughput (e.g., $\approx 80$ bits for OAT). Even if the encoder provides a richer representation ($Z$), the excess information is discarded at the quantization stage. Therefore, encoder upgrades yield diminishing or zero returns.

3. Methodology

The authors conducted a controlled factorial experiment on the LIBERO-10 benchmark using the Franka Emika Panda robot. They isolated the effect of action representation by keeping the observation pipeline and policy backbone architecture constant while varying three key variables:

• Action Representation:
  • Discrete: Ordered Action Tokenization (OAT) with a fixed FSQ codebook ($|V|=1000$).
  • Continuous: Diffusion Policy (DP) with a DDIM scheduler.
• Vision Encoder:
  • Weak: ResNet-18 (64-dim features).
  • Strong: SigLIP (1152-dim semantic features).
  • Extended Gradient: Also tested SigLIP 2 and DINOv2 ViT-L/14.
• Model Size:
  • M (Medium): 4-layer transformer.
  • L (Large): 6-layer transformer (to rule out capacity artifacts).

Supplementary Experiments:

• Encoder Quality Gradient: Tested four encoders to see if the effect is continuous.
• Codebook Size Variation: Systematically increased the OAT codebook size ($|V| = 1000, 1920, 4375$) to test if relaxing the bottleneck restores encoder sensitivity.

4. Key Results

A. Vision Encoder Sensitivity (The Compression Gap)

• Diffusion Policy (Continuous): Upgrading from ResNet-18 to SigLIP resulted in massive performance gains.
  • Size M: $36.4\% \to 57.6\%$ (+21.2%).
  • Size L: $44.0\% \to 70.0\%$ (+26.0%).
• OAT (Discrete): The same upgrade yielded significantly attenuated gains.
  • Size M: $53.8\% \to 57.4\%$ (+3.6%).
  • Size L: $48.0\% \to 58.4\%$ (+10.4%).
• Conclusion: In the discrete pathway, the encoder upgrade is effectively "blocked" by the codebook bottleneck.

B. Encoder Quality Gradient

• Diffusion Policy: Success rate tracked encoder quality monotonically. As encoder quality increased (ResNet $\to$ SigLIP $\to$ SigLIP 2 $\to$ DINOv2), performance consistently rose.
• OAT: Success rate remained flat and fluctuated without a clear trend, regardless of the encoder's representational power.
• Crossover: At low encoder quality (ResNet), OAT outperformed DP. At high encoder quality (DINOv2), DP surpassed OAT by ~12.8 points, demonstrating that discrete methods cannot leverage high-quality upstream features.

C. Codebook Size (Causal Evidence)

• When the OAT codebook capacity was increased (relaxing the bottleneck), the performance gap between ResNet and SigLIP widened significantly.
• At $|V|=1920$, SigLIP maintained high performance while ResNet dropped sharply, exposing the true quality difference that was previously masked by the tight $|V|=1000$ bottleneck.
• This provides causal evidence that the codebook capacity, not the tokenizer architecture itself, is the binding constraint.

5. Key Contributions

1. Identification of the Compression Gap: The paper defines a structural divergence in scaling behavior based on whether the information bottleneck lies at the vision encoder (continuous) or the action tokenizer (discrete).
2. Empirical Validation: Through rigorous factorial experiments, the authors demonstrate that discrete tokenization (OAT) fundamentally limits the scalability of VLA models by capping information throughput, rendering expensive encoder upgrades ineffective.
3. Causal Proof via Codebook Variation: By varying codebook size, the authors prove that the lack of encoder sensitivity in discrete models is a direct result of information compression, not a failure of the policy backbone.
4. Connection to Multimodal Scaling: The findings align with recent pretraining studies (Tong et al., 2026) showing that VAE-based (lossy) representations saturate while semantic encoders continue to scale, suggesting a universal principle in information-processing pipelines.

6. Significance and Implications

• Scaling Strategy: For Physical AI, simply increasing model size or data is insufficient. Researchers must first identify and remove information bottlenecks. If the action representation is discrete, upgrading the vision encoder is a waste of resources until the codebook capacity is expanded.
• Future Architectures: The paper suggests that while discrete tokens are useful for language alignment, they are suboptimal for high-fidelity manipulation scaling. Future work should explore adaptive codebooks or hybrid architectures that combine discrete reasoning with continuous action decoding.
• Encoder Optimization: Once the discrete bottleneck is removed, the vision encoder becomes the new bottleneck. This implies a need for vision encoders specifically optimized for robotic manipulation (contact geometry, affordances) rather than general semantic understanding.

In summary, the paper argues that discrete tokenization creates a "compression gap" that decouples upstream visual improvements from downstream robotic performance, fundamentally altering how VLA models should be scaled.