TaCo: A Benchmark for Lossless and Lossy Codecs of Heterogeneous Tactile Data

Technical Summary: TaCo: A Benchmark for Lossless and Lossy Codecs of Heterogeneous Tactile Data

Published at: ICLR 2026

1. Problem Statement

As embodied AI and robotics advance, tactile sensing has become critical for fine-grained perception and dexterous manipulation. However, tactile sensors (such as vision-based GelSight or force-based sensors) generate high-dimensional, spatiotemporally dense data. This creates a significant bottleneck in three areas:

• Real-time Teleoperation: High bandwidth requirements for low-latency haptic feedback.
• Hardware Constraints: Limited communication bandwidth between hand microcontrollers (MCUs) and motherboards.
• Large-scale Training: The massive storage requirements for collecting and managing datasets for robotic model training.

Existing compression research is fragmented, often focusing on simple signal sparsity or narrow, task-specific scenarios, and lacks a standardized framework to evaluate how compression affects downstream robotic tasks.

2. Methodology

The authors propose TaCo, the first comprehensive benchmark specifically designed for tactile data codecs. The methodology is structured around three pillars:

A. Diverse Datasets and Codecs:

• Datasets: The benchmark utilizes five heterogeneous datasets covering different sensor types (visuo-tactile vs. force-based), resolutions, and scales, totaling over 250,000 frames.
• Codecs: It evaluates 30 different compression methods, categorized into:
  • Off-the-shelf codecs: Traditional signal processing methods (e.g., GZIP, JPEG-XL, VTM) designed for text, images, or video.
  • Pre-trained Neural Codecs: Models trained on natural images/videos (e.g., ELIC, LALIC, DCVC) to test cross-domain generalization.
  • Data-driven Tactile Codecs: The authors' own proposed models, TaCo-LL (lossless) and TaCo-L (lossy), which are trained end-to-end specifically on tactile data.

B. Multi-Dimensional Evaluation Tasks:
The benchmark moves beyond simple reconstruction metrics (like PSNR) to evaluate performance across four functional domains:

1. Lossless Storage: Measuring bits per byte to assess storage efficiency.
2. Human Visualization: Measuring rate-distortion (RD) curves (PSNR/MS-SSIM) for human perception.
3. Semantic Classification: Assessing how compression affects material and object recognition accuracy.
4. Dexterous Robotic Grasping: Evaluating the impact of compressed data on a reinforcement learning (RL) agent's ability to lift and stabilize objects in simulation.

3. Key Contributions

• The TaCo Benchmark: A standardized, open-source framework comprising datasets, metrics, and baselines for the tactile compression community.
• Novel Data-Driven Codecs: The introduction of TaCo-LL (based on DualComp-I) and TaCo-L (based on LALIC), which are the first codecs explicitly optimized for the unique statistical patterns of tactile signals.
• Comprehensive Analysis: A systematic study of how different sensor modalities (vision-based vs. force-based) and object properties (rigid vs. soft) influence compressibility.

4. Key Results

• Superior Compression Efficiency: TaCo-LL and TaCo-L consistently outperformed all off-the-shelf and pre-trained neural baselines. For example, TaCo-LL achieved up to 22× lossless compression on certain datasets.
• Task-Preserving Lossy Compression: In classification tasks, TaCo-L maintained accuracy levels nearly identical to uncompressed data, even at extremely low bitrates (e.g., reducing 24 bpp to ~0.1 bpp).
• Robustness in Robotics: In dexterous grasping simulations, TaCo-L achieved significantly higher compression ratios than traditional video codecs (like VTM) while maintaining a competitive success rate for lifting and disturbance resistance.
• Cross-Dataset Generalization: The data-driven models demonstrated strong robustness when tested on unseen datasets (e.g., ActiveCloth and ObjectFolder-2.0), proving they learn intrinsic tactile patterns rather than just overfitting to specific datasets.

5. Significance

This work bridges the gap between signal processing and embodied intelligence. By providing a rigorous way to measure the trade-off between bitrate and task performance, TaCo enables researchers to develop compression algorithms that are not just mathematically efficient, but "robotics-aware." This is a foundational step toward enabling high-fidelity, real-time tactile feedback in complex, bandwidth-constrained robotic systems and scaling the data infrastructure required for the next generation of embodied AI.