Technical Summary: PICM-Net – Progressive Learned Image Compression for Machine Perception

1. Problem Statement

Traditional image compression standards (e.g., JPEG, VVC) and recent learned image codecs (LIC) are optimized for human visual perception, prioritizing perceptual fidelity metrics like MSE, MS-SSIM, or LPIPS. While deep learning-based codecs have surpassed traditional methods in rate-distortion performance, their objectives remain centered on human visual quality.

Concurrently, the rise of machine vision applications (autonomous driving, surveillance, remote sensing) has created a demand for machine-oriented codecs, which optimize for downstream task performance (e.g., classification accuracy) rather than pixel-level reconstruction. However, existing machine-oriented codecs typically operate in a single-pass, non-progressive manner. They lack Fine Granular Scalability (FGS), a feature well-established in human-oriented compression that allows a single bitstream to be decoded at multiple quality levels. This gap prevents machine vision systems from dynamically adapting to fluctuating network bandwidths or varying confidence requirements without re-encoding or re-transmitting data.

2. Methodology

The authors propose PICM-Net, the first progressive learned image compression codec designed specifically for machine perception. The framework is built upon a frozen, human-oriented hyperprior-based autoencoder backbone (TransTIC) and introduces three core components to enable adaptive, task-driven progressive transmission.

2.1 Progressive Trit-Plane Coding

To enable coarse-to-fine decoding from a single bitstream, the latent representation is decomposed into ternary digits (trits) rather than binary bits.

• Trit-Plane Decomposition: Latent coefficients are quantized and decomposed into a sequence of trits ($d \in \{0, 1, 2\}$).
• Plane-Length Allocation: The number of trit-planes allocated to each coefficient is determined by its predicted scale parameter ($\hat{\sigma}$) from the Gaussian prior. Coefficients with wider distributions receive more planes.
• Symbol Prioritization: Within each plane, symbols are ordered based on their scale parameters ($\sigma$-based ordering). This strategy is computationally efficient and empirically shown to be more effective than expected variance-based ordering for this task.
• Decoding: The bitstream can be truncated at any trit-plane level $\ell$, allowing the decoder to reconstruct an image at varying levels of fidelity.

2.2 Machine-Oriented Progressive Adaptation

Since the backbone is trained for human perception, the authors introduce lightweight adaptation modules to align the codec with machine vision tasks without retraining the entire network:

• Spatial-Frequency Modulation Adapter (SFMA): Inserted into the encoder and decoder, SFMA modulates features in both spatial and frequency domains. It suppresses visually important but task-irrelevant information while amplifying semantically discriminative features.
• Hyper-Synthesis Low-Rank Adapter (HSLoRA): A lightweight LoRA module applied only to the hyper-synthesis network ($h_s$). This realigns the entropy model's mean and scale parameters with the latent distribution shifted by the SFMA, ensuring the entropy coding remains valid without disturbing the fixed hyperprior density model.
• Progressive Decoding-Aware Training: The adaptation modules are trained using a loss function that samples random trit-plane prefixes ($\ell$) during training. This ensures the rate-accuracy trade-off is optimized not just at full reconstruction, but across the entire progressive decoding trajectory, encouraging early trits to carry discriminative semantic content.

2.3 Adaptive Decoding Controller

To address the asymmetry between human and machine perception (where machines do not benefit from bits beyond the point of reliable inference), the authors introduce an Adaptive Decoding Controller.

• Mechanism: A lightweight logistic regression model analyzes classifier output statistics (logits, softmax probabilities, entropy) at each decoding level $\ell$.
• Function: The controller predicts the "suitability" (probability of correct classification) of the current partial reconstruction.
• Inference: During inference, the system decodes progressively. If the predicted suitability $S(\ell)$ exceeds a user-defined confidence threshold $\tau$, decoding stops, and the remaining bits are discarded. This dynamically determines the minimum necessary bitrate per image.

3. Key Contributions

1. PICM-Net Architecture: The first progressive image codec for machine perception, utilizing trit-plane coding to enable multi-level decoding from a single bitstream.
2. Task-Aware Adaptation: A novel adaptation strategy combining SFMA and HSLoRA with progressive decoding-aware training. This shifts symbol prioritization in the bitstream toward task-discriminative regions (e.g., object boundaries) rather than uniform spatial distribution.
3. Adaptive Decoding Controller: A mechanism that dynamically selects the decoding level based on predicted downstream suitability, eliminating unnecessary bit transmission for "easy" samples while allocating more bits to "hard" samples.
4. Empirical Validation: Comprehensive experiments demonstrating superior rate-accuracy performance compared to state-of-the-art human-oriented progressive codecs and competitive performance against non-progressive machine-oriented codecs.

4. Experimental Results

Experiments were conducted on the ImageNet-1K validation set using ResNet-50 and ConvNeXt as downstream classifiers.

• Rate-Accuracy Performance: PICM-Net consistently outperforms existing human-oriented progressive codecs (e.g., PLONQ, DPICT, DeepHQ) across all bitrates. It achieves a BD-rate reduction of −21.6% compared to a static decoder without the adaptive controller in the practical operating region (accuracy $\ge$ 70%).
• Adaptive Efficiency: The adaptive controller successfully reallocates bits. For "easy" images (where the classifier reaches high confidence early), the controller stops decoding after utilizing only ~4% of the progressive trajectory. For "hard" images, it continues decoding up to ~67% of the trajectory.
• Bitrate Savings: At fixed confidence thresholds of 70%, 73%, and 75%, the controller reduces average bitrate by 18.7%, 23.9%, and 26.4%, respectively.
• Ablation Studies:
  • Removing the progressive decoding-aware training causes a significant collapse in average accuracy across bitrates (dropping from 66.5% to 54.1%), confirming its necessity.
  • SFMA contributes the largest performance gain (−66.9% BD-rate), while HSLoRA provides a marginal but necessary refinement (−1.9%).
  • Sigma-based symbol prioritization is preferred over expected variance-based ordering due to a 20.7x speedup in prioritization with negligible performance loss.

5. Significance and Claims

The paper claims that PICM-Net bridges the gap between the flexibility of progressive compression and the efficiency of machine-oriented optimization. By enabling fine-grained scalability for machine vision, the framework allows systems to operate efficiently under dynamic bandwidth constraints without sacrificing task performance.

The authors emphasize that unlike human perception, where more bits always yield better visual quality, machine perception has a saturation point where additional bits provide no utility. PICM-Net's adaptive controller exploits this by dynamically truncating the bitstream, offering a practical solution for real-world deployment where network resources are limited. The work establishes a foundation for future research into classifier-agnostic adaptation and extending progressive compression to dense machine tasks like detection and segmentation.