Parameter-Efficient Architectural Modifications for Translation-Invariant CNNs

1. Problem Statement

Convolutional Neural Networks (CNNs) are theoretically assumed to be translation-invariant due to weight sharing and pooling. However, standard architectures (e.g., VGG, ResNet) exhibit a critical fragility: they rely heavily on Fully Connected (FC) layers at the end of the network. These dense layers flatten spatial dimensions, creating a rigid dependency on absolute pixel coordinates.

• Consequence: Even a single-pixel shift in the input image can drastically degrade classification performance or alter the output.
• Current Mitigation: The industry standard is Data Augmentation (training on shifted crops), which teaches the model to "memorize" invariance rather than structurally enforcing it. This is computationally expensive and treats the symptom rather than the root cause.
• Secondary Issue: In Image Quality Assessment (IQA), standard metrics like LPIPS inherit this sensitivity, penalizing imperceptible shifts as severe errors, which contradicts human visual perception.

2. Methodology

The authors propose a lightweight, structural solution called the "Online Architecture" strategy. Instead of relying on data augmentation, they modify the network architecture to decouple feature recognition from spatial location.

• Core Mechanism: Strategic insertion of Global Average Pooling (GAP) layers.
  • GAP aggregates feature activations spatially across an entire feature map, producing a single value per channel. This effectively removes spatial coordinates from the feature representation.
• Architectural Variants (based on VGG-16):
  1. Base: Original VGG-16 (FC layers intact).
  2. Multi: Inserts GAP layers after every MaxPool operation. The resulting invariant vectors bypass subsequent convolutions and are concatenated before the final classifier.
  3. Final: Replaces the entire block of final dense layers with a single GAP layer followed by one Dense layer.
  4. Flat: Similar to Multi but uses a Flatten layer before concatenation (preserves more structural info but retains high parameter count).
• Experimental Setup:
  • Datasets: MNIST (to visualize aliasing), ImageNet-1K subset (robustness benchmark), and IQA datasets (TID08, TID13, KADID-10k, RAID).
  • Transformations: Primary focus on Translation (using mosaic padding to avoid zero-padding artifacts). Control experiments included rotation and scaling.
  • IQA Integration: The modified backbones were integrated into the LPIPS (Learned Perceptual Image Patch Similarity) framework to create new metrics ($L_{Multi}$, $L_{Flat}$).

3. Key Contributions

1. Structural Invariance: Demonstrated that replacing FC layers with GAP creates a translation-invariant predictor by construction, not by approximation.
2. Extreme Parameter Efficiency: Achieved a 98% reduction in trainable parameters (from ~5.2M to ~82K in the "Final" variant) and a 90% reduction in total model size, while maintaining competitive accuracy.
3. Identification of Aliasing Limits: Identified that while GAP solves macroscopic sensitivity, discrete pooling operations introduce a residual periodic aliasing effect. This prevents perfect pixel-level invariance, causing periodic oscillations in accuracy based on shift magnitude relative to the pooling kernel size.
4. Biologically Plausible IQA: Successfully applied these invariant backbones to LPIPS, creating a metric that aligns significantly better with human psychophysical response curves than the standard retrained baseline.

4. Key Results

A. Classification Robustness (ImageNet)

• Accuracy: The modified models maintained Top-1 accuracy comparable to the baseline (e.g., Final achieved 66.4% vs. baseline 64%).
• Robustness: The average relative loss due to translation dropped from 0.09 (Base) to 0.05 (Multi) and 0.06 (Final).
• Stability: The standard deviation of accuracy across shifts was halved (0.04 $\to$ 0.015–0.018), indicating much more consistent performance regardless of object position.
• Aliasing Observation: Heatmaps revealed periodic vertical bands of higher loss in high-resolution models, confirming that discrete pooling stride causes periodic sensitivity.

B. Image Quality Assessment (IQA)

• Generalization: On the KADID-10k dataset (unseen distortions), the $L_{Flat}$ variant achieved a Spearman correlation of 0.89, significantly outperforming the retrained baseline LPIPS (0.75).
• Psychophysical Alignment: Using the RAID dataset, the $L_{Flat}$ model replicated human suprathreshold response curves with a Spearman correlation of 0.95 (vs. 0.83 for baseline). This confirms the metric correctly mirrors human tolerance to trivial spatial shifts.

C. Specificity

• The improvements were specific to translation. The models did not show improved robustness to rotation or scaling; in some cases, they performed slightly worse than the baseline for rotation, confirming that GAP does not inherently solve all affine transformations.

5. Significance and Conclusion

This work challenges the reliance on brute-force data augmentation for robustness. It proves that architectural constraints (specifically removing spatial dependency via GAP) are a more efficient and biologically plausible path to robustness.

• Efficiency: It demonstrates that massive Fully Connected layers are often redundant for tasks not requiring precise localization, and their removal drastically reduces computational cost without sacrificing performance.
• Perceptual Accuracy: For Image Quality Assessment, enforcing translation invariance is critical. The proposed metric ($L_{Flat}$) avoids penalizing imperceptible shifts, aligning mathematical evaluation with human visual perception.
• Limitations: The study highlights a fundamental theoretical limit: perfect pixel-level invariance is impossible in standard CNNs due to the discrete sampling (aliasing) inherent in pooling layers, suggesting future work may need to address anti-aliasing in downsampling operations.

Availability: The code and datasets are publicly available on GitHub, facilitating further research into invariant architectures.