Technical Summary: Research on Intelligent Compression and Quality Enhancement of Surveillance Videos Based on Lightweight Algorithms

1. Problem Statement

The proliferation of surveillance video applications in fields such as road monitoring and tunnel excavation has created a critical challenge: the need to capture dynamic movements accurately while managing the computational and memory costs associated with complex models. Current state-of-the-art approaches often rely on increasing model complexity to improve recognition accuracy, particularly for distinguishing motion boundaries. However, this leads to excessive memory usage and reduced computational efficiency, hindering deployment on resource-constrained edge devices.

Furthermore, surveillance footage often contains significant redundancy (e.g., static frames in unoccupied corridors), which wastes network resources and user time. While Transformer-based models (e.g., ViT, Swin Transformer) have shown promise in computer vision, many are too parameter-heavy for low-level visual tasks like video enhancement or temporal action detection. Existing lightweight solutions often struggle to balance the trade-off between model size and the accuracy required for precise action boundary localization.

2. Methodology

The authors propose a Lightweight Convolutional Transformer based on Knowledge Distillation (LCT-KD) framework. This approach integrates Convolutional Neural Networks (CNNs) for local feature extraction, Transformers for global context modeling, and knowledge distillation for model compression. The methodology consists of two primary phases: training and inference.

2.1 Overall Architecture

The framework utilizes a multi-teacher hierarchical progressive distillation strategy:

• Teacher Models: Two frozen, pre-trained models—FACFormer (Frame-wise Average Convolutional Former) and ActionFormer—serve as teachers. They generate soft labels to guide the student.
• Student Model (LCT-KD): A compressed student network comprising two core modules:
  1. FACCFM (Frame-wise Average Convolutional Compressed Feature Module): Designed for local temporal modeling. It combines frame-by-frame convolution, average convolution, and residual connections, utilizing grouped convolutions and directional pruning to reduce parameters.
  2. CMAFM (Compressed Mixed-Attention Feature Module): Designed for global context. It employs a hybrid of global sparse attention and local window attention to capture long-range dependencies with minimal parameters.
• Feature Extraction: Input video clips are processed by an I3D backbone to extract spatio-temporal features, which are then down-sampled and fed into the student modules.

2.2 Adaptive Parameter Compression

To achieve lightweighting without significant accuracy loss, the paper introduces a Small-scale Parameter Importance Assessment Network (SAN):

• Dynamic Pruning: The SAN is a lightweight residual MLP that evaluates parameter importance during training based on both gradient magnitude and parameter magnitude.
• Differentiable Masking: Using a differentiable joint mask training strategy, the SAN generates pruning masks dynamically. It retains the top-k parameters (based on a preset retention ratio, e.g., 50%) while discarding less important ones.
• Loss Function: The training objective combines task loss, distillation loss (from entropy-weighted teacher soft labels), and an importance loss that aligns the student's pruning decisions with the teacher's knowledge.

2.3 Training Strategy

The training follows a hierarchical progressive distillation process:

1. Shallow Distillation: The FACCFM module is trained first, guided by the teacher's soft labels.
2. Deep Distillation: The CMAFM module is trained next, with the FACCFM frozen. Both teacher models guide the student.
3. Fine-tuning: A unified fine-tuning stage optimizes the entire student model, allowing all parameters to adapt while maintaining the compressed structure.

3. Key Contributions

The paper claims the following contributions:

• Novel Lightweight Architecture: The proposal of the LCT-KD model, which effectively combines the local adaptability of CNNs with the global feature extraction of Transformers, specifically tailored for temporal action detection.
• Multi-Teacher Hierarchical Distillation: A method that uses FACFormer and ActionFormer as teachers to improve the student model's generalization ability and recall rate while maintaining high accuracy.
• Adaptive Compression Mechanism: The introduction of the SAN, which utilizes adaptive evaluation parameters and dynamic hybrid compression. This allows for directional pruning that balances memory consumption and model performance more effectively than static pruning methods.
• Efficient Feature Modules: The design of FACCFM and CMAFM, which significantly reduce parameter counts through grouped convolutions and sparse attention mechanisms without sacrificing feature modeling capabilities.

4. Experimental Results

The model was evaluated on the THUMOS14 and ActivityNet1.3 datasets, with performance measured by mean Average Precision (mAP) across Temporal Intersection over Union (TIoU) thresholds from 0.3 to 0.7.

4.1 Performance on THUMOS14

• Accuracy: The LCT-KD model achieved an average mAP of 65.90%.
• Efficiency: The model operates with only 26.58M parameters, representing a 47.05% reduction compared to the uncompressed FACFormer teacher (50.20M parameters).
• Trade-off: The accuracy drop from the uncompressed teacher (67.13%) to the compressed student (65.90%) was minimal (1.23%), demonstrating effective knowledge transfer.
• Speed: The model achieved an inference speed of 65 FPS, outperforming the FACFormer (31 FPS) and ActionFormer (42 FPS).

4.2 Performance on ActivityNet1.3

• Accuracy: The model achieved an average mAP of 52.34%.
• Efficiency: The parameter count was reduced to 6.31M.

4.3 Ablation Studies

Ablation experiments confirmed that:

• Adding FACM and FADCM modules to the baseline improved accuracy but increased parameters.
• The SAN-guided compression successfully reduced parameters by nearly half while retaining 98.2% of the teacher model's accuracy.
• The combination of multi-teacher distillation and dynamic masking yielded the best balance of performance and efficiency.

5. Significance and Claims

The paper asserts that the proposed LCT-KD framework successfully addresses the challenge of deploying high-accuracy video action detection models on resource-constrained edge devices. By integrating knowledge distillation with adaptive parameter compression, the model achieves a "sweet spot" where it maintains state-of-the-art accuracy while significantly reducing memory footprint and computational overhead.

The authors claim that this approach is particularly significant for practical applications in surveillance, where transmitting redundant frames is inefficient and where edge devices often lack the capacity to run heavy, parameter-rich models. The study validates that effective compression does not necessarily require a proportional sacrifice in accuracy, provided that the compression is guided by robust knowledge transfer mechanisms.

Limitations and Future Work:
The authors modestly note that while the model improves accuracy, performance in certain complex scenarios remains limited. They suggest that future work could explore self-supervised learning to enhance cross-domain recognition and investigate the integration of large-scale models to further augment representational capacity within memory-constrained environments.