Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervision

1. Problem Statement

The paper addresses the challenge of automated video anomaly detection (VAD) in Closed-Circuit Television (CCTV) surveillance systems. The specific constraints and challenges include:

• Weak Supervision: Anomalies are labeled only at the clip level (the entire video is "normal" or "abnormal") without temporal boundaries or specific anomaly categories. This creates a "weakly supervised" learning scenario.
• Operational Constraints: Real-world deployments require low, predictable latency (real-time processing) and must operate on edge devices with limited computational capacity.
• Class Imbalance: Safety-critical incidents are rare compared to routine activities, leading to severe class imbalance.
• Contextual Ambiguity: An action may be normal in one context but anomalous in another, requiring models to understand semantic context rather than just visual patterns.
• The Gap: While Vision-Language Models (VLMs) show promise, there is a lack of controlled benchmarks comparing compact (small parameter count) VLMs against established methods under a unified protocol. It is unclear if parameter-efficient fine-tuning (PEFT) is necessary for these small models to perform effectively compared to training-free (zero-shot) approaches.

2. Methodology

The authors propose a unified evaluation protocol to benchmark compact VLMs for clip-level binary classification (Normal vs. Abnormal).

Dataset

• UCF-Crime: The sole dataset used, containing 1,900 clips (13 crime categories).
• Split: Official train/test split (1,610 training, 290 testing).
• Preprocessing: Videos are converted to RGB clips. Short videos (<60s) are sampled at 0.5 fps; longer videos are uniformly sampled to 32 frames. No audio or optical flow is used.

Models Evaluated

The study evaluates two capacity tiers of compact VLMs:

1. Tiny Models (2–4B parameters): InternVL3-2B, Qwen2.5-VL-3B, Gemma-3-4B.
2. Small Models (7–8B parameters): InternVL3-8B, Video-LLaVA-7B, Qwen2.5-VL-7B.

Prompting Strategies

Four distinct prompting regimes were tested uniformly across all models to isolate the effect of prompt design:

1. Basic Instruction: Simple binary decision request.
2. Zero-Shot Definition: High-level task framing without examples.
3. Few-Shot: Providing labeled exemplars to guide the model.
4. Chain-of-Thought (CoT): Explicit multi-step reasoning instructions.

Adaptation Strategy (PEFT)

• LoRA (Low-Rank Adaptation): The authors applied LoRA (Rank 8) to fine-tune the models on the UCF-Crime training set.
• Configuration: Base weights and tokenizers were frozen; only LoRA adapters were trained for one epoch using the LLaMA Factory framework.
• Comparison: Models were evaluated in two states: Training-Free (Zero-shot/In-context) and LoRA-Adapted.

Evaluation Metrics

• Accuracy, Precision, Recall, F1-Score, ROC-AUC: Standard classification metrics.
• Latency: Average wall-clock inference time per clip (critical for surveillance).
• Baselines: The study re-ran external state-of-the-art methods (LAVAD, VERA, PEL4VAD) on the same dataset split to ensure fair comparison.

3. Key Contributions

1. Unified Benchmarking Protocol: Established a standardized framework for comparing compact VLMs in surveillance, controlling for preprocessing, prompting, and runtime settings to eliminate "setup noise."
2. Empirical Evidence on PEFT: Demonstrated that parameter-efficient fine-tuning (LoRA) is not optional but essential for compact VLMs in this domain. Training-free small models performed poorly, while LoRA-adapted versions achieved state-of-the-art results.
3. Prompt Sensitivity Analysis: Revealed that complex prompts (Few-Shot, CoT) degrade performance for untuned compact models ("overprompting"), causing accuracy drops and latency spikes. However, LoRA-adapted models became robust to prompt variations, maintaining high performance across all prompt types.
4. Efficiency-Accuracy Trade-off: Showed that compact, LoRA-adapted models (e.g., 2B–4B parameters) can outperform larger general-purpose models (7B–8B) and complex multi-stage pipelines in both accuracy and latency.

4. Key Results

Performance of Untuned Models

• Poor Baseline: Without fine-tuning, compact VLMs struggled significantly. For instance, InternVL3-2B achieved an F1 score of only 0.014 with Few-Shot prompting.
• Overprompting: Adding reasoning (CoT) or examples (Few-Shot) to untuned models increased latency (e.g., InternVL3-2B CoT latency jumped from ~2.7s to ~11.4s) while often reducing accuracy.
• Best Untuned: The best untuned performance was Qwen2.5-VL-7B with Basic prompting (F1 ~0.90), but it still lagged behind adapted smaller models in efficiency.

Impact of LoRA Adaptation

• Transformative Gains: LoRA fine-tuning drastically improved performance.
  • Qwen2.5-VL-3B: F1 score jumped from 0.18 (untuned, Few-Shot) to ~0.90 (LoRA).
  • InternVL3-2B: F1 score improved from 0.01 to ~0.90.
• Latency Improvements: Contrary to expectations, LoRA adaptation often reduced inference latency (e.g., Qwen2.5-VL-3B Basic latency dropped from 5.09s to 2.83s) because the model required less "reasoning" to reach a correct decision.
• Prompt Robustness: LoRA-adapted models achieved consistent F1 scores (0.90–0.92) across all prompt types (Basic, Zero-Shot, Few-Shot, CoT), whereas untuned models were highly volatile.

Comparison with Baselines

• Superiority: The LoRA-adapted InternVL3-2B achieved an F1 of 0.916 and ROC-AUC of 0.919, outperforming established baselines like LAVAD (F1 0.76), VERA (F1 0.72), and PEL4VAD (F1 0.87).
• Efficiency: The adapted 2B model was significantly faster (2.77s/clip) than the 7-8B baselines and multi-stage pipelines.

5. Significance and Conclusion

The paper concludes that parameter-efficient fine-tuning is the critical enabler for deploying compact VLMs in real-world surveillance.

• Practical Implication: Surveillance systems do not need massive 70B+ models or complex multi-stage pipelines. A 2B–4B parameter model, when adapted via LoRA, offers a superior balance of high accuracy, low latency, and low computational cost.
• Design Guidance:
  • Avoid complex prompting (CoT/Few-Shot) for small, untuned models.
  • Use concise instructions (Basic/Zero-Shot) for LoRA-adapted models, as they are robust to prompt variations.
  • Prioritize domain-specific adaptation (LoRA) over architectural complexity.
• Future Outlook: The framework provides a transparent, reproducible foundation for future research into real-time, edge-deployable anomaly detection, with potential extensions to multimodal (audio/visual) integration.

In summary, the study shifts the paradigm from "bigger is better" to "adapted is better," proving that compact, fine-tuned VLMs are the most viable solution for efficient, reliable CCTV anomaly detection.