Technical Summary: COPRA – Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection

1. Problem Definition and Motivation

Video Anomaly Detection (VAD) aims to identify rare and unexpected events in long videos, often under weak supervision where only video-level labels are available. While Vision-Language Models (VLMs) have shown promise in VAD by leveraging semantic priors and offering interpretability, the paper identifies two critical, underexplored limitations in existing approaches:

1. The Shared Parameter Bottleneck: Current VLM-based VAD methods typically rely on a static, shared parameter space (e.g., fixed prompts or globally fine-tuned weights) after training. This forces heterogeneous anomaly types (e.g., traffic accidents vs. shoplifting) and diverse scene contexts into a single compromised solution. Consequently, these models overfit to training distributions and fail to generalize to unseen domains or novel anomaly types.
2. Training-Inference Mismatch: Existing pipelines often train on sparsely sampled frames from long videos with weak video-level supervision but perform inference on densely sampled sliding windows for fine-grained localization. This discrepancy leads to inconsistencies where parameters learned from sparse inputs are applied to dense local decisions, degrading performance under variations in motion, viewpoint, and semantics.

2. Methodology: The COPRA Framework

The authors propose COPRA (Conditional PaRameter Adaptation), a framework that reformulates VAD as an instance-conditioned adaptation problem. Instead of learning a single static update, COPRA dynamically generates input-specific parameter updates for each video segment, modulating a largely frozen VLM on the fly.

Core Architecture

• Instance-Conditioned Parameter Generation: COPRA employs a lightweight generator network, $g_{\phi_2}$, which takes a compact visual conditioning signal derived from the input video $V$ and predicts a specific parameter update $\Delta\theta(V)$.
• Visual Conditioning: The input video is uniformly sampled (e.g., $K=8$ frames) and processed by a frozen vision encoder (a submodule of the VLM) to produce visual tokens. These tokens serve as the sole conditioning signal for the generator.
• LoRA Weight Synthesis: The generator is a Transformer that maps visual tokens into tokenized, instance-conditioned LoRA (Low-Rank Adaptation) weights. These weights are injected as residual updates into the frozen VLM backbone (specifically targeting query-key-value projections, gating modules, and output transformations).
• Hybrid Adaptation: The generated LoRA parameters are initialized with a learnable global latent prior ($p_{global}$), combining dataset-level adaptation patterns with input-specific residuals. This allows the model to retain general knowledge while specializing for the current instance.

Training via Reinforcement Learning

COPRA is trained end-to-end using Group Relative Policy Optimization (GRPO) under video-level supervision:

• Policy: The generator $g_{\phi_2}$ acts as a policy that maps visual signals to parameter updates.
• Reward Design: The system uses binary rewards based on the correctness of the final video-level classification (Normal vs. Anomaly) and format compliance.
• Mechanism: For a given video, the model generates multiple response sequences using the adapted parameters. The advantage is computed via group-wise normalization of these rewards. This encourages the generator to produce parameter updates that lead to correct predictions, effectively learning a "weight space generation" mechanism without requiring explicit supervision on the weights themselves.

Flexible Temporal Adaptation

To address the training-inference mismatch, COPRA supports flexible sampling strategies. During inference, videos are partitioned into overlapping segments. The framework can generate parameters for specific temporal chunks (e.g., 120-second chunks) and reuse them for scoring local segments within that context, enabling context-aware parameter sharing while maintaining temporal granularity.

3. Key Contributions

1. Identification of the Shared Parameter Bottleneck: The paper articulates how static, shared adaptations in VLM-based VAD limit generalization by forcing heterogeneous requirements into a single compromise.
2. Instance-Conditioned Adaptation Framework: COPRA is presented as the first framework to formulate VAD as an instance-conditioned problem, dynamically specializing VLM behavior for each video segment via generated LoRA weights.
3. RL-Driven Parameter Generation: The introduction of a conditional parameter generation framework trained with reinforcement learning (GRPO) under weak supervision, enabling dynamic, per-input adaptation without full fine-tuning.
4. Robust Generalization: The framework is shown to alleviate training-inference mismatch and reduce cross-domain interference, improving robustness across diverse anomaly patterns and temporal dynamics.

4. Experimental Results

The authors evaluated COPRA on standard VAD benchmarks and cross-domain tasks:

• In-Domain Performance: On UCF-Crime and XD-Violence, COPRA consistently outperformed frozen VLM baselines and strong static adaptation methods (e.g., VERA).
  • On UCF-Crime, COPRA improved the AUC of InternVL2-8B from 84.31% to 87.14%.
  • On XD-Violence, it improved the AP of InternVL2-8B from 74.32% to 76.52%.
  • Ablation studies confirmed that instance-conditioned adaptation significantly outperforms static GRPO-based adaptation.
• Cross-Domain Generalization: Trained solely on UCF-Crime, COPRA was evaluated zero-shot on:
  • VRU-Accident: Improved dense captioning semantic metrics (SPICE, METEOR) and multiple-choice VideoQA accuracy.
  • HIVAU-70K: Enhanced natural language explanation generation across clip, event, and video levels.
• Temporal Adaptation: Experiments showed that generating parameters for larger temporal chunks (e.g., 120s) and reusing them for local segments yielded the best performance, validating the flexibility of the temporal adaptation strategy.
• Qualitative Analysis: t-SNE visualizations demonstrated that COPRA produces more discriminative embeddings, better separating normal and anomalous videos compared to the baseline. Generated descriptions were also more coherent and focused on anomaly-relevant cues.

5. Significance and Claims

The paper claims that COPRA provides an effective weight space generation mechanism for foundation models. By moving away from static, shared parameters toward dynamic, input-conditioned adaptation, COPRA enables:

• Scalability: Adapting large frozen models to specific tasks without full retraining.
• Adaptability: Handling significant distribution shifts and novel anomaly types by tailoring the decision boundary for each input.
• Context-Awareness: Bridging the gap between sparse training supervision and dense inference requirements.

The authors acknowledge limitations, including the lack of theoretical grounding for the generated weights, limited interpretability of the LoRA weights themselves, and the computational overhead introduced by the GRPO training process (specifically repeated response sampling). They suggest these areas warrant future investigation.

Code Availability: The code is released at https://github.com/THE-MALT-LAB/COPRA.