CompAgent: An Agentic Framework for Visual Compliance Verification

1. Problem Statement

Visual compliance verification is the process of ensuring digital content adheres to complex, evolving, and region-specific policy rules (e.g., regarding nudity, violence, hate speech, or self-harm).

• Limitations of Current Methods:
  • Traditional Deep Learning: Relies on task-specific classifiers trained on manually labeled datasets. These are costly to build, difficult to scale, and lack generalizability when policies change.
  • Direct MLLM Prompting: While Multimodal Large Language Models (MLLMs) possess broad knowledge, they struggle with fine-grained visual details, structured reasoning over complex policy clauses, and often fail to apply rules consistently without specialized guidance.
• The Gap: There is a need for a system that is adaptable to changing policies, requires no retraining, and can reason over fine-grained visual evidence to make accurate compliance decisions.

2. Methodology: The CompAgent Framework

CompAgent is the first agentic framework designed specifically for visual compliance verification. Instead of a monolithic model, it decomposes the task into a multi-step workflow involving a Planning Agent, a Tool Suite, and a Compliance Verification Agent (CVAgent).

A. Architecture Components

1. Planning Agent:

   • Role: Orchestrates the evidence collection process using a ReAct (Reasoning and Acting) loop.
   • Mechanism: It receives the input image and the compliance policy. It analyzes which policy clauses remain unverified and dynamically selects the most appropriate tools from the Tool Suite.
   • Logic: It does not use a fixed routing table. Instead, it reasons in-context based on three factors: (1) unverified policy clauses, (2) tool capabilities/limitations, and (3) accumulated evidence.
   • Termination: The loop continues until the agent decides to CONCLUDE, passing the full state to the CVAgent.

2. Tool Suite (Modular Evidence Sources):

   • A collection of off-the-shelf, black-box tools that do not require retraining.
   • Summarization Tools: Generate natural language scene descriptions.
   • Content Detection Tools: Face detection (attributes/age), Object detection (bounding boxes), Text detection (OCR), and Content Moderation (safety flags).
   • Specialized Compliance Tools: Includes LlavaGuard (safety ratings/rationales), Safe-CLIP (zero-shot toxic detection), and ICM Assistant (rule-based explainable assessments).

3. Compliance Verification Agent (CVAgent):

   • Role: Performs the final judgment.
   • Input: The original image, the full compliance policy, and the complete chain of evidence (tool outputs) gathered by the Planning Agent.
   • Process: It cross-validates tool outputs against the image and policy. For example, if a summarization tool describes a benign scene but a moderation tool flags violence, the CVAgent re-examines the image to adjudicate.
   • Output: A structured assessment including a binary Rating (Safe/Unsafe), a Category, and a detailed Rationale.

B. Workflow

1. Input: Image + Policy.
2. Planning: The agent iteratively calls tools (e.g., "Call Face Detection because the policy mentions age restrictions").
3. Evidence Accumulation: Tool outputs (JSON metadata, confidence scores) are appended to the state.
4. Verification: The CVAgent synthesizes all data to generate the final compliance decision.

3. Key Contributions

• Agentic Framework: Introduces the first agent-based system for visual compliance that dynamically orchestrates specialized vision tools based on policy requirements, moving beyond static classifiers or simple prompting.
• Policy-Aware Agents: Develops agents capable of parsing complex policies to identify applicable rules, plan tool selection, and perform joint reasoning, significantly improving precision and interpretability.
• Training-Free & Cost-Efficient: The system requires no labeled training data and no fine-tuning. It leverages pre-existing tools and LLM reasoning, making it instantly adaptable to new or evolving compliance policies.
• Explainability: The framework produces detailed rationales linking specific visual evidence (e.g., "detected knife," "text reading 'X'") to specific policy clauses, offering transparency often missing in black-box models.

4. Experimental Results

The authors evaluated CompAgent on two public benchmarks: LlavaGuard (1,290 images) and UnsafeBench (2,037 images).

• Performance Metrics:
  • LlavaGuard: Achieved an Unsafe F1 score of 0.93, outperforming the best zero-shot MLLM (0.61) and the fine-tuned LlavaGuard baseline (0.91).
  • UnsafeBench: Achieved an Unsafe F1 score of 0.76, a 10% improvement over the state-of-the-art (Llama 4 Maverick at 0.71) and significantly outperforming specialized classifiers.
• Policy Responsiveness:
  • CompAgent demonstrated superior adaptability to policy nuances (Policy Exception Rate of 93.98% vs. 93.38% for LlavaGuard).
  • It excelled in categories requiring contextual reasoning, such as Self-Harm (distinguishing educational content from harmful promotion) and Animal Cruelty.
• Tool Analysis:
  • The system showed dynamic routing: Unsafe images escalated to specialized safety tools (e.g., ICM Assistant usage increased from 28.8% to 68.0% for unsafe cases), while safe cases often terminated earlier.
  • Ablation Studies: Removing specific tool sets (e.g., Summarization or Object Detection) caused significant performance drops (F1 dropping to ~0.68–0.71), confirming the necessity of the multi-modal evidence chain.
• Model Sensitivity: Interestingly, the framework performed best with Claude Sonnet 3.5 v2, suggesting that newer, larger models do not always yield better results for agentic orchestration tasks compared to slightly older, more balanced models.

5. Significance and Impact

• Scalability: By decoupling the reasoning engine from the visual analysis tools, CompAgent offers a scalable solution that can handle thousands of images daily without the prohibitive costs of retraining deep learning models for every policy update.
• Real-World Applicability: The framework addresses the critical need for automated content moderation in media, entertainment, and advertising, where policies vary by region and culture.
• Future Directions: While currently focused on single images, the modular design lays the foundation for extending visual compliance verification to video content, where temporal reasoning and dynamic tool selection will be even more critical.

In conclusion, CompAgent represents a paradigm shift from static, training-heavy models to dynamic, tool-augmented agentic systems, proving that combining the reasoning capabilities of LLMs with specialized computer vision tools yields superior accuracy, adaptability, and explainability in visual compliance verification.