ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models

1. Problem Statement

Large Vision-Language Models (LVLMs) have achieved significant progress in multimodal tasks but suffer from two critical reliability issues:

• Hallucinations: LVLMs frequently generate non-existent visual content due to intrinsic model errors, biased priors, or learned co-occurrence patterns. This is particularly dangerous in high-stakes domains like medical diagnosis and remote sensing.
• Adversarial Vulnerability: LVLMs are susceptible to external adversarial attacks (e.g., imperceptible image perturbations) that cause confident but incorrect predictions. These attacks are often transferable across different model architectures.

Existing solutions are limited because they often require:

• Expensive retraining or adversarial training.
• Access to internal model weights (which is unavailable for many proprietary or frozen models).
• Static inference pipelines that cannot adapt to unexpected failures or inconsistencies.

2. Methodology: The ORCA Framework

The authors propose ORCA (Observe–Reason–Critique–Act), an agentic reasoning framework designed to operate entirely at inference time without modifying model weights or requiring additional training data.

Core Architecture

ORCA functions as a ReAct-style agent that orchestrates a plug-in baseline LVLM and a suite of lightweight vision models (≤3B parameters, e.g., DETR, Paligemma, BLIP). The agent interacts with these tools solely via text, never accessing raw images or internal weights.

The Observe–Reason–Critique–Act Loop

The framework operates through an iterative loop defined by three key variables: $M$ (number of visual models), $N$ (number of evidential queries per iteration), and $K$ (maximum iterations).

1. Observe (Initial Evidence Querying):
   • Upon receiving a user query, the agent queries the baseline LVLM and $M-1$ other vision models to gather initial evidence (e.g., captions, object detections).
2. Reason (Per-Response Reasoning):
   • The agent evaluates each model's response individually in the text domain to check for alignment with the user query.
3. Critique (Collective Decision-Making & Consistency Check):
   • The agent aggregates responses to check for semantic agreement.
   • Symbolic Consistency Check: If answers are consistent, the loop terminates. If inconsistencies are detected (e.g., an object detector says "no person" while the LVLM says "person"), the agent enters the reasoning loop. This check uses rule-based templates encoding human knowledge about tool strengths/weaknesses.
4. Act (Evidential Query Generation & Cross-Model Querying):
   • To resolve inconsistencies, the agent generates attribute-guided evidential queries (e.g., "What objects are wearing a brown shirt?" if the LVLM claimed a person was wearing one).
   • These $N$ queries are dispatched in parallel to all $M$ models, creating a diverse evidence pool ($M \times N$ responses).
   • The loop repeats until consistency is reached or the maximum iteration count $K$ is exceeded.

Key Features

• Model Agnostic: Works with any pretrained LVLM and vision tool.
• Traceable: All intermediate reasoning steps, critiques, and decisions are logged, providing an auditable decision trail.
• Emergent Robustness: By leveraging architectural diversity and forcing cross-validation, ORCA naturally resists adversarial perturbations that might fool a single model.

3. Key Contributions

1. Novel Framework: Introduction of ORCA, a training-free, inference-time agentic framework that reduces hallucinations and improves adversarial robustness through structured, iterative reasoning.
2. Zero-Access Operation: Unlike prior methods requiring fine-tuning or internal model access, ORCA operates purely via external API/text interactions with a suite of small vision models.
3. Dual Defense Mechanism: Demonstrates that the framework simultaneously mitigates intrinsic hallucinations and provides emergent robustness against external adversarial attacks without specific adversarial training.
4. Auditable AI: Provides transparent reasoning traces, enabling human verification of the decision-making process.

4. Experimental Results

The authors evaluated ORCA on three LVLMs (mPLUG-Owl, MiniGPT-4, Qwen-VL-Chat) across multiple benchmarks and attack scenarios.

Hallucination Mitigation (Clean Images)

• POPE Benchmark: ORCA improved standalone LVLM performance by +3.64% to +40.67% across Random, Popular, and Concurrence subsets.
  • Example: mPLUG-Owl's accuracy on the Random subset jumped from 54.00% to 94.67%.
• MME Benchmark: ORCA achieved a perfect Total Score of 180.00 across all tested LVLMs, significantly outperforming standalone baselines.
• AMBER Benchmark (Captioning): ORCA drastically reduced hallucination metrics (CHAIR, Hal, Cog).
  • Example: MiniGPT-4's CHAIR score dropped from 16.30% to 8.70%, and the proportion of responses with hallucinations (Hal) dropped from 84% to 26%.

Adversarial Robustness

• Attack Scenarios: Tested against Adversarial Illusions (white-box) and AttackVLM (black-box).
• Performance: ORCA maintained high accuracy under attack where standalone models failed.
  • On POPE, ORCA achieved an average accuracy gain of +20.11% across LVLMs under adversarial conditions.
  • Even under strong attacks, ORCA kept accuracy above 79% for all models, whereas standalone models dropped significantly.
• Defense Compatibility: When combined with traditional defenses (JPEG compression, feature squeezing), ORCA further improved performance, yielding gains of +1.20% to +48.00% over standalone models with defenses.

5. Significance and Conclusion

• Paradigm Shift: ORCA demonstrates that robustness in multimodal AI does not solely depend on model scale or adversarial training. Instead, architectural diversity combined with structured inconsistency-driven reasoning can effectively mitigate both intrinsic errors and external attacks.
• Practical Applicability: The framework is ideal for real-world, high-stakes applications where model internals are inaccessible, retraining is infeasible, and reliability is paramount.
• Trade-offs: The primary cost is increased inference latency (approx. 33× slower than standalone), which the authors argue is an acceptable trade-off for critical domains like medical diagnosis.
• Future Directions: The modular design allows for the integration of specialized tools and faster models, offering a scalable path toward reliable, interpretable, and robust multimodal systems.

In summary, ORCA represents a significant step forward in making LVLMs trustworthy by replacing "black box" generation with a transparent, iterative, and self-correcting reasoning process.