The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering

=== DRAFT ===
Here is a detailed technical summary of the paper "The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering."

Problem Definition

The paper addresses the challenge of cross-domain egocentric video understanding. While existing benchmarks for egocentric vision primarily focus on common daily-life scenarios (e.g., cooking, household tasks), real-world applications often require interpreting specialized domains with distinct visual appearances, terminology, and operational risks. The authors identify a significant domain gap where models proficient in everyday activities fail to recognize surgical instruments, understand industrial assembly, anticipate extreme sports motions, or interpret animal-mounted footage.

To evaluate and advance the ability of multi-modal large language models (MLLMs) to generalize beyond these common scenarios, the paper introduces EgoCross, a benchmark for Cross-Domain Egocentric Video Question Answering (VQA). The task is formulated as a closed-ended multiple-choice question (CloseQA) problem: given an egocentric video clip, a natural language question, and four candidate answers, the model must select the correct option. The benchmark spans four target domains:

1. Surgery
2. Industrial Assembly
3. Extreme Sports
4. Animal Perspectives

Methodology: The Challenge Design

The paper details the organization of the First EgoCross Challenge held at the Third EgoVis Workshop (CVPR 2026). The challenge was structured around two distinct tracks to investigate complementary research questions regarding data and model constraints:

1. Source-Limited Track:

   • Constraints: Participants were restricted to the official baseline model (SFT-Qwen3VL, based on Qwen3-VL-4B) and a small, fixed support set of 80 multiple-choice QA samples (20 per domain).
   • Goal: To evaluate controlled, fair comparisons of adaptation methods under severe data and model resource limitations.
   • Prohibitions: No manual construction of target-domain training data was allowed.

2. Open-Source Track:

   • Constraints: Participants could use broader choices of models and additional public pretraining data.
   • Prohibitions: Manual collection or construction of target-domain training data was strictly prohibited.
   • Goal: To assess how much stronger architectures and broader pretraining resources can advance cross-domain VQA.

Baseline and Resources:
The organizers provided a baseline model, SFT-Qwen3VL, which was a Qwen3-VL-4B-Instruct model fine-tuned on the 80-sample support set using full-parameter supervised fine-tuning. The evaluation metric was average accuracy across 957 test question-answer pairs derived from 798 video clips.

Key Contributions

1. Benchmark Creation: The introduction of the first comprehensive testbed for cross-domain egocentric VQA, covering four challenging, non-daily domains with 957 QA pairs and 798 video clips.
2. Challenge Organization: The successful execution of a large-scale challenge with over 1,500 submissions from 130+ participants, establishing a new standard for evaluating domain adaptation in first-person vision.
3. Analysis of Winning Solutions: The paper provides a technical summary of the top-performing methods, highlighting that significant gains can be achieved through inference-time strategies rather than just heavy retraining.

Results

The challenge demonstrated that substantial performance gains are possible even under strict constraints, though domain-specific challenges remain.

• Participation: 38 teams in the Source-Limited Track and 19 teams in the Open-Source Track.
• Performance Gains: The winning team in the Source-Limited Track (DomainWiseInfer) improved upon the official baseline (46.08%) by 20.90 percentage points, achieving 66.98% overall accuracy.
• Top Performers:
  • DomainWiseInfer: Ranked 1st in both tracks (66.98% accuracy).
  • OmniEgo-R2: Ranked 2nd in both tracks (66.35% in Source-Limited, 66.77% in Open-Source).
  • Reflective Dialogue: Ranked 3rd in the Open-Source Track (65.94%).
  • TokenInj-RAA: Ranked 3rd in the Source-Limited Track (60.61%).
• Domain Variance: Performance varied significantly across domains. Animal perspective questions generally received the highest scores, while Surgery and Extreme Sports remained the most challenging for most teams, indicating that strong average performance does not guarantee uniform cross-domain generalization.

Summary of Winning Solutions

The paper highlights four key methodological approaches used by the top teams:

1. DomainWiseInfer (1st Place): A nearly training-free, domain-wise inference framework. Instead of a universal pipeline, it routes samples by domain and question type, applying specific strategies for input packaging, domain cueing, and answer extraction. It leverages the base model for animal/surgery domains and the SFT checkpoint for industry/sports, focusing on exposing relevant visual/temporal cues.
2. OmniEgo-R2 (2nd Place): A unified routed reasoning framework. It decomposes the task into Temporal Evidence Normalization, Capability-Oriented Routing (identifying if the task requires counting, localization, etc.), Role-Decomposed Reasoning (separating perception from decision-making), and Boundary-Aware Option Verification. It uses domain-specific semantic bases (e.g., tool-centric for surgery) within a shared architecture.
3. Reflective Dialogue (3rd Place, Open-Source): A training-free inference-time adaptation strategy. It transforms support examples into structured multi-turn conversations between a "Teacher" and a "Solver." These dialogues include correctness feedback, visual grounding, and contrastive error analysis. These constructed dialogues are retrieved and prepended to test questions as static context to guide the model.
4. TokenInj-RAA (3rd Place, Source-Limited): A modular framework combining timestamp-aware visual modeling (injecting temporal tokens into frames), parameter-efficient adaptation via LoRA, and retrieval-augmented reasoning (using external surgical knowledge like CholecT50 annotations) to improve temporal reasoning and specialized understanding.

Significance and Claims

The authors claim that the EgoCross Challenge and the resulting technical report serve as a useful technical reference for advancing cross-domain egocentric video understanding.

• Generalization vs. Specialization: The results demonstrate that while robust generalization across specialized egocentric domains remains challenging, significant improvements can be achieved through domain-aware inference, structured reasoning, reflective in-context adaptation, and temporal grounding.
• Efficiency: The success of nearly training-free methods (like DomainWiseInfer and Reflective Dialogue) suggests that carefully designed inference interfaces and reasoning pipelines can yield substantial gains without requiring larger models or extensive retraining.
• Community Resource: All resources, including the challenge data, baseline implementation, and code from the winning teams, are made publicly available to support continued research in this field.

The paper concludes modestly, stating that the challenge highlights the value of specific adaptation strategies while acknowledging that the gap between daily-life and specialized domain understanding is still a significant hurdle for current models.