Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrieval

1. Problem Statement: Noisy Triplet Correspondence (NTC)

Composed Image Retrieval (CIR) aims to retrieve a target image using a reference image and a modification text. However, real-world training data suffers from Noisy Triplet Correspondence (NTC), where the relationship between the reference, text, and target is erroneous or ambiguous.

• The Core Challenge: Existing robust learning methods rely on the "Small-Loss Hypothesis" (assuming clean samples have low loss and noisy samples have high loss). This fails in CIR because NTC noise often manifests as "partial matches" (e.g., a target image that is semantically related but not a perfect match, such as a T-shirt when the text asks for a "short-sleeved shirt").
• The Vicious Cycle: Because partial matches can yield low losses due to superficial semantic overlap, models relying on self-generated confidence (e.g., via Gaussian Mixture Models) misclassify these noisy samples as clean. This leads to "representation pollution," where the model learns incorrect associations, further degrading its ability to distinguish noise, creating a self-dependent vicious cycle.
• The Dilemma: While Multimodal Large Language Models (MLLMs) possess the semantic understanding to act as perfect "arbiters" for noise, their high inference cost prevents their use during online training.

2. Methodology: The Air-Know Framework

The authors propose Air-Know, a novel three-phase decoupled paradigm that separates the "Arbiter" (noise detector) from the "Learner" (retrieval model) to break the self-dependency cycle.

Phase 1: External Prior Arbitration (EPA)

• Goal: Construct a high-precision, small-scale Anchor Dataset ($D_{anchor}$) offline.
• Mechanism: Utilizes an MLLM (e.g., GPT-4o) as an offline expert. Instead of simple classification, it employs a three-stage cross-validation strategy:
  1. Deconstruct Inputs: Independently analyze the reference image, target image, and modification text to establish objective factual anchors.
  2. Compare & Reason: Infer the actual visual change ($\Delta T_I$) between images and cross-validate it against the text instruction to diagnose logical contradictions (e.g., mismatched reference or text).
  3. Judge & Conclude: Output a binary label (Clean/Noisy) with a reasoning chain.
• Outcome: A reliable, small dataset of labeled triplets free from the "small-loss" bias.

Phase 2: Expert-Knowledge Internalization (EKI)

• Goal: Train a lightweight proxy network to internalize the expert's discriminative logic without using the expensive MLLM online.
• Mechanism:
  • Input Engineering: Constructs a Geometric Deconstruction Vector (GDV) combining query features, target features, their difference, and their interaction.
  • Bayesian Geometric Separator: Uses a lightweight Multi-Layer Perceptron (MLP) trained via Variational Inference (VI). Instead of point estimation, it learns a posterior distribution $p(W|D_{anchor})$ to model uncertainty.
  • Objective: Maximizes the Evidence Lower Bound (ELBO), comprising a reconstruction term (fitting the anchor dataset) and a prior matching term (regularization).
  • Output: During inference, the EKI module outputs a matching confidence score ($\hat{c}$) for every training sample, acting as a robust, decoupled arbiter.

Phase 3: Dual-Stream Reconciliation (DSR)

• Goal: Utilize the EKI confidence to guide the main representation model training.
• Mechanism: The confidence score $\hat{c}$ acts as a dynamic gating signal to split training into two streams:
  1. Clean Alignment Stream: High-confidence samples ($\hat{c} \approx 1$) are used to optimize the model via a Robust Contrastive Loss, focusing on aligning clean query-target pairs.
  2. Feedback Reconciliation Stream: Low-confidence samples ($\hat{c} \approx 0$) that exhibit high similarity (hard negatives/partial matches) are routed here. A specific loss function penalizes the model for assigning high similarity to these noisy pairs, effectively "de-polluting" the representation space.
• Result: The base model is calibrated online by the independent EKI arbiter, preventing representation pollution.

3. Key Contributions

1. Novel Decoupled Paradigm: Air-Know is the first framework to successfully decouple the arbiter and learner in CIR under NTC settings, breaking the self-dependent vicious cycle caused by the failure of the small-loss hypothesis.
2. Expert-Knowledge Internalization: Introduces a Bayesian proxy that efficiently internalizes complex MLLM reasoning logic, providing reliable confidence scores without online MLLM overhead.
3. Dual-Stream Reconciliation: Proposes a mechanism to simultaneously optimize clean alignment and correct representation pollution via a feedback stream, specifically targeting "partial match" noise.
4. State-of-the-Art Performance: Demonstrates significant improvements over existing SOTA methods (e.g., TME, HABIT) on standard benchmarks.

4. Experimental Results

The method was evaluated on FashionIQ and CIRR datasets under varying noise ratios (20%, 50%, 80%).

• Robustness: Air-Know significantly outperforms ordinary methods (SPRC) and other robust baselines (TME, HABIT) as noise levels increase. For example, on FashionIQ with 80% noise, Air-Know achieves an average R@10 of 49.23%, compared to TME's 46.67% and SPRC's 33.51%.
• Traditional Performance: While slightly lower than some baselines in noise-free (0%) settings due to its specialized design for robustness, it remains competitive.
• Ablation Studies:
  • Removing the EPA module or the cross-validation steps drastically reduces performance, confirming the necessity of high-quality external priors.
  • Removing the Bayesian uncertainty modeling (Dropout) or the geometric features (GDV) leads to significant drops, proving the importance of probabilistic boundary modeling.
  • Removing the Dual-Stream structure causes catastrophic failure, highlighting the need for both clean alignment and feedback reconciliation.
• Efficiency: Despite the complex architecture, Air-Know is computationally efficient. It achieves a 3x training speedup compared to the robust baseline TME and has a 12.4x faster inference time because the EKI module is discarded during testing.

5. Significance

This paper addresses a critical bottleneck in multimodal retrieval: the inability of current models to handle semantic ambiguity in noisy data. By shifting from a self-supervised noise detection approach (which fails on partial matches) to an external-expert-calibrated approach, Air-Know provides a robust solution for training high-quality CIR models in real-world, noisy environments. The proposed "Expert-Proxy-Diversion" paradigm offers a generalizable strategy for other tasks where reliable ground truth is scarce or expensive to obtain, and where noise exhibits complex semantic structures.