💬 NLP

Source-Modality Monitoring in Vision-Language Models

This paper investigates "source-modality monitoring"—the ability of vision-language models to identify whether information originates from text or images—finding that models rely on both syntactic and semantic signals to solve this binding problem, with semantic cues becoming more dominant when modalities are highly distinct.

Original authors: Etha Tianze Hua, Tian Yun, Ellie Pavlick

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Etha Tianze Hua, Tian Yun, Ellie Pavlick

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a game of "Telephone" with a very smart robot.

In this game, you show the robot a picture of a cat wearing a tiny hat, but you also hand it a piece of paper that says, "A dog is running in the park." Then, you ask the robot: "What is in the picture?"

If the robot is smart, it will ignore the paper and tell you about the cat. If it gets confused, it might tell you about the dog. This ability to know where a piece of information came from—whether it was seen in an image or read in text—is what researchers call Source-Modality Monitoring.

This paper investigates how modern AI models (Vision-Language Models) keep their "eyes" and "ears" straight.

The Core Mystery: How does the AI know?

The researchers wanted to know: When an AI identifies a source, is it following a strict rulebook, or is it just "feeling" the vibe of the data?

To understand this, they used two different metaphors for how the AI might be working:

1. The "Label Maker" Approach (Syntactic/Symbolic)

Imagine every piece of information comes with a bright neon sticker. The image has a sticker that says [IMAGE] and the text has a sticker that says [TEXT].

The Theory: The AI is just a very fast clerk. It sees the question "What is in the image?" looks for the sticker that says [IMAGE], and reads whatever is under it. It doesn't care what the content actually is; it just follows the labels.

2. The "Vibe Check" Approach (Semantic/Distributional)

Imagine there are no stickers. However, images "feel" different from text. An image is a collection of colorful pixels, while text is a sequence of structured words.

The Theory: Even without stickers, the AI can "smell" the difference. It thinks, "This chunk of data feels like a picture, and that chunk feels like a sentence," and it uses that intuition to answer your question.

What the Researchers Found

The researchers performed "surgery" on the AI's brain to see which method it actually uses. They tried three main experiments:

Experiment A: The "Fake Label" Test
They replaced the real stickers with nonsense labels like [DAX] and [WUG].

The Result: The AI was still able to tell them apart! This proves the AI isn't just a mindless clerk following stickers; it actually has the capacity to learn abstract rules.

Experiment B: The "Identity Theft" Test (The most important part!)
They took the stickers and swapped them. They put the [TEXT] sticker on the image and the [IMAGE] sticker on the text.

The Result: The AI didn't get totally fooled! Even though the "labels" were lying, the AI mostly ignored the fake stickers and went with its "gut feeling" (the distributional vibe). It realized, "Wait, this sticker says 'Text,' but this looks like a picture of a cat. I'll trust my eyes."

Experiment C: The "Brain Hack" (The Learned Intervention)
Finally, they used a computer program to mathematically "nudge" the AI's internal thoughts, trying to force it to misidentify the source.

The Result: They found that the "stickers" (the symbolic markers) are actually very powerful. While the AI has a "gut feeling," the stickers act like a GPS signal that helps the AI navigate much more accurately. If you mess with the stickers, the AI's ability to stay on track drops significantly.

Why does this matter?

As we move toward a world of AI Agents—AI that can see your screen, read your emails, and watch your video calls—this becomes a huge deal.

If you tell an AI agent, "Tell me what was said in yesterday's meeting," the AI needs to be able to distinguish between the audio of the meeting, the transcript of the meeting, and the slides shown during the meeting.

If the AI can't reliably "monitor its sources," it might accidentally tell you that a person said something that was actually just written on a slide behind them. This research helps us understand how to build AI that is more reliable, more robust, and less likely to get its "eyes" and "ears" crossed.

Technical Summary: Source-Modality Monitoring in Vision-Language Models

1. Problem Statement

As AI systems evolve into multimodal agents, they must be able to track and communicate the origin of information (e.g., "the image says..." vs. "the text says..."). This capability is termed source-modality monitoring. The authors frame this as a specific instance of the binding problem: the ability of a model to associate specific content (pixels or words) with an abstract role (image or text) and retrieve it based on a query.

The core research question is whether Vision-Language Models (VLMs) rely on syntactic signals (explicit symbolic markers like <image> tokens) or semantic/distributional signals (the inherent nature of the data itself) to perform this binding.

2. Methodology

The researchers designed a series of controlled experiments to decouple these two signals:

Target-Modality Retrieval Task: The authors constructed a dataset of "inconsistent" input pairs where the image and the caption describe different scenes. The model is prompted to retrieve information from one specific source (e.g., "What is in the image?"). Success is measured by selectivity—the tendency to correctly pick the target modality over the non-target one.
Symbolic vs. Distributional Probing:
- Purely Symbolic Task: Replacing "image" and "text" with arbitrary labels (e.g., "Dax" and "Wug") to see if models can bind content to abstract symbols without semantic cues.
- Structural Perturbations: Systematically manipulating the symbolic markers by removing them or swapping them (e.g., putting the <image> tag around the text) to see if performance collapses.
- Linear Probing: Testing if image and text tokens are linearly separable in the embedding space before any contextual processing.
Mechanistic Interventions:
- Freeze-Remove Intervention: To see if marker information is "written" into the content tokens during processing, the authors patched contextualized content activations from a "clean" run into a "corrupt" run where markers were removed.
- Learned Vector Intervention ( $\delta$ ): They applied optimized trainable vectors to either marker tokens or content tokens at different layer depths to see which could more effectively induce "misattribution" (forcing the model to report the wrong modality).

3. Key Contributions

Definition of Source-Modality Monitoring: A formal framework for evaluating how VLMs attribute information to specific input modalities.
Hybrid Binding Mechanism: The discovery that VLMs use a dual-signal approach: they use symbolic markers to contextualize content, but rely heavily on the inherent distributional differences of the modalities when markers are absent or misleading.
Asymmetric Robustness: Evidence that models are more robust at binding "image" content (which is inherently distinct) than "text" content (which is harder to distinguish from other text spans).
Mechanistic Insight: Demonstration that symbolic marker information is propagated into the hidden states of downstream content tokens during the model's forward pass.

4. Key Results

Performance: Most modern VLMs (e.g., Qwen2.5-VL, Gemma-3) perform well on the retrieval task. Models with explicit wrapper tokens (e.g., <image>) generally outperform those without.
Symbolic vs. Semantic:
- Models can perform purely symbolic binding (using "Dax/Wug" labels).
- However, when symbolic markers are swapped, models do not follow the incorrect markers; instead, they follow the distributional signals (the actual content), proving semantic signals often outweigh syntactic ones.
- When markers are removed, selectivity drops but does not hit zero, confirming that the model can still "sense" the modality from the content itself.
The "Document" Effect: Models struggle more with "micromodalities." For example, they can easily bind "text," but require explicit markers to recognize a specific text span as a "document."
Intervention Success: Learned interventions on marker tokens were significantly more effective at inducing misattribution than interventions on content tokens, suggesting markers are a powerful "causal handle" for modality identity.

5. Significance

This work is significant for the development of multimodal AI agents. If agents are to interact with complex, multi-source environments (e.g., "refer to the discussion from yesterday's meeting"), they cannot rely solely on explicit tags. The findings suggest that while current models are capable of using both signals, their reliance on distributional cues makes them susceptible to confusion when modalities are not clearly demarcated. This highlights a need for better "world models" that allow agents to ground linguistic references to the origins of information more robustly.