💬 NLP

Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitration

This paper reveals that modality following in multimodal large language models is governed by a two-stage mechanism where shallow layers aggregate multimodal cues into instruction tokens as a latent buffer, while deep layers utilize a sparse subset of attention heads to selectively resolve modality arbitration based on user intent.

Original authors: Yu Zhang, Mufan Xu, Xuefeng Bai, Kehai Chen, Pengfei Zhang, Yang Xiang, Min Zhang

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yu Zhang, Mufan Xu, Xuefeng Bai, Kehai Chen, Pengfei Zhang, Yang Xiang, Min Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Black Box" Problem

Imagine you have a super-smart robot assistant (a Multimodal Large Language Model) that can see pictures and read text. You give it a picture of a cat and a text description of a dog, then you say, "Ignore the text, tell me about the picture."

If the robot works, it looks at the cat. If it fails, it might get confused and talk about the dog. For a long time, scientists didn't know how the robot made that decision inside its "brain." It was a black box. This paper opens that box to see exactly how the robot decides which information to trust.

The Main Discovery: The "Instruction Token" is the Boss

The researchers found that the robot doesn't just look at the picture and the text and then guess. Instead, there is a specific part of the robot's brain called the Instruction Token (the part that reads your command, like "Look at the picture").

Think of the Instruction Token as the Command Center or the Conductor of an orchestra.

The Musicians: The visual data (the picture) and the textual data (the story) are the musicians.
The Conductor: The Instruction Token is the conductor.

The paper reveals that all the information from the picture and the text flows first to the Conductor. The Conductor gathers all the clues, and then decides what the final answer should be. The final answer isn't made by the picture or the text directly; it's made by the Conductor after it has listened to everyone.

How the Decision Happens: The "Buffer" and the "Judge"

The researchers discovered that the robot's brain works in two distinct stages, like a factory assembly line:

1. The Shallow Layers (The "Waiting Room" or "Buffer")
In the early stages of processing, the robot is just gathering information. It's like a receptionist taking notes.

It listens to the picture.
It listens to the text.
It writes everything down in a "latent buffer" (a holding area).
At this stage, it hasn't decided which one to trust yet. It's just collecting data.

2. The Deep Layers (The "Judge" or "Arbitrator")
In the later, deeper stages of the brain, the robot acts like a Judge in a courtroom.

The Judge looks at the notes from the receptionist.
The Judge reads the instruction (e.g., "Trust the picture!").
The Judge makes a final ruling.
Crucial Finding: This decision happens inside the Instruction Token (the Judge's gavel) before the final answer is even written down.

The Secret Weapon: A Tiny Group of "Special Agents"

The most surprising finding is that this "Judge" isn't a giant, complex machine. It's actually run by a very small, specific team of workers.

Imagine a massive factory with 10,000 workers. The researchers found that only about 5% of these workers (specific attention heads) are actually responsible for making the decision on which modality to follow.

The "Special Agents": These few workers are the ones who actually listen to the instruction and say, "Okay, ignore the text, focus on the image."
The Rest: The other 95% of workers are just doing general tasks or helping with the gathering phase.

Proving It: The "Switch" Experiment

To prove these "Special Agents" were real, the researchers performed two experiments:

The "Silence" Test: They turned off (blocked) just those 5% of Special Agents.
- Result: The robot immediately forgot how to follow instructions. It got confused and started hallucinating or ignoring the user's command. However, its ability to just "see" or "read" normally stayed the same. It was like taking the conductor out of the orchestra; the musicians could still play, but the music made no sense.
The "Volume" Test: They turned up the volume (amplified) on just those Special Agents.
- Result: When the robot failed to follow an instruction, turning up the volume on these specific agents fixed the problem about 60% of the time. It was like giving the Judge a louder gavel, forcing the decision to be made correctly.

Summary

This paper explains that when a multimodal AI follows an instruction:

It gathers all visual and text clues into a Command Center (the Instruction Token).
It first buffers (collects) the info, then arbitrates (decides) based on the instruction.
This decision is made by a tiny, specialized team of 5% of the brain's components.
If you mess with this tiny team, the robot stops following instructions, but if you boost them, you can fix mistakes.

This gives us a clear map of how these robots think, rather than just guessing.

Technical Summary: Dissecting the Mechanistic Dynamics of Modality Arbitration

Problem Statement

Multimodal Large Language Models (MLLMs) possess the capability of Multimodal Instruction Following (MIF), which requires them to selectively leverage specific modalities (e.g., visual vs. textual) based on user directives, particularly when modalities provide conflicting information. While MIF is fundamental for real-world applications like GUI navigation and embodied robotics, the internal decision-making mechanisms governing this "modality arbitration" remain a "black box." Existing interpretability research largely focuses on perception-centric aspects (encoding and retrieving visual information) or output-format compliance, leaving the specific dynamics of how models resolve cross-modal conflicts under instruction largely unexplored. This lack of mechanistic understanding hinders the diagnosis of model failures and the assurance of behavioral reliability.

Methodology

The authors investigate the internal mechanisms of modality following through an information flow perspective, utilizing a controlled diagnostic setting where visual and textual contexts support conflicting answers. The methodology comprises three primary analytical stages:

Attention Knockout Analysis with INSSD:
- The authors employ Attention Knockout Analysis to selectively block attention pathways between token sets (visual tokens $X_{vision}$ , text context $X_{ctx}$ , instruction tokens $X_{inst}$ , and generated tokens $X_g$ ).
- To quantify the impact of these interventions while suppressing noise from task-irrelevant variations, they propose Normalized Signed Structural Divergence (INSSD). This metric measures probability shifts within the decision subspace spanned by instruction-compliant and competing modalities. A negative INSSD shift indicates that the blocked pathway was critical for maintaining instruction compliance.
Latent Decision Tracking:
- Using Logit Lens techniques, the authors track the evolution of internal beliefs across layers.
- They introduce the Latent Decision Alignment Rate (LDAR) to measure the synchronization between the internal state of instruction tokens and the final output. This determines whether the modality decision is crystallized within the instruction tokens before propagating to generated tokens.
Component and Head-Level Attribution:
- The study decomposes the modulation of the modality arbitration margin ( $\Delta S$ ) into contributions from Attention and MLP layers using logit-difference attribution.
- Further granularity is achieved by analyzing individual attention heads to identify sparse subsets driving the arbitration process.
- Targeted Interventions: The authors perform causal interventions by blocking (zeroing output) and amplifying (scaling output) specific attention heads to validate their functional specificity.

Key Contributions

Identification of Instruction Tokens as Structural Anchors: The work establishes that instruction tokens serve as the primary aggregation site for cross-modal cues. Information routing analysis reveals that pathways from modal cues to instruction tokens are the most sensitive components for modality following, acting as a central locus for decision crystallization.
Mechanism of Layer-wise Transition: The paper characterizes a functional stratification within the Transformer architecture:
- Shallow Layers: Act as a "latent buffer," performing undifferentiated information transfer where multimodal cues are aggregated without enforcing selection.
- Deep Layers: Act as the "definitive arbitrator," selectively strengthening the instruction-compliant subspace to resolve modality conflicts.
Sparse Attention Head Specialization: The arbitration process is driven by a remarkably sparse subset of attention heads. While most heads are modality-specific, a small number of "shared" heads exist.
Validation via Causal Intervention: The study demonstrates that blocking only ~5% of the identified critical heads substantially degrades modality following while preserving general visual and language capabilities. Conversely, targeted amplification of these heads can restore modality-following performance on failed samples by up to ~60%.

Results

Information Routing: Cutting attention pathways from modal cues to generated tokens ( $X_{vision} \to X_g$ ) has marginal effects. In contrast, cutting pathways to instruction tokens ( $X_{vision} \to X_{inst}$ ) causes a pronounced negative shift in INSSD, confirming instruction tokens as the convergence site.
Decision Crystallization: LDAR analysis shows that while shallow layers exhibit chance-level alignment, deep layers show over 95% alignment between instruction token states and final outputs, indicating arbitration is resolved within instruction tokens.
Attention vs. MLP: Attention layers consistently exhibit a positive shift in the arbitration margin toward the instruction intent, whereas MLP layers do not show this consistent behavior, suggesting attention is the primary driver of instruction-aligned modulation.
Intervention Efficacy:
- Blocking: Removing the top 40 identified heads (approx. 5% of total) drops the Modality Following Ratio (MFR) by ~60% but leaves general benchmarks (VQA, MMLU, IFEval) largely unaffected.
- Amplification: Amplifying these heads on failure cases recovers ~60% of the lost performance. Excessive amplification (e.g., coefficient > 18) eventually destabilizes the model, collapsing performance.

Significance

The paper provides a mechanistic account of modality following, moving beyond the "black box" of MLLM decision-making. By identifying instruction tokens as structural anchors and revealing a transition from buffering to arbitration driven by sparse attention heads, the work offers a novel lens for understanding multimodal information utilization.

The authors claim these findings inform future efforts to:

Improve Multimodal Evidence Utilization: Understanding the specific pathways allows for more targeted interventions to enhance how MLLMs integrate conflicting evidence.
Design Efficient Architectures: Insights into instruction-anchor-mediated context integration could lead to context-compression strategies and architectural designs that reduce redundant computation in omni-modal models.
Enhance Safety and Reliability: By elucidating the structural foundation of arbitration, the work establishes a basis for developing more robust AI safeguards against modality interference and hallucinations.

The study emphasizes that these insights are derived from mechanistic dissection and causal intervention, providing a structural foundation rather than immediate application-specific solutions.