Mull-Tokens: Modality-Agnostic Latent Thinking

1. Problem Statement

Current Multimodal Large Language Models (MLLMs) struggle with complex spatial and visual reasoning tasks (e.g., solving puzzles, IQ tests, 3D spatial relationships). Existing approaches suffer from several limitations:

• Text-only Chain-of-Thought (CoT): While effective for verbal logic, text-only reasoning often "drifts" from visual inputs, failing to capture spatial causality or geometric constraints.
• Explicit Image Generation: Methods that generate intermediate images or sketches are computationally expensive, slow, and often brittle.
• Interleaved Image-Text Latents: Recent attempts to interleave explicit visual tokens with text reasoning (e.g., MVoT, MIRAGE) have shown conflicting results. They often suffer from modality switching bias, where the model defaults to text even when prompted to use images, or performance degrades when forced to switch modalities explicitly.
• Data Scarcity: Many methods require bespoke, expensive datasets with aligned text-image reasoning traces.

The core challenge is to enable models to "think" in a unified space that can represent both visual and textual information without the overhead of generating explicit images or the brittleness of forced modality switching.

2. Methodology: $\mu$-Tokens

The authors propose $\mu$-Tokens (Multimodal Tokens), a set of modality-agnostic latent tokens that act as an internal "scratchpad" for the model. These tokens can dynamically encode visual concepts (e.g., object layouts, depth maps) or textual concepts depending on the task, without ever being decoded into explicit text or images during the reasoning phase.

Training Framework

The training process follows a three-stage framework inspired by latent reasoning and reinforcement learning:

1. Stage 1: Warm-up (Supervised Fine-Tuning)

   • Goal: Teach the $\mu$-Tokens to anchor to specific multimodal reasoning steps.
   • Data: Uses interleaved text-image reasoning traces (e.g., from Zebra-CoT, Video-R1).
   • Mechanism: The model is trained to predict the next step in a reasoning chain. If the ground truth is text, the token predicts the text token. If the ground truth is a visual sub-goal (e.g., a cropped image or layout), the token is supervised via cosine similarity against the encoded visual features.
   • Outcome: The tokens learn to hold meaningful multimodal representations.

2. Stage 2: Relaxed Training (Answer-Only Supervision)

   • Goal: Allow the model to optimize the latent trajectory freely to maximize final answer accuracy.
   • Mechanism: The explicit supervision on intermediate steps is removed. The model is fine-tuned using only the final answer. The $\mu$-Tokens are inserted as a fixed-length sequence (e.g., 20 tokens) between the input and the answer.
   • Key Design: Unlike continuous recurrent latents (which break parallelism and accumulate errors), $\mu$-Tokens are discrete. They are processed via standard self-attention, allowing for efficient parallel computation and preventing error accumulation over long chains.

3. Stage 3: RL Refinement (GRPO)

   • Goal: Ensure the latent chain causally contributes to the correct answer rather than just co-occurring with it.
   • Mechanism: Uses Group Relative Policy Optimization (GRPO). The model generates multiple trajectories; those leading to correct answers receive higher rewards. The gradients propagate back through the self-attention mechanism to the $\mu$-Tokens, refining the "thinking" process to be causally effective.

3. Key Contributions

• Modality-Agnostic Reasoning: Introduces a paradigm where reasoning tokens are not bound to a specific modality (text or image) but can represent intermediate states in a unified latent space.
• Efficiency: $\mu$-Tokens achieve significant performance gains using only 10–40 tokens, compared to hundreds of tokens required for verbose text CoT or explicit image generation. This offers a Pareto-dominant improvement in speed and compute.
• Two-Stage Training Strategy: Demonstrates that pre-training tokens with multimodal traces (Stage 1) is crucial, followed by answer-only optimization (Stage 2) to prevent the model from being constrained by potentially suboptimal human-generated reasoning traces.
• Discrete vs. Continuous: Proves that discrete latent tokens outperform continuous recurrent embeddings in terms of stability, parallelism, and performance.

4. Experimental Results

The method was evaluated on the Qwen2.5-VL (7B) model across four challenging benchmarks: BLINK (visual puzzles/IQ), SAT-Real (action/motion reasoning), VSI-Bench (video spatial reasoning), and MMSI-Bench (multi-image reasoning).

• Performance Gains:
  • $\mu$-Tokens achieved a +3% average absolute accuracy improvement over the strongest baseline (Direct Answer Fine-tuning).
  • On reasoning-heavy splits (e.g., BLINK Jigsaw and IQ tests), the improvement reached +16%.
  • Outperformed both text-only CoT and interleaved image-text approaches.
• Ablation Studies:
  • Multimodal Warm-up is Critical: Using only text or no warm-up resulted in lower performance. The tokens must be pre-trained to hold both visual and textual information to be effective.
  • Discrete Tokens: Discrete tokens significantly outperformed continuous recurrent embeddings.
  • Token Count: Performance scales with the number of tokens up to a point (around 20), after which "overthinking" degrades performance.
• Generalization: The model showed strong generalization to out-of-domain tasks and multi-image benchmarks (MMSI-Bench, SiteBench).
• Interpretability: While the tokens themselves are latent, the authors showed they can be paired with explicit text rationales, allowing the model to "think" in the latent space first and then verbalize the reasoning if needed, retaining high accuracy.

5. Significance

This work addresses a fundamental bottleneck in multimodal AI: the difficulty of reasoning about space, time, and affordances without generating expensive intermediate outputs.

• Simplicity: It offers a simpler alternative to complex "think-and-sketch" or tool-augmented pipelines.
• Scalability: By using a small number of discrete tokens and standard transformer parallelism, the approach is highly scalable and computationally efficient.
• Future Direction: It suggests that the future of multimodal reasoning lies not in generating more explicit content (images/text), but in optimizing the internal latent representations that bridge perception and decision-making. The approach bridges the gap between human-like spatial imagination and machine reasoning without the overhead of explicit generation.