🤖 AI

MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning

MuCRASP is a novel structured pruning framework for vision-language models that preserves multimodal chain-of-thought reasoning accuracy by identifying and protecting reasoning-critical pivot tokens and addressing cross-modal activation differences, outperforming existing methods even at high compression rates.

Original authors: Aritra Dutta, Somak Aditya

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aritra Dutta, Somak Aditya

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, multi-talented assistant who is incredibly good at solving complex puzzles. This assistant can look at a picture (like a photo of a messy room) and read a question (like "If I knock over the vase, will the cat get wet?"). To answer, the assistant doesn't just guess; it writes out a step-by-step diary of its thoughts, explaining how it connects the dots. This is called Chain-of-Thought (CoT) reasoning.

However, this assistant is huge. It takes up a massive amount of computer memory and energy to run, making it too expensive for many people to use in their daily lives.

The Problem: The "Scissors" Mistake
Scientists have tried to make this assistant smaller by "pruning" it—basically cutting out parts of its brain that they think aren't being used. Think of it like a gardener trimming a giant bush to make it fit in a small pot.

The problem with previous methods is that they were like a gardener using a blindfolded pair of scissors. They cut away branches based on how "heavy" or "active" they looked in general, without understanding what the bush was actually doing.

The Result: When they cut the bush to make it smaller, they accidentally snipped the very specific, tiny twigs that held the logical steps together. The assistant became small and fast, but it lost its ability to think logically. It could still speak fluently, but its reasoning was broken, like a story where the sentences make sense individually but the plot makes no sense.

The Solution: µCRASP (The Smart Gardener)
The authors of this paper, Aritra Dutta and Somak Aditya, created a new method called µCRASP. Instead of a blind gardener, µCRASP is a smart gardener who understands the specific job the bush is doing: reasoning.

Here is how µCRASP works, using three simple metaphors:

Finding the "Pivot Points" (The Turn Signals):
In a long chain of reasoning, most words are just filler. But there are a few critical moments where the assistant switches from one thought to the next (e.g., "I see a vase" $\rightarrow$ "Therefore, it might fall"). The authors call these pivot tokens.
- The Analogy: Imagine a train journey. Most of the track is just straight lines. But the switches (where the train changes tracks) are the most critical parts. If you cut the track at the switches, the train derails. µCRASP identifies these switches and refuses to cut them, even if it means cutting more of the straight, boring track.
Respecting the "Two-Brain" System (Vision + Language):
This assistant has two distinct ways of thinking: one for looking at images (Vision) and one for reading words (Language). These two parts need to talk to each other constantly.
- The Analogy: Imagine a team of two people: a Photographer and a Writer. They have to work together to solve a mystery. Old pruning methods treated them as one big blob of people and cut randomly. µCRASP realizes that if you cut the people who help the Photographer talk to the Writer, the team falls apart. It specifically protects the "meeting room" where these two brains connect.
The Global Budget (The Knapsack):
The goal is to cut a specific percentage of the brain (e.g., 30%) to make it smaller.
- The Analogy: Imagine you have a backpack (the computer memory) and you need to fit a lot of gear in it. Old methods might cut 30% of the heavy boots and 30% of the light socks, leaving you with no boots and too many socks. µCRASP acts like a smart packer. It looks at the value of every single item (neuron) relative to its weight. It might decide to keep the heavy boots (because they are critical for reasoning) and cut a lot of the light socks, ensuring the backpack is small but still fully functional.

The Results
The researchers tested this on several different "assistants" (AI models) and found that:

Old methods broke the reasoning ability very quickly. As soon as they cut more than 25-30% of the model, the logic collapsed completely.
µCRASP kept the logic intact even when they cut away 50% of the model.
Even though the model was half its original size, it could still solve complex visual puzzles (like physics problems or counting objects) with the same logical clarity as the giant, expensive version.

In Summary
µCRASP is a new way to shrink smart AI models without breaking their brains. It does this by carefully identifying and protecting the tiny, critical moments where the AI switches from one thought to the next, and by ensuring the "eyes" and "mouth" of the AI stay connected. This allows powerful reasoning tools to run on smaller, cheaper computers without losing their ability to think step-by-step.

Technical Summary: µCRASP

Problem Statement

Vision-Language Models (VLMs) have achieved significant success in complex multimodal tasks by leveraging Chain-of-Thought (CoT) reasoning, which involves generating explicit intermediate steps before a final answer. However, the large parameter counts of these models make deployment computationally expensive. While structured model pruning offers a path to efficiency, existing methods fail to preserve CoT reasoning accuracy when applied to VLMs.

The authors identify two primary reasons for this failure:

CoT-Agnostic Pruning: CoT reasoning consistency is governed by sparse "pivot tokens" (transition points between reasoning steps). Conventional pruning methods treat all tokens equally, diluting the signal of these critical transition tokens under aggregate loss calculations, leading to a signal-to-noise imbalance.
Cross-Modal Disparity: Traditional pruning methods designed for unimodal Large Language Models (LLMs) do not account for the distinct activation distributions between visual and textual modalities. VLMs exhibit architectural heterogeneity and cross-modal entanglement, making pruning significantly more challenging than in purely textual models.

Methodology: µCRASP

The authors propose µCRASP (Multimodal Chain-of-thought Reasoning aware Structured Pruning), a training-free, structured pruning framework designed to preserve reasoning coherence and cross-modal alignment under a global parameter budget. The method operates through four principal steps:

Global Attribution: Computes a standard global importance score ( $I_{global}$ ) for structural units (neurons, attention heads, GQA groups) using first-order Taylor expansion over all token positions.
Trajectory Pivot Attribution: Identifies "pivot tokens" that mark transitions between reasoning steps (e.g., structural delimiters like "Step 1:" or logical connectives like "Therefore"). It computes a specific attribution score ( $A_{pivot}$ ) restricted to local windows around these detected transition regions. This ensures the pruning process prioritizes units critical for logical continuity rather than just surface-level fluency.
Cross-Modal Dependency Scoring (CMDS): Quantifies the dependency between visual and textual modalities for each decoder layer using a score inspired by Maximum Mean Discrepancy (MMD). This identifies layers where vision-language integration is strongest (typically middle layers), allowing the method to protect these "bottleneck" layers from aggressive pruning.
Global Knapsack Optimization: Fuses the global and pivot attribution scores with a compression-conditioned mixing coefficient ( $\gamma_{dyn}$ ) and layer-level protection factors derived from CMDS and activation sensitivity. The method then solves a global knapsack problem to select the optimal set of structural units to retain, maximizing reasoning-preserving importance while respecting the parameter budget and minimum retention constraints.

Key Contributions

Diagnosis of Failure: The paper demonstrates that existing structured pruning methods (e.g., FLAP, LLM-Pruner, Attribution Pruning) fail to preserve CoT reasoning in VLMs, often causing reasoning chains to collapse under moderate compression despite maintaining acceptable perplexity.
µCRASP Framework: Introduces a novel, training-free pruning method that explicitly targets reasoning-critical components. It integrates reasoning-aware attribution to capture transition-critical tokens, cross-modal dependency modeling to retain vision-language alignment, and layer-wise sensitivity to guide structured pruning.
Comprehensive Benchmarking: Provides extensive evaluation across four VLMs (Llama 3.2-11B, Qwen2.5-VL-7B/3B, Gemma-3-4B) and three reasoning domains (Physical, Quantitative, Commonsense). The study utilizes an LLM-as-Judge metric to evaluate the logical validity of CoT traces, moving beyond standard token-level accuracy metrics.

Experimental Results

Experiments show that µCRASP consistently preserves reasoning quality under increasing compression levels where baselines fail:

Reasoning Coherence vs. Exact Match: µCRASP exhibits a decoupling between reasoning coherence and exact-match accuracy. While exact match (EMa) degrades, the logical validity of the CoT trace (measured by LLM-Judge) remains high. For instance, on Qwen2.5-VL-7B at 30% pruning, µCRASP retains an LLM-Judge score of 8.87 (vs. 9.02 for the dense model) on physical reasoning, whereas the best baseline (LLM-Pruner) drops to 7.32.
High Compression Resilience: µCRASP sustains high reasoning consistency (LLM-J > 7.0) up to 50% pruning, a 20–25 percentage-point improvement over existing methods. In contrast, existing methods often suffer abrupt performance collapse beyond 25–30% pruning.
Cross-Modal Protection: Ablation studies confirm that the CMDS component is essential for preserving vision-language alignment; removing it causes severe degradation in semantic similarity and formatting scores.
Distributional Fidelity: µCRASP maintains a low KL divergence between the pruned and dense model's output logit distributions, indicating that the pruned model preserves the underlying reasoning geometry rather than just surface-level text.

Significance and Claims

The paper claims that µCRASP addresses a critical gap in VLM compression by explicitly modeling the structural integrity of reasoning chains and cross-modal interactions. The authors emphasize that:

Reasoning Coherence is Distinct: Logical chain structure can be preserved even as verbatim answer precision degrades, a phenomenon not captured by traditional metrics like perplexity or exact match.
VLMs Require Specialized Pruning: Pruning VLMs is fundamentally harder than pruning unimodal LLMs due to cross-modal coupling; methods ignoring this lead to catastrophic failure in multimodal reasoning tasks.
Practical Deployment: The method enables the deployment of capable multimodal reasoning models on resource-constrained hardware without requiring fine-tuning, though the authors note that post-pruning recovery (e.g., knowledge distillation) remains a potential future direction.

The authors maintain a modest scope, noting limitations such as reliance on heuristic pivot detection, manual hyperparameter tuning for protection factors, and evaluation restricted to English-language benchmarks. They do not claim to introduce new reasoning capabilities but rather to preserve existing ones under compression.