CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
The paper introduces CRISP, a framework that leverages the intrinsic attention patterns of reasoning models—specifically using the termination token as an information anchor—to prune redundant Chain-of-Thought steps, achieving a 50-60% reduction in token count while preserving logical coherence and accuracy.
Original authors:Yangsong Lan, Hongliang Dai, Piji Li
Original authors: Yangsong Lan, Hongliang Dai, Piji Li
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Token) When Rob finishes thinking and is about to give the final answer, he uses a special mental "anchor" token (a specific signal in the code called`).
The Discovery: The researchers noticed something fascinating. As Rob gets closer to the end, his attention shifts. He stops looking at the long, rambling middle of his essay and focuses intensely on this "anchor."
The Metaphor: Imagine a lighthouse at the end of a long, foggy pier. The light (the </think> token) shines backward. The parts of the pier that the light hits brightly are the critical steps Rob actually needed. The parts in the shadows are just redundant chatter he didn't really need.
2. The "Saliency Pruning" (The Smart Scissors)
Using this "lighthouse" signal, CRISP doesn't just guess what to cut. It knows exactly what matters.
The Process: It looks at every sentence Rob wrote. If the "lighthouse" is shining brightly on a sentence, it keeps it. If the light is dim, it knows that sentence was just filler.
The Action: It uses four smart tools to edit the essay:
KEEP: "This is vital! Keep it."
PRUNE: "This is just fluff. Cut it."
REWRITE: "This is important, but you said it too wordily. Let's say it in 5 words instead of 50."
FUSE: "You said this in step 1 and step 2. Let's merge them into one super-step."
3. The "Polishing" (The Final Touch)
Sometimes, when you cut out the middle of a story, the beginning and end don't flow well together. It might sound robotic or broken.
The Fix: CRISP uses a "Refiner" (a second AI) to smooth out the edges. It takes the chopped-up, high-density version and rewrites it so it flows naturally, like a human speaking, but without the long pauses and repetitions.
The Result: The "High-Octane" Reasoning
After using CRISP, the model is like a Formula 1 car instead of a heavy truck.
Before: It took 3,000 words to solve a math problem.
After: It takes only 1,200 words (a 60% reduction!) but gets the same accuracy.
Why This Matters Think of it like compressing a video file. Old methods would just lower the quality, making the picture blurry. CRISP is like a smart compression algorithm that removes the static and the pauses, keeping the high-definition action scenes perfectly clear.
This means:
Faster Answers: You get the solution much quicker.
Cheaper AI: It costs less money to run these models because they use less computer power.
Smarter AI: By removing the "overthinking" and noise, the AI actually becomes more focused and reliable.
In short, CRISP teaches the AI to stop talking to itself so much, helping it find the "signal" in the "noise" and give you the answer faster, without losing its brilliant reasoning skills.
` Token as an Information Anchor Through an analysis of layer-wise attention maps in models like DeepSeek-R1-Distill-Qwen, the authors discovered a distinct phenomenon:
The </think> token (which marks the transition from reasoning to the final answer) acts as a critical information anchor.
In deep layers, the model's attention concentrates heavily on the </think> token while attending minimally to the raw reasoning chain.
Saliency Correlation: The attention weights from the </think> token to preceding reasoning steps reliably indicate the saliency of those steps. Steps with high attention scores encode critical logic, while low-score steps represent redundancy.
B. Compression as a Structured Search Problem
CRISP formulates CoT compression as a greedy search over a sequence of reasoning steps, guided by intrinsic attention scores. The process involves four atomic operators:
KEEP: Preserves high-salience steps.
PRUNE: Removes low-salience (redundant) steps.
REWRITE: Condenses verbose steps while preserving logic.
FUSE: Merges semantically similar consecutive steps into a single dense unit.
The Algorithm:
Scoring: Calculate the endogenous contribution (Si) of each step ri by aggregating attention weights from the </think> token to tokens within that step.
Heuristic Gating: Based on Si and semantic similarity, a dynamic action space is defined (e.g., if Si is low, allow PRUNE or REWRITE; if high, allow KEEP or REWRITE).
Reward-Guided Selection: The optimal operator is selected by maximizing a reward function that balances prediction fidelity (likelihood of the correct answer) against sequence length: R(a)=logPθ(y∣x,C⊕a(ri))−logPθ(y∣x,C)−β⋅Len(a(ri))
Refinement: Since discrete search operations can cause syntactic fragmentation, an auxiliary LLM-based refiner reconstructs the compressed skeleton (R′) into a fluent, coherent chain (RCRISP), ensuring logical continuity without reverting to verbosity.
C. Training Strategy
The target model is fine-tuned on the refined, high-density trajectories using a multi-task learning objective. A special control token (<|compressed|>) is appended to the input to signal the model to generate compressed reasoning paths during inference.
3. Key Contributions
Intrinsic Saliency Indicator: The paper identifies that attention weights at the </think> token serve as a reliable, model-intrinsic indicator of reasoning step importance, removing the need for external compressors.
CRISP Framework: A novel compression framework utilizing a greedy search over four atomic operators (FUSE, PRUNE, REWRITE, KEEP) guided by intrinsic signals.
Superior Trade-off: Demonstrates that CRISP achieves a significantly better balance between inference efficiency and reasoning accuracy compared to state-of-the-art baselines.
4. Experimental Results
Experiments were conducted on DeepSeek-R1-Distill-Qwen-1.5B and 7B models across three benchmarks: GSM8K (grade-school math), MATH-500 (competition math), and AMC23.
Token Reduction: CRISP achieved a 50–60% reduction in token count compared to the original models.
Example (7B model): Reduced average tokens from 2,971 to 1,235.
Accuracy Preservation: Unlike other methods that sacrifice accuracy for brevity, CRISP maintained or slightly improved accuracy.
Example (7B model): Accuracy increased from 83.6% (Original) to 83.9% (CRISP).
Token Efficiency (TE): CRISP significantly outperformed baselines in Token Efficiency (Accuracy per token).
Example (7B model): TE improved from 2.81 (Original) to 6.80 (CRISP).
Ablation Study: Removing the Refinement module caused a significant drop in accuracy (e.g., 84.2% → 70.6% for the 7B model), proving that semantic reconstruction is critical for maintaining logical coherence after compression.
Robustness: CRISP consistently outperformed baselines (Truncation, CoD, TALE, TokenSkip, A*-Thought) even under strict token budgets (1024 and 2048 tokens).
5. Significance
Efficiency without Compromise: CRISP provides a scalable solution for deploying reasoning models in latency-sensitive or resource-constrained environments without sacrificing the "reasoning capability" that defines modern LLMs.
Paradigm Shift: It moves away from external, misaligned compression logic toward endogenous self-selection, leveraging the model's own attention dynamics to determine what is essential.
Generalizability: The method effectively filters "overthinking" and redundant cognitive computations, suggesting that reasoning models naturally generate more concise paths when guided by their own internal saliency signals.
In summary, CRISP demonstrates that by understanding and utilizing the internal "information anchor" of a reasoning model, it is possible to distill highly efficient, dense reasoning paths that are both shorter and more accurate than the original verbose outputs.