💻 computer science

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy

This paper proposes a method that improves the accuracy and efficiency of reasoning LLMs by retrieving and applying pre-distilled reusable reasoning skills at inference time, thereby significantly reducing token consumption and costs compared to the traditional "reasoning from scratch" approach.

Original authors: Guangxiang Zhao, Qilong Shi, Xusen Xiao, Xiangzheng Zhang, Tong Yang, Lin Sun

Published 2026-04-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guangxiang Zhao, Qilong Shi, Xusen Xiao, Xiangzheng Zhang, Tong Yang, Lin Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a very difficult math problem or write a complex computer program.

The Old Way: "Reasoning from Scratch"
Currently, most advanced AI models (like the "thinking" AIs we see today) approach every new problem as if they have never seen anything like it before. They start from zero. They try a method, realize it's wrong, backtrack, try another, get stuck, and try again. They write out thousands of words of "thinking" just to figure out the basics.

The Analogy: It's like trying to bake a cake for the first time. You don't have a recipe, so you guess. You add too much flour, taste it, realize it's bad, throw it away, start over, add too much sugar, taste it again... By the time you finally get a cake, you've wasted a lot of ingredients (money/tokens) and time.
The Problem: This "trial and error" is expensive. In the AI world, every word the model thinks costs money and takes time.

The New Idea: "Thinking with Reasoning Skills" (TRS)
This paper proposes a smarter way. Instead of forcing the AI to re-invent the wheel every time, we give it a digital toolbox of "Skills."

Here is how it works, step-by-step:

1. The "Grandmaster" Learns (Offline)

First, we take a very smart AI and let it solve thousands of hard problems the old way (with all the trial and error). While it does this, a "Summarizer" (like a wise teacher) watches the process.

The Analogy: Imagine a master chef watching a student cook. The student burns the toast, adds too much salt, and finally makes a great soup. The teacher doesn't just watch; they write down a cheat sheet: "When making soup, always taste before adding salt. If the bread burns, start over immediately."
These cheat sheets are called Skill Cards. They are short, reusable instructions like "Use this math trick for this type of equation" or "Avoid this coding loop."

2. The "Library" is Built

All these cheat sheets are stored in a library.

The Analogy: It's like a library of recipes or a mechanic's manual. It contains both "How to do it right" and "How to fix it when you mess up."

3. The "Smart Student" Solves (Online)

Now, when a user asks a new question, the system doesn't just say "Go think!" Instead, it first looks in the library.

The Analogy: You ask the student, "How do I bake a cake?"
- Old Way: The student guesses and bakes 5 failed cakes.
- New Way (TRS): The system quickly finds the "Cake Recipe" skill card from the library and hands it to the student. The student reads the card: "Preheat oven to 350. Mix dry ingredients first."
- The student skips the guessing, skips the failures, and goes straight to baking a perfect cake.

Why is this a Big Deal?

1. It Saves Money (Fewer Tokens)
Because the AI doesn't have to waste time guessing and backtracking, it writes much less text.

Real-world impact: If you are a company paying for AI, you are paying for every word the AI thinks. TRS cuts the "thinking" words by up to 50% in some cases. It's like getting the same result but paying half the bill.

2. It's Actually Smarter (Better Accuracy)
Usually, when you force an AI to be quick (by telling it to "think less"), it makes more mistakes. It skips important steps.

The Magic: TRS is different. It doesn't just tell the AI to "hurry up." It tells the AI what to do. By giving it the right path, it solves the problem faster AND more accurately. It breaks the rule that says "you have to choose between speed and quality."

3. It Works Like Human Experts
Humans don't re-learn how to drive every time we get in a car. We recall the skill "turn the key, check mirrors, press gas." This paper teaches AI to do the same thing: recall a "skill" instead of re-deriving the logic from scratch.

Summary

Think of TRS as giving the AI a GPS instead of making it wander around a city hoping to find the destination.

Without TRS: The AI drives around, gets lost, takes wrong turns, and burns a lot of gas (tokens) before arriving.
With TRS: The AI looks at the map (the Skill Library), sees the best route, and drives straight there.

The Result: You get the correct answer, faster, and for a fraction of the cost. This makes powerful AI much more practical for real-world use, like coding assistants, math tutors, or business tools.

1. Problem Statement

Reasoning-centric Large Language Models (LRMs), such as OpenAI's o1 and DeepSeek-R1, have demonstrated superior performance in mathematics and coding by generating extensive intermediate "thinking" traces (e.g., Chain-of-Thought). However, this approach introduces significant bottlenecks:

High Cost & Latency: Commercial APIs charge by token, and reasoning traces often contain thousands of tokens, many of which are redundant (e.g., repeated verification, trial-and-error loops).
The Efficiency-Accuracy Trade-off: Existing methods to reduce costs (e.g., enforcing token budgets like TALE, compressing traces like Chain-of-Draft, or suppressing reflection markers like NoWait) often force models to "think from scratch" but faster. On complex problems, this compression leads to catastrophic accuracy drops because models skip crucial reasoning steps.
Lack of Reuse: Unlike human experts who recall reusable skills (heuristics, patterns, anti-patterns) from memory, current LLMs typically treat every query as a blank slate, rediscovering solutions repeatedly.

2. Methodology: Thinking with Reasoning Skills (TRS)

The authors propose TRS, a training-free, retrieval-augmented framework that decouples the acquisition of reasoning logic from its execution. The core philosophy is to replace redundant internal rediscovery with externalized procedural memory.

The framework operates in two phases:

A. Offline: Skill Distillation & Library Construction

Trajectory Generation: A reasoning model ( $M_r$ ) solves a large set of source problems, generating long reasoning traces ( $\tau$ ) and final answers.
Summarization: A stronger summarizer model ( $M_s$ $M_{s}$ ) distills these traces into compact, reusable Reasoning Skill Cards.
- Success Cards: Capture essential patterns (e.g., "Apply u-substitution when integrand has form...").
- Failure Cards: Capture anti-patterns and fixes (e.g., "Avoid integration by parts here; it leads to complexity").
- Schema: Each card is structured with fields: Trigger (applicability cues), Do (actionable steps), Avoid (pitfalls), Check (invariants), and Risk (edge cases).
Indexing: Skills are stored in a Key-Value library. Keys are concatenations of the original problem and generated trigger keywords (10–20 keywords) to facilitate retrieval.

B. Online: Retrieval & Skill Injection

Retrieval: For a new query $q$ $q$ , the system retrieves the top- $k$ $k$ most relevant skill cards using sparse (BM25) or dense (embedding) retrieval, or a hybrid approach.
- Math: Primarily uses BM25 (lexical matching of formulas/keywords).
- Coding: Uses Hybrid retrieval (BM25 + Dense) to capture semantic algorithmic patterns.
Injection: The retrieved skills are prepended to the prompt of the target reasoning model.
Guidance: The model is instructed to use the retrieved skills to navigate the solution space directly, avoiding detours. A lightweight gating instruction ensures the model ignores irrelevant or conflicting advice.

3. Key Contributions

A Practical Token-Efficient Framework: TRS is compatible with black-box LLM APIs and requires no fine-tuning. It shifts the paradigm from "reasoning from scratch" to "reasoning with recalled experience."
Breaking the Efficiency-Accuracy Trade-off: Unlike compression-based methods that sacrifice accuracy for brevity, TRS maintains or improves accuracy while significantly reducing token usage. It achieves this by providing a "navigation map" that prevents the model from getting lost in incorrect branches.
Cross-Model Transferability: The paper demonstrates that skills distilled from one model (e.g., a strong proprietary model) can effectively guide weaker or different models, suggesting that reasoning patterns are model-agnostic.
Comprehensive Analysis: The authors provide extensive ablation studies on retrieval strategies, prompt templates, library coverage, and transferability across diverse benchmarks (DeepMath, competitive programming, and external contest math).

4. Experimental Results

The authors evaluated TRS on DeepMath-103K (Math) and NEMOTRON-COMPETITIVE-PROGRAMMING-V1 (Coding) across multiple models (Gemini-3-Flash, Doubao Seed, GPT-OSS-120B, GPT-4o-mini).

Performance Gains:
- Math: TRS reduced reasoning tokens by 18.5% to 59.1% while improving or maintaining accuracy. For example, on Doubao Seed, costs dropped by 53.8% with negligible accuracy loss (-0.2%).
- Coding: TRS improved pass@1 accuracy (e.g., +4.1% on GPT-OSS-120B) while reducing tokens.
Hard Problem Handling: On difficult subsets (where baseline models "think" for >10k tokens), TRS showed the most significant gains. While baseline compression methods (CoD, TALE) suffered accuracy collapses on hard problems, TRS maintained high accuracy, effectively pruning redundant exploration.
External Transfer: When tested on an external contest-math suite (AIME, HMMT) using a library distilled from a different corpus (AoPS), TRS improved accuracy in 13/25 model-benchmark pairs and reduced costs in 20/25 pairs, proving the generalizability of the skills.
Cost Reduction: Across most configurations, TRS achieved a 15–50% reduction in per-request cost compared to standard Chain-of-Thought (CoT) reasoning.

5. Significance and Implications

Economic Viability: By drastically reducing the token count required for high-quality reasoning, TRS makes advanced reasoning models economically viable for real-world deployment, addressing the "infrastructure strain" caused by current reasoning-heavy models.
Paradigm Shift: It challenges the notion that "more thinking" (longer traces) is always necessary for correctness. Instead, it posits that reusing distilled procedural knowledge is a more efficient path to accuracy.
Scalability: As a training-free, black-box compatible approach, TRS offers a scalable pathway to enhance existing LLMs without the computational cost of retraining. It suggests a future where "Reasoning Skill Libraries" act as a shared, reusable knowledge base for the AI ecosystem.

Conclusion:
The paper successfully demonstrates that "Thinking with Reasoning Skills" allows models to achieve fewer tokens and higher accuracy simultaneously. By externalizing the "trial-and-error" phase into a reusable skill library, TRS eliminates the need for models to rediscover fundamental reasoning patterns for every new query, offering a practical solution to the cost-efficiency bottleneck in advanced AI reasoning.