🤖 machine learning

Procedural-skill SFT across capacity tiers: A W-Shaped pre-SFT Trajectory and Regime-Asymmetric Mechanism on 0.8B-4B Qwen3.5 Models

This paper demonstrates that while procedural-skill SFT yields uniform absolute gains across 0.8B–4B Qwen3.5 models, the resulting performance trajectory follows a W-shaped pattern driven by regime-asymmetric base capabilities, where SFT provides the most significant absolute improvement to models that initially struggle with the procedure.

Original authors: Igor Strozzi

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Igor Strozzi

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Teaching a Recipe to Different Cooks

Imagine you have three cooks of different experience levels:

The Novice (0.8B): A kitchen helper who is eager but easily overwhelmed.
The Apprentice (2B): A cook who knows the basics but sometimes gets lazy or skips steps.
The Sous-Chef (4B): A very competent cook who can usually figure things out on their own.

The researchers wanted to see what happens if they give all three cooks a specific, written 5-step recipe (a "procedural skill") to follow for 200 different cooking challenges. They wanted to know: Does giving them the recipe actually help them cook better, or do they just pretend to follow it?

The Main Discovery: The "W" Shape

The researchers found something surprising. Before they gave anyone the recipe, the cooks' ability to handle the task followed a weird "W" shape when you looked at them from smallest to largest:

The Novice (0.8B): When given the recipe before training, they actually did worse. The 5 steps were too complicated; they got confused at step 1 and messed up the whole dish. The recipe was a burden.
The Apprentice (2B): When given the recipe, they did better. They were in the "sweet spot." The recipe acted like training wheels, helping them check their work and avoid mistakes they usually made.
The Sous-Chef (4B): When given the recipe before training, they did slightly worse (or stayed the same). They were already so good at cooking that reading the recipe felt like extra paperwork. It slowed them down without adding value.

The Twist: After the researchers trained the cooks using the recipe (a process called SFT), something magical happened. The amount of improvement was almost the same for all three cooks.

The Novice learned to follow the steps without getting confused.
The Apprentice got a little sharper.
The Sous-Chef learned to stop treating the recipe as annoying paperwork and start using it as a helpful tool.

The Lesson: The training (SFT) works hardest where the cook is struggling the most. It fixes the Novice's confusion and the Sous-Chef's laziness, resulting in a similar "boost" for everyone, even though they started from very different places.

The "Format" Trap (A Major Mistake They Fixed)

The paper highlights a huge mistake they almost made.

Imagine a strict judge who only accepts answers written on a specific line: "ANSWER: [The Dish Name]".

The Novice and Sous-Chef often wrote the answer in a sentence like, "I conclude that the dish is Pizza."
The strict judge (a computer script) couldn't find the word "ANSWER" and marked them as failures, even though they got the right answer.
The Apprentice happened to write in the exact format the judge wanted, so they looked like the winner.

The researchers realized their "judge" was biased. It wasn't measuring cooking skill; it was measuring typing skill.

The Fix: They switched to a human-like judge (an AI that reads the whole sentence).

Result: The Novice and Sous-Chef's scores jumped up!
The Bias: The strict judge had been unfairly punishing the cooks who wrote in "free-form" sentences. The researchers found that the "curated" (recipe-following) group was actually penalized more by the strict judge because their detailed explanations often buried the final answer in the text.

The "Ceiling" and The "Frontier"

The researchers compared their trained Sous-Chef (4B) to a World-Class Celebrity Chef (called "Haiku-4-5").

Before training, the Sous-Chef was good, but not quite at the Celebrity Chef's level.
After training with the recipe, the Sous-Chef tied the Celebrity Chef. They both got 197 out of 200 dishes perfect.
This proves that the training didn't just make the Sous-Chef "look" like they were following steps; it actually made them as good as the best in the world for these specific tasks.

What Didn't Work (The "Negative" Results)

The researchers tried to make the Novice (0.8B) better by changing the training recipe (changing the text, removing parts of the steps, etc.).

Result: Nothing worked. The Novice still couldn't cook the dishes perfectly.
Why: The Novice simply didn't have enough brainpower (capacity) to hold all the steps in their head at once. No matter how they tried to teach them, the "ceiling" was the Novice's own limits, not the teaching method.

Summary of Key Takeaways

Training helps everyone, but for different reasons: It rescues the confused, sharpens the competent, and re-engages the over-confident. The "boost" is surprisingly similar across the board.
Don't trust rigid judges: If you only check for specific formatting (like "ANSWER: X"), you might miss the fact that a model actually solved the problem correctly.
Size matters, but not how you think: A tiny model can learn to use a procedure just as well as a big one, but it can't necessarily execute the whole task perfectly if the task is too hard for its brain size.
The "W" Curve: Small models get confused by procedures; medium models love them; big models find them annoying until they are trained to see the value.

In short: The paper shows that teaching a step-by-step process is a powerful tool, but you have to measure it fairly (ignoring rigid formatting rules) and understand that a model's size determines how well it can execute the steps, even if it learns to follow them just as well as a bigger model.

Technical Summary: Procedural-Skill SFT Across Capacity Tiers

Problem Statement
The paper investigates whether explicit procedural instructions during Supervised Fine-Tuning (SFT) teach smaller language models to effectively utilize reasoning procedures at inference time, or if they merely induce the models to imitate the response shape (format learning). Specifically, it examines the contribution of procedural-skill SFT across different model capacity tiers (0.8B, 2B, and 4B parameters) using the Qwen3.5 dense series. A central challenge addressed is the potential for evaluation artifacts, particularly format-compliance biases in deterministic judges, which may conflate format learning with genuine procedural application.

Methodology
The study employs a self-contained pipeline involving:

Data Construction: A hand-curated catalog of 40 declarative skills (covering logical fallacies, deductive forms, cognitive biases, etc.) rewritten into procedural forms by Opus 4.7. A synthesizer generated 600 tasks (400 train, 200 holdout), ensuring a 1:1 task-to-skill mapping.
SFT Corpus: A 353-row curated chat-format corpus generated by tracing Opus 4.7 responses with injected procedural skill blocks. The corpus consists of (system prompt with skill block, user task, assistant Chain-of-Thought + ANSWER line) demonstrations.
Models: Three Qwen3.5 dense models (0.8B, 2B, 4B) fine-tuned using LoRA. A frontier reference, Claude Haiku 4.5, was included for comparison.
Evaluation Protocol:
- Conditions: Baseline (solver prompt without skill block) vs. Curated (solver prompt with matching procedural skill block).
- Scoring: Two paths were used: (1) Deterministic-mixed (native bench dispatch using regex extractors for structured answers and Opus 4.7 for free-form), and (2) LLM-only (Opus 4.7 judge for all task types to bypass format-compliance gates).
- Metrics: Pass rates and the differential lift ( $\Delta = \text{pass\_rate}_{\text{curated}} - \text{pass\_rate}_{\text{baseline}}$ ).
- Validation: Cross-family validation using GPT-5.4 on 2,800 paired episodes to bound judge-overlap bias.

Key Findings

Uniform SFT Contribution ( $\Delta$ -lift): Under matched-path LLM-only scoring, the SFT-attributable procedural lift is roughly uniform across model sizes: +0.070 (0.8B), +0.040 (2B), and +0.075 (4B). This suggests the differential work SFT performs (teaching the model to use the procedure vs. not using it) is capacity-invariant within a 4 percentage point band.
W-Shaped Pre-SFT Trajectory: The variation in post-SFT performance is dominated by a non-monotone pre-SFT base trajectory.
- 0.8B: Deep trough ( $\Delta = -0.075$ ). The base lacks capacity to follow the 5-step procedure, leading to compounding errors.
- 2B: Positive peak ( $\Delta = +0.060$ ). The base is in a "sweet spot" where the procedure scaffolds checks the model would otherwise skip.
- 4B: Shallow trough ( $\Delta = -0.010$ ). The base is competent enough to solve tasks directly; the procedure adds overhead without signal, sometimes leading to less defensible conclusions.
- Frontier (Haiku-4-5): Modest positive ( $\Delta = +0.030$ ).
Regime-Asymmetric Mechanism: SFT works hardest in absolute terms where the base struggles with the procedure. It rescues the 0.8B model from a procedure-incompetent regime to neutral and flips the 4B model from a procedure-overhead regime to a procedure-helpful one. The mechanism is regime-asymmetric rather than strictly capacity-conditional.
Format-Compliance Artifact: The study identifies a systematic bias in the deterministic ANSWER-line extractor used by 83.5% of the bench. This extractor under-counts free-form prose conclusions, systematically penalizing the curated condition (where conclusions are often embedded in procedural narration) and the frontier reference (Haiku). The LLM-only re-judge revealed this bias, showing that Haiku's performance improved by +18.5 pp on the curated condition when format constraints were removed.
0.8B Capacity Ceiling: Iterative recipe variations at 0.8B (corpus composition, partial fine-tuning, skill-block stripping) failed to lift the absolute pass rate beyond a 2 pp band. This confirms that the 0.8B absolute ceiling is bound by base reasoning capacity, not the SFT recipe. However, the differential lift ( $\Delta$ ) remained consistent with larger models.
Frontier Parity: The 4B model with SFT (v2.0) ties the Haiku-4-5 reference at the bench's effective ceiling (0.985 pass rate) under fair scoring. This is attributed to SFT plus base scaling, as the pre-SFT 4B base (0.840) did not match Haiku.

Significance and Claims
The paper claims to supersede two earlier framings: "format-only learning at 0.8B" (revealed as a path-mismatch artifact) and "SFT contribution shrinks at 4B" (revealed as a misunderstanding of the pre-SFT baseline trajectory).

Empirical Contribution: It provides evidence that procedural-skill SFT delivers a consistent differential lift across scales, but the net benefit depends heavily on the pre-existing procedural regime of the base model.
Methodological Contribution: It highlights a critical format-compliance artifact in standard benchmarks that biases SFT claims against curated conditions and frontier models. It demonstrates the necessity of format-tolerant evaluation (LLM-only judging) to separate format learning from procedural application.
Falsifiable Prediction: The paper predicts that at larger scales (8B/14B), the pre-SFT trajectory will recover toward positive values (similar to Haiku), and the SFT $\Delta$ -lift will shrink toward 2B levels as the base becomes naturally procedure-friendly.

Limitations
The authors explicitly note several limitations:

Single-seed evaluation: Results are point estimates without multi-seed replication.
Corpus-Judge Overlap: Opus 4.7 was used for task synthesis, trace generation, and judging. While cross-family validation (GPT-5.4) bounds the judging-stage bias, the corpus-generation overlap remains structurally unbounded.
Task Alignment: The bench measures procedural application on procedure-aligned tasks, not general skill routing or compositional skill use.
No Generic Control: The study did not include a control training on generic instruction-tuning data to quantitatively decompose format vs. procedure contributions, though the differential lift is argued to be evidence of the latter.