💬 NLP

Nanbeige4.1-3B: A Small General Model that Reasons, Aligns, and Acts

Nanbeige4.1-3B is a pioneering open-source 3B-parameter language model that unifies strong reasoning, code generation, and agentic tool-use capabilities through advanced reward modeling and deep search training, outperforming both its predecessors and significantly larger models.

Original authors: Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Xiyun Xu, Yang Song, Yiming Jia, Yuntao Wen, Yunzhi Xu, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen

Published 2026-02-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chen Yang, Guangyue Peng, Jiaying Zhu, Ran Le, Ruixiang Feng, Tao Zhang, Xiyun Xu, Yang Song, Yiming Jia, Yuntao Wen, Yunzhi Xu, Zekai Wang, Zhenwei An, Zhicong Sun, Zongchao Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a smartphone (a small, efficient device) and a supercomputer (a massive, power-hungry machine). Usually, we think the supercomputer can do everything better: solve complex math, write perfect code, and plan long trips. The smartphone? It's good at basic tasks but gets confused when things get too complicated or require a long chain of steps.

Nanbeige4.1-3B is a new kind of "smartphone" that breaks this rule. It's a tiny AI model (only 3 billion "brain cells," or parameters) that somehow manages to do the heavy lifting usually reserved for giant supercomputers.

Here is how the researchers taught this tiny model to think like a genius, explained through simple analogies:

1. The "Swiss Army Knife" Problem

Most small AI models are like specialized tools. One is a great screwdriver (good at math), another is a great hammer (good at coding), but if you ask the screwdriver to build a house, it fails. They are "fragmented."

Nanbeige4.1-3B is the first small model that is a true Swiss Army Knife. It can:

Reason: Solve hard logic puzzles.
Code: Write efficient computer programs.
Act: Go on long "deep search" missions to find information on the internet, just like a human researcher.

2. How They Trained It: The "Coach" Analogy

To make this tiny model so smart, the researchers didn't just feed it data; they acted like strict, clever coaches using three specific training methods:

A. The "Double-Check" System (Reasoning & Alignment)

Imagine a student writing an essay.

Old Way: The teacher just says, "Good job" or "Bad job" (Point-wise).
New Way: The teacher gives the student two essays to compare. "Essay A is okay, but Essay B is much better because it's more creative and polite."
The Result: By constantly comparing "Good" vs. "Better" answers, the model learns not just to be correct, but to be helpful, safe, and human-like. It stops rambling and starts giving high-quality answers.

B. The "Efficiency Coach" (Coding)

When writing code, a model might write a program that works, but it's slow and clunky—like driving a tank to go to the grocery store.

The Innovation: The researchers added a rule: "If your code works, great! But if you can also make it run faster and use less energy, you get extra points."
The Result: The model learned to write code that isn't just "correct," but elegant and efficient. It stopped wasting time and started writing like a pro programmer.

C. The "Long-Haul Navigator" (Deep Search)

This is the most impressive part. Usually, small models get lost if they have to take more than 2 or 3 steps to find an answer. They get tired or confused.

The Analogy: Imagine asking a small dog to find a specific book in a library. It might bark at the first shelf and give up.
The Fix: The researchers trained the model to take hundreds of steps (up to 600!) without losing its place. They broke the journey down into tiny checkpoints.
- Step 1: Did you ask the right question? (Reward)
- Step 2: Did you find the right page? (Reward)
- Step 3: Did you read the right sentence? (Reward)
The Result: The model can now act like a detective, digging through layers of information to solve complex problems that usually require a much bigger AI.

3. The "David vs. Goliath" Results

The paper shows that this tiny 3B model is beating giants:

It beats other small models (like the 4B Qwen models) easily.
Shockingly, it often beats models that are 10 times larger (30B or even 80B parameters).
In a real-world test (LeetCode coding contests), it ranked 1st place in some rounds, solving problems faster and better than much larger competitors.

The Big Takeaway

For a long time, we thought "bigger is always better" in AI. This paper says: "Not if you train them right."

By using smart training techniques—comparing answers, rewarding speed, and breaking long tasks into small steps—you can squeeze "supercomputer" intelligence into a tiny, efficient package. It's like taking a compact car and tuning the engine so well that it can race on the same track as a Formula 1 car.

In short: Nanbeige4.1-3B proves that you don't need a massive brain to do big things; you just need the right training.

1. Problem Statement

Small Language Models (SLMs), typically with around 3 billion parameters, have shown promise in specialized tasks like mathematical reasoning or code generation. However, they suffer from fragmented capabilities:

Reasoning-focused models often fail at long-horizon interactions (e.g., deep search).
Agent or code-specialized models often lack robust general reasoning, creative writing, or human preference alignment.
Core Challenge: Can a 3B parameter model be trained to be a truly unified generalist that excels in reasoning, coding, and agentic behavior simultaneously without compromising its existing strengths?

2. Methodology

The authors propose Nanbeige4.1-3B, built upon the Nanbeige4-3B-Base, utilizing a structured post-training pipeline that combines large-scale Supervised Fine-Tuning (SFT) and cascaded Reinforcement Learning (RL).

A. Enhanced SFT and Data Construction

Context Scaling: Extended the context window from 32k/64k to 256k tokens to support long-horizon reasoning and deep search.
Data Mixture: Redesigned the SFT data to include higher ratios of challenging math, code, and deep-search data. The final 256k stage mixture includes: Code (27%), Deep-Search (26%), STEM (23%), Tool-use (13%), and General domains (10%).
Refinement: Improved the Solution Refinement and Chain-of-Thought (CoT) Reconstruction frameworks to generate higher-quality reasoning traces.

B. General Reasoning & Alignment (RL Strategy)

To address issues like redundancy and misalignment, the authors employ a two-stage RL approach:

Point-wise RL: A general reward model trained on human preference data evaluates individual responses. This suppresses repetitive and low-readability outputs, optimizing for formatting and stability (e.g., reducing truncation errors in code).
Pair-wise RL: A pairwise reward model is trained on comparisons between strong and weak model responses. This leverages preference data more efficiently, using a swap-consistency regularizer to mitigate position bias. It significantly improves performance on multi-turn dialogue and adversarial prompts.

C. Deep Search (Agentic Behavior)

Data Synthesis: Constructed a large-scale dataset using Wiki-graph random walks to create complex multi-hop QA pairs.
Turn-Level Supervision: Implemented a rigorous filtering mechanism where a critic model evaluates every turn of a search trajectory based on logical soundness, tool-call accuracy, and informational gain. Only high-fidelity trajectories are used for training.
Outcome: This enables the model to execute up to 600 tool-call turns reliably, a capability rarely seen in models of this size.

D. Code Generation (Complexity-Aware RL)

Moving beyond simple correctness, the authors introduced a complexity-aware reward system:

Two-Stage Code RL:
- Stage 1: Optimizes for execution correctness (pass rate).
- Stage 2: Introduces a gated time-complexity reward. This reward is only activated if the solution passes all test cases. It compares the model's solution against an optimal reference bound.
Judge System: Uses a sandbox for execution and a specialized instruct model to estimate time complexity, encouraging solutions that are not just correct but computationally efficient (e.g., $O(N)$ vs. $O(N \log N)$ ).

3. Key Contributions

First Unified 3B SLM: Nanbeige4.1-3B is the first open-source small language model to simultaneously achieve state-of-the-art performance in reasoning, coding, and long-horizon agentic search.
Novel Training Paradigms:
- Integration of Point-wise and Pair-wise RL for superior alignment.
- Turn-level supervision for deep search trajectories.
- Gated time-complexity rewards for code generation.
Data Pipeline Innovations: Open-sourced a complex multi-hop QA dataset and a rigorous turn-level filtering mechanism for agent training.
Performance Leap: Demonstrated that strict capacity constraints (3B parameters) do not preclude deep-search capabilities comparable to specialized 30B+ models or general 100B+ models.

4. Experimental Results

The model was evaluated across three main domains:

A. General Reasoning & Coding

Outperforms Larger Models: Nanbeige4.1-3B surpasses Qwen3-30B-A3B and Qwen3-32B on multiple benchmarks, including LiveCodeBench-V6 (76.9 vs 66.0) and Arena-Hard-V2 (73.2 vs 60.2).
Real-World Coding: In LeetCode Weekly Contests (484–488), it achieved an 85.0% pass rate, ranking 1st in Contest 487 and 3rd in 488, outperforming Qwen3-32B (50%) and Qwen3-30B-A3B (65%).

B. Deep Search & Agentic Tasks

State-of-the-Art for SLMs: On benchmarks like GAIA (69.90), xBench-DeepSearch-05 (75.00), and SEAL-0 (41.44), Nanbeige4.1-3B significantly outperforms other small models (e.g., Qwen3-4B scores ~28-34 on these).
Competitive with Large Models: Its performance rivals or exceeds specialized "Research Agents" (e.g., Tongyi-DeepResearch-30B) and large foundation models with tools (e.g., Minimax-M2, DeepSeek-V3.2) on specific search tasks.

C. Efficiency

The model successfully executes complex, multi-step tool interactions (up to 600 turns) without the degradation in performance typically seen in smaller models.

5. Significance

Redefining SLM Potential: The paper proves that small models can achieve "broad competence" and "strong specialization" simultaneously, challenging the notion that deep search and complex reasoning require massive parameter counts.
Efficiency: It offers a highly efficient alternative to 30B–100B models for tasks requiring tool use, code generation, and complex reasoning, reducing inference costs and latency.
Community Impact: By open-sourcing the model, the dataset, and the training recipes, the authors provide a blueprint for the community to jointly train reasoning, coding, and agentic behaviors under strict capacity constraints.

In conclusion, Nanbeige4.1-3B represents a significant milestone in the development of compact, generalist AI agents, demonstrating that with the right training signals (complexity-aware rewards, turn-level supervision, and hybrid RL), small models can punch well above their weight class.