Sabiá-4 Technical Report

This technical report introduces Sabi'a-4 and Sabiazinho-4, a new generation of Brazilian Portuguese language models featuring a 128K token context window and specialized training in legal and agentic tasks, which demonstrate superior cost-performance and capabilities in legal drafting, dialogue, and tool use compared to previous generations.

The Gist

Imagine you have a brilliant, multilingual assistant who speaks Portuguese perfectly, but until now, they were a bit like a generalist librarian: they knew a little bit about everything, but if you asked them to draft a complex legal contract or navigate a 100-page document, they sometimes got lost or made mistakes.

Enter Sabiá-4 and Sabiazinho-4. Think of these as the "specialized experts" of the AI world, specifically trained to be the ultimate Brazilian Portuguese assistants.

Here is the story of how they were built and why they matter, explained without the jargon.

1. The Four-Step Training Camp

The creators didn't just teach these models from scratch; they took a smart, general-purpose AI and gave it a rigorous four-step boot camp:

• Step 1: The Legal Library (Continued Pre-training): Imagine taking a smart student and locking them in a library filled only with Brazilian laws, court cases, and Portuguese literature. They read everything. This ensures the model doesn't just speak Portuguese; it thinks like a Brazilian lawyer.
• Step 2: The Long-Story Challenge (Context Extension): Previous models had short attention spans—they could only read a few pages before forgetting the beginning. These new models were trained to hold a 128,000-token context in their "head." That's like reading an entire novel, a legal case file, and a history book simultaneously without forgetting the first sentence.
• Step 3: The Role-Play Workshop (Supervised Fine-Tuning): Now, the model practices specific jobs. It learns how to chat naturally, write code, draft legal documents, and even act as an "agent" that can use tools (like clicking buttons on a website or checking a bank account).
• Step 4: The Personality Polish (Preference Alignment): Finally, human trainers gave the model feedback. "No, don't sound like a robot," or "Yes, be more polite here." This step taught the model to understand subtle nuances, avoid being overly sycophantic (sucking up), and follow strict formatting rules.

2. The "Price vs. Performance" Sweet Spot

The paper includes a chart (Figure 1) that is the most important takeaway for businesses. Imagine a graph where the Y-axis is "How smart is the model?" and the X-axis is "How much does it cost?"

• Most top-tier models are like Ferraris: incredibly fast and smart, but they cost a fortune to run.
• Cheaper models are like bicycles: cheap, but they might struggle with heavy loads.
• Sabiá-4 is the Tesla. It sits in the "upper-left" corner of the chart: it is incredibly smart (rivaling the most expensive models) but costs a fraction of the price. It offers the best "bang for your buck."

3. What Can They Actually Do?

The team tested these models on six different "obstacle courses" to see how they performed:

• The Legal Gauntlet: They were asked to write legal briefs and judge decisions. They scored higher than many expensive competitors, proving they can handle the tricky language of Brazilian law.
• The "Needle in a Haystack" Test: Imagine hiding a single specific fact inside a 100-page document. Can the model find it? Yes. They can read massive documents and find the exact piece of information needed without getting confused.
• The "Follow Instructions" Game: If you ask a model to "write a poem, then summarize it, then change the tone to sad, and use no commas," older models often forget a step. Sabiá-4 remembers the whole chain of commands perfectly.
• The "Agent" Test: This is the coolest part. The models were tested on tasks like buying a football ticket or sending money via Pix (Brazil's instant payment system). They successfully navigated websites, filled out forms, and completed tasks just like a human would.

4. Why Does This Matter?

For a long time, if you wanted an AI that understood Brazilian law or could handle complex, long documents, you had to pay a premium price to big US-based companies.

Sabiá-4 changes the game. It proves that you can have a model that:

1. Speaks Brazilian Portuguese natively (not just a translation).
2. Understands the specific legal and cultural context of Brazil.
3. Can handle massive amounts of text.
4. Is affordable enough for everyday businesses to use.

In a nutshell: Sabiá-4 is like hiring a highly specialized, incredibly fast, and very affordable Brazilian lawyer-assistant who never forgets a detail, can read a whole library in seconds, and is ready to help you buy tickets, send money, or draft a contract—all for a price that doesn't break the bank.

Technical Summary

Here is a detailed technical summary of the Sabiá-4 Technical Report:

1. Problem Statement

The report addresses the need for high-performance, cost-effective Large Language Models (LLMs) specifically optimized for Brazilian Portuguese, with a specialized focus on the Brazilian legal domain. Previous generations of models often struggled with:

• Domain Specificity: Lack of deep understanding of Brazilian legislation, judicial decision-making, and legal drafting nuances.
• Long-Context Limitations: Inability to effectively process and retrieve information from very long documents (e.g., full legal case files).
• Agentic Capabilities: Poor performance in multi-turn instruction following, tool use (function calling), and web navigation.
• Cost vs. Performance: A trade-off where high-accuracy models were prohibitively expensive for production deployment, while cheaper models lacked the necessary precision for complex tasks.

2. Methodology

The authors developed Sabiá-4 and Sabiazinho-4 using a four-stage training pipeline on Google Cloud TPUs (v5p and v6e) using the JAX framework. The pipeline is designed to specialize a generalist base model without the massive cost of training from scratch.

• Stage 1: Continued Pre-training:
  • The model underwent continued pre-training on a massive Portuguese corpus combined with a specialized Brazilian legal corpus.
  • This stage was critical for grounding the model in Brazilian legal terminology, norms, and the structure of federal legislation.
• Stage 2: Long-Context Extension:
  • The context window was extended to 128K tokens.
  • Training utilized a curated dataset of naturally long documents to ensure the model could maintain coherence and retrieve information ("Needle in a Haystack") across extended contexts.
• Stage 3: Supervised Fine-Tuning (SFT):
  • The model was fine-tuned on a diverse instruction dataset covering:
    • Chat & Dialogue: Multi-turn conversations.
    • Code: Programming tasks.
    • Legal Tasks: Drafting and analysis.
    • Function Calling/Agentic Tasks: Synthetic data pipelines were used to generate diverse examples for tool usage and web navigation.
• Stage 4: Preference Alignment:
  • The final stage aligned model outputs with human preferences, focusing on refining writing style, interpreting subtle nuances, and adhering to strict formatting requirements (crucial for legal documents).

3. Key Contributions

• Specialized Legal Models: Introduction of Sabiá-4 and Sabiazinho-4, which set new benchmarks for Brazilian legal drafting (attorney and judge styles) and legislative knowledge.
• 128K Context Capability: Successful extension of the context window to 128K tokens, validated through rigorous testing beyond standard "Needle in a Haystack" tests.
• Agentic Benchmarking: Development and application of new benchmarks for Portuguese agentic capabilities, including Ticket-Bench (football ticketing), Pix-Bench (financial transfers), MARCA (web navigation), and CLIMB (multi-hop search).
• Cost-Performance Optimization: The models are positioned to offer a superior cost-to-performance ratio compared to frontier models from major providers (OpenAI, Google, Anthropic, etc.).

4. Results

The models were evaluated across six categories using specific benchmarks. Key findings include:

• Legal Domain Superiority:
  • OAB-Bench (Attorney Style): Sabiá-4 scored 7.49/10, outperforming Qwen3-235b (6.33) and DeepSeek-v3.2 (6.40), and approaching the performance of GPT-5.2 (8.73) at a fraction of the cost.
  • Magis-Bench (Judge Style): Sabiá-4 achieved 5.08/10, significantly outperforming Qwen3-235b (4.52) and DeepSeek-v3.2 (4.88).
  • Legal Knowledge: Sabiá-4 achieved 97.4% accuracy on Brazilian Federal Laws, vastly outperforming competitors (e.g., Qwen3 at 65.9%).
• Long-Context Understanding:
  • On the MRCR (Multi-Round Co-reference Resolution) benchmark, Sabiá-4 and Sabiazinho-4 demonstrated robust performance across context lengths up to 128K, maintaining accuracy where previous generations degraded.
• Agentic & Instruction Following:
  • Multi-IF (Instruction Following): Sabiá-4 achieved 82.0% strict accuracy on multi-turn instruction accumulation.
  • Agentic Tasks: While Sabiá-4 scored 72.2% on agent capabilities (Pass@3), it showed significant improvement over Sabiá-3.1 (43.1%). It notably outperformed many larger models in cost-effective scenarios, though frontier models like GPT-5.2 (85.7%) and Gemini-3 Pro (90.4%) still lead in raw agent success rates.
• General Knowledge:
  • On Brazilian Standardized Exams (13 exams including ENEM, OAB, Revalida), Sabiá-4 achieved 86.6% accuracy, competing closely with GPT-5.2 (92.9%) and Gemini-3 Pro (95.0%).
• Cost-Performance Trade-off:
  • As illustrated in Figure 1, Sabiá-4 and Sabiazinho-4 occupy the upper-left quadrant of the pricing-accuracy chart. They achieve competitive benchmark scores at a significantly lower cost (e.g., Sabiazinho-4 input cost is $0.19/1M tokens vs. $2.00 for GPT-4.1).

5. Significance

• Production Viability: The models offer a viable alternative for production deployments in Brazil, particularly for legal tech, government services, and enterprise RAG (Retrieval-Augmented Generation) systems where cost efficiency and domain accuracy are paramount.
• Domain Specialization Strategy: The report validates that continued pre-training on domain-specific corpora (legal) followed by targeted fine-tuning is a highly effective strategy for achieving state-of-the-art performance in specific verticals without the computational overhead of training massive foundation models from scratch.
• Regional Language Advancement: By focusing on Brazilian Portuguese nuances and local legislation, these models bridge the gap between global LLM capabilities and local regulatory requirements, reducing hallucination in legal contexts.
• Future Roadmap: The authors outline plans for 256K context windows, enhanced reasoning capabilities, and stable releases based on user feedback, indicating a commitment to continuous improvement in the agentic and reasoning domains.