🤖 AI

Toward Cybersecurity-Expert Small Language Models

The paper introduces CyberPal 2.0, a family of cybersecurity-expert small language models (4B–20B parameters) trained on an enriched dataset via the SecKnowledge 2.0 pipeline, which outperforms or matches frontier models like GPT-4o and o1 on key threat investigation and intelligence tasks while maintaining a significantly smaller size.

Original authors: Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche

Published 2026-07-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a library of giant, super-smart robots (Large Language Models, or LLMs) that can write poetry, solve math problems, and chat about almost anything. But if you ask them to act like a cybersecurity expert—hunting down digital threats, understanding complex hacking techniques, or fixing security holes—they often stumble. They might guess, hallucinate facts, or lack the deep, specific training needed for high-stakes security work.

This paper introduces CyberPal 2.0, a new family of "smaller" but highly specialized robots designed specifically to be cybersecurity experts. Here is how they did it, explained simply:

1. The Problem: The "Generalist" vs. The "Specialist"

Think of the big, famous AI models (like GPT-4 or o1) as general practitioners in a hospital. They are incredibly smart and can handle almost any medical question. However, for a complex, rare disease (like a specific type of cyber-attack), you really want a specialist surgeon who has spent years studying just that one thing.

The problem is that big models are expensive to run, and sending sensitive company data to them is often against privacy rules. So, the authors wanted to build a "surgeon" that is small enough to fit in a hospital's own basement (on-premises) but smart enough to handle the toughest security cases.

2. The Secret Sauce: "SecKnowledge 2.0" (The Training Camp)

You can't just teach a robot to be a security expert by giving it a stack of textbooks. The authors built a special training camp called SecKnowledge 2.0.

The Old Way: Imagine a teacher asking a student to explain a security concept, and the student gives a short, vague answer.
The New Way (Expert-in-the-Loop): The authors created a system where human security experts act as coaches. They don't just write the questions; they design the exact steps the robot must take to answer.
- Analogy: Instead of just asking "How do I fix this leak?", the coach says, "First, check the pipe material. Second, look at the pressure gauge. Third, consult the manual for that specific valve. Only then give the answer."
The Fact-Checkers: To stop the robot from making things up (hallucinating), the system forces the robot to go out and find real evidence (like searching a database or reading a specific security report) before it answers. It's like a detective who must show their evidence board before solving the case.

3. The Result: The CyberPal 2.0 Family

The result is a family of models ranging from 4 billion to 20 billion "neurons" (parameters).

Small but Mighty: Even the smallest model (4B) is tiny compared to the giant models (which can be hundreds of billions of parameters), but it punches way above its weight class.
The Performance: When tested on difficult security exams (like matching a bug report to a specific weakness or identifying hacker tactics), these small models:
- Beat their own "untrained" versions by a huge margin (7–14% better).
- Crushed other open-source security models.
- Matched or beat the biggest, most expensive "frontier" models (like Google's Sec-Gemini v1 and OpenAI's o1) on core tasks, despite being a fraction of their size.

4. Why This Matters (According to the Paper)

The paper claims this approach solves three big headaches for companies:

Privacy: You can run these models on your own servers without sending sensitive data to the cloud.
Cost: They are much cheaper to run than the giant models.
Reliability: Because they are trained with human experts and forced to find real evidence, they are less likely to make dangerous mistakes or "make things up."

Summary Analogy

If the big AI models are like Olympic athletes who can run, swim, and jump, CyberPal 2.0 is like a specialized rescue dog. It might not be able to do everything an Olympic athlete can do, but if you need to find a specific person in a collapsed building (a specific cyber threat), this dog is faster, cheaper to feed, and better trained for that exact job than the athlete.

The authors have made their training methods and the best model (the 20B version) open source, so other companies can build their own "rescue dogs" without starting from scratch.

Technical Summary: Toward Cybersecurity-Expert Small Language Models

Problem Statement

While Large Language Models (LLMs) are transforming general applications, they lag in specialized domains like cybersecurity due to a scarcity of high-quality, domain-specific models and training datasets. Adopting frontier commercial models for security operations presents significant challenges: strict safety guardrails often limit their utility in real-world workflows, compliance requirements restrict the integration of sensitive organizational data, and the cost of processing massive volumes of security telemetry is prohibitive. Consequently, there is a critical need for domain-specific Small Language Models (SLMs) that can operate on-premises, ensure data privacy, and deliver deep technical reasoning across network, cloud, and application security without the overhead of frontier-scale models.

Methodology

The authors propose CyberPal 2.0, a family of cybersecurity-expert SLMs ranging from 4B to 20B parameters. The core innovation lies in the training data pipeline, SecKnowledge 2.0, which enhances the original SecKnowledge dataset through a multi-stage reformatting and enrichment process.

The SecKnowledge 2.0 Pipeline

The pipeline moves beyond standard data reformatting by integrating expert-in-the-loop steering with LLM-driven automation:

Expert-in-the-Loop Format Definition: Instead of using generic templates, the system employs a semi-automated workflow where experts define specific reasoning structures (formats) for distinct cybersecurity tasks. An LLM generates candidate formats based on task descriptions and illustrative examples, which experts then evaluate and refine. This ensures the reasoning traces align with expert intent and domain logic.
Multi-Step Grounding and Retrieval: To mitigate hallucinations and ensure factual accuracy, the pipeline incorporates a retrieval-augmented generation (RAG) component. For each task, the system generates search queries, filters them for relevance, retrieves results from vector databases or the web, and parses them to anchor the reasoning steps in external evidence.
Adaptive Reasoning Depth: The training data is curated to include both long-form, step-by-step chain-of-thought (CoT) examples for complex reasoning and concise, fast-response instructions for lighter tasks. This mixture teaches the models to adapt their reasoning effort based on task difficulty.

Training Strategy

The models are fine-tuned using the Qwen3 base models (4B, 8B, 14B) and gpt-oss-20b as starting points. Notably, the authors found that initializing from base models (pre-trained but not post-trained/aligned) yielded significantly better performance than starting from post-trained counterparts, particularly on reasoning-heavy benchmarks. The training utilizes a learning rate of $4 \times 10^{-5}$ , a context length of 8192, and a batch size of 3072 for two epochs.

Key Contributions

SecKnowledge 2.0 Pipeline: A novel data enrichment pipeline that combines schema-driven, expert-defined reasoning formats with multi-source document grounding. This approach yields higher-fidelity, task-grounded reasoning traces compared to standard reformatting methods.
CyberPal 2.0 Model Suite: A family of open-source SLMs (4B–20B parameters) specialized for cybersecurity operations, capable of handling detection, investigation, threat hunting, and data classification.
Comprehensive Evaluation: The models were evaluated against a rigorous suite of benchmarks covering governance, threat intelligence, incident response, and secure code generation, including comparisons against state-of-the-art frontier models (e.g., GPT-4o, o1, o3-mini, Sec-Gemini v1).

Results

The evaluation demonstrates that CyberPal 2.0 models consistently outperform their baselines and compete with or surpass much larger frontier models:

Baseline Performance: Across diverse benchmarks, CyberPal 2.0 models achieved average gains of 7–14% over their respective base models. On specific tasks like CTIBench-RCM (Root Cause Mapping), gains reached 16–31%.
Frontier Model Comparison:
- Root Cause Mapping (RCM): The 20B parameter model ranked first, outperforming Sec-Gemini v1, o1, o3-mini, and GPT-4o. The 4B model ranked second.
- Cyber Threat Intelligence (CTI) Knowledge: The models ranked second only to Sec-Gemini v1, outperforming almost all other tested frontier models, including o1.
- Efficiency: Despite being a fraction of the size of frontier models, the 20B model matched or exceeded their performance, while the 4B model outperformed significantly larger models like Mistral Large and DeepSeek-v3.
Real-World Use Cases: In applied workflows such as mapping threat reports to MITRE ATT&CK techniques and reassessing CVE severity scores under specific deployment constraints, the models showed consistent improvements over baselines (e.g., +8.75% accuracy on threat-to-TTP mapping for the 4B model).
Secure Code Generation: While specialized for security reasoning, the models maintained robust secure code generation capabilities, with pass rates remaining close to their instruction-tuned baselines, indicating that specialization did not compromise code safety.

Significance and Claims

The paper claims that CyberPal 2.0 demonstrates that compact, domain-specific SLMs can deliver frontier-level capabilities for security operations without the associated frontier-level costs. By leveraging the SecKnowledge 2.0 pipeline—specifically the combination of expert-defined reasoning structures and evidence grounding—the authors show that high-quality, specialized data is more impactful than simply scaling model size.

The work addresses the practical constraints of enterprise security, offering models that are:

Deployable On-Premises: Meeting privacy and data residency obligations.
Cost-Effective: Efficiently handling massive security data volumes.
Actionable: Providing defensible, evidence-grounded conclusions for threat investigation and response.

The authors emphasize that their approach prioritizes data quality and domain-specific alignment over sheer parameter count, providing a viable path for enterprises to adopt advanced AI for defensive security operations. The models, pipeline, and benchmarks are released as open source to foster reproducibility and practical adoption.