💬 NLP

Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation

This paper introduces a decoupled, lightweight LLM framework that utilizes distillation, Rank-Stabilized Low-Rank Adapters, and a custom Chain-of-Verification aggregation strategy to achieve state-of-the-art smart contract security auditing performance with significantly fewer parameters than existing models.

Original authors: Bagus Rakadyanto Oktavianto Putra, Muhamad Risqi Utama Saputra, Widyawan, Guntur Dharma Putra

Published 2026-06-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bagus Rakadyanto Oktavianto Putra, Muhamad Risqi Utama Saputra, Widyawan, Guntur Dharma Putra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very complex, digital vault (a smart contract) that holds people's money. Once this vault is built on the blockchain, you can't change the locks or add new security bars later. If there's a flaw, hackers can steal everything. So, before you open the vault, you need a security expert to check it.

Traditionally, this check is done by humans, but there are too many vaults to check manually. Recently, people tried using giant "super-intelligences" (Large Language Models or LLMs) to do the checking. But these giants are like trying to hire a team of 100 PhDs to do a single job: they are incredibly expensive, require massive computers to run, and often get confused when asked to do too many things at once.

This paper introduces a smarter, lighter way to do this. Here is how they did it, explained simply:

1. The "Specialized Team" vs. The "Jack-of-All-Trades"

Instead of hiring one giant, expensive AI to do the whole job at once, the authors broke the security audit down into four small, specialized steps, like a relay race:

The Detector: A tiny, fast AI that just looks at the code and says, "Is there a bug here? Yes or No?"
The Explainer: If there is a bug, a slightly larger AI explains how a hacker could use it. It's like a detective writing a report on how the lock was picked.
The Judge: Another small AI looks at that explanation and decides: "Is this a minor scratch (Low), a broken hinge (Medium), or a total collapse (High)?"
The Fixer: Finally, a smart AI suggests exactly how to patch the hole.

The Analogy: Imagine trying to fix a car. A "Unified" approach is like asking one person to diagnose the engine, explain the physics of the failure, grade the severity, and weld the new part all in one breath. They might get overwhelmed and make mistakes. The authors' approach is like having a specialized mechanic, a physics professor, a safety inspector, and a welder working in a line. Each does one thing perfectly.

2. The "Small Brain" with a "Big Teacher"

The authors used very small AI models (only 0.6 to 4 billion "neurons," compared to the giants which have 30+ billion). Usually, small brains aren't smart enough for hard tasks.

To fix this, they used two tricks:

Knowledge Distillation (The Teacher-Student): They took a massive, super-smart "Teacher" AI (Qwen3-30B) and taught the small "Student" AI how to think step-by-step. The student didn't just memorize answers; it learned the reasoning process.
The "Voting" System (Consensus): When the AI has to make a decision (like "Is this a bug?"), instead of asking it once, they ask it five different ways. If four out of five say "Yes," they go with "Yes." This stops the AI from guessing randomly.

3. The "Fact-Checker" to Stop Hallucinations

AI models sometimes "hallucinate"—they make up facts that sound real but are wrong. To stop this, the authors used a Chain-of-Verification method.

The Analogy: Imagine a student writes an essay. Before handing it in, they have to write down three questions about their own essay ("Did I prove this point? Is this fact true?"). They answer those questions honestly. If the answers don't match the essay, they fix the essay. This ensures the final report is actually true.

4. The Big Discovery: The "Middle-Child" Bias

The researchers found something surprising about how AI judges severity. When asked to decide if a bug is "Low," "Medium," or "High," the AI gets scared of being wrong.

The Finding: If the AI is unsure, it almost always defaults to "Medium." It's like a student who doesn't know the answer to a test question and just circles "C" because it feels like the safest middle ground.
The authors found that the giant models were terrible at spotting the extreme cases (very low or very high risks) because they kept shoving everything into the "Medium" bucket. Their smaller, specialized team was much better at spotting the extremes.

5. Why This Matters (The Results)

Cheaper: Their system runs on a standard laptop or a modest computer card. The giant models they compared against need super-expensive, data-center-grade computers that cost thousands of dollars.
Smarter: Despite being much smaller, their system was more accurate than the giant models. They got 98% accuracy in finding bugs, while the giants struggled with the complex reasoning parts.
Actionable: They didn't just find bugs; they provided clear explanations and specific code fixes, which is what developers actually need.

In short: The paper proves that you don't need a giant, expensive AI to secure your digital vault. By using a team of small, specialized, well-trained AI assistants that check each other's work, you can get better results for a fraction of the cost.

Technical Summary: Decoupled Smart Contract Audits via Lightweight LLMs

Problem Statement

Smart contracts, being immutable once deployed on blockchains, are highly susceptible to security vulnerabilities that can lead to catastrophic financial losses (e.g., the DAO hack). While Large Language Models (LLMs) have shown promise in automated security auditing, existing approaches face three critical limitations:

Computational Inefficiency: Current state-of-the-art (SOTA) solutions rely on massive, monolithic LLMs (7B–34B parameters) that require substantial GPU VRAM (often exceeding 12–24 GB), making them inaccessible for independent developers and small teams using standard consumer hardware.
Lack of Actionable Output: Many approaches focus primarily on binary vulnerability detection, neglecting the critical need for severity classification and specific, actionable remediation recommendations.
Architectural Flaws: Unified prompting strategies force a single model to simultaneously perform diverse reasoning tasks (detection, explanation, severity, remediation), leading to error propagation, format hallucinations, and high latency.

Methodology

The authors propose an efficient, decoupled end-to-end framework that utilizes lightweight, open-source LLMs (0.6B–4B parameters) to replicate the workflow of human security experts. The framework divides the audit process into four specialized, sequential modules:

Vulnerability Detector: A classification task to identify if code contains vulnerabilities.
Vulnerability Explanator: A generative task to explain the nature of the vulnerability and how it can be exploited.
Severity Determinant: A classification task to assign a severity level (Low, Medium, High).
Recommender: A generative task to provide actionable code patches for remediation.

Key Technical Components

Rank-Stabilized Low-Rank Adapters (rsLoRA): Instead of full fine-tuning or standard LoRA, the authors employ rsLoRA. This technique modifies the scaling factor to $\gamma_r = \alpha / \sqrt{r}$ , preventing gradient collapse and allowing for larger ranks without sacrificing parameter efficiency.
Multi-Prompt Voting Mechanism: For classification tasks (Detection and Severity), the model processes five distinct template prompts. The final output is determined by majority voting, with a confidence score calculated based on the consensus.
Modified Chain-of-Verification (CoVe) Aggregation: For generative tasks (Explanation and Recommendation), the framework uses a "Candidate Generator" to produce multiple independent draft responses. A custom "Aggregator" then employs a modified CoVe strategy to:
1. Ground the context in the code.
2. Generate verification questions for each candidate.
3. Independently answer these questions to fact-check claims.
4. Synthesize a single, verified report, discarding hallucinations.
Knowledge Distillation: To enhance the reasoning capabilities of the student models (Qwen3-4B), the authors distill Chain-of-Thought (CoT) reasoning from a powerful teacher model (Qwen3-30B-A3B). The teacher's CoT outputs are used as part of the training data to teach the student model step-by-step reasoning.

Dataset Construction

The study utilized two distinct datasets derived from reputable sources (Code4rena, Shieldify Security, and top TVL projects):

Detection Dataset: 930 vulnerable and 969 benign code samples. Benign samples were processed using a Graph-Based Stratified Sampling algorithm to ensure length parity with vulnerable snippets, preventing the model from learning superficial length biases.
Vulnerability Dataset: 1,000+ samples containing code paired with audit metadata (explanations, severity, recommendations). This dataset was split using a Project-Aware Stratified strategy to prevent data leakage across training and testing sets.

Experimental Results

The framework was evaluated against SOTA dense coder models (e.g., CodeLlama-34B, DeepSeek-Coder-33B, Qwen2.5-Coder-32B) under both Zero-Shot and Fine-Tuned settings.

Vulnerability Detection: The proposed 0.6B model achieved 98.25% accuracy and an F1-score of 0.9824, outperforming all baselines, including fine-tuned 34B models.
Severity Classification: The framework achieved an accuracy of 64.58% and a macro F1-score of 0.5877.
Generative Tasks (Explanation & Recommendation): Using an LLM-as-a-Judge metric (LMUnit 70B), the framework achieved alignment scores of 0.4375 for explanations and 0.7639 for recommendations, significantly outperforming the best baselines (which scored ~0.22 and ~0.37 respectively).
Resource Efficiency: The decoupled approach operates within ~3.11 GB VRAM for classification and ~8.64 GB VRAM for generation, fitting comfortably on consumer hardware (8–12 GB VRAM), whereas SOTA baselines require 21–24 GB.

Key Contributions and Findings

Decoupled Architecture: The paper demonstrates that separating complex audit workflows into specialized modules significantly outperforms unified monolithic prompting, reducing error propagation and hallucination.
Lightweight Superiority: The study empirically validates that highly optimized, lightweight models (0.6B–4B) with targeted fine-tuning and aggregation strategies can consistently outperform much larger (7B–34B) dense models in specific security tasks.
Severity Centrality Bias (SCB): A novel empirical finding reveals that current LLMs (both zero-shot and fine-tuned) exhibit a "Severity Centrality Bias," where they disproportionately default to "Medium" severity to minimize loss, failing to accurately distinguish "Low" and "High" edge cases. The proposed framework mitigates this but does not fully eliminate it.
Synergistic Optimization: Ablation studies confirm that the combination of rsLoRA, Knowledge Distillation, and Modified CoVe aggregation creates a synergistic effect, where removing any single component leads to significant performance degradation.

Significance

The paper claims to establish a critical benchmark for future research in LLM-assisted auditing by proving that democratized smart contract security is feasible on standard consumer hardware. By moving away from resource-heavy monolithic models toward a modular, lightweight pipeline, the authors provide a practical pathway for integrating continuous, "shift-left" security auditing into everyday development workflows without requiring expensive cloud GPU instances. The identification of the Severity Centrality Bias further highlights a specific, unaddressed challenge in automated risk assessment that future research must tackle.