💻 computer science

Enhancing Smart Contract Vulnerability Detection in DApps Leveraging Fine-Tuned LLM

This paper proposes a novel approach that leverages fine-tuned Large Language Models (Llama3-8B and Qwen2-7B) trained on a comprehensive dataset of 215 real-world DApp projects to significantly enhance the detection of smart contract vulnerabilities, particularly complex logical errors like token price manipulation that traditional tools often miss.

Original authors: Jiuyang Bu, Wenkai Li, Zongwei Li, Zeng Zhang, Xiaoqi Li

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiuyang Bu, Wenkai Li, Zongwei Li, Zeng Zhang, Xiaoqi Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Smart Contract as a digital vending machine that lives on a blockchain. Once you put money in, it automatically gives you a soda (or a digital token) based on the rules you programmed into it. The problem is, if the programmer made a mistake in the code, the machine might give you free soda, steal your money, or get stuck forever. Because these machines are "immutable" (you can't change the rules once they are built), finding those mistakes before they go live is critical.

This paper is about building a super-smart security guard to check these vending machines for flaws before they open.

The Problem: Old Guards vs. New Tricks

Traditionally, security guards used "rulebooks" (static analysis tools). They would look for specific, known mistakes, like "Is there a hole in the floor?" or "Is the door locked?"

The Issue: Hackers are getting clever. They aren't just breaking locks; they are finding weird, logical loopholes. For example, "If I buy a token at 9:00 AM and sell it at 9:01 AM, can I trick the machine into thinking the price is higher than it is?"
The Limitation: The old rulebooks can't see these complex, logical tricks. They are like a guard who only knows how to check for broken windows but doesn't understand how a thief might pick a lock.

The Solution: A "Specialized" AI Guard

The authors decided to use a Large Language Model (LLM)—a type of AI that is really good at reading and understanding human language and code. Think of a standard LLM as a generalist librarian. They know a lot about everything, but they haven't specifically studied "blockchain security."

If you ask a generalist librarian, "Is this code safe?" they might guess, but they often get it wrong because they haven't seen enough specific examples of bad code.

The authors' innovation: Instead of just asking the librarian to guess, they trained the librarian specifically on a massive library of real-world blockchain disasters. They turned the generalist into a specialized security expert.

How They Built the Expert

The Training Data (The "Case Files"):
Most previous studies used tiny, simple code examples (like a 40-line script). That's like training a security guard on a toy vending machine.
- What they did: The team collected 215 real-world projects containing nearly 5,000 contracts. They even included tricky, hard-to-detect "logical errors" like token price manipulation (where hackers trick the system into changing the value of money).
- The Tool: They built a robot tool called "SmartCollect" to gather all the messy, interconnected files that make up a real app, ensuring the AI saw the whole picture, not just isolated snippets.
Fixing the Imbalance (The "Rare Disease" Problem):
In the real world, most code is safe, and very little is broken. This is like a doctor trying to learn how to diagnose a rare disease when 99% of their patients are healthy. The AI gets confused and just says "Everything is fine" to be safe.
- The Fix: They used a technique called Random Over Sampling (ROS). Imagine taking the few examples of "broken machines" and making photocopies of them so the AI sees them just as often as the "working machines." This forces the AI to pay attention to the flaws.
The Training Methods (Two Ways to Teach):
- Full-Parameter Fine-Tuning (FFT): This is like rewriting the entire brain of the AI to specialize in security. It's heavy and takes a lot of energy, but it makes the AI a true expert.
- LoRA (Low-Rank Adaptation): This is like adding a specialized notebook to the AI's existing brain. It's lighter and faster, but the paper found it wasn't quite as good at catching the tricky bugs as the full rewrite.
The "Dual Audit" System:
The AI doesn't just guess. It plays two roles:
- The Auditor: Looks at the code and says, "I think there's a problem here."
- The Verifier: Double-checks the Auditor's work to make sure it's not just hallucinating.

The Results: Did it Work?

The paper compares their specialized AI against the "generalist" AI (just asking it questions without training) and other security tools.

Generalist AI: Got it right only about 20% of the time. It was mostly guessing.
Specialized AI (with Full Training): Got it right about 83% of the time (F1-score).
Beating the Competition: Their method was better than other top tools (like GPTLENS and GPTSCAN).
The "Unauditable" Wins: The biggest victory was catching price manipulation flaws. These are the "logical errors" that computers usually miss. Their AI caught these with 97% precision (when it said "this is broken," it was almost always right).

The Bottom Line

The paper argues that to catch the smartest hackers, you can't just use a generic AI or a simple rulebook. You need to take a powerful AI, feed it a massive amount of real-world, messy, complex code, and train it specifically to spot the subtle, logical tricks that humans and old computers miss. By doing this, they created a much more reliable security guard for the world of Decentralized Applications (DApps).

Technical Summary: Enhancing Smart Contract Vulnerability Detection in DApps Leveraging Fine-Tuned LLM

Problem Statement
Decentralized applications (DApps) face critical security risks due to vulnerabilities in their underlying smart contracts. While traditional detection methods, such as static analysis and state transition verification, are effective for known patterns (e.g., reentrancy, arithmetic errors), they struggle with emerging threats and "machine-unauditable" logical flaws. Recent statistics indicate that nearly 80% of contract vulnerabilities are logical errors, including complex issues like token price manipulation, which evade conventional program analysis. Furthermore, existing benchmarks often rely on simplified datasets (contracts with fewer than 40 lines), failing to represent the complexity of real-world DApp projects. Current Large Language Model (LLM) approaches using prompt engineering or Retrieval-Augmented Generation (RAG) also show limitations, achieving only 40–60% accuracy due to a lack of domain-specific adaptation to the Solidity programming language.

Methodology
The paper proposes a novel framework that leverages fine-tuned LLMs to enhance vulnerability detection in DApps. The methodology consists of four core components:

Comprehensive Data Collection:
The authors constructed a new dataset comprising 215 real-world DApp projects and 4,998 smart contracts. Unlike previous datasets, this collection was gathered using a custom Python tool named SmartCollect, which recovers complete project dependencies, including public libraries and external contracts. The dataset specifically addresses the scarcity of logical error examples by including 23 contracts with token price manipulation vulnerabilities, alongside standard vulnerabilities like reentrancy, arithmetic issues, and timestamp dependence.
Data Augmentation (ROS):
To address the severe class imbalance inherent in vulnerability detection (where vulnerable cases are a minority), the study employs Random Over Sampling (ROS). This technique duplicates samples from the minority class to balance the dataset, thereby mitigating the performance degradation often seen in deep learning models trained on imbalanced real-world data.
Dual-Role LLM Architecture:
The proposed system utilizes a two-step audit process where the LLM assumes two distinct roles activated via prompt engineering:
- Smart Contract Auditor: Reviews code to identify potential vulnerabilities and generate reasoning.
- Verifier: Further validates the identified vulnerability types to ensure accuracy and rigor.
  The prompts are designed with Chain-of-Thought (CoT) reasoning, requiring the model to emulate an auditor's workflow by understanding context, analyzing logic, and providing structured outputs.
Fine-Tuning Strategies:
The study evaluates two fine-tuning approaches on mainstream open-source models (Llama3-8B and Qwen2-7B):
- Full-Parameter Fine-Tuning (FFT): Updates all model parameters via gradient descent to achieve optimal task-specific performance.
- Low-Rank Adaptation (LoRA): A parameter-efficient method that freezes the base model weights and trains low-rank matrices ( $A$ and $B$ ) to approximate weight updates, reducing computational overhead.

Key Contributions

Real-World Dataset: The creation of a large-scale, representative dataset of 215 DApp projects (4,998 contracts) that includes complex logical errors and token price manipulation flaws, addressing the gap left by simplified existing benchmarks.
Fine-Tuning Framework: A demonstration that domain-specific fine-tuning (FFT and LoRA) significantly outperforms prompt-based LLMs and state-of-the-art tools (GPTLENS, GPTSCAN) in detecting smart contract vulnerabilities.
Logical Error Detection: A specific focus on "machine-unauditable" vulnerabilities, proving that fine-tuned LLMs can effectively detect logical flaws like price manipulation, which traditional tools miss.
Data Augmentation Validation: Empirical evidence showing that ROS significantly improves model performance on imbalanced vulnerability datasets.

Experimental Results
The study conducted comparative experiments to answer four research questions regarding performance, fine-tuning impact, logical error detection, and data augmentation:

Performance vs. Baselines: Fine-tuned models significantly outperformed non-fine-tuned prompt-based approaches. While a standard Qwen2-7B prompt achieved an F1-score of only 0.20, the fine-tuned Llama3-8B-FFT achieved an F1-score of 0.73 (Precision: 0.78, Recall: 0.70).
Comparison with SOTA: The fine-tuned models (Llama3-8B-FFT and Qwen2-7B-FFT) achieved a precision of 0.78, surpassing GPTLENS (0.64) and GPTSCAN (0.57).
Logical Error Detection: The approach demonstrated exceptional capability in detecting price manipulation vulnerabilities (PMV). The Llama3-8B-FFT model achieved 0.97 precision and 0.68 recall for PMV, highlighting its ability to handle non-machine-auditable flaws.
Impact of Data Augmentation: Applying ROS to the fine-tuning process yielded further improvements. The Llama3-8B-FFT-ROS model achieved an F1-score of 0.83 (Precision: 0.84, Recall: 0.79), representing a 6–7% increase in precision and a 9–16% increase in recall compared to the non-augmented FFT baseline.

Significance
The paper asserts that domain-specific fine-tuning of LLMs, combined with data augmentation techniques like ROS, offers a robust solution for the evolving security challenges in the blockchain ecosystem. By moving beyond predefined vulnerability patterns and general-purpose prompting, this approach effectively addresses the detection of complex, logical, and machine-unauditable vulnerabilities in real-world DApps. The results underscore the potential of fine-tuned LLMs to serve as a critical layer in blockchain protection, particularly for identifying the sophisticated logical errors that constitute the majority of modern smart contract risks.