💻 computer science

An Effective and Cost-Efficient Agentic Framework for Ethereum Smart Contract Auditing

This paper introduces Heimdallr, a cost-efficient agentic framework for Ethereum smart contract auditing that leverages function-level code reorganization, heuristic reasoning, and cascaded verification to achieve high-precision vulnerability detection and exploit chaining using lightweight open-source models, significantly outperforming existing tools in speed, cost, and accuracy while successfully identifying real-world attacks and zero-day vulnerabilities.

Original authors: Xiaohui Hu, Wun Yu Chan, Yuejie Shi, Qumeng Sun, Wei-Cheng Wang, Chiachih Wu, Haoyu Wang, Ningyu He

Published 2026-01-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaohui Hu, Wun Yu Chan, Yuejie Shi, Qumeng Sun, Wei-Cheng Wang, Chiachih Wu, Haoyu Wang, Ningyu He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Unscalable, Noisy, and Expensive" Security Guard

Imagine Smart Contracts as the automated rules of a digital bank. If the code has a bug, hackers can steal millions. Currently, checking these rules is a nightmare:

Manual Auditing (The Human Expert): This is like hiring a team of elite detectives to read every single page of a library. It's incredibly thorough, but it costs a fortune (tens of thousands of dollars) and takes weeks. Small projects can't afford it.
Static Analyzers (The Metal Detector): These are automated tools that scan for known "bad shapes" in the code. The problem? They are like a metal detector that beeps at everything—coins, keys, and even a soda can. They generate so many false alarms (false positives) that human auditors get overwhelmed and ignore them.
Early AI Auditors (The Hallucinating Intern): New tools using AI (Large Language Models) were introduced to speed things up. But they are like an overconfident intern who makes things up. They often invent bugs that don't exist (hallucinations) or miss complex tricks because they don't understand the full story of the bank's rules. They are also very expensive to run.

The Solution: Heimdallr (The "Super-Intern" with a Checklist)

The authors built Heimdallr, a new automated system designed to be the "Goldilocks" of security: not too expensive, not too noisy, and actually smart. They call it an "Agentic Framework," which is a fancy way of saying it's an AI agent that doesn't just guess; it plans, checks, and verifies.

Think of Heimdallr as a three-stage security process:

Stage 1: The Organizer (Contextual Profiling)

The Problem: Imagine trying to read a 1,000-page novel all at once. You'd get lost. Similarly, AI models get confused if you feed them too much code at once.
Heimdallr's Trick: Instead of reading the whole book at once, Heimdallr acts like a librarian who groups the story into logical chapters. It looks at how different parts of the code talk to each other (like who calls whom) and bundles them into small, manageable "chunks" that make sense together. This ensures the AI doesn't miss the connection between two distant parts of the code.

Stage 2: The Detective (Model-Agnostic Auditing)

The Problem: A standard AI might just say, "This looks suspicious." But is it actually a crime?
Heimdallr's Trick: Heimdallr uses a "Plan-Remind-Solve" strategy:

Plan: It looks at the code chunk and asks, "Where could a hacker strike?"
Remind: Before guessing, it pulls out a "Cheat Sheet" (a Knowledge Base) of real historical hacks and recent news. It reminds the AI, "Hey, remember how this specific type of bank was robbed last year? Check if this looks like that."
Solve: It doesn't just guess. It runs three different types of tests:
1. The "What If" Test: It imagines a hacker with infinite money or control over the weather (blockchain time) to see if the system breaks under extreme pressure.
2. The Math Test: It uses a strict math calculator (Z3 solver) to check if the numbers add up correctly, because AI is bad at math but great at logic.
3. The Chain Reaction Test: It checks if a small bug in one room can trigger a chain reaction that blows up the whole building.

Stage 3: The Filter (False Positive Filtration)

The Problem: Even a good detective might flag innocent people.
Heimdallr's Trick: This is the most important part. Before showing the results to a human, Heimdallr runs a second, stricter check. It asks: "Is this actually a real threat, or did we just imagine it?" It filters out the noise, ensuring that when it says "There is a bug," it is almost certainly true.

The Results: Why It Matters

The paper claims Heimdallr is a massive leap forward:

It's a Detective, Not a Guessing Machine: When tested on 20 real-world hacks that happened recently (after June 2025), Heimdallr successfully figured out 17 of them and explained exactly how the hackers did it. It also found 4 brand-new (zero-day) bugs in live systems that could have saved $400 million.
It's Cheap: While other tools might cost $15–$ 50 to scan a project, Heimdallr can do it for as little as $0.22 (using smaller, open-source AI models) or $2.31 (using the most powerful models).
It's Fast: It can scan a project in minutes, whereas older tools took hours.
It Beats the Competition: In a head-to-head test against other top tools, Heimdallr found far more real bugs and reported far fewer fake ones. In a contest against 548 human auditors, the automated Heimdallr ranked in the top 4%.

The Catch (Limitations)

The authors are honest about what Heimdallr can't do yet:

Language: It currently only speaks "Solidity" (the language of Ethereum). It can't read code written for other blockchains yet.
Long-Term Tricks: It's great at catching immediate hacks, but it might miss very slow, long-term attacks that take weeks to unfold (like a slow poison).
Safety: Because it is so good at finding holes, the authors decided not to release the code for free. They don't want hackers to use it to find holes in banks. Instead, they offer it as a secure service that vetted professionals can use.

Summary

Heimdallr is like a super-intelligent, cost-effective security guard that organizes the code, studies past crimes, runs strict math tests, and double-checks its own work before ringing the alarm. It solves the old problem of "expensive but slow" vs. "cheap but full of lies," offering a solution that is fast, cheap, and actually reliable.

Technical Summary: Heimdallr – An Effective and Cost-Efficient Agentic Framework for Ethereum Smart Contract Auditing

1. Problem Statement

The rapid expansion of Decentralized Finance (DeFi) has been accompanied by a surge in security incidents, with over 700 attacks causing approximately $3.35 billion in losses in a single year. Existing smart contract auditing solutions face critical limitations:

Manual Auditing: While effective, it is prohibitively expensive (often $70,000–$ 150,000+ per protocol) and unscalable.
Static Analyzers (e.g., Slither, Mythril): These tools suffer from high false positive rates and a limited scope, failing to detect complex, context-dependent business logic vulnerabilities.
LLM-Based Auditing: While Large Language Models (LLMs) offer reasoning capabilities, current implementations are plagued by hallucinations, context window constraints, and high operational costs. Direct LLM application often results in negligible detection capabilities (F1 Score < 0.02) or unmanageable noise (False Positive Rate > 97%).

The core challenge is to leverage the automation and reasoning of LLMs to achieve scalable, highly reliable contract audits while minimizing false positives and balancing cost-efficiency without compromising detection accuracy.

2. Methodology: The Heimdallr Framework

Heimdallr is a modular, model-agnostic agentic framework designed to mimic the rigorous reasoning of human experts through three coupled stages: Contextual Profiling, Model-Agnostic Auditing, and False Positive Filtration.

2.1 Contextual Profiling

To address the trade-off between scalability and context preservation, Heimdallr partitions the codebase into semantically coherent batches rather than analyzing files arbitrarily.

Graph Construction: It constructs a static directed dependency graph ( $G$ ) using Slither, capturing control flow and data flow (state variable access) between functions.
Community Detection: The Louvain algorithm partitions the graph into densely connected subgraphs (communities), grouping functions that interact densely regardless of file location.
Importance Analysis: Within each batch, contracts are scored based on betweenness centrality and PageRank to identify "Anchor Contracts" (core business logic) versus peripheral components.
LLM-Driven Refinement: An LLM refines these batches by merging those with related semantic tags (e.g., "Swap") and pruning standardized library contracts to fit within token limits while preserving custom business logic.

2.2 Model-Agnostic Auditing

The auditing engine employs a "Plan-Remind-Solve" workflow to generate vulnerability hypotheses ( $V_{raw}$ ).

Plan: The agent identifies potentially vulnerable functions and contracts within a batch, prioritizing them based on structural importance and potential consequences.
Remind: The system retrieves relevant knowledge from a hierarchical Knowledge Base ( $K$ ) containing historical attack reports and patterns. It also utilizes a Model Context Protocol (MCP) server to perform real-time searches for derivative protocols and recent incidents, augmenting the context with live data.
Solve: This phase integrates three distinct strategies to generate hypotheses:
1. Adversarial State Assumption Analysis: The agent simulates extreme hostile conditions (e.g., miner control over block variables, interaction hijacking, infinite resources via flash loans) to uncover vulnerabilities that only manifest under stress.
2. Targeted Symbolic Constraint Solving: For functions involving sensitive arithmetic, the system transpiles Solidity code to Z3 constraints to mathematically verify precision loss or boundary conditions, combining formal rigor with semantic reasoning.
3. Global Synthesis: An iterative combination process links isolated vulnerabilities to construct multi-step exploit chains, addressing the "cognitive tunneling" limitation of models.

2.3 False Positive Filtration

A dedicated verification layer ( $\Psi$ ) processes raw hypotheses to produce a final, high-precision report ( $V_{final}$ ).

Contextual Aggregation: The agent re-evaluates each hypothesis under a global context to ensure local risks are not neutralized by global safeguards (e.g., nonReentrant modifiers).
Semantic Deduplication: Using embeddings and clustering (DBSCAN), redundant reports triggered by different entry points but sharing the same root cause are merged, retaining the highest-confidence version.
Threat Model Assessment: Hypotheses relying on technically infeasible external assumptions (e.g., compromised infrastructure or dishonest developers with backdoors) are discarded.

3. Key Contributions

Novel Auditing Methodology: Heimdallr combines graph-theoretic profiling with neuro-symbolic reasoning. Its "Plan-Remind-Solve" workflow detects complex logic vulnerabilities, while the adversarial verification module rigorously suppresses false positives.
Production-Ready Service: The framework is released as a standardized API service for CI/CD integration and supports privacy-preserving local deployment using open-source models (e.g., GPT-oss-120B) in air-gapped environments.
Comprehensive Empirical Evaluation: The authors benchmark Heimdallr against four state-of-the-art baselines (including industrial tools and academic prototypes), demonstrating superior F1 scores and cost-efficiency.
High-Fidelity Benchmarks: The paper constructs and releases three datasets covering high-value zero-day exploits, standardized benchmarks, and crowdsourced contest findings to address the scarcity of ground-truth data.

4. Experimental Results

Evaluations were conducted on three datasets: Real-world High-value Exploits (D1), Proprietary Benchmarks (D2), and Crowdsourced Auditing Projects (D3).

Detection Effectiveness: Powered by Claude-Sonnet-4.5, Heimdallr achieved an 86.7% detection rate on high-value exploits (D1 & D2), significantly outperforming the Claude baseline (51.1%) and static tools like Hound (20.0%).
False Positive Reduction: Unlike academic tools which exhibited false positive rates exceeding 97%, Heimdallr maintained a false positive rate between 31.27% and 49.12% across various models.
Cost-Efficiency: Heimdallr reduced analysis time by up to 97.59% and financial costs by 98.77% compared to baselines. The average cost was $0.22 per project or $2.31 per 10K LOC for high-precision configurations.
Real-World Validation:
- Zero-Day Discovery: Heimdallr identified 3 critical zero-day vulnerabilities in a live protocol with $400M TVL and flagged 4 medium-severity issues in a collaboration with an auditing firm.
- Contest Performance: In Sherlock auditing contests, Heimdallr achieved a 92.45% detection rate at a negligible cost, ranking 19th out of 548 human auditors (top 4%).
- Retrospective Analysis: The system successfully reconstructed 17 out of 20 real-world attacks occurring after June 2025 (totaling $384M in losses), which were not part of its training data.

5. Significance and Claims

The paper claims that Heimdallr resolves the scalability-precision trade-off in DeFi security. By achieving high detection rates (86.7%) with low false positives and minimal cost ($0.22/project), it shifts the paradigm from episodic "audit-once" checks to continuous DevSecOps.

Crucially, the authors emphasize that the framework's model-agnostic architecture allows for effective auditing using lightweight, open-source models (like GPT-oss-120B) on consumer-grade hardware. This capability addresses critical institutional needs for data sovereignty, enabling rigorous, air-gapped security analysis without exposing proprietary code to third-party servers. The work demonstrates that with proper agentic orchestration and verification layers, LLMs can be made reliable and cost-effective for detecting intricate business logic vulnerabilities that traditional tools miss.