Detecting Malicious Intents in Smart Contracts with Pre-trained Programming Language Models

1. Problem Statement

Smart contracts are the backbone of decentralized applications (DApps), but they face significant security threats not only from accidental vulnerabilities but also from deliberately embedded malicious developer intents. Unlike standard vulnerabilities (e.g., reentrancy), malicious intents are intentional design choices by developers to exploit users (e.g., disabling trading, manipulating fees, or creating honeypots).

While vulnerability detection is well-studied, detecting these specific unsafe developer intents remains under-researched. The authors' previous work, SmartIntentNN, achieved an F1 score of 0.8633 but suffered from performance bottlenecks, particularly with minority-class intents due to data imbalance and the limitations of using the Universal Sentence Encoder (USE) for code representation.

2. Methodology: SmartIntentV2

The paper introduces SmartIntentV2, a state-of-the-art deep learning model designed to detect ten distinct categories of unsafe intents (Fee, DisableTrading, Blacklist, Reflect, MaxTX, Mint, Honeypot, Reward, Rebase, MaxSell). The architecture consists of three main stages:

A. Domain-Adaptive Pre-training (SmartBERT)

• Base Model: The authors initialized the model with CodeBERT (a BERT-based model pre-trained on code and natural language).
• Domain Adaptation: They performed further pre-training on a corpus of 16,000 real-world smart contracts using a Masked Language Modeling (MLM) objective.
• Granularity: Unlike previous approaches that treated contracts as single sequences, this model processes code at the function level to respect the 512-token limit and capture finer semantic details.
• Representation:
  • Function Level: Mean pooling is applied to token embeddings to generate a fixed-size vector for each function.
  • Contract Level: Function vectors are stacked into a matrix. Contracts with fewer than 256 functions are padded; longer ones are truncated.

B. Multi-Label Classification Network

• Encoder: A Bidirectional Long Short-Term Memory (BiLSTM) network processes the contract-level matrix to capture inter-function dependencies.
• Output Layer: A dense layer with a Sigmoid activation function produces probabilities for each of the 10 intent categories (multi-label classification).
• Loss Function: To address severe class imbalance (where some intents are rare), the model utilizes Binary Focal Loss. This loss function down-weights easy negative examples and focuses training on hard-to-classify samples.

C. Training Strategy

• Two-Phase Training:
  1. Primary Training: Standard training on the full dataset.
  2. Class-Balanced Training: A secondary phase where the model is fine-tuned by randomly sampling 10 instances from each intent category per batch. This ensures the model sees balanced data for minority classes without sacrificing majority class performance.

3. Key Contributions

1. SmartIntentV2 Model: A new architecture achieving an F1 score of 0.9279, surpassing its predecessor (SmartIntentNN) and all baselines.
2. SmartBERT: A domain-adapted pre-trained language model specifically for smart contracts, publicly available on Hugging Face. It captures syntactic and semantic patterns unique to Solidity that general models miss.
3. Open-Source Ecosystem: The authors released the dataset (30,000 quality-controlled contracts), code, documentation, and models via GitHub.
4. Comprehensive Evaluation: The study provides a rigorous comparison against traditional deep learning models (LSTM, CNN) and Large Language Models (LLMs) like GPT-4.1.

4. Experimental Results

The model was evaluated on a held-out test set of 10,000 real-world smart contracts.

• Performance Metrics:
  • Accuracy: 0.9789
  • Precision: 0.9090
  • Recall: 0.9476
  • F1 Score: 0.9279 (Micro-averaged)
• Comparison with Predecessor: SmartIntentV2 improved the F1 score by 7.48% over SmartIntentNN (0.8633).
• Comparison with LLMs:
  • GPT-4.1 achieved an F1 score of only 0.5606 in a zero-shot setting.
  • SmartIntentV2 showed a 65.5% relative improvement over GPT-4.1.
  • Efficiency: SmartIntentV2 processed 100 contracts in 2.63 seconds with negligible cost, whereas GPT-4.1 took 11.07 seconds and cost $2.88 in API fees.
• Ablation Study: Replacing SmartBERT with general-purpose models (RoBERTa or standard CodeBERT) resulted in a significant drop in performance (F1 dropped to ~0.89), confirming the necessity of domain-adaptive pre-training.

5. Significance and Impact

• State-of-the-Art Performance: SmartIntentV2 establishes a new benchmark for smart contract intent detection, proving that specialized, domain-adapted models outperform general-purpose LLMs for highly specialized security tasks.
• Cost-Effectiveness: The model offers a lightweight, low-latency, and free alternative to expensive LLM APIs, making it viable for real-time integration into auditing pipelines and DApp interfaces.
• Handling Imbalance: The combination of Focal Loss and Class-Balanced Training effectively mitigates the "long-tail" problem in security data, ensuring rare but dangerous intents (like Honeypots) are detected with high recall.
• Future Directions: The paper identifies limitations regarding closed-source contracts (lack of source code) and the need for code localization (pinpointing exactly where the malicious intent is). Future work aims to train on bytecode and extend the model to locate specific malicious code segments.

In conclusion, this work demonstrates that integrating domain-specific pre-training with robust sequence modeling is the most effective current approach for detecting malicious developer intents in the blockchain ecosystem.