🤖 AI

AttestLLM: Efficient Attestation Framework for Billion-scale On-device LLMs

AttestLLM is a novel, efficient attestation framework that leverages algorithm/software/hardware co-design to embed robust watermarks into on-device LLM activation distributions, ensuring model legitimacy and hardware IP protection within Trusted Execution Environments without compromising inference performance.

Original authors: Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar

Published 2026-02-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you just bought a brand-new, high-performance smartphone. The manufacturer didn't just put a standard app on it; they installed a super-smart AI assistant (a Large Language Model, or LLM) that was specifically tuned to work perfectly with your phone's unique hardware. This AI is the phone's "brain," and the manufacturer spent millions of dollars designing it.

The Problem: The "Counterfeit Brain" Risk
Now, imagine a shady third-party seller (like a reseller) gets their hands on your phone before you do. They could swap out the manufacturer's special AI with a cheap, illegal, or even dangerous version.

For the Manufacturer: They lose money because their expensive "brain" is being used without permission.
For You: The phone might start acting weird, leaking your private data, or performing poorly because the "fake brain" wasn't built for your specific hardware.

Traditionally, security systems could check if the software was real, but they were too slow and clumsy for these massive, billion-parameter AI brains. They were like trying to inspect a whole library by reading every single book one by one—it takes too long and slows everything down.

The Solution: AttestLLM (The "Smart Security Guard")
This paper introduces AttestLLM, a new security system designed specifically to protect these giant AI brains on your phone. Think of it as a highly efficient, invisible security guard that checks the AI's ID card without ever stopping the AI from doing its job.

Here is how it works, using simple analogies:

1. The Invisible Tattoo (Offline Watermarking)

Before the phone even leaves the factory, the manufacturer gives the AI a "tattoo."

The Analogy: Imagine the AI is a giant, complex machine. The manufacturer doesn't just paint a logo on the outside (which a thief could scrape off). Instead, they subtly tweak the internal gears and springs (the AI's layers) in a way that is invisible to the user but creates a unique, unbreakable signature.
The Trick: They are smart about where they put the tattoo. They put more "ink" on the parts of the machine that are sturdy and less "ink" on the delicate parts that might break if tweaked too much. This ensures the AI still works perfectly (high quality) but carries a hidden, unforgeable ID.

2. The Random Spot Check (Online Attestation)

Once the phone is in your hands, the AI is running in the "Main World" (doing your tasks). But a secure, locked-down "Safe Room" (called a TEE) inside the phone's processor is constantly watching.

The Analogy: Instead of stopping the AI to check its whole body (which would be slow), the security guard in the Safe Room randomly picks a few gears to inspect every few seconds.
The Process:
1. The guard asks the AI to run a specific, secret test phrase (a "trigger").
2. The guard quickly checks if the AI's reaction matches the hidden tattoo.
3. If the reaction is perfect, the guard says, "All clear, keep working!"
4. If the reaction is wrong (meaning the AI was swapped or tampered with), the guard immediately pulls the plug and stops the phone.

3. The Speed Trick (Efficiency)

The biggest challenge was speed. Checking a billion-parameter AI usually takes forever. AttestLLM uses a "conveyor belt" method.

The Analogy: Imagine a factory assembly line. While one worker is checking the current gear, the next worker is already grabbing the next gear. They overlap the work so there is no waiting time.
The Result: This system is 62 times faster and uses 30 times less battery than previous methods. It's so efficient that you won't even notice it's happening.

Why This Matters

For You: Your phone stays fast, private, and safe. You know the AI you are talking to is the real deal, not a scammer's fake version.
For Manufacturers: They can finally sell their expensive, custom-tuned AI on devices without worrying that a reseller will swap it out for a cheaper copy.

In a Nutshell:
AttestLLM is like a super-fast, invisible bouncer for your phone's AI. It checks the AI's secret ID card in the blink of an eye, ensuring that only the authorized, high-quality brain is allowed to run, while blocking any imposters trying to sneak in. It protects the manufacturer's investment and keeps your device safe, all without slowing you down.

1. Problem Statement

The rapid deployment of Large Language Models (LLMs) on consumer edge devices (e.g., smartphones, IoT) offers benefits like low latency and enhanced privacy. However, this introduces a critical security vulnerability: unauthorized model substitution.

Threat: Untrusted third parties in the supply chain (e.g., resellers) can access the device's Rich Execution Environment (REE) and replace vendor-optimized, proprietary LLMs with unauthorized or malicious models.
Consequences: This infringes on hardware vendors' Intellectual Property (IP), erodes user trust, and can lead to degraded performance or unsafe behaviors.
Limitations of Existing Solutions:
- Scalability: Current attestation frameworks (e.g., DeepAttest) are designed for small neural networks (millions of parameters) and fail to scale to billion-parameter LLMs.
- Hardware Constraints: On-device Trusted Execution Environments (TEEs), such as ARM TrustZone, have limited secure memory (10–32 MB), insufficient to load full LLM parameters for verification.
- Efficiency: Running full model inference inside the TEE causes massive latency and energy overhead due to frequent world switches and secure memory operations.

2. Methodology: AttestLLM Framework

AttestLLM is the first framework to protect hardware-level IP for billion-scale on-device LLMs using an algorithm/software/hardware co-design approach. It consists of two main phases:

A. Offline Watermarking

This phase embeds robust, device-specific signatures into the LLM before deployment.

Adaptive Allocation: The system profiles the LLM using a dataset to analyze layer sensitivity. Layers with high activation magnitudes (more sensitive) receive fewer watermark bits, while less sensitive layers receive more. This preserves model utility.
Optimization-Based Insertion: A two-stage insertion process is used to handle quantization (a necessary step for edge deployment):
1. Pre-Quantization: Uses gradient-based optimization on full-precision weights to embed the signature.
2. Post-Quantization: Uses zeroth-order optimization (finite-difference) to refine the signature on discrete, quantized weights (INT4/INT8).
Layer-wise Verification: Instead of verifying the whole model at once, AttestLLM enables independent verification of individual layers. It saves input activations and checks the output activations against the embedded signature.

B. Online Attestation (in TEE)

This phase verifies the model's legitimacy periodically during inference within the TEE.

Virtualized Enclave: To overcome memory limits, AttestLLM uses a virtualization-based enclave (pKVM) to enlarge the available secure memory on ARM TrustZone.
Dynamic Sampling: Instead of verifying all layers, the system randomly samples a subset of $k$ layers from the total $|M|$ layers every $f$ generated tokens.
Pipelined Execution: To minimize latency, the framework overlaps communication and computation:
- One TEE thread handles secure copying and decryption of sampled layers from the REE.
- Remaining threads perform watermark extraction and verification in parallel.
- Early Exit: If a mismatch is detected in any layer, the process immediately aborts the REE inference.
Security Mechanisms:
- Randomization: Model weights are shuffled in untrusted memory to prevent brute-force tampering.
- Hypervisor Monitoring: The hypervisor tracks OS page allocations to detect if the running model differs from the attested one (preventing "model swapping" attacks).
- Trusted Timer: Ensures attestation requests occur at fixed intervals, preventing adversaries from halting the process.

3. Key Contributions

First Billion-Scale Framework: Introduces the first attestation system capable of protecting hardware IP for LLMs with billions of parameters on resource-constrained edge devices.
Co-Design Approach: Combines adaptive watermarking algorithms, virtualization-based TEE extensions, and pipelined verification to solve memory and efficiency bottlenecks.
Robustness & Fidelity: Achieves 100% watermark extraction rates while maintaining near-perfect model quality (minimal degradation in perplexity and accuracy).
Efficiency: Demonstrates significantly lower overhead compared to existing TEE-shield inference methods.

4. Experimental Results

The framework was evaluated on various on-device LLMs (Llama3 1B-8B, Qwen3 4B-8B, Phi-4 15B) quantized to INT4 and INT8 on an ARM-based edge device.

Reliability:
- Watermark Extraction Rate (WER): 100% for all tested models.
- Attest Pass Rate (APR): 100% for authorized models; 0% for unauthorized/tampered models.
Fidelity Preservation:
- Perplexity (PPL) Degradation: Average increase of only 0.62% (INT4) and 0.07% (INT8).
- Zero-Shot Accuracy: Average decrease of only 0.09% (INT4) and 0.15% (INT8).
Efficiency (vs. TEE-shield Inference):
- Latency Overhead: Reduced by 62× (AttestLLM: ~9.3% vs. TEE-shield: ~472%).
- Energy Overhead: Reduced by 30× (AttestLLM: ~7.3% vs. TEE-shield: ~87.8%).
Robustness Against Attacks:
- Watermark Forgery: Resistant due to encrypted keys and randomized memory layouts.
- Model Replacement: Successfully detected partial (4/32 layers) and full model replacements, aborting execution with high probability ( $P_{evasion} < 10^{-3}$ ).
- TEE System Attacks: Mitigated via trusted timers and hypervisor monitoring, preventing adversaries from bypassing attestation by halting interactions.

5. Significance

AttestLLM addresses a critical gap in the security of the AI supply chain. By enabling efficient, scalable, and robust attestation for billion-parameter models on edge devices, it allows hardware vendors to:

Protect their substantial investments in model compression and hardware optimization.
Ensure that only authorized, safe, and high-quality models run on their devices.
Maintain user trust by preventing the deployment of malicious or degraded models by third-party resellers.

The framework sets a new standard for securing on-device AI, proving that high-security verification is feasible even under strict hardware constraints without sacrificing inference performance.