🤖 AI

Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs

This paper proposes a commit-open protocol that utilizes Merkle-tree commitments to sparse autoencoder (SAE) feature traces to effectively detect silent model substitution in hosted LLMs, outperforming existing probe-after-return schemes by rejecting diverse adaptive attacks while maintaining minimal computational overhead.

Original authors: Ziyang Liu

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ziyang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you order a gourmet steak from a high-end restaurant. You pay for a premium cut, but the kitchen secretly swaps it for a cheaper, frozen patty. In the world of AI, this is a real risk: a cloud provider might advertise a powerful, expensive AI model but secretly send you a cheaper, weaker one to save money.

This paper proposes a new way to catch these "fake steaks" without needing to trust the restaurant.

The Problem: The "Magic Mirror" Trick

Current security methods try to catch cheaters by sending a secret test question (a "probe") to the AI alongside your real request. If the AI answers the test question correctly, the provider claims, "See? We used the good model!"

But a dishonest provider can cheat this system using a "magic mirror" trick. They can route your secret test question to the expensive, high-quality model (to pass the test) while routing your actual request to the cheap, weak model. You get a bad result, but the security check says everything is fine.

The Solution: The "Sealed Recipe Box"

The authors propose a new system called a Commit-Open Protocol. Think of it like a sealed recipe box that the chef must lock before they start cooking.

The Commitment (Locking the Box): Before the AI generates any answer, it creates a digital "fingerprint" of its internal thought process. It uses a special tool (called a Sparse Autoencoder, or SAE) to take a snapshot of its brain activity at every step. It then locks these snapshots into a digital "Merkle tree" (a secure, unchangeable list) and publishes the lock.
The Opening (Checking the Receipt): After the AI gives you the answer, the security checker picks a few random steps from the process and asks the provider to "open" the box for those specific moments.
The Verification (The Taste Test): The checker compares the opened snapshots against a public library of what a "good" AI brain looks like. If the snapshots match the high-quality model, the answer is accepted. If they look like the cheap model, the answer is rejected.

Why Cheaters Can't Win

The paper tested this against 17 different types of cheaters, including those trying to swap models, those trying to tweak the model slightly, and even those trying to mathematically trick the system.

The "Parallel Serve" Fail: In the old "magic mirror" method, cheaters could easily pass the test by showing the good model only the test questions. In this new system, because the "fingerprint" is locked before the answer is fully generated and covers the whole process, the cheater can't swap models mid-stream without breaking the lock.
The "Fake Brain" Fail: Even if a cheater tries to forge the fingerprints (pretend they have the good model's brain activity), the math shows it's impossible to fake the complexity of the real model's internal thoughts without actually running the real model.
The "Hacker" Fail: Even if a hacker tries to work backward from the fingerprints to trick the system, the gap between the real model and the fake one is too wide to bridge.

The Cost

The authors ran this on three different AI models. They found that adding this security lock only slows down the AI by about 2.1%. It's a tiny price to pay to ensure you aren't being served a frozen patty when you paid for a steak.

In short: This paper introduces a "lock-and-check" system that forces AI providers to prove they used the specific model they promised, making it mathematically impossible to swap in a cheaper version without getting caught.

Technical Summary: Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs

Problem Statement

Hosted Large Language Model (LLM) providers face a "silent-substitution" incentive: the economic motivation to advertise a high-capability model while secretly serving users a cheaper, lower-quality substitute. Existing detection mechanisms, such as "probe-after-return" schemes (e.g., SVIP), are vulnerable to a parallel-serve side-channel. In these schemes, a dishonest provider can route a verifier's specific probe request to the advertised (strong) model to pass the audit, while simultaneously serving ordinary users from the substitute (weak) model. This creates a gap where substitution remains undetectable by standard verification methods.

Methodology: The Commit-Open Protocol

To close this verification gap, the authors propose a commit-open protocol that prevents the provider from selectively routing probes. The methodology operates as follows:

Commitment Phase: Before any opening request occurs, the provider commits to a "feature-trace sketch" of its served output. This sketch is generated using a Sparse Autoencoder (SAE) at a published probe layer. The provider constructs a Merkle tree over per-position SAE feature traces, creating a cryptographic commitment to the actual model behavior during the session.
Opening and Verification Phase: A verifier requests the opening of random positions within the committed trace. The provider reveals the corresponding SAE features.
Scoring and Decision: The verifier scores these opened features against a public named-circuit probe library. This library is calibrated to account for cross-backend noise. The final decision is made using a fixed-threshold joint-consistency z-score rule, determining whether the revealed features are consistent with the advertised model.

Experimental Instantiation and Results

The protocol was instantiated and tested on three distinct backbone architectures:

Qwen3-1.7B
Gemma-2-2B
A scaled-up Gemma-2-9B variant utilizing a 131k-feature SAE.

The evaluation involved 17 distinct attackers representing various substitution strategies, including:

Same-family model lifts.
Cross-family substitutes.
Rank-<=128 adaptive LoRA (Low-Rank Adaptation) attacks.

Key Findings:

Detection Efficacy: All 17 attackers were rejected at a shared, scale-stable threshold using the proposed protocol.
Comparison to Baseline: The same set of attackers successfully evaded a matched SVIP-style parallel-serve baseline, demonstrating the superiority of the commit-open approach in preventing selective routing.
Robustness Against Advanced Attacks:
- A white-box end-to-end attack, which attempted to backpropagate through the frozen SAE encoder to forge features, failed to close the detection margin.
- A feature-forgery attacker (one that never runs the honest model $M_h$ ) is bounded in closed form by an intrinsic-dimension argument, theoretically limiting the feasibility of successful forgery.
Performance Overhead: The commitment mechanism adds a negligible computational cost, increasing forward-only wall-clock time by ≤2.1% at a batch size of 32.

Significance and Claims

The paper claims that this protocol effectively eliminates the parallel-serve side-channel vulnerability inherent in probe-after-return schemes. By forcing the provider to commit to the SAE feature traces of the actual served output before verification, the system ensures that the model used for auditing is the same model used for user inference. The authors position this as a practical, low-overhead solution for detecting silent model substitution in hosted LLM environments, validated across multiple model scales and attack vectors without requiring changes to the underlying model architecture beyond the SAE integration.