🤖 AI

TensorCommitments: A Lightweight Verifiable Inference for Language Models

The paper proposes TensorCommitments, a lightweight, tensor-native proof-of-inference scheme that enables efficient and robust verification of large language model outputs on untrusted clouds with minimal computational overhead compared to existing cryptographic and non-cryptographic methods.

Original authors: Oguzhan Baser, Elahe Sadeghi, Eric Wang, David Ribeiro Alves, Sam Kazemian, Hong Kang, Sandeep P. Chinchali, Sriram Vishwanath

Published 2026-02-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Oguzhan Baser, Elahe Sadeghi, Eric Wang, David Ribeiro Alves, Sam Kazemian, Hong Kang, Sandeep P. Chinchali, Sriram Vishwanath

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are ordering a complex, custom-built meal from a famous chef who lives in a different country. You send them your order (the prompt), they cook the dish (the inference), and send you the final plate (the output).

The problem? You can't see the kitchen. You have to trust that they didn't:

Swap the expensive truffles for cheap mushrooms.
Burn the food but serve it anyway.
Use a different, cheaper recipe that tastes slightly off.

Currently, if you want to check if the chef did the job right, you'd have to fly to their country, watch them cook the exact same dish again, and compare the results. This is too slow, too expensive, and impossible to do for every single order.

TensorCommitments (TCs) is a new system that lets you verify the chef cooked your meal correctly without needing to watch them cook or fly to their country. It's like a magical, unbreakable "tamper-evident seal" that proves the cooking happened exactly as promised.

Here is how it works, broken down into simple concepts:

1. The "Magic Receipt" (Tensor Commitments)

Usually, when we try to prove a computer did math correctly, we treat the data like a long, flat list of numbers (like a grocery list). But Large Language Models (LLMs) don't think in lists; they think in tensors (multi-dimensional blocks, like a stack of pancakes or a 3D cube of data).

The Old Way: Imagine trying to prove a 3D cake was baked correctly by listing every single crumb in a long line. It takes forever to check.
The TC Way: This paper says, "Let's keep the cake in its 3D shape!" Instead of flattening the data, they wrap the whole 3D block in a cryptographic "magic receipt." This receipt is a single, tiny tag that represents the entire complex structure. If anyone tries to swap a crumb (tamper with the data), the receipt instantly breaks and becomes invalid.

2. The "Smart Tree" (Terkle Trees)

An LLM doesn't just cook one step; it cooks hundreds of steps (layers) at once. Checking every single step is still too much work.

The Analogy: Imagine a massive library where every book is a step in the cooking process.
- Old Trees (Merkle Trees): To prove a specific book is real, you have to check the spine of every book on the shelf above it, all the way to the top. It's a long chain of checks.
- Terkle Trees: This paper builds a special tree where the branches are shaped like the data itself. Because the tree matches the shape of the "cake" (the tensor), you can check a whole section of the library with just one quick glance at the branch. It's like checking a whole shelf of books by looking at the shelf label, rather than checking every single book spine.

3. The "Smart Inspector" (Layer Selection)

Even with the magic receipt, checking every step is overkill. The authors realized that not all steps in cooking are equally important.

The Insight: In an LLM, some layers (steps) are like "seasoning" (tiny changes matter a lot), while others are like "boiling water" (changes don't matter as much).
The Solution: They created a "Smart Inspector" algorithm. Instead of checking the whole kitchen, the inspector looks at the recipe and says, "Hey, the chef is most likely to mess up at Step 42 and Step 89. Let's just check those two spots."
The Result: This saves massive amounts of time. The inspector only checks the most critical parts, making the verification incredibly fast (lightweight) while still catching 99% of the cheating.

Why is this a Big Deal?

It's Fast: In tests, adding this verification only slowed down the "chef" (the server) by 0.97% and the "customer" (the verifier) by 0.12%. That's like adding a fraction of a second to a 10-second process.
It's Secure: It catches "slow-burn" attacks that other methods miss. For example, if a chef subtly changes the recipe to make the food slightly less healthy but keeps the taste the same, old methods might miss it. TCs catch these subtle changes because they verify the structure of the cooking, not just the final taste.
No Heavy Lifting for You: You don't need a supercomputer to verify the work. A simple laptop can check the "magic receipt" and be 100% sure the work was done correctly.

The Bottom Line

TensorCommitments is like giving every AI answer a digital fingerprint that is impossible to forge. It allows us to trust AI systems in critical fields (like medicine, finance, or law) without needing to trust the company running the servers. It proves the AI did the math right, without needing to re-do the math ourselves.

1. Problem Statement

As Large Language Models (LLMs) are increasingly deployed on external cloud infrastructure for critical tasks (e.g., medical diagnosis, financial trading, autonomous agents), users face a "trust gap." They must rely on remote servers to execute inference correctly without being able to inspect the internal states (activations, hidden layers) or re-run the massive models due to prohibitive computational costs.

Existing solutions for Verifiable Inference suffer from significant trade-offs:

Cryptographic approaches (e.g., ZK-SNARKs): Provide strong security guarantees but are too slow for LLMs, often requiring minutes of prover time per query and massive overhead.
Non-cryptographic/Learning-based approaches: Often require the verifier to have a powerful GPU to re-run the model or auxiliary models, violating privacy by exposing internal states or requiring retraining.
Vector-centric commitments: Previous methods flatten tensor data into 1D vectors and use univariate polynomials. This destroys the natural tensor structure of LLMs (layers, heads, spatial grids), leading to inefficient interpolation and large proof sizes as model dimensions grow.

The Core Challenge: How to design a lightweight, privacy-preserving mechanism that allows a client with limited resources to verify that a complex LLM inference was executed correctly, without re-running the model or exposing its internal states.

2. Methodology

The authors propose TensorCommitments (TCs), a tensor-native proof-of-inference scheme that leverages the multi-dimensional structure of LLM activations. The solution consists of three main components:

A. TensorCommitments (TCs)

Instead of treating model activations as a flat vector, TCs treat them as tensors (multi-dimensional arrays).

Multivariate Interpolation: The prover interpolates the activation tensor $T$ into a multivariate polynomial $f_T$ of order $m$ (where $m$ corresponds to tensor dimensions like layers, heads, etc.).
Commitment: The prover commits to this polynomial using a structured reference string (SRS) derived from a trusted setup. The commitment is a single group element $C_T = g^{f_T(\tau_1, \dots, \tau_m)}$ , where $\tau$ are secret trapdoors.
Opening & Verification: To prove a specific value at a point $\omega$ , the prover performs iterative univariate polynomial division (peeling off variables) to generate quotient polynomials. These are committed to form the proof. The verifier checks the consistency using bilinear pairings ( $e(\cdot, \cdot)$ ) without seeing the full tensor or re-running the model.
Key Insight: Multivariate interpolation is significantly faster than univariate interpolation for high-dimensional data. As shown in Figure 1, moving from 1D to 2D interpolation reduces runtime by over 30x for the same data size.

B. Terkle Trees

To handle long sequences and multi-turn dialogues, the authors introduce Terkle Trees, a tensor-native authentication structure.

Structure: Unlike Merkle Trees (hash-based) or Verkle Trees (1D vector commitments), Terkle Trees organize commitments in a high-arity tree where each node commits to a tensor-shaped grid of parameters or features.
Efficiency: A single root commitment represents the entire dialogue state. The prover only needs to open a few paths (selected by the layer policy) to verify specific parts of the computation.
Scalability: Terkle Trees reduce proof size and verification time compared to Verkle trees by exploiting the tensor structure, allowing for $O(1/m)$ fewer levels in the tree for the same data volume.

C. Layer Selection Algorithm

Verifying every layer of a massive model (e.g., 70B parameters) is still too expensive. The authors design a robustness-aware layer selection strategy.

Spectral Analysis: They analyze the "heavy-tailed" spectral scores of layer weight matrices. Layers with heavier tails (lower $\alpha$ -scores) are empirically more sensitive to perturbations and critical for output integrity.
Optimization: Using an Integer Linear Program (ILP) solved via Dynamic Programming, the system selects a contiguous interval of layers that maximizes the "benefit" (robustness) per unit of verification cost (budget). This ensures the verifier focuses resources on the most vulnerable and impactful parts of the model.

3. Key Contributions

Tensor-Native Commitment Scheme: A novel cryptographic primitive that binds LLM inference to multivariate polynomials, preserving tensor structure and enabling efficient verification.
Terkle Trees: A new authentication tree structure that aggregates evolving LLM states into a single root while supporting localized, tensor-shaped queries.
Robustness-Aware Layer Selection: An algorithm that identifies and verifies the most critical layers of an LLM, optimizing the trade-off between verification cost and security against tailored attacks.
System Implementation & Evaluation: A full implementation demonstrating that TCs add minimal overhead (0.97% prover, 0.12% verifier) while significantly outperforming state-of-the-art methods in robustness.

4. Experimental Results

The authors evaluated TensorCommitments on LLaMA2 models (7B, 13B, 70B) against baselines like zkLLM (ZK-proofs), SVIP (learned inspector), and TOPLOC (last-layer activation hashing).

Overhead:
- Prover Time: Adds only 0.97% to the inference time.
- Verifier Time: Adds only 0.12% to the inference time.
- Memory: The verifier requires 0 GB of GPU memory (unlike TOPLOC which requires ~71 GB).
Robustness:
- TCs improved detection accuracy against tailored attacks by up to 48% compared to TOPLOC (the previous SOTA).
- Specifically, TCs detected Prompt Tampering (entity/adjective swaps) and Noise Injection attacks that TOPLOC missed because TOPLOC only checks the final layer's top- $k$ activations.
Scalability:
- Terkle Trees achieved up to 67x faster proof generation and 1416x faster verification compared to Verkle trees for large datasets, while maintaining near-Merkle tree prover costs.
Privacy: Unlike methods requiring the verifier to re-run the model or see activations, TCs are fully cryptographic and preserve model privacy.

5. Significance

This work bridges the gap between cryptographic security and practical scalability for LLMs.

Trust in AI: It enables a future where users can trust remote LLM inferences for high-stakes decisions without needing to own the hardware or trust the provider blindly.
Efficiency: By leveraging the mathematical structure of tensors (multivariate interpolation) rather than flattening data, the authors achieve a paradigm shift in efficiency, making verifiable inference feasible for billion-parameter models.
Defense Mechanism: The layer selection algorithm provides a new defense against "slow-burn" attacks that manipulate internal states without altering the final output signature, a vulnerability in previous fingerprinting methods.

In conclusion, TensorCommitments offers a lightweight, privacy-preserving, and highly robust framework for verifiable LLM inference, making it a critical step toward the safe deployment of autonomous AI agents in critical infrastructure.