Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AI

The paper introduces AFTUNE, a framework that ensures the integrity of cloud-based large language model fine-tuning and inference by employing a lightweight recording and spot-check mechanism to generate verifiable execution traces, thereby enabling clients to practically audit proprietary AI processes without prohibitive overhead.

The Gist

Imagine you hire a master chef to cook a very expensive, secret recipe for your restaurant. You give them your special ingredients and a specific set of instructions (like "add salt at step 3"). However, the chef works in a locked kitchen that you can't enter. You have to trust them completely that they didn't:

1. Use cheaper, frozen ingredients instead of your fresh ones.
2. Skip steps to save time.
3. Secretly add a "poison" to the food that makes people sick only when they order a specific dish.

In the world of AI, Cloud Providers are the chefs, Large Language Models (LLMs) are the secret recipes, and Clients are the restaurant owners. The problem is that modern AI models are so huge that they don't fit in your own kitchen; you must send them to the cloud. But because the model is proprietary (the chef's secret), you can't peek inside to see if they actually followed your instructions.

This paper introduces AFTUNE, a system that lets you audit the chef's work without ever entering the locked kitchen or seeing the secret recipe.

The Core Problem: The "Black Box"

Currently, if you ask a cloud provider to fine-tune an AI (teach it new things), you get a result. But you have no proof that the AI was actually trained on your data or that it wasn't tampered with.

• Old Solution 1 (Math Magic): Some tried to use complex math (Zero-Knowledge Proofs) to prove the work was done. But for giant AI models, this math is so heavy it slows the computer down to a crawl. It's like trying to weigh a truck by counting every single grain of sand on it.
• Old Solution 2 (The Glass Box): Others suggested putting the whole AI inside a "Trusted Execution Environment" (TEE)—a super-secure digital vault. But modern AI models are too big to fit inside these vaults. It's like trying to park a semi-truck in a tiny garage.

The AFTUNE Solution: The "Spot-Check" System

AFTUNE solves this by changing the rules of the game. Instead of trying to watch the entire cooking process in real-time (which is impossible), it creates a digital receipt system that allows for random, high-stakes spot checks.

Here is how it works, using our kitchen analogy:

1. Breaking the Recipe into "Blocks"

Imagine the cooking process is a long movie. AFTUNE doesn't try to record every single frame. Instead, it cuts the movie into blocks (chunks).

• Layer Blocks: Groups of steps (e.g., "chopping," "sautéing").
• Time Blocks: Groups of time steps (e.g., "the first 10 minutes").

2. The "Boundary Receipt"

At the end of every block, the chef (the cloud provider) takes a snapshot of the state of the food (the "activations" and "gradients"). They don't send you the food; they just send you a cryptographic hash.

• Analogy: Think of this like a sealed wax stamp on a letter. You can't see what's inside, but if the wax is broken or the seal doesn't match the master key, you know the letter was tampered with.
• The chef records these "seals" at the boundaries of every block.

3. The "Secret Re-Cook" (Verification)

Later, when you want to check if the chef did a good job, you don't ask them to show you the whole kitchen. Instead, you say: "I want to check the block where you sautéed the onions between minutes 10 and 20."

• The chef sends you the "seals" (hashes) for that specific block.
• You take your own secure, tiny kitchen (a TEE) and re-cook just that small part of the recipe from scratch using the original ingredients.
• You compare your result with the chef's "seal."
  • If they match: The chef likely did it right.
  • If they don't match: The chef cheated.

Why This is a Game-Changer

1. It's Fast (The "Map-Reduce" Trick)
Calculating those "seals" (hashes) for huge data is usually slow. AFTUNE uses a trick called Map-Reduce.

• Analogy: Instead of one person reading a 1,000-page book to find a typo, you give 100 people one page each. They all read their page at the same time, then combine their notes. This makes the process incredibly fast, so the chef doesn't have to slow down their cooking.

2. It's Random (The "Spot-Check" Strategy)
You don't need to check every block. That would take too long.

• Analogy: Think of a police officer checking a bus. They don't check every single passenger. They randomly pull over the bus and check a few people.
• The chef knows you might check any block at any time. If they try to cheat on even one block, they risk getting caught. This uncertainty stops them from cheating in the first place.

3. It Handles Giant Models
Because you only re-cook a tiny "block" inside your secure vault, the vault doesn't need to be huge. You can verify a massive AI model even if your secure vault is small.

The Bottom Line

AFTUNE turns the "Black Box" of cloud AI into a "Glass Box" without breaking the glass.

• Before: You had to blindly trust the cloud provider.
• Now: You can randomly spot-check their work with mathematical certainty. If they try to swap your data, skip steps, or hide a backdoor, the "seals" won't match your re-calculation, and you'll catch them.

It creates a world where you can use powerful, proprietary AI models from the cloud with the same confidence as if you were cooking the meal yourself in your own kitchen.

Technical Summary

Here is a detailed technical summary of the paper "Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AI" (AFTUNE).

1. Problem Statement

The deployment of Large Language Models (LLMs) and other AI models has shifted to cloud-based infrastructure, where clients outsource fine-tuning and inference to third-party providers. This creates a fundamental trust gap:

• Lack of Visibility: Clients cannot verify if the provider executed the training or inference according to the agreed configurations (e.g., specific datasets, hyperparameters, or model architectures).
• Proprietary Constraints: For proprietary models (e.g., GPT, Gemini), clients cannot inspect model weights, making traditional verification impossible.
• Limitations of Existing Solutions:
  • Zero-Knowledge Proofs (ZKPs): While mathematically sound, they incur prohibitive computational overhead (thousands of times slower than standard execution) for large-scale models.
  • Trusted Execution Environments (TEEs): Current TEE approaches require loading the entire model and its state (including optimizer moments for training) into the secure enclave. Modern LLMs exceed the memory capacity of single TEE instances, making full encapsulation impractical.
  • Attacks: Untrusted providers could downgrade services, embed biases, insert backdoors, or skip fine-tuning entirely without detection.

2. Methodology: AFTUNE Framework

AFTUNE is a framework that decouples execution from verification, enabling auditable fine-tuning and inference without requiring the entire model to reside in a TEE. It relies on block decomposition and compositional verification.

Core Design Principles

1. Two-Dimensional Block Structure:

   • Instead of recording every layer at every step (which is $O(L \times T)$), AFTUNE partitions the training/inference process into a grid of Layer Blocks ($B_L$) and Step Blocks ($B_S$).
   • Boundary States: The system records only the "boundary states" (activations, gradients, parameters, and optimizer states) at the edges of these blocks.
   • Compositional Verification: Because neural network computation is deterministic, the output of one block serves as the input to the next. Verifying the boundary states ensures the continuity of the entire chain.

2. Map-Reduce Hashing Scheme:

   • To avoid bottlenecks in generating cryptographic evidence, AFTUNE uses a parallel hashing strategy.
   • Large tensors are partitioned into chunks, hashed in parallel on the accelerator (GPU), and then aggregated into a single commitment hash. This minimizes the impact on the primary training/inference throughput.

3. Sampling-Based Spot-Checking:

   • Clients do not need to verify every block. They randomly select a subset of blocks to audit.
   • Probabilistic Guarantees: Even a small sampling rate provides high detection probabilities for malicious tampering. The uncertainty of which blocks will be audited acts as a strong deterrent.

4. TEE-Based Recomputation Protocol:

   • Execution: The heavy lifting (training/inference) happens on standard, high-performance accelerators outside the TEE.
   • Verification: Upon a client's request, a lightweight TEE instance (on the provider's side) is provisioned.
   • Process: The TEE loads the specific block's boundary states and the model parameters, recomputes the block execution, and compares the results against the recorded hashes and values.
   • Tolerance: Verification allows for small numerical differences (floating-point tolerance) to account for hardware variations, but requires bitwise integrity for hashes.

5. Storage Optimization:

   • Sparse Checkpointing: To reduce storage costs, the system can skip saving checkpoints at every step, reconstructing missing states via recomputation during verification.
   • Zero-Storage Mode: In extreme cases, only hashes are stored. Verification requires re-running the entire training process from the start for the requested blocks.

3. Key Contributions

• AFTUNE Framework: The first system to enable verifiable fine-tuning and inference for proprietary LLMs in the cloud, overcoming the memory constraints of TEEs and the overhead of ZKPs.
• Block Decomposition Strategy: A novel method to break down massive models into independently verifiable units, allowing verification to scale with available TEE resources rather than model size.
• Efficient Commitment Generation: A map-reduce hashing scheme that leverages GPU parallelism to generate cryptographic commitments with minimal overhead.
• Sampling-Based Auditing: A strategy that balances cost and security, providing probabilistic detection guarantees that compound over time.
• Implementation & Evaluation: Full integration into CUDA-based pipelines (using Gramine for SGX/TDX) and evaluation on real-world models (Llama-3.1-8B, Qwen2.5-14B, DINOv2-Giant, ViT-Large).

4. Results and Evaluation

The authors evaluated AFTUNE on diverse models (LLMs and Vision Transformers) using real TEE hardware (Intel SGX/TDX).

• Performance Overhead:
  • AFTUNE imposes practical overhead (14%–36% for training, ~10% for inference) compared to baseline training, significantly lower than "Full TEE" or "Step TEE" approaches which are often infeasible for large models.
  • Larger block sizes reduce overhead but increase verification time; the system offers a tunable trade-off.
• Storage Efficiency:
  • Storage costs are reduced by over 50% compared to step-by-step verification methods.
  • Storage can be further managed via sparse checkpointing or zero-storage strategies.
• Verification Accuracy:
  • Numerical Tolerance: The system correctly distinguishes between legitimate floating-point variations and malicious tampering.
  • Precision Requirement: Experiments show that bfloat16 precision is insufficient for security (adversarial perturbations can hide within its error margin), whereas float32 provides a clear security margin.
• Security Against Attacks:
  • Adversarial Examples: Attacks injecting perturbations into activations are detectable because the required perturbation magnitude to succeed is orders of magnitude larger than natural numerical noise (in float32).
  • Parameter Poisoning: Similarly, injecting backdoors via weight perturbations requires changes far exceeding the system's numerical tolerance, ensuring detection.
  • Backdoor/Model Substitution: Sampling-based verification effectively detects these attacks with high probability.

5. Significance

• Bridging the Trust Gap: AFTUNE enables clients to audit proprietary AI services without needing access to model weights, solving a critical bottleneck in the "Model-as-a-Service" economy.
• Scalability: By decoupling verification from execution, it makes auditable AI feasible for models that are too large to fit in secure enclaves.
• Economic Viability: The low overhead (near-baseline performance) means providers can offer auditable services without prohibitive costs, encouraging adoption.
• Accountability: It shifts the paradigm from "trusting the provider" to "verifying the evidence," creating a more secure and accountable AI ecosystem.

In summary, AFTUNE demonstrates that trustworthy model services are achievable in today's cloud environments by combining cryptographic commitments, block-based decomposition, and selective TEE-based recomputation.