💻 computer science

"Show Me You Comply... Without Showing Me Anything": Zero-Knowledge Software Auditing for AI-Enabled Systems

This paper introduces ZKMLOps, a modular verification framework that integrates Zero-Knowledge Proofs into Machine Learning Operations to enable regulatory compliance auditing of AI systems by cryptographically proving model integrity and correctness without exposing sensitive proprietary data or model details.

Original authors: Filippo Scaramuzza, Renato Cordeiro Ferreira, Giovanni Quattrocchi, Damian Andrew Tamburri, Willem-Jan van den Heuvel

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Filippo Scaramuzza, Renato Cordeiro Ferreira, Giovanni Quattrocchi, Damian Andrew Tamburri, Willem-Jan van den Heuvel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Black Box" Dilemma

Imagine you own a secret recipe for the world's best chocolate chip cookie. You sell these cookies to a giant supermarket chain. The supermarket's regulators say, "We need to verify that you aren't using banned ingredients or unsafe ovens. Show us your recipe and your kitchen logs."

Here is the catch: If you show them your secret recipe, your competitors might steal it. If you show them your kitchen logs, you might reveal private customer data. But if you don't show them anything, they can't trust that you are following the rules.

This is the exact problem facing AI systems today. Governments (like the EU with its new AI Act) are saying, "We need to know your AI is safe, fair, and honest." But AI models are like "black boxes"—they are so complex that even their creators don't fully understand how they make every decision. To prove they are working correctly, companies usually have to hand over their secret code and data, which they don't want to do.

The Solution: ZKMLOps (The "Magic Envelope")

The authors of this paper created a new framework called ZKMLOps. Think of it as a Magic Envelope system.

In this system, the company (the "Prover") wants to prove to the regulator (the "Verifier") that their AI is working correctly without opening the envelope to show the secret ingredients.

They use a mathematical trick called Zero-Knowledge Proofs (ZKPs).

The Analogy: Imagine you have a locked box with a red ball inside. You want to prove to a friend that the ball is red without showing them the ball. You could use a special machine that takes the box, checks the color, and prints a receipt that says "CONFIRMED: The ball is red." The receipt proves the statement is true, but your friend never actually sees the ball or the inside of the box.

ZKMLOps puts this "Magic Envelope" technology into the standard workflow that AI teams use to build and update their models (called MLOps). It turns the complex math of these proofs into a repeatable, automated process.

How It Works (The "Assembly Line")

The paper describes ZKMLOps as a factory assembly line with three main parts:

The Blueprint (The Guidelines): Before starting, the team decides what they are proving (e.g., "Did this AI correctly calculate a credit score?") and which mathematical tool to use. They write this down in a "Traceability Specification," which is like a recipe card that says, "We used Tool A to prove Property B."
The Factory Floor (The Hexagonal Architecture): This is the core software. It doesn't care which specific math tool is used; it just manages the process. It takes the AI model, locks it into a "circuit" (a mathematical representation), and runs the proof generation.
The Delivery (The Proof): The system generates a tiny digital receipt (the proof). The regulator checks this receipt. If the math checks out, the regulator knows the AI did what it was supposed to do, without ever seeing the secret model or the private data.

What They Tested (The "Stress Test")

The researchers didn't just build the system; they tested it to see if it was practical.

Speed Test (RQ1): They asked, "Does this system slow things down too much?"
- Result: They found that for small AI models, the system adds a little bit of extra time (about 30-40% overhead), mostly because of the "paperwork" (setting up the connection). However, as the AI models get huge and complex, that extra time becomes almost invisible. It's like paying a $5 entry fee to a theme park: if you stay for 10 minutes, the fee is huge; if you stay for 10 hours, the fee doesn't matter.
Comparison Test (RQ2): They asked, "When is this the right tool to use?"
- Result: They compared ZKMLOps to other methods like "Trusted Hardware" (using special secure chips) or "Sampling" (checking only 1 out of 100 results).
- The Verdict: ZKMLOps is the best choice when you need absolute certainty for a specific event (like an audit) and you cannot trust the hardware or the people running the system. It is slower than just "guessing" or "sampling," but it provides a much stronger guarantee of truth without revealing secrets.

The "Catch" (What the Paper Admits)

The paper is honest about one limitation: The "Deployment Gap."

The system proves that the blueprint (the model file) is correct. However, it cannot automatically prove that the company is actually running that exact blueprint in the real world. A sneaky company could prove they have a safe model, but then secretly run a different, unsafe model in production.

To fix this, the paper suggests combining ZKMLOps with other tools (like secure hardware attestation) to ensure the "blueprint" and the "real-world machine" are the same.

Summary

ZKMLOps is a new software framework that allows companies to prove their AI systems are safe and compliant with laws without having to reveal their trade secrets or private data. It uses advanced math (Zero-Knowledge Proofs) to create a "receipt of truth" that auditors can check, solving the conflict between the need for transparency and the need for privacy. The paper shows that while it takes some extra computing power, it is fast enough to be practical and is the best tool for high-stakes, one-time audits where trust is low.

Technical Summary: ZKMLOps – Zero-Knowledge Software Auditing for AI-Enabled Systems

1. Problem Statement

The increasing deployment of AI-enabled systems in safety-critical and high-stakes domains has triggered a regulatory demand for verifiable accountability, exemplified by the EU AI Act. However, a fundamental conflict exists between the requirement for deep transparency (to satisfy auditors and regulators) and the necessity to protect proprietary assets (such as model weights and training data) and privacy-sensitive information.

Traditional software verification and validation (V&V) techniques—ranging from manual process audits to formal methods—are ill-suited for modern AI models. These models often function as "black boxes" with millions of learned parameters, where direct probing risks intellectual property leakage, and disclosing training data may violate data protection laws. Consequently, organizations face a dilemma: they must prove compliance with strict transparency and robustness obligations without revealing the underlying artifacts that enable such transparency.

2. Methodology

The paper introduces ZKMLOps, a framework designed to operationalize Zero-Knowledge Proofs (ZKPs) within Machine Learning Operations (MLOps) lifecycles. The methodology bridges the gap between cryptographic protocol research and software engineering practices through four distinct steps:

Toolchain Survey: The authors reviewed existing ZKP implementations (e.g., ezkl, snarkjs, Cairo/Stwo) against five selection criteria: non-interactivity, transparent setup, standard representation, succinctness, and post-quantum security.
Use Case Analysis: The framework was driven by the specific requirements of regulatory compliance in financial risk modeling, identifying necessary artifacts, properties to be proven, and evidence requirements.
Framework Design: ZKMLOps employs a Hexagonal Architecture to decouple core auditing logic from specific ZKP backend implementations. It utilizes the Orchestrated Saga pattern to manage audit workflows as a sequence of stateful steps (Setup, Key Exchange, Proof Generation, Verification) and the Strategy pattern to handle interchangeable script behaviors for different protocols.
Empirical Evaluation: The framework was evaluated on two dimensions:
- Viability: Measuring orchestration overhead across heterogeneous backends (ezkl/Halo2 and snarkjs/Groth16) and models of increasing size (from tabular models to MobileNet-v2 variants).
- Cost-versus-Assurance Trade-offs: Comparing full ZK auditing against lighter-weight alternatives, including sampling with Verifiable Computation (VC), Trusted Execution Environments (TEE), and Multi-Party Computation (MPC).

3. Key Contributions

A. The ZKMLOps Framework

ZKMLOps provides a modular, repeatable process for generating verifiable cryptographic evidence. It allows a prover (e.g., a financial institution) to convince a verifier (e.g., a regulator) that a specific computational statement is true—such as "the approved model produced this inference on approved input data"—without revealing the model parameters or the input data itself.

The framework consists of two main layers:

Methodic Layer: A set of guidelines for protocol selection, resulting in a ZKP Traceability Specification. This document acts as an Architecture Decision Record (ADR), detailing the audit purpose, the MLOps lifecycle phase, the selected protocol, a property matrix, and a trade-off justification (e.g., accepting non-post-quantum security for faster verification).
Practical Layer: A service-oriented implementation ("Trustworthiness as a Service") that orchestrates the audit workflow. It manages artifact storage, executes protocol-specific scripts via adapters, and maintains an internal state machine to track the audit lifecycle.

B. Trust and Threat Model

The paper clarifies that ZKP-based auditing does not eliminate trust but relocates it. Trust is shifted from the prover's honesty to the soundness of the cryptographic construction and the integrity of the artifact-binding mechanism.

Artifact Binding: The ZKP statement is bound to a cryptographic commitment $H(M)$ of the artifact bundle (compiled circuit, weights, configuration). Any change to the bundle invalidates prior proofs.
Deployment Integrity Gap: The authors explicitly acknowledge a limitation: the framework proves properties of a committed artifact bundle, not necessarily the running production service. Closing this gap requires complementary runtime attestation (e.g., TEE-based remote attestation), which is outside the scope of ZKMLOps but documented as a necessary integration.

C. Comparative Analysis

The study positions full ZK auditing against alternatives:

Sampling with VC: Offers lower per-request costs but only provides statistical guarantees for sampled data, not full confidentiality for the entire stream.
TEE: Provides hardware-based trust but relies on microcode integrity and is susceptible to side-channel attacks, unlike crypto-only ZK.
MPC: Requires non-collusion assumptions and interactive protocols, making it unsuitable for offline, replayable evidence.

4. Results

Framework Viability (RQ1)

Bounded Overhead: The orchestration overhead imposed by ZKMLOps is stable and backend-invariant. For small tabular models, the overhead was approximately 32–38% of total wall-clock time across both ezkl (Halo2) and snarkjs (Groth16) backends.
Scalability: As the cryptographic workload increases (tested via MobileNet-v2 variants from 0.4M to 3.5M parameters), the orchestration overhead becomes a vanishing fraction of the total time. The fixed cost of HTTP round-trips and subprocess management does not scale with circuit size, making the framework responsive even for large models.

Cost-versus-Assurance Trade-offs (RQ2)

Audit-on-Demand: Full ZK auditing is identified as the appropriate tool for "audit-on-demand" settings where a single inference must be verified with full confidentiality and replayable evidence. It provides crypto-only trust without relying on hardware vendors.
High-Throughput Streaming: For scenarios requiring high-throughput streaming inference, lighter-weight alternatives like sampling with VC are more cost-effective, though they sacrifice the guarantee of full confidentiality for every request.

5. Significance and Claims

The paper claims that ZKMLOps resolves the tension between audit transparency and asset confidentiality by enabling organizations to provide verifiable cryptographic evidence regarding specific computational properties of their AI systems without exposing proprietary models or sensitive data.

The authors emphasize that the framework is not a theoretical construct but a practical integration of existing ZKP primitives into standard software engineering patterns. By decoupling the orchestration layer from specific cryptographic backends, ZKMLOps allows for the modular adoption of new protocols (e.g., switching from SNARKs to STARKs for post-quantum security) without altering the core auditing workflow.

The significance lies in operationalizing the "show me you comply... without showing me anything" paradigm, offering a path for regulatory compliance (such as the EU AI Act) that respects intellectual property and privacy constraints. The paper modestly notes that while the framework enables governance throughout the MLOps lifecycle, the current evaluation focuses primarily on inference verification, with training and data-preprocessing audits identified as future extensions.