Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors

Technical Summary: Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors

Problem Statement
As international agreements on artificial intelligence (AI) development and deployment become increasingly likely, there is a critical need for verification infrastructure capable of confirming that AI data centers are not executing undeclared workloads. The core challenge addressed by this paper is Subgoal 2A of the AI verification agenda (Baker et al., 2025): enabling a "Prover" (the data center operator) to cryptographically prove to a "Verifier" (an auditor) that no undisclosed computations were performed within a cluster.

The specific technical hurdle is the potential for covert exfiltration of results from undisclosed workloads. Even if a data center claims to be running only authorized tasks, a malicious actor could hide the results of unauthorized computations in the traffic leaving the facility. This can be achieved through:

1. Steganography: Hiding data in protocol headers (e.g., padding bytes, port numbers, sequence numbers) or within the plaintext payloads of network packets.
2. Side-Channels: Encoding information in the timing of packet transmission (inter-packet gaps) or analog properties of the optical signal.
3. Non-Determinism: Exploiting the inherent non-determinism of Large Language Model (LLM) inference outputs to encode hidden bits.

Existing solutions often rely on Trusted Execution Environments (TEEs) or hardware components that require mutual trust between the Prover and Verifier. However, establishing trust in a single hardware component is difficult due to supply chain vulnerabilities (e.g., malicious IP, dopant-level Trojans) that cannot be patched post-deployment.

Methodology: The Secure Gateway Device (SeGaDev)
The authors propose a "Secure Gateway Device" (SeGaDev) placed at the north-south network boundary of an AI data center (between the TLS termination point and the internal routing fabric). The device captures all ingress and egress traffic, sanitizes it to remove covert channels, and generates cryptographic hashes for later verification.

The architecture is designed to function without any processors mutually trusted by both parties. Instead, it relies on physics-based observability and cryptographic protocols:

1. Placement and Passive Observation:

   • The SeGaDev intercepts plaintext traffic after encryption termination.
   • Passive Optical Splitters (using Fused Biconical Taper or Planar Lightwave Circuit) are used on the links carrying data to the Verifier. This allows the Prover to independently observe the hashes being sent to the Verifier without being able to interfere with the Verifier's monitoring leg. Trust is grounded in the physics of beam splitting rather than hardware integrity.

2. Sanitization (Active Warden):

   • The device employs an Active Warden to scrub steganographic degrees of freedom from network protocol headers.
   • Coin-Flip Protocol: To prevent either party from controlling the random values used to overwrite header fields (e.g., ephemeral source ports, TCP Initial Sequence Numbers), the Prover and Verifier contribute to a shared random number generation via a commit-reveal coin-flip protocol. The resulting pseudorandom values are used to overwrite header fields, eliminating the ability to encode data in them.
   • Re-timing: The device buffers all packets and releases them on a fixed schedule at the maximum link rate. This eliminates timing side-channels (e.g., modulating inter-packet gaps) and analog signal modulation.

3. Hashing and Commitment:

   • After sanitization, the device computes cryptographic hashes (e.g., SHA-256) over the payload data (application layer).
   • These hashes, along with timestamps, are transmitted to the Verifier over a dedicated fiber link.
   • The Prover retains the original plaintext data and the mapping of which data corresponds to which hash.

4. Verification Workflow:

   • The Verifier can retroactively challenge the Prover by requesting the preimage data for specific hashes.
   • The Prover submits the plaintext data to a secure, mutually trusted verification facility (outside the scope of the SeGaDev itself) to replay the computation and confirm the hash matches.

Key Contributions

• Architecture for Mutual Verifiability: A detailed design for a verification mechanism that removes the need for mutually trusted processors, leveraging passive optical splitters and coin-flip protocols to ensure neither party can tamper with the evidence or the monitoring process.
• Covert Channel Elimination: A specific methodology for erasing steganographic channels in Ethernet protocol headers and timing side-channels using an Active Warden architecture adapted for high-speed data centers.
• Feasibility Assessment: An analysis of the storage and bandwidth requirements for a large-scale deployment (e.g., 100,000 GPUs), estimating that the costs are manageable relative to the scale of modern AI data centers.

Results and Feasibility Analysis
The paper provides a theoretical and architectural assessment rather than empirical experimental results from a full-scale deployment. Key findings include:

• Residual Covert Channels: Even after sanitization, the authors identify three remaining potential channels:

  1. Timing Modulation: Mitigated by fixed-frame counting and canonical ordering, reducing capacity to connection lifecycle events.
  2. Steganography in ML Outputs: Identified as the most significant residual risk. Non-determinism in LLM inference could allow ~0.1 bits of hidden data per token. For a 200k GPU cluster, this could theoretically allow exfiltration of ~40 Mbit/s. The authors suggest this can be mitigated by enforcing deterministic inference (via seed synchronization and hardware control) or adding noise to scrub hardware-induced entropy.
  3. Failure Channels: Encoding data in the timing or occurrence of "hardware failures." This is estimated to be a low-bandwidth channel (~1 kbit/day) but can be screened by analyzing failure rates against hashed data.

• Storage Requirements: For a hypothetical 100k GPU cluster streaming 200M tokens/second, the authors estimate a worst-case annual storage requirement of ~1,260 PB for plaintext data and ~200 PB for hashes. They note this is a pessimistic upper bound (assuming 1 token per JSON packet) and that standard batching and compression would reduce this significantly. The cost is estimated to be less than 1% of the total data center cost.

• Hardware Implementation: The components (FPGAs, optical splitters, hash processors) are commercially available and used in high-stakes environments like financial regulation (e.g., MiFID II) and military networks. The authors estimate the bill of materials for a demonstration device is low compared to engineering costs.

Significance and Claims
The paper claims to provide a practical, retrofit-able solution for verifying AI compliance at the network level without relying on the integrity of specific hardware components. By shifting the root of trust to physical observability (optical splitters) and cryptographic commitments, it addresses the "trust" bottleneck in international AI verification.

The authors emphasize that while the SeGaDev prevents the exfiltration of results from undisclosed workloads, it does not prevent the execution of such workloads internally. Therefore, it is a critical component of a broader verification pipeline that must be combined with other measures (e.g., backend network taps, compute utilization accounting) to fully secure AI infrastructure. The work serves as a foundational design for "Level 2" verification (capturing raw evidence of computation) in the broader AI verification hierarchy.