💻 computer science

Self-Verifying Measurement Records: Hash-Linked Evidence Graphs for Hardware Benchmarking

This paper proposes a tamper-evident, hash-linked transparency log for hardware benchmarking that binds reported performance metrics to verifiable evidence using probabilistic identities and cryptographic challenges, enabling offline, trustless verification of results across diverse GPU architectures while accounting for floating-point noise and adversarial threats.

Original authors: Faruk Alpay, Baris Basaran

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Faruk Alpay, Baris Basaran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you buy a car and the salesperson hands you a piece of paper saying, "This car goes 200 miles per hour." You have to take their word for it. You can't test it yourself because the car is gone, the track is closed, and even if you could, the engine might be secretly broken in a way that only shows up under specific conditions.

This paper proposes a new way to report hardware performance (like how fast a computer chip is) so that you don't have to trust the person who wrote the report. Instead, the report comes with its own "proof of life" and "proof of truth" attached to every single number.

Here is how they do it, explained through simple analogies:

1. The "Tamper-Proof Diary" (The Evidence Graph)

Think of the performance report not as a static PDF, but as a digital diary where every entry is chained to the next one using a special digital lock (a hash).

How it works: If someone tries to change a number in the middle of the diary (like changing "100 miles" to "200 miles"), the digital lock breaks, and the whole chain becomes invalid.
The Benefit: You can look at the diary offline (without internet or the original computer) and verify that the numbers haven't been faked. It's like a notary public who stamps every page, but the stamp is mathematical and impossible to forge.

2. The "Spot Check" for Math (Linear Quantities)

When the computer does a massive math problem (like multiplying two giant grids of numbers), checking the whole answer takes forever. The authors use a clever trick called a "probabilistic identity."

The Analogy: Imagine a baker claims to have baked 1,000 perfect cookies. Instead of tasting all 1,000, the inspector picks a random handful, adds a secret "tasting spice" (a random probe), and checks if the flavor matches the recipe.
The Catch: If the baker cheated, the inspector will almost certainly catch them. If the inspector checks 8 times, the chance of a cheater getting away is less than 1 in 256.
The "Floor" Calibration: Computers aren't perfect; they make tiny rounding errors (like a scale that is off by a gram). The authors measured exactly how "off" their specific chips get naturally. They set a "tolerance zone" based on this natural error. If the math is slightly off, it's accepted as normal. If it's way off, it's flagged as a lie or a broken chip.

3. The "Double-Check" (Cross-Device Verification)

To be extra sure, they used two identical chips to do the exact same math at the same time.

The Analogy: It's like having two identical twins take the same test. If they both get the exact same answer down to the last digit, you know the answer is right. If one gets a different answer, you know one of them is lying or confused.
The Bonus: Even though the twins are identical, they run at slightly different speeds (like one twin is slightly faster at running). The report records this tiny speed difference as a "fingerprint" for that specific chip, proving it's the real deal and not a fake.

4. The "Stress Test" (Physical Limits)

The authors wanted to know: Can a hacker trick the chip into giving a wrong answer without us noticing?

The Experiment: They tried to "stress" the chip by making it run hot and fluctuating its power usage (like a power surge).
The Result: The chip didn't break or give wrong answers. Instead, it just slowed down to protect itself. This suggests that "silent errors" (where a chip gives a wrong answer without complaining) are rare and usually only happen on defective chips, not on healthy ones being stressed by a normal user.

5. The "Time Machine" (Reconstruction)

What happens when the computer chip breaks or is thrown away? Can we still trust the old report?

The Solution: The report includes "seeds" (like a recipe) that allow anyone to re-run the math on a different computer.
The Limit: You can re-verify the math and the logic perfectly. However, you cannot re-verify the speed of the old chip, because that speed depended on the specific physical condition of that dead chip (its heat, its age, its specific manufacturing quirks). The "truth" of the math survives; the "speed" of the old machine is lost to time.

Summary

This paper builds a self-verifying receipt for computer performance.

No Trust Needed: You don't trust the manufacturer; you trust the math and the digital locks.
Spot Checks: It uses random math tricks to catch lies instantly.
Double Witnesses: It uses two chips to confirm each other.
Honesty: It admits that while the math can be proven true forever, the specific speed of a chip is a fleeting moment that can't be perfectly recreated later.

The goal isn't to make the chips faster, but to make the numbers we read about them impossible to fake and easy to verify.

Technical Summary: Self-Verifying Measurement Records for Hardware Benchmarking

Problem Statement
Hardware performance benchmarks are traditionally accepted on trust. Reported numbers (e.g., FLOP/s rates) are detached from their underlying evidence: the specific device state, driver version, and exact input shapes are often lost by the time a result is published. Furthermore, hardware is not infallible; fleet studies indicate that a small but steady fraction of cores (approximately one in a thousand) can produce incorrect arithmetic results without raising errors. Consequently, a reported performance number lacks evidence of both its measurement origin and the correctness of the underlying computation. The paper addresses this as an integrity problem, aiming to create a measurement record that is tamper-evident and independently checkable without requiring the reader to trust the producer.

Methodology
The authors propose a hash-linked evidence graph, a transparency log for hardware measurements where every reported quantity is bound to its observation and verification via content hashes. The system operates through several interconnected mechanisms:

Evidence Graph Structure: The record is an acyclic graph where nodes represent observations (raw measurements), reductions (pure functions like medians or dispersions), and claims (the final reported numbers). Edges are content hashes (SHA-256). An offline audit can verify the entire graph by re-computing hashes and reductions without needing the original hardware or network access.
Verification of Linear Quantities (Matrix Products): For linear operations like matrix multiplication ($C=AB$), the system employs a probabilistic identity check (Freivalds' algorithm). A random probe matrix $X$ $X$ is used to compute the residual $\|A(BX) - CX\|$ $∥ A (B X) - C X ∥$ .
- Calibrated Tolerance: Instead of a theoretical bound, the acceptance tolerance ( $\tau$ ) is derived from floating-point error analysis and calibrated to the specific device's measured "residual floor" (the lowest residual observed for correct runs). This distinguishes between legitimate low-precision roundoff and genuine errors.
- Multi-Stage Transcript: Verification is recorded as a sequence of stages (acquire, witness, decide, repair). If a result exceeds tolerance, it is flagged, repaired via higher-precision recomputation, and re-witnessed.
Verification of Nonlinear Quantities: For operations lacking cheap algebraic checks (e.g., attention mechanisms, atomic accumulations), the system uses:
- Algebraic Guards: Decomposing attention into matrix products (checked via Freivalds) and a softmax invariant (checking row sums).
- Weighted Checksums: For atomic accumulations, a weighted sum check detects consistent faults that run-to-run divergence checks would miss.
- Reproducibility Classes: Quantities are classified as bit-stable ( $S_0$ ) or bounded by measured divergence ( $S_{nd}$ ).
Cross-Device Corroboration: Using two identical cards, the system measures the differential $D[q] = q(\text{card}_1) - q(\text{card}_0)$ . Deterministic kernels yield a zero differential, providing a bit-for-bit independent witness. Throughput differentials serve as a per-part fingerprint.
Security Enhancements: To prevent an adversary from exploiting a committed probe seed (which could hide errors in the probe's null space), the system derives the probe seed from a Fiat-Shamir challenge based on the claimed output. This ensures the challenge depends on the output being verified.

Key Results
The authors demonstrated the construction on Blackwell (RTX PRO 6000 Max-Q) and Hopper architectures, as well as consumer (RTX 5090) and server configurations.

Residual Floors: The study measured the residual floor of matrix products across precisions (FP16, TF32, BF16, FP8) and sizes. These floors were found to be architecture-independent invariants (stable across different Blackwell devices) but distinct from the device's throughput.
Detection Capabilities:
- A single bit flip in a matrix product was rejected with high probability ( $1 - 2^{-k}$ ).
- Consistent faults in nonlinear operations (which pass reproducibility checks) were caught by algebraic guards.
- An FP8 product exceeding the FP16-calibrated tolerance was successfully flagged and repaired via FP32 accumulation.
Device Independence: The "transferable invariants" (residual floors and numerical classes) were successfully reconstructed on different devices (RTX 5090, RTX PRO 6000 Server) using committed seeds, achieving 100% recovery of these specific claims. Device-local metrics (throughput) were not recoverable, as expected.
Physical Stress Testing: Under thermal soak and $di/dt$ power virus stress, the residual floor remained invariant, and no natural silent errors were observed in 139,264 checks. This suggests silent errors are rare (defective parts) or require privileged access, rather than being induced by standard tenant workloads.
Security Analysis: The Fiat-Shamir challenge successfully closed the attack surface where a probe-aware adversary could hide 50% of an error in the null space. A coordinated attack across 4 bit-identical witnesses was also caught by the output-derived challenge.

Significance and Claims
The paper claims to shift hardware benchmarking from a "trust-based" model to an "evidence-based" model. Its primary significance lies in:

Tamper-Evident Records: Providing a structure where a reader can verify a number's correctness and origin offline, without trusting the producer.
Calibrated Trust: Establishing that trust in a number should be proportional to its attached evidence. The method separates appropriate reliance from misplaced faith by calibrating tolerances to the physical reality of the silicon (the residual floor).
Hardware as Witness: Treating the hardware itself as the witness and signer of the measurement, rather than just the subject of the test.
Reconstructibility: Demonstrating that even after a device is retired, its "transferable invariants" (the mathematical properties of its computation) can be re-derived on successor hardware, preserving the integrity of the historical record.

The authors position this not as a replacement for hardware roots of trust (which are needed if the host producing the record is physically subverted) but as a disciplined application of probabilistic identities and content addressing to make hardware measurements self-verifying. The work is presented as a complete, auditable archive where the code, observations, and verification transcripts are bound to a single root hash, allowing for a single-pass offline audit.