Hazel: Secure and Efficient Disaggregated Storage

Hazel is a storage management system that extends the NVMe-over-Fabrics protocol to provide secure, high-performance disaggregated storage with confidentiality, integrity, and freshness guarantees through novel mechanisms like counter-leasing, a disaggregated Merkle Tree, and smart NIC offloading.

The Gist

Imagine you are a high-security bank manager. In the old days, you kept your gold in a heavy safe inside your office. It was secure, but if you grew too big, you’d run out of room, and your office would become a cluttered mess of safes.

To grow, you decide to move your gold to a massive, high-tech warehouse across town (this is Disaggregated Storage). This is much more efficient because you can have a tiny office for your staff and a giant, cheap warehouse for the gold.

The Problem:
Moving the gold creates three massive security headaches:

1. The Spy (Confidentiality): How do you make sure no one can see what the gold looks like while it's being driven to the warehouse?
2. The Counterfeiter (Integrity): How do you know someone didn't swap your real gold bars for painted lead ones during the trip?
3. The Time Traveler (Freshness): This is the sneakiest one. Imagine a thief steals a gold bar today, but then replaces it with a gold bar you used to have three years ago. It’s real gold, but it’s "stale." It’s not the gold you currently own. Traditional security often misses this "replay attack."

Current security methods are like hiring a massive army of guards to inspect every single grain of gold. It works, but it’s so slow that your bank practically stops functioning.

---

Enter "Hazel": The Smart Security System

The researchers created Hazel, a system that makes high-security storage fast and scalable. They did this using three clever "tricks":

1. The "Lease" System (Scalable Control)

Instead of having a guard manually check every single gold bar's ID every time, Hazel uses a "Counter-Leasing" system.
Think of it like a digital ticket dispenser at a deli. The central office gives a local branch a "range" of tickets (e.g., "You are authorized to use tickets #1,000 to #5,000"). The local branch can hand these out instantly without calling the main office every single time. This prevents a massive "phone call" bottleneck and allows the bank to grow to massive sizes.

2. The "Smart Label" (Optimized Data Path)

In old systems, security info was kept in a separate, bulky ledger that had to be checked constantly. Hazel uses "Metadata Encapsulation."
Imagine if every gold bar had a tiny, microscopic, unforgeable sticker attached directly to it that contained its ID, its "freshness" code, and its secret key. Because the info is on the object, you don't have to go running back to the ledger every time you pick up a bar. You just look at the sticker and go. This makes the "reading" process incredibly fast.

3. The "Lazy Accountant" (The Hazel Merkle Tree)

To solve the "Time Traveler" (Freshness) problem, Hazel uses a special math structure called the Hazel Merkle Tree (HMT).
Traditional security is like an accountant who insists on re-counting every single coin in the vault every time one person buys a coffee. It’s perfect, but it’s exhausting.
Hazel’s accountant is "eventually consistent." They let the transactions happen quickly and then do the heavy math in the background in organized batches. It’s like a restaurant that gives you your receipt immediately but settles the complex bookkeeping at the end of the shift. It stays accurate, but it doesn't slow down the service.

---

The Result

The researchers tested Hazel using high-end hardware (NVIDIA chips). Even though they added layers of heavy-duty security, the "speed penalty" was almost invisible—only about 1% to 2%.

In short: Hazel gives you the security of a fortress with the speed of a highway.

Technical Summary

Technical Summary: Hazel: Secure and Efficient Disaggregated Storage

Problem: The Security-Performance Gap in Disaggregated Storage

Modern data centers and supercomputers are moving toward disaggregated storage using NVMe-over-Fabrics (NVMe-oF), which decouples compute from storage to improve resource utilization. Simultaneously, Confidential Computing (CC) is becoming the standard for protecting sensitive workloads via Trusted Execution Environments (TEEs).

However, existing security solutions (like Linux’s dm-crypt, dm-integrity, and dm-x) were designed for local storage and fail in disaggregated, PB-scale environments due to three critical issues:

1. Scalability: Traditional methods struggle to manage unique Initialization Vectors (IVs) and keys across multiple nodes without massive synchronization overhead.
2. Performance/Resource Exhaustion: Implementing encryption, integrity, and freshness (protection against replay attacks) on the host CPU leads to massive throughput degradation (up to 50%) and high CPU utilization (over 80%).
3. Freshness Bottlenecks: Maintaining a Merkle Tree (MT) for freshness requires global consistency, which causes network congestion and high latency in distributed settings.

Methodology: The Hazel Architecture

Hazel is a storage management system designed to extend NVMe-oF capabilities to satisfy the CC threat model (confidentiality, integrity, and freshness) without requiring hardware changes to the NVMe protocol itself. Its design is built on three pillars:

1. Scalable Control Path (Counter-Leasing & Key Derivation):

• Counter-Leasing: To avoid the "birthday paradox" collisions of random IVs and the synchronization bottlenecks of global counters, Hazel uses a Key Broker Service (KBS). The KBS leases large ranges of counters (e.g., 1 TB chunks) to local Hazel instances, allowing them to operate independently without constant network coordination.
• Key Management: Instead of storing massive amounts of per-partition keys, Hazel uses key derivation. A unique user-device key is derived from a master user storage key and the device ID using HMAC, minimizing the exposure of master keys.

2. Optimized Data Path (Metadata Encapsulation & HMT):

• Metadata Leveraging: Hazel uses the existing 64B NVMe metadata field to encapsulate security information (IVs, hashes, and key IDs). This avoids the "read-modify-write" overhead of storing security data in separate sectors.
• Hazel Merkle Tree (HMT): To solve the freshness problem, Hazel introduces the HMT. Unlike traditional MTs, the HMT:
  • Uses IVs as leaves instead of raw data to reduce memory footprint.
  • Employs Eventual Consistency (EC): It completes write requests before the tree update is finished, moving hashing out of the critical path.
  • Uses a Metadata Cache on the SSD to enable a "fast-path" read, where freshness is verified by comparing a sector's local authentication hash against its parent in the tree.

3. Resource Efficiency (SmartNIC Offloading):

• Hazel is designed to be offloaded to CC-capable SmartNICs (sNICs), specifically the NVIDIA BlueField-3. By moving cryptographic operations from the host CPU to the sNIC's hardware accelerators (GGAs), Hazel achieves line-rate processing and reduces host CPU pressure.

Key Contributions

• Novel Control Flow: Introduced the counter-leasing protocol and a decentralized key derivation framework.
• HMT Innovation: Developed a new Merkle Tree structure optimized for disaggregated storage that supports asynchronous updates and zero-overhead read verification.
• Protocol Extension: Achieved security through request encapsulation within the NVMe-oF framework, ensuring compatibility with existing hardware.
• Hardware-Software Co-design: Prototyped a high-performance implementation using SPDK and NVIDIA DOCA on BlueField-3 DPUs.

Results

The authors evaluated Hazel using synthetic benchmarks, the IO500 suite, YCSB (database workload), and ML training pipelines.

• Minimal Overhead: Hazel demonstrated as little as 1–2% performance degradation for synthetic patterns and AI training.
• Throughput & IOPS: In IO500 benchmarks, Hazel maintained an average overhead of only 6.3% across various patterns.
• Database Performance: For YCSB/RocksDB, throughput overhead averaged 0.6%, and latency overhead averaged 2.2%.
• Scalability: The system successfully handled PB-scale storage simulations with manageable memory requirements for the HMT (e.g., ~23 GB for a 1 PB drive).

Significance

Hazel represents a significant advancement in the convergence of High-Performance Computing (HPC) and Cloud Security. By proving that strong security guarantees (confidentiality, integrity, and freshness) can be achieved with negligible performance costs, Hazel provides a blueprint for the next generation of secure, disaggregated data centers. It enables sensitive industries (finance, healthcare, AI) to leverage the cost benefits of the cloud without compromising the rigorous security requirements of Confidential Computing.