NetCAS: Dynamic Cache and Backend Device Management in Networked Environments

NetCAS is a framework that dynamically optimizes I/O distribution between local caches and remote backend storage by leveraging real-time network feedback and a precomputed performance profile, achieving significantly higher throughput than traditional caching and existing converging schemes in fluctuating network environments.

The Gist

Imagine you are running a busy coffee shop. You have two ways to get coffee to your customers:

1. The Barista (The Cache): A super-fast expert standing right behind the counter. They can make a cup in 5 seconds, but they can only hold a few cups at a time.
2. The Delivery Truck (The Backend): A slower truck that drives to a warehouse across town. It takes 30 seconds to get there, but it has an infinite supply of coffee.

The Old Way: The "Barista Only" Rule

In the past, coffee shops (computer systems) followed a strict rule: "Only use the Barista."
If the Barista had the coffee, great! If not, you had to wait for the Barista to go get it from the truck, bring it back, and then serve you. The truck was considered too slow to be useful for anything else. This worked fine when the Barista was lightning fast and the truck was a slow, broken-down wagon.

The New Problem: The Traffic Jam

Today, technology has changed. The "Barista" (fast memory) is still fast, but the "Truck" (remote storage) has also gotten much faster. However, the road to the warehouse (the network) is unpredictable. Sometimes it's a clear highway; other times, it's a massive traffic jam.

If you keep sending all your orders to the Barista, you waste the Truck's speed. But if you send orders to the Truck during a traffic jam, your customers wait forever.

The old systems were like a manager who didn't look out the window. They would keep sending trucks down the road even when it was gridlocked, or they would ignore the truck entirely even when the road was clear.

The Solution: NetCAS (The Smart Traffic Manager)

The paper introduces NetCAS, a smart system that acts like a real-time traffic manager for your coffee shop.

Here is how it works, using simple metaphors:

1. The "Memory Map" (The Perf Profile)

Before the shop even opens, NetCAS studies the Barista and the Truck. It creates a Memory Map (called a Perf Profile).

• What it knows: "If we have 10 customers, the Barista is best. If we have 100 customers, the Truck can actually help speed things up."
• Why it matters: It doesn't guess. It knows exactly how fast each worker is under different conditions.

2. The "Window Watcher" (Real-Time Monitoring)

While the shop is open, NetCAS has a Window Watcher looking at the road.

• What it does: It doesn't just guess the traffic; it sees it. If the truck is stuck in a jam (network congestion), the Watcher shouts, "Hey, the road is slow! Stop sending trucks!"
• The Reaction: It instantly tells the manager to send fewer orders to the Truck and more to the Barista. As soon as the traffic clears, it says, "Okay, send more trucks!"

3. The "Fair Queue" (The Splitter)

NetCAS doesn't just shout orders randomly. It uses a Smart Queue (called Batched Weighted Round Robin).

• How it works: Imagine a conveyor belt. Instead of sending 100 cups to the Barista and then 100 to the Truck (which causes one to sit idle while the other is overwhelmed), NetCAS interleaves them perfectly.
• The Analogy: It's like a DJ mixing two songs. It ensures the Barista and the Truck are both working at the same time, keeping the line moving smoothly without anyone getting stuck waiting.

Why is this a Big Deal?

The paper tested this system in a "disaggregated" environment (where the computer and the storage are in different buildings, connected by a network).

• The Result: NetCAS was up to 174% faster than the old "Barista Only" method.
• The Comparison: It was 3.5 times faster than other smart systems that try to "guess" the best mix but react too slowly to traffic jams.

The Bottom Line

In the past, computers treated fast storage and slow storage as a strict hierarchy (Fast first, slow never).
NetCAS realizes that in the modern world, "slow" storage is often fast enough to help, if the road isn't jammed.

It's a system that dynamically splits the work between the fast local worker and the remote warehouse based on real-time traffic conditions, ensuring that no matter what happens on the road, your coffee (data) gets to you as fast as possible.

Technical Summary

1. Problem Statement

Modern storage systems are evolving from strict hierarchical designs (where a fast cache strictly dominates a slow backend) to hybrid models where cache and backend devices (e.g., PMem and NVMe SSDs) exhibit overlapping throughput and latency characteristics. While Non-Hierarchical Caching (NHC) strategies have shown that splitting I/O requests between these devices can maximize aggregate bandwidth, existing solutions face critical limitations in disaggregated data center environments:

• Network Variability: In disaggregated storage, backend devices are accessed remotely over networks (e.g., NVMe-oF/RDMA). Network congestion, cross-traffic, and fabric contention cause unpredictable fluctuations in backend latency and throughput.
• Static/Slow Adaptation: Traditional splitting policies rely on static ratios or slow convergence mechanisms (like OrthusCAS) that monitor hit rates or local device stats. These fail to react quickly enough to rapid network changes, leading to either overloading a degraded backend or underutilizing available bandwidth.
• Performance Degradation: When a backend is congested, sending requests to it causes queue buildup and latency inflation, which can degrade overall system performance below that of a simple cache-only approach.

2. Methodology: The NetCAS Framework

NetCAS is a lightweight, network-aware caching framework designed to dynamically split I/O requests between a local cache and a remote backend based on real-time network feedback and precomputed performance models. It integrates directly into the OpenCAS kernel module.

Core Components:

1. Performance Profile (Perf Profile):

   • A precomputed lookup table (LUT) indexed by block size, in-flight requests, and thread count.
   • It stores empirically measured standalone throughputs for both cache and backend devices under various workload configurations.
   • This allows NetCAS to determine an ideal baseline split ratio ($\rho_{base}$) without costly online exploration.

2. Real-Time Network Monitoring:

   • NetCAS instruments the NVMe-oF RDMA host kernel module to measure throughput and latency at the request completion path.
   • It calculates a severity score (drop_permil) based on normalized deviations from baseline throughput and latency. This score quantifies network congestion without requiring cross-node coordination or intrusive signaling.

3. Dynamic Split Ratio Calculation:

   • The system uses an analytical model to balance service times. Under congestion, the backend throughput ($I_{back}$) is scaled down by the severity score.
   • The new split ratio ($\rho$) is computed as:
     $$\rho = \frac{I_{cache}}{I_{cache} + I_{back} \cdot (1 - d/1000)}$$
   • This ensures the system shifts more load to the local cache when the network is congested.

4. Batched Weighted Round Robin (BWRR) Scheduler:

   • To enforce the calculated split ratio efficiently, NetCAS employs a BWRR scheduler.
   • Unlike random dispatch or simple round-robin, BWRR uses a "batched" approach with a repeating pattern derived from the Greatest Common Divisor (GCD) of the split weights.
   • This ensures the target ratio is maintained even within short time windows, preventing burstiness and ensuring both devices remain saturated without introducing per-request locking overhead.

5. Mode-Based Control:

   • The system operates in four states: No Table (profiling), Warmup (stabilizing baselines), Stable (using precomputed ratios with near-zero overhead), and Congestion (recalculating ratios based on live metrics).
   • Transitions are event-driven, ensuring the system incurs negligible overhead during steady-state operation.

3. Key Contributions

• Network-Aware Splitting: First framework to extend non-hierarchical caching to remote storage by dynamically adjusting split ratios based on real-time network fabric conditions rather than just cache hit rates.
• Hybrid Control Plane: Combines a precomputed Perf Profile for rapid baseline estimation with host-local end-to-end monitoring for immediate reaction to network anomalies, avoiding the latency of centralized controllers.
• Low-Overhead Enforcement: Introduces the BWRR scheduler to enforce split ratios with minimal CPU overhead, avoiding per-request decision costs and extra threads/locks.
• Transparency: Integrates directly into the OpenCAS kernel path, requiring no changes to applications or file systems, and maintains consistency by only splitting cache-hit read requests (writes remain pass-through).

4. Experimental Results

The authors evaluated NetCAS on a dual-socket Intel Xeon server with an Optane PMem cache and a remote Samsung NVMe SSD over a 100 Gbps RDMA network.

• Performance Gain (No Contention): NetCAS achieved up to 174% higher throughput compared to traditional cache-only (OpenCAS) and 125% higher than the state-of-the-art OrthusCAS in contention-free scenarios.
• Robustness Under Contention:
  • Under fluctuating network conditions, NetCAS outperformed OrthusCAS by up to 3.5×.
  • While OrthusCAS suffered sharp throughput drops (up to 20%) when the backend was congested due to its static split ratio, NetCAS adaptively shifted load to the cache, limiting throughput reduction to only 17%.
• Real-World Workloads: Using Filebench workloads (random reads, sequential scans, mixed I/O), NetCAS consistently maintained the highest throughput. In mixed read/write scenarios, it outperformed OpenCAS by 1.65× and OrthusCAS by 1.29× under contention.
• Overhead: The framework introduced negligible CPU overhead, increasing total utilization from 12.46% (OpenCAS) to only 12.79% (NetCAS).

5. Significance

NetCAS addresses a critical gap in modern cloud and data center storage architectures. As storage becomes increasingly disaggregated to improve elasticity and cost-efficiency, the assumption of stable backend performance is no longer valid.

• Scalability: By operating autonomously on each host without a global control plane, NetCAS scales efficiently with cluster size.
• Efficiency: It unlocks the aggregate bandwidth of heterogeneous storage tiers even in unstable network environments, moving beyond the "cache hit rate" paradigm to an "end-to-end throughput" paradigm.
• Practicality: The solution is lightweight, requires minimal code changes (approx. 1,200 LOC), and is deployable in existing Linux kernel environments, making it a viable solution for next-generation hybrid storage systems.