Next Generation Cloud-native In-Memory Stores: From Redis to Valkey and Beyond

This study provides a comprehensive experimental benchmark of emerging cloud-native in-memory key-value stores—including Valkey, KeyDB, and Garnet—within Kubernetes environments to evaluate their performance trade-offs, resource efficiency, and long-term viability as alternatives to Redis.

The Gist

Imagine you run a bustling, high-speed coffee shop. Your customers (data requests) are demanding, and they need their drinks (data) instantly. For the last decade, Redis has been the gold standard barista. It's fast, reliable, and everyone knows how to order from it.

But recently, the owner of the original coffee shop changed the rules (licensing), making some people nervous about relying on them forever. So, three new baristas stepped up to take over: Valkey, KeyDB, and Garnet.

This paper is like a blind taste test and efficiency audit of these new baristas, run inside a simulated "cloud" kitchen (Kubernetes) to see who actually performs best when the rush hour hits.

Here is the breakdown of what they found, using simple analogies:

1. The Contenders: Who Are They?

• Redis (The Veteran): The original. It's a single barista working incredibly fast but can only do one thing at a time. If the line gets too long, it gets overwhelmed.
• Valkey (The Community Successor): Imagine the original barista's best apprentice who took over the shop, kept the exact same menu (100% compatible), but hired a team of helpers to carry trays. It's fast, familiar, and backed by a huge group of supporters (the Linux Foundation).
• KeyDB (The Multitasker): Another apprentice who decided to hire many baristas to work at once. It's fast, but the paper suggests the shop might be running out of steam because the original owner hasn't updated the training manual in a while.
• Garnet (The High-Tech Innovator): A completely new shop built from scratch by a tech giant (Microsoft). It uses futuristic equipment (modern .NET tech) and is incredibly fast and efficient. However, the menu is slightly different, so customers have to learn new ways to order.

2. The Test: The "Rush Hour" Simulation

The researchers didn't just ask the baristas how fast they were; they simulated a chaotic lunch rush.

• The Setup: They put all four systems in identical "cloud kitchens" (Kubernetes) with the same amount of space and power.
• The Crowd: They simulated two types of crowds:
  • Write-Heavy: A crowd constantly placing new orders (like a live sports betting app).
  • Read-Heavy: A crowd just checking the menu or looking up prices (like a caching system for a news site).
• The "Hot" Items: Just like in real life, 20% of the menu items get ordered 80% of the time. They tested how the systems handled this "hot key" traffic.

3. The Results: Who Won the Race?

🏆 The Speed Demon: Garnet

Garnet was the undisputed speed king.

• The Analogy: If Redis could serve 100 cups an hour, Garnet could serve 200.
• The Catch: It's like a Formula 1 car. It's incredibly fast and uses less fuel (CPU/Memory), but you can't just drive it on a normal road. You have to rebuild your garage (refactor your code) because the steering wheel is in a different spot (only 70% compatible with Redis).
• Verdict: Best for brand-new shops that want maximum speed and don't mind the construction work.

🥈 The Smart Switch: Valkey

Valkey was the most practical winner.

• The Analogy: It's like upgrading your old car with a turbo engine and better tires, but keeping the same steering wheel and pedals. You can swap it in overnight without changing how you drive.
• The Performance: It was about 30-40% faster than the original Redis and used less fuel (memory/CPU).
• Verdict: The best choice for existing shops that want to upgrade without the headache of moving to a new building.

🥉 The Struggling Multitasker: KeyDB

KeyDB tried to be fast by hiring more staff, but it didn't quite pay off.

• The Analogy: It hired a team of baristas, but they kept bumping into each other in the kitchen. It was slightly faster than the original, but it used more energy to do it.
• The Problem: The shop owner (developers) seems to have stopped updating the training manual. The paper worries this shop might close down soon.
• Verdict: Not recommended. It's a bit slower than Valkey and risky for the long term.

4. The Big Takeaways

• If you want to stay safe and fast: Go with Valkey. It's the "drop-in" replacement. You get a speed boost without the migration nightmare.
• If you are building a new, high-speed empire: Go with Garnet. It's the most efficient and powerful, but you have to be willing to learn a new language to talk to it.
• If you are thinking about KeyDB: The paper says to be careful. It's a bit of a gamble because the developers aren't very active right now.

The Bottom Line

The world of "in-memory" data storage is evolving. You don't have to stick with the old standard (Redis) anymore.

• Valkey is the sensible, reliable upgrade.
• Garnet is the high-performance rocket ship.
• KeyDB is a bit of a wild card that might not be around in five years.

The paper concludes that for most businesses, Valkey is the sweet spot: it gives you the speed you need without the stress of moving house.

Technical Summary

1. Problem Statement

In-memory key-value datastores are critical for modern cloud-native architectures, with Redis historically serving as the de facto standard. However, recent licensing changes by Redis Inc. (shifting to RSALv2/SSPLv1) have created a need for open-source alternatives. While three primary contenders have emerged—Valkey (a Linux Foundation-backed fork), KeyDB (a multithreaded fork), and Garnet (a Microsoft Research implementation)—the literature lacks an independent, comprehensive experimental evaluation. Existing comparisons often rely on vendor claims or narrow use cases, failing to address critical production concerns such as migration complexity, operational compatibility, long-term sustainability, and resource efficiency under realistic cloud-native workloads.

2. Methodology

The authors conducted a systematic benchmarking study within a Kubernetes environment to simulate realistic cloud-native deployment scenarios.

• Test Environment:
  • Infrastructure: Two VMs on the same physical host (one for the Kubernetes cluster, one for benchmarking).
  • Configuration: Ubuntu 24.04 LTS, Kubernetes v1.31.5, 4 vCPUs/12 GiB RAM per node.
  • Deployment: All systems (Redis v7.2.7, Valkey v8.0.2, KeyDB v6.3.4, Garnet v1.0.62) were deployed in standalone mode with identical resource constraints (2 CPU cores, 8 GiB RAM) and persistence disabled to isolate in-memory performance.
• Workload Design:
  • Tool: Modified memtier_benchmark to support Zipfian key-access distributions (simulating "hot keys").
  • Data Pressure: Pre-loaded with ~4.8 million keys to achieve 75% memory utilization (1 KiB value size + overhead).
  • Workload Types:
    • Write-Heavy (Workload A): 50% SET / 50% GET, Zipfian skew $\alpha = 0.9$.
    • Read-Heavy (Workload B): 5% SET / 95% GET, Zipfian skew $\alpha = 1.4$.
  • Concurrency: Tested from 50 to 500 connections.
• Metrics: Throughput (ops/sec), Tail Latency (P99, P999), CPU/Memory efficiency, and statistical significance testing (t-tests).

3. Key Contributions

• Independent Benchmarking: Provides the first comprehensive, unbiased comparison of Valkey, KeyDB, and Garnet against Redis in a containerized environment.
• Holistic Evaluation: Moves beyond raw throughput to evaluate migration complexity (API compatibility), resource efficiency (cost implications), and project viability (community support and development velocity).
• Architectural Analysis: Correlates specific architectural choices (e.g., Valkey's I/O multithreading, Garnet's .NET lock-free structures, KeyDB's multithreading) with observed performance trade-offs.
• Practical Guidelines: Offers scenario-specific recommendations for organizations deciding on a Redis replacement.

4. Key Results

The study reveals distinct trade-offs among the systems, summarized below:

A. Performance (Throughput & Latency)

• Garnet (The Performance Leader):
  • Achieved the highest throughput: +108% over Redis in write-heavy and +110% in read-heavy workloads.
  • Lowest latency: 40–55% lower P99 latency than Redis under high concurrency.
  • Reason: Built from scratch on modern .NET with lock-free data structures and optimized memory management.
• Valkey (The Balanced Performer):
  • Consistent improvements: +32% (write) and +38% (read) throughput over Redis.
  • Latency: 25% lower P99 than Redis; excellent tail latency in read-heavy scenarios.
  • Reason: Enhanced I/O multithreading while maintaining single-threaded command execution safety.
• KeyDB (The Modest Improver):
  • Modest gains: +12% (write) and +15% (read) over Redis.
  • Latency: Higher P99 and P999 latency than Redis (up to +35% variance), suggesting thread contention issues.
  • Reason: Multithreaded design that increases CPU overhead without proportional throughput gains.

B. Resource Efficiency

• Garnet: Most efficient, using 30–40% less CPU and 25–30% less memory than Redis for equivalent throughput.
• Valkey: Improved efficiency, using 15–20% less CPU and 8–10% less memory.
• KeyDB: Less efficient, increasing CPU usage by 10–15% and memory by 2% compared to Redis.

C. Compatibility & Viability

• Valkey: 100% API compatible with Redis. Backed by the Linux Foundation and major cloud providers (AWS, Google, Oracle). High development velocity.
• Garnet: Only 70% API compatible. Requires significant code refactoring for advanced Redis features. Backed by Microsoft but has a narrower ecosystem.
• KeyDB: 95% API compatible. However, development activity is low/stagnant, raising serious concerns about long-term sustainability.

5. Significance and Recommendations

The paper concludes that the "best" choice depends entirely on the deployment context:

1. For Existing System Migration: Valkey is the optimal choice. It offers a "drop-in" replacement with full API compatibility, significant performance gains, and strong long-term viability, minimizing migration risk.
2. For Greenfield/High-Performance Projects: Garnet is recommended if maximum throughput and resource efficiency are critical and the team can manage the migration effort (refactoring for 30% API incompatibility). It offers the best cost-performance ratio for cloud infrastructure.
3. For Multi-Core Optimization: KeyDB is not recommended for production. Despite its multithreaded architecture, it offers only marginal performance gains, suffers from higher latency variance, and lacks active development support.

Overall Impact: This study fills a critical gap in the literature by providing data-driven evidence that Valkey is the most practical successor for general adoption, while Garnet represents a high-performance niche alternative, and KeyDB faces viability risks. The findings help organizations make informed architectural decisions balancing performance, cost, and operational stability.