Clove: Object-Level CXL Memory Management in Managed Runtimes

This paper presents Clove, a system that extends managed language runtimes with profile-guided hotness tracking and object relocation policies to enable efficient object-level CXL memory management, significantly outperforming traditional page-based systems by reducing application slowdown by 22–84%.

The Gist

The Big Problem: The "Mixed Bag" of Memory

Imagine your computer's memory is like a house with two rooms:

1. The Fast Room (Local Memory): This is right next to your desk. You can grab things from here instantly.
2. The Slow Room (CXL Memory): This is in the basement. It's much bigger, but it takes a long time to walk down there and grab something.

For years, computers have managed these rooms by moving things in boxes (called "pages"). If you want to move a box from the basement to the desk, you have to move the entire box, even if it only contains one item you need right now and 99 items you don't.

The Flaw: This is inefficient. Imagine a box containing a single, very popular book (hot data) and 99 old newspapers (cold data). If you move that whole box to the Fast Room just to get the book, you waste precious space on the newspapers. This is what the paper calls "intrapage hotness skew."

The Old Solution vs. The New Idea

• The Old Way (Unmanaged Languages): Some researchers tried to fix this by asking programmers to manually label their data and rewrite their code to tell the computer, "Move this specific book, not the whole box." This works, but it's hard work for the programmer and isn't transparent.
• The New Way (Managed Runtimes): The authors of this paper, Clove, realized that many modern applications (like those written in Java or C#) already run inside a "manager" (a runtime environment like the JVM). These managers are already experts at moving individual items around to keep the house tidy. They just haven't been taught how to manage the basement (CXL) yet.

The Goal: Teach the existing manager to move individual items (objects) instead of whole boxes, without the programmer having to change a single line of code.

How Clove Works: The Three-Step Strategy

Clove acts like a super-organized housekeeper who knows exactly what you need. It uses three main tricks:

1. The "Spy" (Profile-Guided Tracking)

To know what to move, Clove needs to know what you are using.

• The Problem: If the housekeeper checks every single item in the house every second to see if you touched it, she would spend all her time checking and no time cleaning. This is too slow.
• The Clove Solution: Instead of checking everything, Clove uses a hardware "spy" (called PEBS) to listen for the specific sounds of you reaching for the most popular items. It identifies the few specific "commands" (instructions) that cause the most trips to the basement.
• The Result: It only tracks the items accessed by those specific commands. It's like putting a motion sensor only on the door to the kitchen, rather than on every single drawer. This gives it perfect data with almost zero effort.

2. The "Packer" (Hybrid Relocation)

Once Clove knows what is "hot" (used often), it needs to move it.

• The Problem: The computer's operating system (OS) only understands how to move whole boxes (pages). The Java manager (Runtime) only understands how to move individual items (objects). They speak different languages.
• The Clove Solution: Clove acts as a translator.
  1. The Java manager gathers all the "hot" items and packs them tightly together into a single, neat stack of boxes.
  2. It tells the OS, "Hey, this specific stack of boxes is now full of hot stuff. Please move this whole stack to the Fast Room."
• The Result: The OS moves the boxes efficiently, but because Clove packed them so well, the boxes are now 100% full of useful stuff, with no wasted space on old newspapers.

3. The "Selector" (Smart Policies)

Clove doesn't just move everything that is warm. It has to be smart about what it moves.

• The Problem: If you move items that are only slightly popular, you might waste space in the Fast Room. If you move too few, you don't fill the room enough.
• The Clove Solution: It uses a "cutoff" rule. It only moves items that are very popular. It also checks the "neighborhoods" (memory regions) to see if a neighborhood has enough popular items to make the trip worth it. If a neighborhood is mostly empty or mostly cold, Clove leaves it alone to save energy.

The Results: Why It Matters

The authors tested Clove on real-world applications (like a database, a graph algorithm, and a cache) and compared it to the best existing systems that move whole boxes.

• Speed: Clove made applications run 22% to 84% faster than the old box-moving systems.
• Efficiency: It filled the "Fast Room" with useful data much better, meaning the computer rarely had to run to the basement.
• Transparency: The programmers didn't have to change their code. Clove did all the work automatically in the background.

Summary Analogy

Imagine a library where you have a small, fast desk (Fast Memory) and a huge, slow archive room (CXL Memory).

• Old Systems: They move entire bookshelves from the archive to the desk. If the shelf has one bestseller and 1,000 boring textbooks, the desk gets cluttered with textbooks, and you can't fit the bestseller.
• Clove: It's a smart librarian. It sees you reading the bestseller, pulls just that book off the shelf, packs it neatly with other bestsellers, and puts that specific collection on your desk. It leaves the boring textbooks in the archive. You get faster access, and your desk stays organized, all without you having to tell the librarian what to do.

Technical Summary

Technical Summary: Clove – Object-Level CXL Memory Management in Managed Runtimes

1. Problem Statement

The advent of Compute Express Link (CXL) has accelerated the development of tiered memory systems, offering a performance gap of only 2–4× between local memory and CXL-attached memory. However, existing management strategies rely on page-based granularity (e.g., 4 KB or 2 MB pages) to track memory hotness and relocate data. This approach suffers from intrapage hotness skew, where a single page contains objects with vastly different access frequencies. Consequently, even with optimal page placement, cold objects are forced into the fast tier alongside hot ones, reducing the fast-tier hit ratio.

While object-level management has been explored for unmanaged languages (e.g., C/C++) using bespoke runtimes or compiler support, it remains underexplored for managed languages (e.g., Java, .NET). Existing object-level approaches for CXL often lack transparency, requiring developers to rewrite applications. Furthermore, CXL's load/store interface provides no natural software interception point, making transparent object-level tracking challenging without incurring prohibitive overhead.

2. Methodology: The Clove System

Clove addresses these challenges by extending existing managed runtimes (specifically the Java Virtual Machine, or JVM) to support transparent, object-level CXL management. The system leverages the runtime's inherent capabilities—object-level visibility, moving garbage collection (GC), and Just-In-Time (JIT) compilation—to solve three core technical problems: hotness tracking, relocation mechanisms, and placement policies.

2.1 Profile-Guided Object Hotness Tracking

Directly tracking every object access via hardware sampling (e.g., Intel PEBS) is inefficient due to the high volume of objects and the resulting CPU overhead. Clove introduces a profile-guided approach:

• Instruction Identification: The system uses hardware-assisted sampling to identify a small set of "delinquent" load instructions responsible for the majority of Last-Level Cache (LLC) misses.
• Selective Instrumentation: Instead of tracking all objects, Clove uses the JIT compiler to insert hotness-tracking logic only at these specific delinquent load instructions.
• Low-Overhead Counters: The tracking logic increments a counter stored in unused bits of the object header (e.g., the top 16 bits in the JVM). This avoids separate metadata storage and minimizes cache misses.
• Periodic Activation: To balance accuracy and overhead, the tracking logic is periodically activated (e.g., every 1 ms) by a background thread rather than on every instruction invocation, ensuring the branch predictor handles the check efficiently.

2.2 Hybrid Relocation Approach

Clove employs a hybrid strategy that decouples logical object placement from physical page migration:

• Hot-Object Compaction: The managed runtime compacts "hot" objects into contiguous virtual pages within the heap. This is achieved by reusing the GC's existing object relocation machinery. During GC, Clove checks object hotness counters and redirects hot objects to a designated "hot-object space," separating them from cold objects.
• Page-Based Migration: Once hot objects are compacted into specific virtual pages, the underlying page-based system (e.g., OS-level tiering) detects these pages as hot and migrates the corresponding physical pages to the fast tier.
• Large Object Handling: Objects spanning multiple pages are handled at page granularity by the underlying system, while the runtime focuses on compacting smaller, hot objects.

2.3 Relocation Policies

To ensure efficiency, Clove implements specific policies for selecting objects and regions for compaction:

• Hotness Cutoff: Clove dynamically determines a hotness threshold based on the available fast-tier capacity. It constructs a histogram of object hotness during the GC scan and selects only objects exceeding the cutoff to fill the fast tier.
• Region Selection: In region-based GCs, Clove selects regions for compaction based on the ratio of hot bytes within them. It uses low and high watermarks to avoid selecting regions that are either too sparse (low benefit) or too dense (unnecessary work), thereby bounding the relocation overhead.

3. Key Contributions

The paper identifies extending managed runtimes as a viable design point for transparent object-level CXL management. The specific contributions include:

1. Design Point Identification: Proposing the extension of existing managed runtimes (like JVM and .NET CLR) to manage CXL tiered memory, leveraging their existing object-level visibility and GC mechanisms.
2. Novel Mechanisms:
   • Profile-Guided Tracking: A method to identify delinquent loads and instrument only relevant accesses, achieving accurate tracking with near-zero overhead.
   • Hybrid Relocation: A separation of concerns where the runtime compacts hot objects logically, and the OS handles physical page migration.
   • Systematic Compaction Policies: Algorithms to select objects and regions that maximize fast-tier utilization while minimizing GC overhead.
3. Prototype and Evaluation: A prototype implementation in OpenJDK 21 (using ZGC) that demonstrates significant performance improvements over state-of-the-art page-based systems without requiring source code changes.

4. Results

The authors evaluated Clove on a dual-socket system using three memory-intensive Java applications: Ehcache (key-value cache), JGraphT (PageRank), and H2 (in-memory database).

• Performance Improvement: Compared to state-of-the-art page-based systems (TPP, Memtis, HybridTier), Clove reduced application slowdown by 22% to 84% across various local memory ratios (10%, 20%, 50%).
  • In synthetic workloads with HotWarm distributions, Clove reduced latency by 29–59% at a 20% local memory ratio.
  • In real-world Twitter traces, slowdown reductions ranged from 29–63%.
• Hit Ratio: Clove achieved 11–45 percentage points higher fast-tier hit ratios than page-based baselines (Memtis). Its hit ratio was within 4–5 percentage points of an "oracle" placement policy.
• Overhead: The runtime overhead for hotness tracking was negligible (often <1%) due to selective instrumentation and periodic activation. Additional GC overhead from object-graph scans and relocation phases was minimal, occurring roughly once every six minutes and taking 15–26 seconds, which was hidden by concurrent execution in ZGC.

5. Significance and Claims

The paper claims that Clove demonstrates the potential of managed runtimes as a foundation for efficient and transparent CXL memory management. By leveraging existing mechanisms like moving garbage collection and JIT compilation, Clove overcomes the "intrapage hotness skew" problem that plagues page-based systems without the transparency costs associated with unmanaged-language approaches.

The authors emphasize that while the prototype is in the JVM, the principles apply to other managed runtimes with similar capabilities (e.g., .NET CLR, PyPy). The work suggests that for managed-language applications, which constitute a large class of production software, extending the runtime is a more effective path to CXL optimization than building bespoke runtimes or requiring application rewrites. The paper remains modest regarding extreme workloads where object hotness shifts every few seconds, noting that Clove is designed for workloads where hot sets evolve over minutes or hours.