OBASE: Object-Based Address-Space Engineering to Improve Memory Tiering

OBASE is a compiler-runtime system for unmanaged languages that improves memory tiering efficiency by dynamically reorganizing the address space to cluster hot and cold objects into uniform pages, thereby eliminating hotness fragmentation and reducing memory footprints by up to 70% with minimal overhead.

The Gist

The Big Problem: The "Hot and Cold" Mix-Up

Imagine you run a massive, high-speed library (your computer's memory).

• The Fast Aisle (DRAM): This is the expensive, super-fast section where you keep the books people are reading right now.
• The Slow Basement (SSD/Slower Memory): This is the cheap, slower section for books nobody has touched in years.

The Goal: You want to keep the Fast Aisle full of only the books people are currently reading, and move the dusty, unused books to the Basement to save money.

The Reality (The "Hotness Fragmentation" Problem):
In most computer systems, the librarian (the Operating System) doesn't know which specific pages of a book are being read. They only know that a whole book (a memory page) is on the shelf.

Here is the tragedy:
Imagine a single book on the Fast Aisle.

• Page 1 is being read constantly (Hot).
• Pages 2 through 100 are never touched (Cold).

Because Page 1 is hot, the librarian marks the entire book as "Active" and refuses to move it to the Basement.

• Result: You are paying for expensive Fast Aisle space to store 99 pages of dust just to keep 1 page of fresh news.
• The Paper's Finding: In real-world data centers, up to 97% of the data sitting in the expensive Fast Aisle is actually "cold" dust that could be moved, but the system is too dumb to see it because it's mixed up with the hot stuff.

---

The Solution: OBASE (The Smart Librarian)

The authors propose a system called OBASE. Think of OBASE as a super-intelligent, reorganizing librarian who works inside the application (the software) before the OS ever sees the data.

OBASE doesn't try to fix the OS; it fixes the layout of the data so the OS can do its job perfectly.

How OBASE Works (The Analogy)

1. The "Guide" System (The Name Tag)
In normal computer code, if you move a book, everyone holding a bookmark to it gets lost. OBASE solves this by giving every object a "Name Tag" (called a Guide).

• Instead of pointing directly to a book, your bookmark points to the Name Tag.
• The Name Tag says, "The book is currently on Shelf A."
• If OBASE moves the book to Shelf B, it just updates the Name Tag. The bookmarks don't break; they just look at the tag to find the new location.

2. The "Temperature Check" (Tracking)
OBASE watches the library. It sees which books (or parts of books) are being opened constantly.

• Hot Objects: Books being read right now.
• Cold Objects: Books gathering dust.

3. The Great Reorganization (Clustering)
This is the magic. OBASE physically moves the data around in the computer's memory:

• It gathers all the Hot pages and stacks them tightly together in one big, dense pile.
• It gathers all the Cold pages and stacks them in a separate, empty pile.

The Result:
Now, the Fast Aisle is 100% full of hot, active data. The Basement is 100% full of cold, unused data. There is no mixing.

• The OS (the librarian) can now look at the "Cold Pile" and say, "Ah, this whole pile is useless. I can move it to the cheap basement immediately!"
• The OS doesn't need to be smart; it just needs to see the clean piles OBASE created.

---

Why This is a Big Deal

1. It Saves Massive Money
The paper tested this on real data from Google, Meta, and Twitter.

• Before OBASE: They had to keep huge amounts of expensive memory to avoid slowing things down.
• After OBASE: They could shrink their expensive memory usage by up to 70% while keeping the system just as fast.
• Analogy: It's like realizing you don't need a 50-car garage because you only drive 2 cars. You can sell the other 48 parking spots.

2. It Works with Old Systems
The best part? OBASE doesn't require rewriting the Operating System (Linux, Windows, etc.). It acts as a "translator" or a "frontend." It prepares the data so that existing tools (like kswapd or TMO) work better than they ever have before.

3. It's Safe and Fast
Moving data while people are using it is dangerous (like moving a shelf while someone is climbing it). OBASE uses a clever "lock-free" trick. It only moves a book when no one is currently holding it. If someone does grab it, OBASE pauses the move, waits, and tries again. This happens so fast you don't even notice.

• Overhead: It only slows the computer down by about 2% to 5%, which is a tiny price to pay for saving 70% on memory costs.

Summary in One Sentence

OBASE is a smart system that rearranges computer memory to group "frequently used" data together and "rarely used" data together, allowing the computer to dump the cheap stuff into slow storage without slowing down the expensive stuff, saving data centers massive amounts of money.

Technical Summary

1. Problem Statement: Hotness Fragmentation

Despite the widespread deployment of memory tiering mechanisms (moving cold data to slower, cheaper tiers like SSDs or CXL-attached memory), data centers continue to overprovision expensive DRAM. The paper identifies the root cause as hotness fragmentation.

• The Mismatch: Applications allocate and access data at the object granularity (variable sizes), while the OS manages memory at the page granularity (fixed 4KB or 2MB).
• The Phenomenon: Modern allocators place objects based on size rather than access patterns. Consequently, "hot" (frequently accessed) and "cold" (rarely accessed) objects become interleaved within the same physical pages.
• The Consequence: A single hot object renders an entire page "active" in the eyes of the OS. This traps surrounding cold data in expensive DRAM, preventing it from being reclaimed or tiered.
• Evidence: Analysis of Google production workloads reveals that in active pages, up to 97% of the bytes are cold and unreclaimable. Even when only 3.2% of bytes are accessed, 91.8% of pages are marked active, making them poor candidates for reclamation.
• Dynamic Nature: Object hotness is not static; it changes continuously over time. Static allocation hints or one-time profiling cannot solve this because an object's access pattern evolves.

2. Methodology: OBASE System

The authors propose OBASE (Object-Based Address-Space Engineering), a compiler-runtime system designed for unmanaged languages (specifically C/C++) that acts as an intelligent frontend for page-aware OS backends.

Core Philosophy

OBASE decouples memory management into two orthogonal problems:

1. Frontend (Layout): Dynamically reorganizing the virtual address space to cluster hot objects together and segregate cold objects.
2. Backend (Reclamation): Existing OS mechanisms (kswapd, TMO, TPP, Memtis) that decide which pages to move based on page-level activity.

By ensuring pages are uniformly hot or uniformly cold, OBASE allows existing backends to make optimal decisions without needing to understand object-level semantics.

Key Technical Components

1. Lightweight Pointer Instrumentation (Guides):

   • OBASE introduces a "Guide" abstraction. Developers mark pointers that can be relocated.
   • Dereferencing a guide triggers a lightweight check that records access activity and resolves the object's current physical location.
   • Metadata Encoding: OBASE repurposes unused upper bits of 64-bit pointers to store metadata (Active Thread Count, Consecutive Inactive Window, Heap ID) directly in the pointer, avoiding external side tables.

2. Access Tracking (SODA):

   • Uses a Sparse Object Data Activity (SODA) bitmap to track which virtual regions contain managed objects.
   • The Object Collector (OC) periodically scans this metadata to classify objects as NEW, HOT, or COLD based on access frequency.

3. Spatially-Aware Memory Allocator (SAMA):

   • SAMA reserves contiguous virtual address ranges for three logical heaps:
     • NEW: Freshly allocated objects.
     • HOT: The current working set.
     • COLD: Objects inactive for a sustained period.
   • This contiguity allows the OS to apply coarse-grained hints (e.g., MADV_PAGEOUT to the whole COLD region) rather than individual pages.

4. Safe Concurrent Migration (Lock-Free Protocol):

   • Challenge: Moving objects in C++ without stopping the world or breaking pointer semantics.
   • Solution: OBASE uses an Active Thread Count (ATC) mechanism.
     • Threads register their activity via Thread-local Active Scope Guards (TAGs) when entering public data structure operations.
     • Migration only occurs when ATC reaches zero (no thread is actively using the object).
   • Optimistic Migration: The OC copies the object and attempts an atomic Compare-And-Swap (CAS) to update the guide. If a thread accesses the object during the copy, the CAS fails, and the migration is aborted. This ensures safety without blocking threads.

3. Key Contributions

• Identification of Hotness Fragmentation: Quantified the severe inefficiency in current memory tiering caused by the object/page granularity mismatch, showing that active pages are mostly cold.
• Address-Space Engineering: A novel approach to dynamically reorganize virtual memory layouts to align page boundaries with object access patterns.
• OBASE Implementation: A working system for unmanaged languages that achieves safe, lock-free object migration with minimal overhead.
• Backend Agnosticism: Demonstrates that reorganizing the layout significantly improves the effectiveness of existing OS tiering mechanisms without modifying them.

4. Evaluation Results

The system was evaluated across ten concurrent data structures (Hash Tables, Skip Lists, B+ Trees), six different backends, and production traces from Meta and Twitter.

• Page Utilization: OBASE improved page utilization by 2–4×. In read-only workloads, utilization reached ~80% (up from ~20%).
• Memory Footprint Reduction: By enabling effective tiering, OBASE reduced the Resident Set Size (RSS) by 65–72% (up to 70% in some cases).
• Backend Synergy:
  • Reclamation: Without OBASE, backends faced a trade-off between memory savings and performance (e.g., aggressive reclamation caused 38% throughput loss). With OBASE, backends achieved maximum memory savings (matching the working set size) with zero throughput degradation.
  • Tiering: In tiered configurations (DRAM + CXL/PMEM), OBASE allowed systems to achieve the same throughput with half the DRAM capacity. For example, OBASE+TPP at a 1:16 DRAM:CXL ratio performed comparably to TPP alone at a 1:8 ratio.
• Overhead:
  • Runtime overhead is 2–5% on throughput and p90 latency.
  • Scalability is linear; overhead remains bounded (1–8%) even as thread counts increase from 2 to 32.
• Adaptability: The system successfully adapted to dynamic production traces (Meta, Twitter), automatically adjusting the "cold threshold" to maintain a low promotion rate and handle shifting hotsets.

5. Significance

• Cost Reduction: OBASE offers a path to significantly reduce the dominant cost driver in data centers (DRAM) by enabling the use of cheaper, slower memory tiers without sacrificing performance.
• Practicality: Unlike previous research that required custom runtimes or kernel modifications, OBASE works with standard OS backends and unmanaged languages, lowering the adoption barrier for hyperscalers.
• Generalizability: The principles of address-space engineering extend beyond tiering. They could improve garbage collection (grouping by access intensity), reduce false sharing in NUMA systems, and enhance security isolation.
• Paradigm Shift: The paper argues that the virtual address space is a communication channel between applications and the OS. By engineering this channel, applications can express object-level semantics that page-based systems can effectively utilize.

In conclusion, OBASE solves the fundamental inefficiency of memory tiering by ensuring that the "hot" and "cold" data are physically separated at the page level, allowing existing OS mechanisms to reclaim memory aggressively without performance penalties.