🤖 AI

Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV

This paper introduces "semantic materialization," demonstrating that sparse event-KV caches can retain and serve as independent views of specific, bounded state even after their source observations are evicted, provided the events are deliberately phrased to encode information without explicitly stating values.

Original authors: Zefeng Cai, Zerui Cai

Published 2026-07-28

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Zefeng Cai, Zerui Cai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a spaceship that has been traveling for years. To navigate, your ship's computer keeps a massive log of every star it passed, every sensor reading, and every decision made. But the ship's memory is limited; it can't keep the whole log in its active brain. So, the computer has a clever trick: it keeps a "snapshot" of the most important moments in a special, high-speed memory bank called a KV cache. Think of this cache not just as a diary of words, but as a set of sticky notes where the computer has already done the hard math. When the ship needs to know something, it doesn't re-read the whole history; it just looks at these sticky notes.

The big question for future AI explorers is: What happens when you throw away the original page of the log that created a sticky note? If the computer deletes the page that said "The engine is hot," but keeps the sticky note that says "Engine Status: Critical," can the computer still know the engine is hot? Or does the note become a meaningless scribble? This is the puzzle of sparse event-KV serving. It's about whether an AI can remember the result of a thought even after it has forgotten the ingredients of that thought. If the answer is yes, we could build super-smart agents that remember years of conversation without needing a library the size of a planet. If the answer is no, we might be throwing away the very things that make our AI smart.

The Magic of the "Ghost Note"

In this paper, researchers Zefeng Cai and Zerui Cai decided to test this idea with a very specific, almost magical experiment. They treated the AI's memory like a game of "Telephone," but with a twist. They set up a scenario where an AI agent was told, "The switch is ON." Later, the agent was told, "The mirror reflects the switch." Then, the researchers deleted the first message ("The switch is ON") from the AI's memory, leaving only the second message ("The mirror reflects the switch"). Finally, they asked the AI: "Is the mirror ON or OFF?"

Here is the mind-blowing part: Among the specific cases where the AI's answer actually depended on the hidden switch, it got it right almost every time.

To be precise, the researchers found that when the AI's answer did change based on the hidden switch (a "donor-sensitive" item), it followed the hidden value 99 times out of 99 on the Qwen3-8B model. However, this perfect record applies only to the items where the AI was capable of changing its mind; it does not mean the AI got every single question in the universe right. In the full set of 256 test pairs, there were 125 cases where the AI gave the same answer regardless of the hidden switch (because the answer was already obvious or the AI was indifferent). But for the 130 cases where the hidden switch mattered, the AI followed the deleted source perfectly.

Even though the AI was never told the value of the switch in the messages it was allowed to see, it knew the answer was ON. The researchers call this "Semantic Materialization." It's as if the AI wrote the answer to the math problem on a sticky note before it threw away the textbook. The "sticky note" (the cached memory of the second sentence) secretly held the answer, even though the sentence itself didn't say it.

The "Donor Pair" Test: How They Knew It Wasn't a Guess

You might think, "Maybe the AI just guessed 'ON' because it's a common word." To prove this wasn't a lucky guess, the researchers used a clever trick called a "Donor Pair."

Imagine they ran the exact same experiment twice, but with a tiny difference:

Run A: The hidden switch was ON.
Run B: The hidden switch was OFF.

In both runs, the messages the AI could see were identical. The only difference was the secret message they deleted. When they asked the AI about the mirror, it answered ON in Run A and OFF in Run B.

On the Qwen3-8B model, this happened 99 times out of 99 for the items where the answer actually depended on the switch. The AI followed the hidden, deleted message perfectly. This proves the AI wasn't guessing; it was reading a "ghost note" hidden inside its memory that carried the secret value.

The Catch: It's Not Magic, It's a Contract

While this sounds like a superpower, the paper reveals that this "ghost note" ability is tricky and has strict rules. It's not a universal magic trick that works for everything.

1. You have to write the note the right way.
The researchers tried 16 different ways to phrase the "mirror" sentence. Some worked like a charm, and others failed completely.

The Winner: Phrases like "The register mirrors the source" worked great (about 80% success).
The Loser: Phrases like "The register was written from the source" barely worked at all (around 50%, which is just guessing).
The Lesson: It's not about what the sentence means, but how it sounds. Two sentences that mean the same thing can have totally different results depending on the specific words used. It's like a secret handshake; if you get the rhythm wrong, the door doesn't open.

2. It only works for simple secrets.
The "ghost note" is great at remembering simple Yes/No answers (like ON/OFF).

Success Rate: For binary choices, the AI remembered correctly 93% of the time.
Failure: If you asked the AI to remember a 4-choice option, it dropped to 22% (barely better than guessing).
Total Failure: If you asked it to remember a 3-digit number (like "123"), it got it right 0% of the time. The "note" can't hold big, heavy data; it only holds tiny, compact states.

3. Don't trust "accidental" memory.
The researchers tested if this happens naturally in long conversations (like a real chat between friends). They found that if you just let the AI chat and hope it "accidentally" writes a good ghost note, it doesn't work. The success rate was the same as if the note didn't exist.

The Fix: You have to deliberately write a special "carrier" sentence designed to hold the secret. When they did this, the success rate jumped from 6% to 51%. It's not something that happens by accident; it's a tool you have to build.

The Big Warning for AI Builders

The most important takeaway for anyone building AI systems is a warning about how we test memory.

Usually, if you delete a piece of information from an AI's memory and the AI still gets the answer right, engineers assume, "Great! That piece of info wasn't important. We can delete it to save space."

This paper says: STOP.

If the AI gets the answer right after you delete the source, it might be because the AI already wrote the answer into a "ghost note" earlier. You didn't prove the source was useless; you just proved the AI was smart enough to save the answer for later. If you delete the source and the note, the AI might fail.

The Bottom Line

This paper discovered a hidden layer of AI memory where the computer can "pre-compute" answers and store them in a way that survives even when the original question is gone. It's like a chef who writes the recipe on a napkin, throws away the ingredients list, and still knows exactly how to cook the dish.

However, this power is fragile. It only works if you use the right "secret handshake" (phrasing), it only holds small secrets (Yes/No), and it doesn't happen by accident. For anyone trying to build AI that remembers long stories, the lesson is clear: Don't just delete the past; check if the AI has already written the answer down on a sticky note. If you do, you might find that the "memory" is much more powerful—and much more complex—than we thought.

Technical Summary: Compute Globally, Materialize Locally

1. Problem Statement

Long-horizon agents increasingly rely on sparse KV cache serving, where systems retain a subset of cached event spans and evict the rest to manage memory. A critical, untested premise underlies current eviction and episodic-memory policies: that a retained event span remains informative even after the specific observations (source events) that generated its internal state have been dropped.

Standard text-retrieval logic suggests that if the source text is gone, the information is lost. However, KV-backed serving posits a stronger capability: during the initial global prefill, downstream tokens are contextualized by upstream sources via causal computation. Consequently, the cached rows ($KV$) of a downstream event may encode the result of that computation, acting as a "materialized" state that persists even if the source text is never served again. This paper investigates whether these independently served spans carry hidden, source-omitted state, and under what conditions this "semantic materialization" is reliable.

2. Methodology

The authors employ a controlled causal comparison using donor pairs to isolate semantic materialization from text-based retrieval or guessing.

Donor Pair Construction: Two agent histories are constructed that are byte-identical in every served token and position. They differ only in an omitted source event ( $e_{src}$ ) which sets a value $v$ (e.g., "Register S is ON") versus $v'$ ("Register S is OFF").
The Test: The system serves a downstream "root" event ( $e_{root}$ ) whose text references the source (e.g., "M mirrors S") but does not explicitly name the value. The source event $e_{src}$ is omitted from the serving set.
The Query: The model is asked for the state of the mirrored register.
The Metric: If the model's answer follows the omitted donor value ( $v$ or $v'$ ) significantly more than chance, it proves the served $KV$ rows of $e_{root}$ carry materialized state derived from the missing source.
Controls: The study uses independent decoys to prevent the model from simply eliminating wrong answers. It distinguishes between "roots" (events that directly attend to the source) and "reference edges" (events that attend to the root).
Models: Experiments are conducted on frozen checkpoints of Qwen3-8B, Ministral-3-8B, and Gemma-4-12B, alongside exploratory diagnostics on 2024-era models (Gemma-2, Llama-3.1, etc.).

3. Key Contributions and Findings

A. Discovery of Semantic Materialization

The paper establishes that selecting rows for serving does not select only their visible token content.

Causal Evidence: On Qwen3-8B, among donor-sensitive items where the source is omitted, the answer follows the hidden donor value with a ratio of 99:0. The served text never states the value, yet the model recovers it from the cached rows of the downstream event.
State vs. String: The materialized information is semantic state, not verbatim text. In matched swap probes, verbatim transfer occurred in only 9.7% of cases (indistinguishable from control), while semantic polarity transferred in 64%.

B. The "Memory Contract"

The paper defines a contract governing what can be written, where it lands, and what survives:

Trigger (Surface Form): Materialization is not a universal property of meaning but of surface form.
- Across 16 different linguistic constructions (e.g., "M mirrors S" vs. "M is kept equal to S"), write-through rates vary wildly (from chance to 0.95) even when comprehension is near-ceiling.
- No single construction works across all models. For instance, "mirrors" is a strong writer on Qwen3 but fails on Gemma-4.
- Calibration is required: Systems cannot assume materialization; they must calibrate specific constructions per checkpoint.
Landing (Roots Carry, Edges Route):
- Roots: The event that directly attends to the source carries the dominant donor-aligned signal.
- Edges: Downstream events that refer to the root (e.g., "T mirrors M") mostly route the query back to the root rather than carrying the state themselves. On Gemma-4, asking about a chain of length 3 yields the correct answer only if the root is served; serving only the edge yields chance performance.
Access (Compact State Envelope):
- Binary State: Recovery of binary states (ON/OFF) is robust (0.934 accuracy vs. 0.5 chance).
- Higher Cardinality: Recovery degrades rapidly as payload complexity increases. Four-way and eight-way payloads fall near chance rates. Three-digit numbers are never recovered exactly (0/192).
- Derived State: Without a write-time prompt, derived conclusions are not natively recovered.

C. Programming the Primitive

Active vs. Passive: Passive harvesting of natural mentions from long-term dialog yields no detectable advantage over isolated encoding.
Deliberate Carriers: A deliberately phrased, answer-free event (e.g., "determine the verdict now and record it here") raises donor-aligned recovery from 6% to 51% on Qwen3-8B.
Explicit Text: An explicit textual record remains the only reliable fallback that recovers state across all models.

D. Implications for Eviction Policies

The paper challenges the standard validation metric for memory systems: Dropping a source event and observing no accuracy loss does not prove the source was unnecessary. The system may have retained a downstream row that already materialized the answer. Corrections should be served as new events (patches) rather than cache edits to avoid hijacking historical queries.

4. Significance and Claims

The paper claims to identify a semantic memory substrate within the KV cache that operates independently of the visible text.

Beyond Retrieval: Sparse event-KV serving is not merely retrieval over rows; it is the retrieval of precomputed, materialized results of upstream computation.
Engineering Handle: The primitive is programmable. By calibrating specific "carrier" constructions per model, systems can intentionally write compact state into downstream events to survive source eviction.
Limitations: The authors are modest about the scope. The primitive is limited to compact state (binary/low-cardinality). It does not support exact numeric recovery or complex derived conclusions without explicit prompting. Furthermore, the phenomenon is highly model- and construction-specific, requiring per-checkpoint calibration rather than offering a universal syntax.

In summary, the paper argues that for long-horizon agents, the KV cache acts as a "compute globally, materialize locally" engine. However, relying on this requires a deliberate "memory contract" where systems explicitly write state into calibrated carrier events, rather than passively hoping natural dialog mentions will suffice.