The Sleeping Agent: What Gist-Based Context Compression Loses and Why
This paper demonstrates that while gist-based context compression improves multi-hop reasoning and factual retrieval in long-horizon language agents, it significantly impairs temporal question answering by discarding specific dates and times, a deficit that can be precisely remedied with a simple prompt modification to preserve temporal expressions.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine your brain is a massive, bustling library that never closes. Every time you talk to a friend, read a book, or watch a movie, a new book gets added to the shelves. But here's the catch: your library has a strict rule. It can only keep a few books open on the desk at once to read them. If you try to read a story that's too long, the librarian has to make a tough choice: throw away the old books, or try to shrink them down into tiny, pocket-sized summaries so they fit. This is the daily struggle for "AI agents"—computer programs designed to have long conversations. They need to remember what happened hours or days ago, but their "desk space" (called context window) is limited. If they forget the details, they can't answer tricky questions like "What did we decide to do last Tuesday?" or "Who was the third person we met?" Scientists have been trying to figure out the best way to shrink these memories without losing the most important parts. The big question is: when you summarize a long chat, do you keep the gist (the main story) or do you accidentally throw away the specific dates and times that make the story make sense?
This paper, titled "The Sleeping Agent," dives into that exact problem. The researchers built a clever system called Salience-Weighted Consolidation (SWC). Think of SWC as a super-smart librarian who doesn't just randomly shrink books. Instead, this librarian reads every page, rates how important it is (giving high scores to big decisions or funny stories and low scores to small talk), and then decides what to keep. For the "medium importance" stuff, the librarian writes a short summary, or "gist," to save space. The team tested this on ten long conversations, asking the AI hundreds of questions to see how well it remembered things.
Here is the twist they discovered: The librarian was doing a great job keeping the story but was terrible at keeping the calendar. When the AI used the standard summary method, it became a master at remembering "who did what" (like "Alice decided to meet Bob") but completely forgot "when" it happened (like "last Tuesday at 5 PM"). In fact, the standard method only kept about 3% of the time-related words (like dates and times) in the summaries. It was like summarizing a mystery novel by saying, "The detective solved the case," but leaving out the crucial detail that it happened before the moon rose.
The researchers realized this wasn't a random mistake; it was a design flaw. The instructions given to the summarizer didn't explicitly say, "Hey, don't delete the dates!" So, the AI treated dates as "boring details" to be thrown away. To fix this, they tried a tiny, one-sentence change to the librarian's instructions: "You MUST preserve verbatim: all specific dates, times, durations, ages, and temporal expressions."
The result was like flipping a light switch. With this tiny tweak, the number of time-related facts that survived the summary jumped from 3.05% to 62.39%—a twenty-fold increase! And the best part? It didn't hurt the AI's ability to remember the story. The "gist" of the conversation stayed just as good. When they tested the AI on questions about time (Category 2), the accuracy jumped by 0.314 points, a huge improvement that held true across all ten conversations.
The paper rules out the idea that "compression" generally makes AI dumber. In fact, for questions about logic or simple facts, the summarized version was actually better than just cutting off the old chat (truncation). The failure was specific: the generic summarization process was accidentally deleting the "time anchors" of the memory. The author is very sure about this because they measured it directly, showing that the fix works specifically for time-related questions without messing up other types of memory.
So, what's the takeaway? If you want an AI to remember a long conversation, you can't just ask it to "summarize the story." You have to explicitly tell it to keep the calendar and the clock. The paper suggests that for any system trying to compress long memories, treating "time" as a special, protected category is the key to keeping the AI from becoming a "sleeping agent" that knows the plot but has no idea what day it is.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.