Stop When Memory Suffices: Evidence-Conditioned Progressive Execution for LLM Agents

Imagine you are trying to solve a massive, multi-day mystery. You have a giant notebook filled with every conversation, clue, tool used, and mistake made during the investigation. To solve a new riddle, you could either read the entire notebook from cover to cover every single time (which takes forever and is exhausting), or you could try to summarize the whole thing into a tiny reference sheet beforehand (which is fast, but you might miss a crucial detail hidden in the fine print). This is the daily struggle for Artificial Intelligence (AI) agents—computer programs designed to think, act, and remember like humans. As these AI agents get smarter and tackle longer, more complex tasks, they need a way to remember things without getting bogged down by the sheer volume of information. The big question scientists are asking is: How can an AI know exactly how much of its memory it needs to look at to solve a problem, so it doesn't waste time reading the whole book when just one page would do?

This is where a new idea called Router-Mem comes in. Think of it as a super-smart librarian who doesn't just fetch books; they also decide how much of the library you actually need to visit. In the past, AI memory systems were stuck in a "two extremes" trap: either they were super-fast but missed important clues (the "reference sheet" approach), or they were thorough but incredibly slow because they read everything every time (the "read the whole book" approach). Router-Mem breaks this rule by acting like a progressive detective.

Here is how it works: When you ask the AI a question, it first does a quick, cheap "sniff test" of its memory to grab a few relevant clues. Then, a tiny, lightweight "traffic cop" (called a router) looks at those clues and the question. It asks a simple yes-or-no question: "Is this enough to solve the mystery right now?" If the answer is yes, the AI instantly gives you the answer and stops, saving a huge amount of time. If the answer is no, the system doesn't start over; instead, it uses those initial clues as a map to dive deeper into the specific sections of the memory that need more attention, gathering more evidence before answering.

The researchers tested this on two challenging memory benchmarks called AMA-Bench and BEAM. They found that Router-Mem is a game-changer. On the AMA-Bench test, it achieved a score of 55.17%, and on the BEAM test, it scored 38.77%. But the real magic is in the speed. Compared to the old method of reading the entire memory history every time, Router-Mem cut the time it takes to think by 27.3% on one test and 25.5% on the other.

The paper suggests that this "stop when you have enough" approach is the sweet spot. It proves that you don't need to choose between being fast and being accurate. By training a smart router to recognize when evidence is sufficient, the system can skip the heavy lifting for easy questions while still doing the deep dive for hard ones. It's like having a detective who knows exactly when to close the case and when to keep digging, making long-term memory for AI both efficient and powerful.