🤖 AI

Trajectory-Aware Retrieval Agents for Temporal Decision- Making

This paper introduces TLM, a closed-loop agentic framework that leverages latent growth curve models over retrieved embeddings to preserve temporal structure and iteratively refine evidence, thereby significantly outperforming standard retrieval-augmented approaches in temporally grounded decision-making tasks like medical diagnosis and financial prediction.

Original authors: Jing Wang, Jie Shen, Xing Niu

Published 2026-07-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Jing Wang, Jie Shen, Xing Niu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a mystery, but instead of a single clue, you have a massive, chaotic pile of notes from a detective who wrote them over several years. Some notes describe the beginning of the case, others describe the middle, and some describe the very end. If you just grab a handful of these notes at random and ask a super-smart robot to solve the mystery, the robot might get confused. It might mix up a clue from 1990 with a clue from 2024, or miss the fact that the suspect's behavior changed slowly over time. This is the problem scientists face when they try to use Artificial Intelligence (AI) to make decisions based on long stories, like medical records or stock market reports. The AI is great at reading, but it often forgets when things happened.

To fix this, researchers are building "agents"—AI systems that can think, look for information, and then decide. A common tool they use is called Retrieval-Augmented Generation (RAG). Think of RAG like a librarian who runs to the shelves to find books that match your question, brings them back, and hands them to the AI. The problem is, standard librarians just grab books that sound similar, ignoring the fact that the story inside them has a timeline. They might hand you a page from the end of a novel before the page where the character falls in love. This paper introduces a new way to help the AI understand the story of the data, not just the words. It treats the information like a movie reel, watching how the plot moves forward, spots sudden twists, and fills in missing scenes, all to make better predictions about the future.

The Story of the "Trajectory" Detective

The researchers behind this paper, Jing Wang, Jie Shen, and Xing Niu, noticed that standard AI tools are terrible at handling time. If you ask a normal AI to predict what will happen next based on a patient's medical history or a company's earnings call, it often treats the information like a bag of marbles. It picks out the "important" marbles (sentences) but ignores the order they were picked up. In the real world, time matters. A patient's fever rising over three days is a different story than a fever that spikes and then drops. A CEO's tone shifting from confident to nervous during a call is a critical clue that gets lost if you just read the sentences out of order.

To solve this, the team built a new system called TLM (Trajectory Language Model). You can think of TLM as a detective who doesn't just read the case file; they watch the movie of the case file.

Step 1: The Smart Search
First, TLM acts like a hybrid librarian. It uses two methods to find clues: one that looks for exact word matches (like searching for a specific name) and one that looks for the "vibe" or meaning of the words. It grabs a few initial pages from the document to start.

Step 2: The Growth Curve (The "Movie Reel" Trick)
This is the paper's big magic trick. Instead of just reading the pages, TLM looks at the shape of the story. It uses a mathematical tool called a Latent Growth Curve Model (LGCM). Imagine you are watching a plant grow. You don't just look at the plant on Tuesday and the plant on Friday; you look at the line connecting them. Is the plant growing steadily? Is it wilting? Did it suddenly stop growing?
TLM does this with the text. It turns the sentences into a line on a graph.

Trend: Is the story getting better, worse, or staying the same?
Turning Points: Did the story suddenly change direction? (Like a patient getting worse after feeling better).
Gaps: Did the story skip a beat? (Like a missing day in a diary).
If TLM sees a "gap" in the story, it knows to go back to the library and find the missing pages to fill the hole.

Step 3: The "SHAP" Scorecard
Once TLM has a pile of clues, it needs to decide which ones matter most. It uses a method called SHAP, which is like a game where every sentence gets a score for how much it helped solve the puzzle. TLM uses a tiny, fast "scorer" robot to do this math quickly. If a sentence is super important, TLM uses it to ask the librarian for more related clues. Crucially, TLM never throws away a clue it already found; it only adds new ones. This ensures the story keeps getting richer, never poorer.

Step 4: The Final Decision
Finally, TLM feeds this organized, time-aware story to a large AI brain (a Large Language Model) to make the final call. Because the story is now ordered and complete, the AI can make a much smarter guess.

What They Found

The team tested this new detective on three very different types of mysteries:

Medical Questions: They asked the system to answer difficult medical questions based on patient histories. The standard AI (without TLM) got about 34.6% of the answers right. The new TLM system got 64.2% right. That's nearly double the success rate! The paper suggests this is because medical stories are all about how a patient changes over time, and TLM is the only one that noticed the pattern.
Earnings Calls: They tried to predict if a company's stock would go up or down after a CEO's speech. This is notoriously hard because the stock market is very smart and reacts instantly. The standard AI was barely better than guessing (about 50.7%). TLM improved this to 58.0%. While that doesn't sound like a huge jump, in the world of finance, beating the market even a little bit is a big deal.
Stock Gaps: They tried to predict if a stock price would jump up or down the next morning. They simulated a trading game starting with $10,000. The standard AI ended up with $12,067. TLM, by spotting the trends better, grew the money to $20,400.

The Takeaway

The paper shows that when you give an AI a way to understand the shape of a story over time, it becomes a much better detective. By using math to track trends and fill in missing pieces, TLM doesn't just read the words; it understands the journey. The authors are careful to say this isn't a magic wand that solves everything—it works best when the information has a clear timeline, and it still needs to be tested in more real-world situations. But for now, it proves that if you want an AI to make good decisions about the future, you have to teach it how to look at the past in order.

Technical Summary: Trajectory-Aware Retrieval Agents for Temporal Decision-Making

Problem Statement

The paper addresses the limitations of standard Retrieval-Augmented Generation (RAG) pipelines when applied to high-stakes decision-making tasks involving temporally structured, long-form text (e.g., clinical histories, earnings calls, stock market data).

Standard RAG approaches retrieve text chunks based solely on semantic similarity, fragmenting the chronological context into an unordered "bag of text." This structural loss is critical because many decision problems rely on the trajectory of information—such as a patient's worsening symptoms, a CEO's shifting guidance, or serial stock price dependence. Discarding this temporal structure can lead to conditioning on contradictory evidence or missing critical transitions. Furthermore, standard autoregressive LLMs produce token sequences rather than bounded classification decisions, leading to high variance and sensitivity to prompt formatting when forced to output discrete labels.

Methodology: TLM (Trajectory Language Model)

The authors propose TLM, a closed-loop agentic framework that integrates classical statistical methods into the RAG pipeline to model the latent trajectory of retrieved evidence. The framework operates in five stages:

Hybrid Retrieval: The source document is partitioned into overlapping chunks. Initial retrieval uses a hybrid approach combining sparse lexical matching (BM25) and dense semantic similarity (via a pretrained sentence encoder) to form an initial evidence set $Z^{(0)}$ .
Latent Growth Curve Modeling (LGCM): The ordered retrieved chunks are treated as observations of an evolving latent state. The embeddings of these chunks are fitted to a linear LGCM ( $e_i = \mu + \beta t_i + \epsilon_i$ $e_{i} = μ + β t_{i} + ϵ_{i}$ ), where $\mu$ $μ$ represents the baseline state and $\beta$ $β$ the rate of change. This decomposition yields three actionable signals:
- Trend Classification: Categorizing the trajectory as stable, progressive, or changing direction.
- Turning Point Detection: Flagging chunks with residuals exceeding the mean as deviations from the expected trajectory.
- Gap Detection: Identifying sharp differences between successive embeddings to trigger targeted re-retrieval of missing intermediate information.
Learned Re-Ranking and Retrieval: A lightweight, attention-based scorer $g_\theta$ is trained jointly with the LLM. It re-ranks chunks by importance and constructs "learned queries" (averaged embeddings of top-scoring chunks) to retrieve additional, diagnostically relevant evidence that may not lexically match the original query.
LLM Classification: The final evidence set (re-ranked chunks, LGCM summary, query, and options) is passed to a causal LLM. The model scores answer options using next-token logits to produce a discrete classification.
SHAP-Guided Iterative Refinement: Post-initial prediction, the system estimates the contribution of each chunk using a leave-one-out approximation of Shapley values, computed efficiently via the lightweight scorer. Chunks with positive contributions seed new queries to retrieve additional evidence. Crucially, this refinement is additive only; previously retrieved chunks are never discarded.

The authors provide a theoretical proposition stating that under a scorer-calibration assumption, this additive refinement procedure is monotonically non-decreasing in the probability assigned to the correct label.

Key Contributions

Latent Growth Curve Modeling for RAG: The paper introduces the first application of LGCMs to RAG evidence trajectories. This provides a principled decomposition of evidence into baseline, trend, and residuals, enabling the detection of trends, turning points, and information gaps.
Joint LLM-Scorer Architecture: The authors propose a lightweight scorer trained jointly with the LLM to enable efficient SHAP value estimation. This replaces expensive LLM forward passes with cheap scorer passes, making iterative evidence refinement practical.
Empirical Validation Across Domains: The framework is evaluated on three distinct temporal decision tasks: medical question answering, earnings call surprise prediction, and overnight stock gap prediction.

Experimental Results

The TLM framework was evaluated on three datasets using a Qwen3-14B backbone and BAAI/bge-large-en-v1.5 embeddings:

Medical Question Answering (MedQA): TLM achieved 64.2% accuracy, significantly outperforming the zero-shot LLM baseline (34.6%) and an embedding-only classifier (15.3%). This demonstrates the necessity of multi-hop inference over temporally structured evidence in clinical reasoning.
Earnings Call Surprise Prediction: On a binary classification task predicting market reaction to earnings, TLM achieved 58.0% accuracy. This surpasses zero-shot LLMs (50.7%) and standard retrieval baselines, which barely exceeded random chance, highlighting the difficulty of the task due to the efficient market hypothesis.
Overnight Stock Gap Prediction: In a 3-way classification task (Buy/Hold/Sell) simulated as a trading strategy, TLM achieved 38.3% accuracy compared to 27.5% for the zero-shot baseline. In terms of economic impact, TLM grew a simulated portfolio from $10,000 to $20,400, compared to $12,067 for the baseline.

Significance and Claims

The paper claims that TLM bridges a gap between statistical methodology for longitudinal data analysis and modern LLM agents. By treating retrieved evidence not as an unordered set but as a temporal trajectory, the framework allows agents to reason about trends and transitions rather than static facts.

The authors emphasize that while the gains in financial tasks are modest in percentage terms, they are economically meaningful given the difficulty of predicting market reactions. The approach is presented as a generalizable method for improving decision-making in any domain where the chronological order and evolution of information are critical to the correct outcome. The work does not claim to solve all temporal reasoning challenges but offers a specific mechanism (LGCM + SHAP refinement) to mitigate the structural loss inherent in standard RAG pipelines.