🤖 machine learning

TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series

The paper introduces TimeRLM, a recursive language model framework that leverages code and vision capabilities to achieve state-of-the-art precision in localizing anomalies within long-context time-series data, outperforming existing models on both a new synthetic benchmark (AnomalyXL) and real-world recordings.

Original authors: Nicolas Zumarraga, Lorenzo Steno, Ning Wang, Max Rosenblattl, Thomas Kaar, Maxwell A. Xu, Kevin O'Sullivan, Markus Kreft, Elgar Fleisch, Paul Schmiedmayer, Patrick Langer, Robert Jakob

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicolas Zumarraga, Lorenzo Steno, Ning Wang, Max Rosenblattl, Thomas Kaar, Maxwell A. Xu, Kevin O'Sullivan, Markus Kreft, Elgar Fleisch, Paul Schmiedmayer, Patrick Langer, Robert Jakob

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to find a single, tiny clue hidden inside a library that stretches for miles. This isn't just any library; it's a library of time, where every book is a continuous stream of data streaming from sensors—like a heart monitor beeping for days, a factory machine humming for weeks, or a sleep tracker recording every breath for years. In the world of computer science, this is called time-series data. Usually, when we want to find something unusual (an "anomaly") in this data, we ask a super-smart computer brain, known as a Large Language Model (LLM), to read the whole story at once and tell us what's wrong.

But here's the problem: these computer brains have a short attention span. If you hand them a story that is too long, they get overwhelmed and start missing the tiny, crucial details buried in the middle. It's like asking someone to read a 1,000-page novel in one second; they might remember the beginning and the end, but the secret message hidden on page 450 gets lost. This paper tackles that exact headache: how do we find a needle in a haystack when the haystack keeps growing?

The researchers behind this study, a team from institutions like ETH Zürich and Stanford, propose a clever new way to solve this puzzle. They introduce a system called TimeRLM. Instead of forcing the computer to swallow the entire massive data stream in one giant bite, they let the computer act like a detective with a magnifying glass and a notebook. The computer keeps the massive data file outside its brain, in a "sandbox" where it can run code, zoom in on specific sections, and even draw graphs to look at the data visually. It can ask itself, "Let me check the last hour," or "Let me zoom in on this specific channel," and then report back. This method is called a Recursive Language Model, meaning the computer can break a big problem into smaller, manageable steps, solving them one by one rather than guessing the whole answer at once.

To test if this idea actually works, the team built a new training ground called AnomalyXL. Think of this as a giant, synthetic video game where they programmatically hide different types of "glitches" (like a sudden spike in temperature or a weird heartbeat) inside long, boring streams of data. They created two ways to play: one where the computer just guesses multiple-choice answers (like "Is there a glitch? Yes/No"), and a harder version where the computer must pinpoint the exact start and end time of the glitch, like a surgeon identifying the precise moment a heart skipped a beat.

The results were striking. When the researchers tested their new "detective" system against other top-tier computer models, the TimeRLM system was far superior at finding the exact location of these hidden glitches. In the hardest tests, where the computer had to point to the specific seconds where an anomaly happened, TimeRLM achieved a score of 0.682 (a measure of how well the predicted time matches the real time), while the best traditional models only managed 0.329. Even more impressively, when the computer had to not only find the glitch but also explain what kind it was, TimeRLM scored 0.745, whereas the other models barely scored above 0.072.

The paper suggests that this "recursive" approach—where the computer interacts with the data over multiple turns, using code to dig deeper—is a much more effective way to reason about long periods of time than just trying to read everything at once. To make the system even better, the team used a technique called reinforcement learning, which is like giving the computer a reward every time it successfully finds a hidden glitch. After this training, the system became even sharper, finding anomalies with high precision even when tested on real-world data it had never seen before, such as actual heart recordings (ECG) and sleep studies.

Interestingly, the paper also points out that the old way of testing these models—using simple multiple-choice questions—might be tricking us. The researchers found that while many models could guess "Yes, there is a glitch" correctly, they often failed to say where it was. It's like a student who can guess the right answer on a test by looking at the options but can't actually solve the math problem. By forcing the models to provide precise evidence and exact timestamps, TimeRLM revealed a true ability to navigate long, complex data streams, suggesting that for tasks like monitoring patient health or industrial safety, the future of AI might not be about bigger brains, but about smarter, more patient detectives.

Technical Summary: TimeRLM

Problem Statement

Precise anomaly localization in long-context time-series is a critical challenge in domains such as clinical care, industrial operations, and financial services. While Time Series Language Models (TSLMs) have emerged to process raw time-series data and verbalize findings, recent benchmarks indicate a significant degradation in retrieval performance as context length increases. Similar to failure modes observed in text, vision, and audio long-context retrieval, single-pass TSLMs often miss fine-grained events or fail to localize anomalies precisely when buried within high-frequency data spanning hours or days. Existing approaches typically ingest time-series into a context window (either via full-resolution patching, latent compression, or rendered plots) and attempt to answer in a single forward pass, which limits their ability to perform the iterative, fine-grained scanning required for precise temporal retrieval.

Methodology

TimeRLM Formulation

The authors propose TimeRLM, a Recursive Language Model (RLM) formulation designed to manipulate time-series signals as external context rather than static input. Unlike single-pass models, TimeRLM treats the time-series as a queryable environment variable within a sandboxed Python REPL (Read-Eval-Print Loop).

The system operates as a finite-horizon sequential decision process:

Orchestration: A root LLM (policy $\pi_\theta$ ) receives a question and the recording metadata.
Action Space: At each step, the model can either:
- Execute Code: Emit Python code to the sandbox to query, transform, or visualize specific segments of the signal (e.g., calculating rolling statistics, spectral decomposition, or plotting sub-segments).
- Recursion: Invoke sub-RLMs to analyze specific segments of the signal in depth.
- Finalize: Submit a structured JSON answer.
Feedback Loop: The model receives the execution output (text or visual plots) as new context, allowing it to iteratively refine its hypothesis and narrow down the anomaly location.
Vision Integration: The framework supports a vision variant where the model can render time-series as plots and perceive them visually, alongside the text-based code execution.

Reinforcement Learning (RL)

To further enhance performance, the authors implement a post-training strategy using Group Relative Policy Optimization (GRPO).

Training Loop: The generator is integrated directly into the training loop, synthesizing fresh batches of recordings at every step to prevent sample repetition.
Reward Signal: The reward is derived from the verifiable outcome of the decision process (e.g., Intersection over Union for localization).
Cost Penalties: The reward function includes penalties for token usage (relative to the group's maximum) and for forced finalization, encouraging efficient exploration.
Curriculum: Anomaly-free cases are withheld during the initial training steps to prevent the model from over-relying on "no anomaly" answers before learning localization behaviors.

The AnomalyXL Benchmark

To rigorously evaluate these capabilities, the authors introduce AnomalyXL, a synthetic long-context anomaly localization benchmark.

Scale: Samples span 16k to 131k data points with up to 16 channels.
Anomaly Injection: Anomalies (transient spikes, level shifts, trend changes, variance changes, seasonality changes) are injected into stochastic noise processes with programmatically controlled magnitude and duration.
Task Families: The benchmark includes five task categories:
1. Localize: Detect presence and return the precise time window.
2. Classify: Identify the anomaly type and window.
3. Magnitude: Quantify the deviation in standard deviations ( $\sigma$ ).
4. All Channels: Identify anomalies across all 16 channels.
5. Lead-Lag: Determine the temporal offset between anomalies in two channels.
Formulations: Tasks are evaluated in two modes: AnomalyXL-MCQ (multiple-choice, coarse-grained) and AnomalyXL-Localize (structured JSON, precise retrieval).

Key Contributions

TimeRLM Architecture: A novel RLM formulation for time-series that enables iterative, code-driven interaction with raw signals, bypassing the context window limitations of single-pass models.
AnomalyXL Benchmark: The first synthetic long-context anomaly benchmark released with its generator and RL environment, featuring programmatically injected anomalies and configurable difficulty across five task types.
Reinforcement Learning Integration: A demonstration that post-training RLMs on synthetic data using GRPO significantly improves performance and generalization, even with smaller backbone models.
Comprehensive Evaluation: Extensive testing against state-of-the-art TSLMs (ChatTS, OpenTSLM-Flamingo, ITFormer, Toto) and frontier single-pass models (GPT-4.1-mini, GPT-5.5) across synthetic and real-world clinical datasets.

Results

Performance on AnomalyXL

Precise Localization: TimeRLM significantly outperforms all baselines on AnomalyXL-Localize. On the localization task, TimeRLM (image variant, GPT-5.5 backbone) achieves an IoU of 0.682, compared to a maximum of 0.329 for the best single-pass baseline.
Classify-with-Evidence: TimeRLM reaches 0.745 on classify-with-evidence, vastly superior to the best baseline of 0.072.
Recursion Benefits: Enabling recursion (depth-1) further improves performance, particularly on evidence-bearing tasks, with the model making an average of 1.14 sub-RLM calls per task.
Efficiency: The RL-trained TimeRLM achieves these results using an LLM backbone up to eight times smaller (Qwen3.5-4B) than the 32B Toto baseline, while requiring approximately one-third as many agent interaction turns to produce a final answer.

Generalization to Real-World Data

Classification vs. Localization: When evaluated on unseen real-world Electrocardiogram (ECG) (LTAF dataset) and Sleep (Sleep-PSG) recordings, the post-trained TimeRLM shows mixed results. While it improves classification performance (F1) over the base model and surpasses TSLMs in some cases (e.g., 0.446 F1 vs 0.331 for ITFormer on ECG; 0.340 F1 vs 0.083 for OpenTSLM on Sleep-PSG), localization performance remains near zero. Specifically, the RL-trained model achieves an IoU of only 0.016 on ECG and 0.002 on Sleep-PSG, failing to precisely localize anomalies despite the improved classification scores.
TSLM Failure: Supervised TSLMs (fine-tuned on AnomalyXL) also fail on localization in these unseen clinical domains, with no model exceeding an IoU of 0.004. However, TSLMs retain some ability to classify anomalies (e.g., ITFormer reaches 0.331 F1 on ECG), whereas the RL model's localization capability does not transfer effectively to real-world data despite synthetic training.
ARFBench: TimeRLM leads on the Tier 2 tasks of ARFBench (real telemetry data), achieving 61.4% accuracy and 57.9% macro-F1, outperforming single-pass GPT-5.5 and Toto variants.

Ablation and Analysis

Context Length: Single-pass models show a steep performance degradation as context length increases (e.g., localization IoU drops from 0.48 at 16k to 0.16 at 131k for single-pass image). TimeRLM maintains stable performance (0.61–0.79 IoU) across the entire length range.
MCQ vs. Localization: The paper notes that MCQ benchmarks (AnomalyXL-MCQ) often over-report model capabilities because they can be solved using aggregate statistics without precise retrieval. TimeRLM's advantage is most pronounced in the precise retrieval tasks (Localize, Classify-with-Evidence).

Significance and Claims

The paper claims that recursive interaction with time-series is an effective approach for long-horizon temporal reasoning and retrieval. The authors argue that:

Retrieval is the Bottleneck: The primary failure mode of current TSLMs in long contexts is not understanding the data, but the inability to precisely retrieve specific segments.
Code as a Tool: Using code to query external context allows models to overcome the "needle in a haystack" problem inherent in long time-series, enabling precise localization that single-pass architectures cannot achieve.
Synthetic-to-Real Transfer: Reasoning in code provides a signal-agnostic learning environment. Post-training on synthetic data preserves the model's ability to classify anomalies in real-world, out-of-domain clinical signals, suggesting that the learned procedures generalize better than standard supervised fine-tuning for classification tasks. However, the study highlights a critical limitation: this transfer does not extend to precise localization (IoU) on real-world data, where performance remains negligible despite synthetic training.
Benchmarking Necessity: The authors assert that existing multiple-choice benchmarks may under-investigate retrieval capabilities and that precise, evidence-grounded localization metrics are necessary to accurately evaluate time-series reasoning models.

The work concludes that while TSLMs are promising, the recursive, agentic approach of TimeRLM offers a superior pathway for precise anomaly localization in long-context scenarios, particularly when combined with reinforcement learning, though its ability to localize precisely on real-world clinical data remains an open challenge.