💻 computer science

Bounded Context Management for Tabular Foundation Models on Stream Learning

This paper introduces CURE, a context management policy for Tabular Foundation Models in stream learning that optimizes prediction performance under distribution shift by preserving recent and uncertain examples while removing redundant ones through entropy-gated admission and redundancy-aware eviction.

Original authors: Jinmo Lee, Doyun Choi, Moongi Choi, Jaemin Yoo

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jinmo Lee, Doyun Choi, Moongi Choi, Jaemin Yoo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery that changes every day. New clues (data) arrive one by one in a continuous stream. You have a very powerful, pre-trained "super-brain" (a Tabular Foundation Model) that can solve these mysteries, but it has a strange rule: it doesn't learn by updating its own brain. Instead, it learns by looking at a specific list of past clues you give it right before it makes a guess. This list is called the "Context."

The problem? You only have a small notebook (limited memory) to write down these clues. If you fill the notebook, you have to throw something out to make room for new clues.

The Big Question: Which clues should you keep, and which should you throw away, so your super-brain makes the best guesses possible?

This paper introduces a new strategy called CURE to answer that question. Here is how it works, broken down into simple concepts:

1. The Problem: The "FIFO" Mistake

Before this paper, the standard way to manage the notebook was FIFO (First-In, First-Out). Imagine a line at a coffee shop: the person who arrived first gets served first, and when the line is full, the person at the back leaves.

The Flaw: In a changing world, the "oldest" clue isn't always the least useful. Sometimes, an old clue is a rare, confusing case that teaches the super-brain a lot. Sometimes, a new clue is just a copy of something you already have. Throwing things away just because they are "old" is inefficient.

2. The Solution: The CURE Strategy

The authors propose a new way to manage the notebook based on three simple rules: Keep the Recent, Keep the Confusing, and Throw Away the Duplicates.

They split the notebook into two sections: a Short Bank and a Long Bank.

Step A: The Short Bank (The "Recent" Rule)

How it works: Every new clue goes into the Short Bank first. This is a small, fast-moving area that holds the very latest examples.
Why: Things that happened recently are usually the best guess for what will happen next. It's like checking the weather forecast for today rather than last year to decide what to wear.
The Flow: When the Short Bank gets full, the oldest item in it is pushed out to be considered for the Long Bank.

Step B: The Long Bank (The "Confusing" Rule)

How it works: When a clue moves from the Short Bank to the Long Bank, the system asks: "Was this clue hard to predict?"
The Analogy: Imagine you are teaching a student. If a student gets a question right instantly, they probably already know the answer. But if they struggle and get it wrong (high uncertainty), that moment of confusion is a goldmine for learning.
The Action: The system only lets "confusing" clues (high uncertainty) into the Long Bank. If a clue was easy to predict, it gets discarded. This ensures the Long Bank is full of valuable, high-information examples.

Step C: The Cleanup (The "Redundancy" Rule)

How it works: Eventually, the Long Bank gets full. Now you have to throw something out.
The Analogy: Imagine you have a photo album. If you have 50 photos of your cat sitting in the exact same pose, you don't need all 50. You only need one.
The Action: The system looks for clues that are identical twins (same category and very similar features). If it finds two very similar clues, it throws away the one that is "farther" from the current trend (based on the Short Bank) and keeps the one that is closer to the "center" of the group. This removes duplicates without losing important information.

3. The Results: Why It Matters

The authors tested this "CURE" strategy on seven different data streams (like weather data, sensor data, and game data) and compared it against:

Old-school methods: Algorithms that constantly retrain themselves (like updating a tree's branches).
Previous "Context" methods: The old "First-In, First-Out" notebook strategy.

The Outcome:

CURE won. It made more accurate predictions than the old methods in almost every test.
It's flexible. It worked well with different types of "super-brains" (different foundation models), proving that the strategy itself is the key, not just the specific model used.
It's efficient. The extra math required to decide what to keep or throw away was so small that it barely slowed down the system.

Summary

Think of CURE as a smart librarian for a library that is constantly overflowing.

Instead of just kicking out the oldest books (FIFO), the librarian:
1. Keeps the newest books on the front desk (Short Bank).
2. Only shelves the most interesting and puzzling books in the main archive (Long Bank).
3. Throws away duplicate copies of the same boring book to save space.

By doing this, the "super-brain" always has the most helpful, up-to-date, and diverse set of clues to look at, allowing it to predict the future much better than before.

Technical Summary: Bounded Context Management for Tabular Foundation Models on Stream Learning

1. Problem Definition

The paper addresses tabular stream learning, a setting where a model must make supervised predictions on sequentially arriving examples under constraints of bounded memory, real-time response, and potential distribution shifts.

While standard stream learners (e.g., Hoeffding Trees, Adaptive Random Forests) adapt by updating their internal model states (e.g., tree statistics or ensemble members), Tabular Foundation Models (TFMs) operate differently. TFMs are pre-trained models that make predictions via in-context learning: given a query $x_t$ and a labeled context $D_t$ , they output a posterior predictive distribution $q_\theta(\cdot | x_t, D_t)$ without updating the model parameters $\theta$ .

Consequently, for TFMs applied to data streams, the adaptation mechanism shifts from updating model weights to managing the bounded context $D_t$ . The core challenge is defined as bounded context management: designing an online policy $U$ to select which labeled examples to retain in a context of fixed budget $B$ as new data arrives, such that the context remains maximally useful for future queries.

2. Methodology: The Future-Information View and CURE

The authors propose a theoretical framework called the Future-Information View to determine which examples should be kept or discarded. They define the "future usefulness" of a context $D_t$ as the mutual information it provides about the labels of near-future queries. Since the exact future distribution is unknown, they decompose this objective into three practical signals:

Recency: Recent examples serve as a proxy for the near-future feature distribution.
Uncertainty: Examples that were uncertain at the time of prediction (high entropy) are likely to provide high information gain once their labels are revealed.
Redundancy: Examples that are close in representation space and share the same label provide overlapping evidence; removing one incurs minimal information loss.

Based on these signals, the authors introduce CURE (Context management via Uncertainty-aware admission and Redundancy-aware Eviction). CURE implements a dual-bank architecture:

Short Bank ( $S_t$ ): A FIFO (First-In-First-Out) buffer that stores the most recent labeled examples. This ensures the context always covers the current data regime.
Long Bank ( $L_t$ ): A buffer for older examples selected for long-term retention.

CURE's Update Policy:

Admission: New labeled examples enter the Short Bank. When the Short Bank overflows, the oldest item ( $z^+$ $z^{+}$ ) becomes a candidate for the Long Bank.
- Entropy-Gated Admission: The candidate is admitted to the Long Bank only if its stored prediction-time entropy exceeds a threshold $\tau$ . This ensures the Long Bank retains examples that were informative (uncertain) at the time of observation.
Eviction: When the Long Bank exceeds its budget, an example must be removed.
- Redundancy-Aware Eviction: The policy identifies the most represented class in the Long Bank. Within that class, it finds the pair of examples closest in the normalized raw-feature space. It then removes the example from this pair that is farther from the centroid of the current Short Bank (acting as a reference for the current regime). This removes redundant evidence while preserving examples that are distinct relative to the current data distribution.

3. Key Contributions

Paradigm Shift: The paper reframes stream learning for TFMs from model-state adaptation to context management, identifying that the utility of a TFM in a stream depends entirely on the quality of its retained context.
Theoretical Framework: It formalizes the "Future-Information View," providing a theoretical justification for using recency, uncertainty, and redundancy as proxies for maximizing information about future queries.
CURE Policy: It proposes a concrete, efficient policy (CURE) that combines entropy-gated admission and redundancy-aware eviction within a dual-bank structure.
Empirical Validation: The authors demonstrate that CURE significantly outperforms classical stream learners and prior context management baselines (like DualFIFO) across multiple datasets and TFM backbones.

4. Experimental Results

The authors evaluated CURE on seven data streams (six real-world, one synthetic) using four TFM backbones (TabICL-v2, LimiX-v1, TabPFN-v2.5, TabDPT-v1).

Performance vs. Classical Baselines: CURE achieved the best prequential accuracy on all seven streams when using TabICL-v2. It showed relative improvements of up to 27.0% over classical stream learners (e.g., ARF, BOLE, EFDT). Notable absolute gains included +19.59 points on the RIALTO dataset and +19.31 on METER.
Backbone Robustness: CURE improved over the DualFIFO baseline in 17 out of 18 comparisons across different TFM backbones, indicating that the context management strategy is transferable and not specific to a single model architecture.
Ablation Studies: Comparisons with controlled variants (e.g., using only entropy or only redundancy) showed that CURE's combination of signals yields the best average rank, suggesting the components are complementary.
Efficiency: The additional computational overhead of CURE's eviction logic is minimal (approx. 0.0024 seconds per step), with the majority of runtime still dominated by the TFM's prediction and context fitting.

5. Significance and Claims

The paper claims that context management is a central mechanism for making Tabular Foundation Models effective on evolving data streams. Unlike classical methods that require complex online training or state updates, CURE allows a frozen TFM to adapt simply by curating its context.

The authors position CURE as a practical solution that bridges the gap between the static nature of foundation models and the dynamic requirements of stream learning. By leveraging the "future-information view," they demonstrate that a simple policy combining recency, uncertainty, and redundancy can yield substantial performance gains, making TFMs competitive with, and often superior to, state-of-the-art classical stream learners. The work suggests that for TFMs, the "model" is effectively the context, and managing this context is the primary optimization problem in streaming scenarios.