Technical Summary: Forecasting Future Behavior as a Learning Task

Problem Statement

Trust in Artificial Intelligence systems relies on the ability to forecast how a model will behave on new inputs or under specific conditions. Traditionally, this forecasting is anchored by explanations of the model's computation, which are then used to predict behavior in new scenarios. However, for Large Reasoning Models (LRMs)—which emit long sequences of reasoning tokens before producing a final answer—this conventional approach faces two critical limitations:

1. Generalization Failure: Explanation methods designed for single token generations do not naturally extend to the long, stochastic reasoning trajectories emitted by LRMs.
2. Unfaithfulness: The reasoning tokens, despite their natural language appearance, are often not a faithful account of the underlying computation. They may omit factors (e.g., biased answer positions, user hints) or diverge from the actual computation steps, rendering "naive reading" of the trajectory unreliable for forecasting.

Consequently, existing methods for behavior forecasting on LRMs are either prohibitively expensive (requiring massive resampling to estimate statistics) or blind to the trajectory's internal dynamics (relying on single-location probes like attention or log-probabilities).

Methodology

The authors propose an alternative framework: treating behavior forecasting as a learnable task. Instead of relying on human-readable explanations, they train an external model, termed a Behavior Forecaster, to predict behavioral statistics directly from a single observed reasoning trajectory.

Core Setup

• Input: A single observed trajectory $\tau = (P, R, A)$, consisting of a prompt $P$, reasoning tokens $R$, and final answer $A$.
• Output: A forecast of a specific behavioral statistic $b(M, P, A)$ regarding the model's future behavior, without re-running the target LRM.
• Training Data Generation: The training labels are generated by querying the target LRM itself at scale. For each prompt, the LRM is run multiple times (e.g., 10 times) to empirically estimate ground-truth behavioral statistics. This process is computationally expensive but is performed only once during data generation, not during inference.
• Inference: The trained Behavior Forecaster operates in a single forward pass over the observed trajectory, making it orders of magnitude cheaper than resampling or using frontier models as naive readers.

Target Tasks

The paper instantiates this approach on two specific forecasting tasks:

1. Rerun Consistency: The probability that a fresh run on the same prompt $P$ will produce the same answer $A$.
2. Counterfactual Sensitivity: For each removable segment $s$ of the prompt, the degree to which removing $s$ reduces the probability of the model producing answer $A$.

Architecture and Training

• Initialization: The Behavior Forecaster's backbone is initialized from the weights of the target LRM itself. This ensures the model starts with representations already attuned to the specific reasoning patterns of the target.
• Fine-Tuning: The backbone is trained end-to-end (not just the prediction head) jointly with a task-specific prediction head.
• Input Arrangements:
  • Counterfactual Sensitivity: Uses a prompt-echo arrangement ($P-R-A-P$). Since the backbone is an autoregressive decoder, predictions for prompt segments must be made at the echoed prompt positions to allow the model to attend to the reasoning ($R$) and answer ($A$) when evaluating the prompt ($P$).
  • Rerun Consistency: Uses a cross-attention pooling mechanism over the entire trajectory ($P-R-A$) to produce a single scalar score, as there is no per-segment structure to attribute.

Key Results

The authors evaluate the Behavior Forecaster across three diverse reasoning datasets (FEVEROUS, RuleTaker, TreeCut) and two target LRMs (OLMo-3-7B-Think and Qwen3.5-2B).

1. Superior Accuracy: Trained Behavior Forecasters significantly outperform naive readers (GPT-5.4 and Claude Opus 4.6) that attempt to forecast behavior by reading the trajectory text.
   • On OLMo-3-7B-Think, the forecaster achieves a Spearman correlation of 0.731 for counterfactual sensitivity (vs. 0.672 for GPT-5.4) and 0.571 for rerun consistency (vs. 0.305 for GPT-5.4).
   • The forecaster also outperforms standard single-location probes (e.g., attention attribution, final-answer log-likelihood), which perform near random baselines.
2. Cost Efficiency: The Behavior Forecaster achieves these results at less than 1/10,000th of the estimated inference cost of the frontier naive readers.
3. Generalization:
   • Zero-shot: Forecasters generalize to held-out variants within a dataset family (e.g., FEVEROUS numerical subset) without fine-tuning.
   • Transfer Learning: A forecaster trained on two datasets can be adapted to a held-out third dataset with as few as 600 fine-tuning steps, significantly improving performance over the source model.
   • Cross-Task Transfer: A forecaster trained for counterfactual sensitivity can be adapted to predict hint-sensitivity (whether a model relied on a user-provided hint), outperforming direct training from scratch.
4. Ablation Insights:
   • Initialization: Initializing from the target LRM is necessary for strong performance; random initialization yields significantly lower scores.
   • End-to-End Training: Freezing the backbone and training only the head results in poor performance, indicating that adapting the backbone representations is crucial.
   • Input Content: Removing reasoning tokens ($R$) from the input drastically reduces performance, confirming that the trajectory carries information beyond the prompt and answer.

Significance and Claims

The paper claims that reasoning trajectories contain information about an LRM's future behavior that is not accessible through naive reading. By treating behavior forecasting as a supervised learning task, the authors demonstrate that:

• Trust can be anchored by learned patterns: One can forecast model reliability (consistency) and robustness (sensitivity) without needing the reasoning text to be faithful or human-interpretable.
• Efficiency: High-fidelity behavioral forecasting is possible at a fraction of the cost of resampling or using larger frontier models.
• New Research Direction: This work motivates studying behavior forecasting as a distinct learning task, suggesting that any LRM behavioral property that can be automatically labeled is a candidate for low-cost forecasting at deployment scale.

The authors remain modest, noting that generalization to Out-of-Distribution (OOD) tasks substantially different from training data requires further study and broader training coverage, which is currently limited by the high cost of generating labeled data. They also acknowledge that if future LRMs become more faithful in their reasoning, the gap between naive reading and learned forecasting may narrow.