💬 NLP

Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling

This paper proposes a target-adaptive text-tabular modeling framework that leverages hidden representations from a frozen "LLM-as-Observer" to effectively predict the decisions of unfamiliar AI negotiation counterparts from limited interaction history, outperforming direct prompting and standard feature-based baselines.

Original authors: Eilam Shapira, Moshe Tennenholtz, Roi Reichart

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Eilam Shapira, Moshe Tennenholtz, Roi Reichart

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a bustling marketplace where everyone is trying to sell or buy things, but the people you are talking to are actually robots. You don't know their code, their secret instructions, or what kind of "brain" they have inside. You only see what they say and what they offer.

The big question this paper asks is: Can you figure out what a robot will do next just by watching it make a few deals?

Here is a simple breakdown of how the researchers tried to solve this, using some everyday analogies.

The Problem: The "Black Box" Shopper

In the real world, AI agents (like shopping bots) are becoming common. They negotiate prices with each other. But their internal logic is a "black box." You can't peek inside to see their code.

The Challenge: If you meet a new robot seller, can you predict if they will accept your price or make a counter-offer, just by seeing them negotiate with other people a few times before?

The Solution: A "Text-Tabular" Detective

The researchers built a system to act as a detective. Instead of asking a giant, powerful AI to "guess" the answer (which they call LLM-as-Predictor), they treated the problem like a fill-in-the-blank puzzle using a spreadsheet.

Think of it like this:

The Spreadsheet (The Table): Every time a robot makes a decision, it's like a row in a spreadsheet. This row contains:
- The Game Stats: The round number, the current price, the time left.
- The Chat Log: The text messages the robots exchanged.
- The "Observer" Clue: A special new feature (explained below).
The Adaptation (The "K" Games): Before the detective tries to predict the new robot's move, they are shown K (a small number, like 2, 8, or 16) past games where that specific robot played. These act as "study guides" to learn that specific robot's personality.

The Secret Weapon: The "Observer"

This is the paper's most creative idea. Usually, when you use an AI to understand a situation, you ask it, "What will happen?" and it gives you an answer.

The researchers did something different. They used a small, frozen AI (called the Observer) to read the situation and the chat, but they threw away its answer.

The Metaphor: Imagine a detective (the Observer) reading a crime scene. Instead of writing a report saying "The butler did it," the detective just gets a sudden feeling or a gut instinct about the scene. The researchers took that "gut feeling" (the AI's internal hidden state) and used it as a clue in their spreadsheet.
Why it works: The paper found that the AI's "gut feeling" contained hidden signals about how the robot would behave that the AI's actual spoken words missed. It's like knowing someone is nervous by their posture, even if they say "I'm fine."

The Experiment: Two Groups of Robots

To test this, they used two groups of robots:

The "Source" Group (The Training Class): 13 robots made by top tech companies (like Google, OpenAI, etc.). They all used the same instructions but had different "brains." The system learned from them.
The "Target" Group (The Test Class): 91 robots built by students in a hackathon. These students all used the same "brain" but built different "bodies" (different rules and prompts).

The system had to learn from the "Source" group and then predict the moves of the totally new "Target" group.

The Results: The Spreadsheet Wins

The researchers compared three methods:

Direct Guessing: Asking a giant AI, "Here is the chat history and the robot's past games; what will it do next?"
Basic Spreadsheet: Using the game stats and chat logs in a spreadsheet model.
The Full Detective (Their Method): The spreadsheet model + the "Observer's" gut feeling.

The Winner: The Full Detective method won.

It was better at guessing if a robot would say "Yes" or "No" (Accept/Reject).
It was better at guessing what price the robot would offer next.
Crucially: The "Observer's" hidden feeling added about 4% more accuracy than just using the chat logs and game stats alone.

The Takeaway

The paper concludes that when trying to predict what a strange AI will do next, you shouldn't just ask a giant AI to guess. Instead, you should:

Turn the situation into a structured data row (stats + chat).
Use a small AI just to "read" the situation and extract its internal "vibe" (hidden state).
Feed that vibe into a smart spreadsheet model that learns from past examples.

This approach is cheaper, more accurate, and better at adapting to new, unknown robots than just asking a big AI to "think" about it directly.

Technical Summary: Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling

Problem Formulation
The paper addresses the challenge of predicting the next decision of an unfamiliar, language-based AI agent (the "target agent") given only a limited number of prior interaction histories ( $K$ games). In scenarios such as automated negotiation or procurement, the target agent's underlying Large Language Model (LLM), system prompts, control logic, and rule-based fallbacks are hidden. Each decision point involves a mix of structured game states (offers, round numbers, payoffs) and free-text dialogue. The authors formulate this as a target-adaptive text-tabular prediction problem. The goal is to predict two complementary tasks:

Response Prediction: A binary classification task determining if the target will accept or reject a current offer.
Proposal Prediction: A regression task estimating the value of the next offer the target will make.

The setting requires adapting to a new agent without access to its internal code or prompts, relying solely on $K$ labeled examples of its past behavior and a large source population of previously observed agents.

Methodology
The proposed approach departs from direct "LLM-as-Predictor" prompting (where a large LLM is asked to predict the next move directly). Instead, the authors employ a text-tabular foundation model (TabPFN) that conditions on a multimodal representation of the decision point. The core components are:

Feature Modalities: Each decision point is represented as a single table row combining three feature blocks:
- Game-State Features: Structured variables including configuration parameters (e.g., discount factors, valuations), round index, current and historical offers, and decision history.
- Dialogue Representation: Generic semantic embeddings of the conversation history, encoded via a sentence transformer (e.g., all-MiniLM-L6-v2) and reduced via PCA.
- Observer Representation (LLM-as-Observer): A novel feature block where a small, frozen LLM (e.g., Gemma-2-2B, Qwen3-1.7B) reads the public decision-time state and dialogue. Crucially, the LLM's direct output (its predicted decision) is discarded. Instead, its hidden states (specifically from the upper-stack layers, relative depth 0.6–0.9) are extracted and used as a high-dimensional feature vector. This treats the LLM as an encoder of decision-relevant signals rather than a direct predictor.
Target-Adaptive Learning: The TabPFN model is trained on a source population of agents (13 frontier LLMs from the GLEE benchmark) and adapts to the target agent using its $K$ observed games as in-context labeled examples. The model combines population-level regularities with target-specific evidence without gradient-based retraining.
Baselines: The authors compare their full model against:
- Game+Text Features: The same tabular model but without the Observer hidden-state features.
- LLM-as-Predictor: A frontier LLM prompted directly with the game state, dialogue, and $K$ examples to output a prediction.

Key Contributions

Formulation: The paper reframes few-shot prediction of language-based agents as a target-adaptive text-tabular task, utilizing $K$ prior games as labeled adaptation examples.
LLM-as-Observer: The introduction of a decision-oriented feature block where a frozen LLM acts as an encoder. The authors demonstrate that the hidden states of a small frozen LLM provide complementary, decision-relevant signals that are not reliably exposed in the model's direct output.
Cross-Population Dataset and Evaluation: The authors introduce a held-out dataset of 91 "scaffolded" agents (built by students using a shared underlying LLM but varying in prompts and control logic) and evaluate transfer from a source population of frontier LLMs (varying in underlying model). This tests the robustness of predictors across different axes of agent heterogeneity.

Results
Experiments were conducted on bargaining and negotiation games from the GLEE framework.

Performance: The full text-tabular model with Observer features outperforms both the "Game+Text" baseline and the direct "LLM-as-Predictor" approach across all values of $K$ (0 to 16).
Observer Contribution: At $K=16$ , the Observer features improve Response Prediction AUC by approximately 4 percentage points over the best baseline and reduce Proposal Prediction error (measured by median $R^2$ ) by 14% in bargaining tasks.
Hidden States vs. Direct Output: Ablation studies confirm that the predictive power comes from the LLM's hidden states, not its direct logits. The hidden states consistently outperform the LLM's own accept/reject probabilities, suggesting they encode strategic information that the output head fails to surface.
Robustness: The gains from Observer features are stable across different frozen LLM providers (Gemma, Qwen, Llama) and across mid-to-late layers, indicating the signal is an intrinsic property of the representation.
LLM-as-Predictor Limitations: Direct prompting with large frontier LLMs performs poorly on proposal prediction (regression), often yielding negative $R^2$ scores, likely due to the difficulty of calibrating autoregressive token generation for continuous numerical values compared to the tabular model's regression capabilities.

Significance and Claims
The paper claims that predicting the decisions of unfamiliar AI agents is better framed as a target-adaptive text-tabular learning problem than as direct few-shot LLM prediction. The significance lies in demonstrating that:

Separating representation (extracted via a frozen LLM encoder) from adaptation (performed by a supervised tabular learner) yields superior performance.
Hidden LLM representations expose decision-relevant signals that direct prompting does not reliably surface, even when using smaller, frozen models compared to large frontier models.
This approach enables effective adaptation to new, engineered agents (scaffolded agents) using a reusable source population, suggesting a general recipe for modeling the decision-making of unfamiliar AI agents in language-mediated commerce.

The authors note limitations, including that the games are controlled abstractions rather than real markets, and the method assumes access to a relevant source population. They emphasize that the approach is a step toward prediction-aware agents that can adapt to partners without sharing private state.