🤖 AI

Scaffolding the Strategist: Architecture-Dependent Reasoning Interventions in Hotelling Spatial Markets

This paper demonstrates that structured reasoning interventions in Hotelling spatial markets have architecture-dependent effects, where commitment scaffolding benefits standard models but hinders reasoning-optimized ones, while principled separation shows the opposite pattern, ultimately revealing that adversarial stress degrades reasoning models more severely and that a persistent gap between identifying and executing strategies can only be closed for reasoning models through specific architectural alignment.

Original authors: Pratyush Singh

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pratyush Singh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine two different kinds of brainy robots trying to solve a tricky game of "Hotelling's Hot Dog Stand." In this game, two vendors have to pick the perfect spot on a beach to sell hot dogs, knowing that customers will walk to the closest one, but they hate walking too far. It's a puzzle that requires thinking ahead, guessing what the other guy will do, and doing some serious math.

The researchers at Caltech wanted to see if giving these robots a "cheat sheet" or a special set of instructions (called scaffolding) would help them play better. They tested two specific robots:

The Standard Robot (GPT-4.1-mini): A smart, fast worker who follows instructions well but doesn't have a built-in "thinking" mode.
The Reasoning Robot (GPT-5-mini): A super-smart worker who has a built-in "thinking" mode, constantly chatting with itself to solve problems before answering.

Here is the big surprise they found: What helps one robot actually hurts the other. It's like trying to teach a chess grandmaster to play by forcing them to write down every single move on a piece of paper before they make it, while telling a beginner to just "think out loud" to stay focused.

The Great Crossover

The researchers tried four different ways to help the robots, but two of them created a perfect "crossover" effect:

The "Commitment" Trick: This asked the robots to write down their rules and promise to stick to them before solving the problem.
- Result: The Standard Robot got better! It scored +0.21 points higher. It needed that extra structure to stay on track.
- Result: The Reasoning Robot got worse! It dropped -0.63 points. Why? Because this robot was already thinking deeply on its own. Forcing it to write down a "commitment" first was like putting a speed bump in front of a race car—it just got in the way and confused the robot.
The "Separation" Trick: This forced the robots to break the problem into three strict steps: 1) State the rules, 2) Make predictions, 3) Give the answer.
- Result: The Reasoning Robot loved it! It scored +0.31 points higher. The structure helped organize its powerful internal thinking.
- Result: The Standard Robot hated it! It dropped -0.40 points. It didn't have the brainpower to use this fancy three-step process, so the extra steps just slowed it down and caused mistakes.

This wasn't a fluke. The researchers ran 720 different tests (8 questions, 3 different ways of asking, 3 times each, for 2 robots). The difference was so clear that the math says there's only a 0.2% chance this happened by accident.

The "Talk Too Much" Trap

The researchers also tried a "Stress Test." They asked the robots to argue against their own answers and try to prove themselves wrong.

The Result: Both robots got worse, but the Reasoning Robot crashed harder, dropping -1.47 points compared to the Standard Robot's -0.57.
The Lesson: The more the Reasoning Robot thought about its own correct answers, the more it started to doubt them. It's like a person who knows the answer to a riddle but starts overthinking it until they get it wrong. The researchers found that the easier the question was, the more the robot messed up when forced to doubt itself.

Knowing vs. Doing

There was another weird thing the researchers noticed called the "Declarative–Procedural Gap."

The Standard Robot could often say the right strategy (like "I should move to the center of the beach"), but when it tried to actually do the math to prove it, it failed. It knew the "what" but couldn't do the "how."
The Reasoning Robot was better at doing the math, but it still struggled to connect its knowledge to action—unless they used the "Separation" trick. When they forced the Reasoning Robot to separate "stating the rule" from "doing the math," it finally closed the gap, achieving a 25.9% success rate where its ability to state the right strategy perfectly matched its ability to execute it.

What This Means

The main takeaway isn't that one robot is "better" than the other. It's that one size does not fit all.

If you have a robot that doesn't think much on its own, you need to give it structure (like "Commitment").
If you have a robot that thinks too much on its own, you need to give it a clear process to organize those thoughts (like "Separation").

The researchers are careful to say this is based on simulations with two specific robots from one company. They haven't proven this works for every robot in the world, but it strongly suggests that the way we talk to AI needs to change depending on how the AI is built. If you treat a super-thinker like a simple worker, or a simple worker like a super-thinker, you might just make things worse.

Technical Summary: Scaffolding the Strategist

Problem Statement

While Large Language Models (LLMs) have demonstrated proficiency in mathematical and logical benchmarks, their capacity for strategic economic reasoning—specifically the ability to integrate domain knowledge, anticipate competitor actions, and translate understanding into concrete strategic decisions—remains poorly understood. Existing game-theoretic benchmarks (e.g., GTBench, EconArena) largely rely on discrete action spaces and simultaneous-move games, leaving continuous strategy spaces, multi-stage backward induction, and counterintuitive equilibria unexplored.

A critical, unaddressed dimension is the interaction between reasoning interventions (scaffolding) and model architecture. With the emergence of reasoning-optimized models that possess built-in chain-of-thought capabilities, it is unclear whether external scaffolding (such as prompting for step-by-step reasoning) provides additive value, offers diminishing returns, or actively interferes with internal deliberation. Furthermore, the "declarative–procedural gap"—where models can articulate correct principles but fail to execute them—requires investigation across different architectural classes.

Methodology

The authors utilize Hotelling's linear city model (Hotelling, 1929) as a diagnostic vehicle. This framework was selected because it demands multi-step strategic reasoning, separates deductive and abductive reasoning subtypes within a unified structure, resists training-data contamination via non-standard parameterizations, and naturally exposes the gap between knowing a strategy and executing it.

Experimental Design

Models: Two architectures from OpenAI were evaluated:
- GPT-4.1-mini: A standard instruction-following model.
- GPT-5-mini: A reasoning-optimized model with built-in deliberation capabilities.
Tasks: Eight questions organized into two components:
- Component A (Deductive): Five questions requiring forward derivation from premises (e.g., parameter sensitivity, location changes, cost structure identification).
- Component B (Abductive): Three questions requiring inference from observations to best explanations (e.g., competitor inference, strategy shift detection).
Conditions: Each question was tested under five conditions:
1. Baseline: No scaffolding.
2. Commitment: The model must state and commit to analytical principles before solving (premise-locking).
3. Contradiction Detection: The model must identify and resolve internal contradictions in its reasoning chain.
4. Principled Separation: A three-phase pipeline requiring the model to state principles, generate predictions, and then decide (declare-then-execute).
5. Adversarial Stress-Test: The model is prompted to argue against its own conclusions and defend or revise them.
Evaluation: The study generated 720 responses (2 models × 5 conditions × 3 prompt framings × 3 repetitions × 8 questions). Responses were scored by GPT-5.2 on a 0–10 scale (conclusion accuracy + reasoning quality), calibrated against human judgment ( $\kappa = 0.97$ ).

Key Contributions

Crossover Interaction: The paper documents a statistically significant crossover interaction ( $p = 0.002, d = 1.69$ ) between scaffolding type and model architecture. Interventions that improve a standard model degrade a reasoning model, and vice versa.
Adversarial Stress-Test Paradox: The study shows that adversarial self-examination universally degrades performance, with the reasoning model suffering 2.6× greater damage than the standard model. This degradation correlates negatively with baseline difficulty ( $R^2 = 0.36$ ), indicating that "thinking too much" about correct answers introduces doubt and errors.
Declarative–Procedural Gap Quantification: The authors quantify the gap between identifying a correct strategy and executing it. They find that principled separation fully closes this gap for the reasoning model (converging declarative and procedural rates to 25.9%), whereas no intervention helps the standard model bridge this gap.
Diagnostic Framework Validation: The paper validates Hotelling spatial competition as a contamination-resistant "microworld" for isolating intervention $\times$ architecture interactions.

Results

The Crossover Interaction

Commitment Scaffolding: Improved the standard model (GPT-4.1-mini) by +0.21 points but degraded the reasoning model (GPT-5-mini) by −0.63 points.
- Interpretation: The standard model lacks internal deliberation, so external premise-locking provides necessary structure. The reasoning model already performs internal deliberation; forcing external commitment creates interference (paralleling the "verbal overshadowing" effect).
Principled Separation: Showed the opposite pattern. It degraded the standard model by −0.40 points but improved the reasoning model by +0.31 points.
- Interpretation: The reasoning model benefits from phase-gating that organizes its internal deliberation. The standard model lacks the reasoning depth to exploit this structure, incurring processing overhead without benefit.
Consistency: These crossover patterns held across 7 out of 8 questions.

Adversarial Stress-Testing

Both models suffered performance degradation under stress-testing.
The reasoning model suffered significantly more ( $−1.47$ vs. $−0.57$ ; $p = 0.038$ ).
The damage was inversely correlated with baseline difficulty: the easiest problems (where models were already correct) suffered the most degradation, suggesting that self-critique on correct intuitive judgments introduces error.

Declarative–Procedural Gap

Standard Model: Identified correct strategies ~30% of the time but executed them 0% of the time. No intervention improved procedural correctness above 3.7%.
Reasoning Model: Under the separation protocol, the gap closed completely (declarative and procedural rates both reached 25.9%). Under commitment, the gap widened (declarative 33.3% vs. procedural 11.1%).

Significance and Claims

The paper claims that reasoning interventions do not have uniform effects; their impact is critically and predictably dependent on model architecture.

Design Principle: The authors propose a guiding principle for scaffolding: "Provide what the architecture lacks; do not duplicate what it already has."
- For capability-limited models (standard), scaffolding cannot substitute for reasoning capacity.
- For organizationally limited models (reasoning-optimized), scaffolding choice matters as much as model choice.
Implications for Evaluation: The study cautions against the intuition that self-criticism (adversarial stress-testing) universally improves robustness, showing it can be detrimental, particularly for advanced models on well-understood problems.
Limitations: The authors acknowledge that the study uses two proprietary models from a single provider, limiting generalizability to other model families. They also note that the eight questions span a single economic domain, and generalization to other strategic contexts remains open.

The paper concludes that the "declarative–procedural gap" has qualitatively different origins in different architectures: a fundamental capability limitation in standard models versus an organizational problem solvable by appropriate scaffolding in reasoning models.