Technical Summary: Learning to Prompt – Improving Student Engagement with Adaptive LLM-based High-School Tutoring

Problem Statement

While Large Language Models (LLMs) offer significant potential for personalizing education, current tutoring systems face three primary limitations:

1. Static Prompting: Most existing systems rely on static prompts designed for a single subject (e.g., mathematics), failing to adapt to the diverse requirements of different academic disciplines (e.g., French, Geography, Physics).
2. Simulation-to-Reality Gap: Research often validates pedagogical efficacy in synthetic environments where simulated students may exhibit "sycophancy" or idealized behaviors, leading to distributional shifts when deployed with real students.
3. Feedback Sparsity: In real-world scenarios, users often skip formal exercises after interacting with a tutor, resulting in a lack of immediate, explicit feedback (e.g., test scores) necessary for training adaptive models. Existing approaches often rely on single black-box scores that are prone to high variance and reward gaming.

Methodology

The authors propose a Subject-aware Adaptive Tutoring System that frames prompt selection as a contextual bandit routing problem. The framework consists of three core components:

1. Input Representation and Prompt Space

• Hybrid Input ($X$): To mitigate representation degeneration (anisotropy) in pre-trained embeddings, the system uses a dual-path architecture. It fuses a 768-dimensional topic embedding (from an E5 encoder) with a learnable 64-dimensional subject ID embedding. This explicitly injects subject boundaries into the latent space.
• Prompt Pool ($P$): The system selects from a pool of 20 distinct pedagogical prompts. These prompts, authored by human experts and varied via LLMs, cover strategies such as Socratic scaffolding, Feynman role-reversal, and empathetic support.

2. Evaluative Score Calibration ($Y$)

To address the lack of immediate exercise scores, the authors employ an LLM Evaluator (GPT-5-mini) that decomposes instructional efficacy into 14 binary pedagogical criteria grounded in educational theory (e.g., tutor scaffolding, student curiosity, correct answers within 2 turns).

• Weighting: The raw score is a weighted sum of these 14 features, where weights are derived from Pearson correlations with human-labeled learning outcomes.
• Calibration: To bridge the sim-to-real gap, a sigmoid calibration function is applied to the raw score ($y_{raw}$) to align the distribution of simulated feedback with real-world empirical distributions, preventing the router from over-optimizing for synthetic behaviors.

3. Routing Architecture and Optimization

• Actor-Critic Network: The router uses a residual subject injection mechanism. The subject embedding is injected directly into the final output layers (via skip-connections) to prevent dilution during deep non-linear transformations, ensuring subject-specific requirements govern the strategy selection.
• Training: The model is optimized using Proximal Policy Optimization (PPO) adapted for a single-step contextual bandit setting. It utilizes Experience Replay and importance sampling to handle the high latency of LLM environments.
• Deployment Strategy: In live deployment, the system operates in two modes: Exploitation (80% of traffic, greedily selecting the highest probability strategy) and Exploration (20% of traffic, weighted sampling to gather diverse training data).

Experimental Setup

• Simulation: A controlled environment with three simulated student profiles (Motivated, Mediocre, Unmotivated) was used to train the router over 16,000 steps.
• Real-World A/B Test: The system was deployed with 359 Dutch high-school students (656 conversations) in March and April 2026.
• Baselines: Comparisons were made against a Production Baseline (expert-refined static Socratic prompt) and a Literature Baseline (static adaptation of a prior pedagogical prompt).

Key Results

1. Simulation Performance

• The adaptive router significantly outperformed both static baselines in simulation (Mean Score: 0.694 vs. 0.647 and 0.64, $p < 0.001$).
• The router successfully eliminated the "tail" of poor interactions (scores < 0.4) observed in static approaches.
• Subject Alignment: In 8 out of 13 evaluated subjects, the router achieved a perfect match with the empirically optimal strategy. In others, the regret margin was minimal (< 0.02).

2. Real-World Deployment

• Instructional Efficiency: The adaptive router reduced the number of turns required to reach deep interactions (turns $\ge$ 5) by approximately 3 turns compared to the baseline ($p = 0.007$).
• Strategy Shift: The router successfully adapted from simulation-preferred strategies (e.g., Feynman/deep reasoning) to real-world optimal strategies (e.g., Coach/scaffolding), demonstrating successful sim-to-real transfer.
• Conversion Rates:
  • The Exploitation router achieved a conversion rate comparable to the static baseline (19.1% vs. 19.6%).
  • The Exploration router (stochastic sampling) achieved a significantly higher conversion rate of 28.1%, suggesting that strategy diversity may be more beneficial than a single "optimal" policy in early deployment.
• Score Validity: The LLM evaluator's pedagogical scores were statistically predictive of exercise conversion (0.599 for converted vs. 0.560 for non-converted, $p = 0.037$), validating the proxy metric.

Significance and Claims

The paper claims to demonstrate that:

1. Decomposed Feedback is Robust: Breaking down instructional efficacy into 14 observable criteria provides a reliable, human-aligned proxy for learner success, even in the absence of downstream exercise scores.
2. Adaptive Routing Works: Formulating tutoring as a prompt routing problem allows the system to specialize pedagogical strategies across different subjects, outperforming static baselines in both simulation and real-world settings.
3. Sim-to-Real Transfer is Achievable: Through score calibration and residual subject injection, the system can bridge the gap between idealized simulations and sparse, noisy real-world interactions, shifting strategies to match actual student needs (e.g., moving from deep reasoning to scaffolding).

The authors note limitations, including data sparsity in long-tail subjects, the difficulty of measuring immediate knowledge gain without formal exercises, and the constraint of a discrete prompt pool which prevents the generation of novel hybrid strategies. They suggest future work could focus on longitudinal studies, balancing domain imbalances, and incorporating conversation history into the routing decision.