Technical Summary: PIVOT (Preference-based Intervention Vectors for Pedagogical Tutor Steering)

Problem Statement

Large Language Models (LLMs) are increasingly deployed for conversational tutoring, yet effective pedagogy requires more than merely generating correct answers. Successful tutoring involves dynamic strategic choices, such as when to scaffold reasoning, provide hints, offer feedback, or invite reflection. While existing prompting and training-based alignment methods (e.g., Supervised Fine-Tuning, Reinforcement Learning) improve general pedagogical behavior, they suffer from critical limitations:

1. Prompting: Is adaptable but fragile under distribution shifts or adversarial requests.
2. Training-based Alignment: Produces stable behavior but lacks reliable, explicit control over specific pedagogical moves at inference time without retraining.
3. Activation Steering: Prior work in steering LLMs typically relies on binary contrasts (e.g., truthful vs. untruthful). Pedagogical tutor moves are not naturally binary; behaviors like feedback, hinting, and direct explanation often overlap, lacking clear contrastive negative classes. Furthermore, the field lacks a unified taxonomy or labeled dataset for tutor moves across different systems.

Consequently, there is a need for a framework that enables inference-time control over specific, nuanced pedagogical strategies without modifying model weights or relying solely on static prompts.

Methodology: PIVOT Framework

The authors propose PIVOT, an online preference-based activation steering framework designed for frozen LLM tutors. The system learns multi-layer residual-stream steering vectors through a generate–label–optimise loop.

1. Tutoring Move Taxonomy

The authors define a seven-category taxonomy grounded in prior work on intelligent tutoring and classroom discourse:

• Scaffolding Moves:
  • Feedback: Evaluates correctness without adding reasoning.
  • Pump: Invites broad elaboration without narrowing the response space.
  • Focusing: Asks the student to justify or explain specific parts of their reasoning.
• Answer-Revealing Moves:
  • Hint/Funnelling: Narrows the solution space with clues or corrections without giving the answer.
  • Prompt: Nearly completes a step, leaving a short completion for the student.
  • Assertion: Directly states the answer or explanation.
• Reflection:
  • Metacognition: Asks the student to reflect on their understanding, confidence, or learning state.

2. Learning Steering Vectors

PIVOT learns move-specific vectors ($V^{(m)}$) across selected residual-stream layers ($S$) using an iterative loop:

• Generate: Given a tutoring context, the frozen LLM generates candidate responses. Steering strength is controlled by scalars ($\alpha_{prompt}$ and $\alpha_{gen}$).
• Judge: A human-validated LLM judge assigns a move label to each response. Separate judges evaluate relevance and fluency.
  • Target Pool: Responses matching the target move ($\hat{m} = m$) and passing relevance/fluency checks.
  • Non-Target Pool: Responses with different move labels ($\hat{m} \neq m$).
• Optimise: Preference pairs are constructed from target and non-target generations. The system optimizes a weighted preference loss to update the steering vectors. Crucially, the system employs dynamic weighting to upweight non-target moves that the model frequently confuses with the target move, refining the decision boundary.

3. Inference-Time Steering

At inference, PIVOT injects the learned vectors into the residual stream of the frozen model.

• Single Move: A specific vector is added to selected layers with a scaling factor $\gamma$.
• Composition: Multiple vectors can be combined (added or subtracted) to create blended pedagogical behaviors (e.g., combining Assertion and Feedback).

Key Contributions

1. PIVOT Framework: An inference-time framework for controllable pedagogical steering that learns multi-layer residual-stream vectors via online preference optimization over the model's own tutor-move confusions.
2. Taxonomy and Evaluation Pipeline: Introduction of a seven-category tutor-move taxonomy and a human-validated LLM-based evaluation pipeline for assessing tutor moves, relevance, and fluency.
3. Analysis of Controllability: Characterization of how steering strength, intervention layers, datasets, and model variants affect control, relevance, and fluency.
4. User Study: A study with 30 teachers demonstrating that controllable tutor moves are pedagogically interpretable and preferred over neutral baselines.

Results

Offline Evaluation

• Control Accuracy: PIVOT achieves high target-move hit rates (e.g., ~88.9% on MathDial, ~88.2% on ConvoLearn) at moderate steering strengths ($\alpha \approx 1.0 - 2.0$).
• Quality Trade-off: Relevance and fluency remain near baseline at moderate strengths but degrade sharply when steering is too strong ($\alpha > 2.0$).
• Generalization:
  • Cross-Dataset: Vectors trained on MathDial and ConvoLearn generalize to the unseen StudyChat dataset (80.6% hit rate).
  • Cross-Model: Vectors transfer effectively to a non-instruction-tuned base model (Qwen3.5-9B-Base), indicating the learned directions are not dependent on chat-oriented post-training.
• Layer Ablation: Multi-layer steering (layers 6–28) significantly outperforms single-layer steering. Early layers (6–12) are most critical for reliable control, while late layers contribute less.
• Comparison: PIVOT outperforms an offline BiPO-style baseline trained on the same preference pairs, suggesting online preference construction is vital for separating overlapping pedagogical behaviors.

User Study

• Participants: 30 teachers with primary/elementary math experience.
• Preference: 73.3% of participants preferred the steered conversations over a neutral baseline generated with the same prompt.
• Perception: Teachers rated the controls as clear, usable, and pedagogically meaningful. They valued the ability to guide students without over-directing or revealing answers prematurely.
• Usage Patterns: Teachers dynamically adjusted controls based on student states (e.g., using Focusing and Hint for stuck students, and Assertion for passive/unsure students), treating the controls as responsive pedagogical actions rather than arbitrary generation knobs.

Significance and Claims

The paper claims that PIVOT provides a promising mechanism for giving educators finer-grained control over LLM tutor responses without the cost of retraining model weights or the fragility of prompt engineering alone. By turning tutor behaviors into steerable residual-stream directions, the framework allows for the precise modulation of pedagogical strategies (e.g., shifting from direct explanation to scaffolding) while preserving the model's relevance and fluency.

The authors emphasize that while the system improves controllability and teacher preference, it does not guarantee improved student learning outcomes. They position PIVOT as a tool to support, not replace, human educators, requiring teacher oversight and safeguards against over-reliance on generated feedback. The work highlights that activation steering can effectively manage the nuanced, non-binary nature of pedagogical interactions, a challenge that previous binary steering methods could not address.