💻 computer science

INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators

The paper introduces INSIDE, a student modeling framework that fine-tunes Large Language Models to generate both student actions and their underlying internal reasoning grounded in Bloom's Taxonomy, thereby significantly improving simulation fidelity and reasoning alignment compared to existing prompting methods.

Original authors: Rose Niousha, Minwoo Kang, Narges Norouzi

Published 2026-08-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Rose Niousha, Minwoo Kang, Narges Norouzi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to be a student. You show it thousands of examples of kids typing code, getting stuck, asking for help, and trying again. Most robots today are like excellent actors who can memorize the script perfectly; they can type out the exact same code a human student would type. But they are terrible at understanding why the human typed it. They don't know if the student is confused, frustrated, guessing, or having a "lightbulb moment." They only see the final performance, not the messy, internal monologue happening in the student's head. This paper, titled "INSIDE the Student's Mind," comes from the field of Artificial Intelligence and Education. It tackles a specific problem: how to make AI simulators that don't just copy what students do, but also think like them. The authors build on the idea that to truly understand learning, we need to model the invisible reasoning behind the visible actions, much like a detective trying to figure out not just what a suspect did, but what they were thinking when they did it.

The researchers introduce a new framework called INSIDE (Internal Student Dialogue). Think of it as giving the AI robot a "secret voice" or a "thought bubble." Instead of just asking the AI to "write the next line of code," they ask it to first whisper its internal thoughts, then write the code. To teach the AI how to do this, the team used a clever trick: they took real data from a computer science class where students submitted code, got feedback from an AI tutor, and tried again. Since no one recorded the students' actual thoughts, the researchers used a super-smart "teacher" AI to look at the code changes and the feedback, and then reconstruct what a student was probably thinking at that moment. They created a dataset where every code submission is paired with a made-up but realistic "think-aloud" story.

When they trained their AI models using this new "thought + action" data, something interesting happened. The paper finds that these models, which they call INSIDE, became much better at mimicking real students than models that only looked at the code. In fact, the INSIDE models matched the real students' coding patterns (like how many lines they wrote or how often they made mistakes) even better than previous methods. But the real magic was in the "thoughts." The researchers measured how well the AI's internal reasoning matched the actual changes the student made to their code. They found that the INSIDE models achieved a reasoning alignment of up to 57.9%, which was the highest score among all the models they tested.

The paper explicitly argues against the idea that just making an AI "think harder" or "think logically" is enough. They show that if you just ask a standard AI to reason before it acts, it often produces perfect, logical thoughts that lead to perfect code—which is exactly what real students don't do. Real students often have confused, incomplete, or slightly wrong thoughts that lead to buggy code. The paper suggests that to build a truly realistic student simulator, the AI must be allowed to be uncertain, make mistakes in its reasoning, and sometimes misunderstand the feedback, just like a human. The results suggest that by forcing the AI to generate this "messy" internal dialogue first, it creates a much more accurate picture of how a student learns, offering a new way to test and improve AI tutors without needing real humans in the loop.

Technical Summary: INSIDE – Jointly Modeling Latent Reasoning and Action in LLM Student Simulators

Problem Statement

Large Language Model (LLM)-based simulators in education often replicate observable student actions (e.g., code submissions) but fail to capture the underlying latent reasoning processes. In educational contexts, two students may produce identical code for fundamentally different reasons, or arrive at the same incorrect solution via distinct cognitive paths. Existing approaches typically treat student behavior as a black box, focusing solely on outcome prediction. This limitation hinders the ability to diagnose misconceptions, generate targeted feedback, or evaluate tutoring systems effectively, as observable actions provide only a partial view of cognition. Furthermore, while LLM reasoning research often focuses on improving correctness, human learning involves frequent errors, misconceptions, and incomplete strategies that current models struggle to simulate authentically.

Methodology

The paper proposes INSIDE (INTERNAL STUDENT DIALOGUE), a framework that jointly models student actions and their internal reasoning processes. The methodology consists of three primary components:

1. Data and Context

The study utilizes data from an introductory programming course at UC Berkeley, covering two semesters (Spring 2024 and Spring 2025). The dataset includes ~900 students, 10 homework assignments, and multi-attempt submission streams. Each attempt captures student code, autograder results, and natural-language feedback from an AI tutor. The data is partitioned into training (Spring 2025) and testing (Spring 2024) sets, with further splits for "Old Problems" (familiar problems, new students) and "New Problems" (unseen problems).

2. Internal Dialogue Generation (Teacher Model)

Since ground-truth internal reasoning is unobservable, the authors employ a teacher model (GPT-5) to perform retrospective inference. Given the interaction context (problem, prior submissions, feedback) and the observed next submission, the teacher model generates:

Structured State Summaries: Third-person analyses of the student's cognitive (understanding/misconceptions), affective (emotions/motivation), and action (concrete steps) states, grounded in Bloom's Taxonomy.
First-Person Internal Dialogue: A "think trace" ( $z_t$ ) simulating the student's internal monologue prior to the submission. This dialogue is designed to reflect novice reasoning, including uncertainty, partial understanding, and potential errors.

3. Student Modeling Framework

The core contribution is a student model trained to generate both the internal dialogue and the subsequent code submission ( $c_t$ ) conditioned on the interaction history.

Fine-tuning (INSIDE): Models (Qwen2.5, Qwen3, LLaMA-3) are fine-tuned using LoRA on paired sequences of (Context $\to$ Internal Dialogue $\to$ Code). The internal dialogue serves as an intermediate cognitive layer.
Baselines: The framework is compared against:
- SFT: Fine-tuning without internal dialogue (direct context $\to$ code).
- Prompting (CoT): Standard Chain-of-Thought prompting.
- Prompting (BloomCoT): Structured prompting inspired by Bloom's Taxonomy.

Key Contributions

Reconstruction of Pedagogically Grounded Reasoning Traces: The paper introduces a method to reconstruct latent reasoning traces from interaction data using retrospective inference, enabling models to generate interpretable internal dialogue despite the absence of ground-truth reasoning.
Joint Modeling Framework (INSIDE): A novel framework that conditions code generation on inferred internal dialogue, explicitly modeling the latent reasoning process preceding observable actions.
Two-Dimensional Evaluation: A comprehensive evaluation metric assessing both Action Fidelity (similarity of generated code distributions to real students) and Reasoning Quality (alignment between generated dialogue and actual code edits).

Results

The evaluation demonstrates that INSIDE outperforms both fine-tuned baselines without reasoning (SFT) and various prompting strategies.

Action Fidelity

Using the Wasserstein distance to measure distributional similarity between model-generated and real student code:

INSIDE consistently achieves the lowest Wasserstein distances across metrics (Pass Rate, Lines of Code, AST depth/width, PEP 8 violations) on the "Old Problems" test set ( $test\_OP$ ).
On "New Problems" ( $test\_NP$ ), INSIDE performs comparably to SFT, though the gap is smaller. The authors attribute this to the higher success rate in $test\_NP$ , where SFT is already close to the target distribution.
Pass Rate Trajectories: Real students exhibit low initial pass rates with sharp increases near the final steps. SFT models track this well, but prompting-based models often maintain artificially high pass rates from the start. INSIDE models closely match the incremental progress of real students, with the lowest Mean Absolute Error (MAE) in pass rate trajectories.

Reasoning Quality (Alignment)

Alignment measures the fraction of atomic claims in the generated internal dialogue that are reflected in the ground-truth code edits:

INSIDE achieves the highest alignment scores among the tested models. Specifically, Qwen3-8B-INSIDE reaches 57.9% on $test\_NP$ , while Qwen2.5-7B-INSIDE reaches 51.8% on $test\_OP$ , outperforming the best prompting baselines in both settings.
Notably, larger, more capable models (e.g., GPT-5, Qwen3-32B) often achieve lower alignment scores, suggesting that superior general reasoning ability does not automatically translate to simulating novice-like, error-prone reasoning.
Self-Consistency: While prompting models show high self-consistency (reasoning matches their own generated code), they often fail to match real student behavior. INSIDE maintains competitive self-consistency while achieving superior alignment with actual student edits.

Significance and Claims

The paper claims that modeling reasoning beyond observable actions is essential for building realistic student simulators. By incorporating internal dialogue, INSIDE bridges the gap between explanation and behavior, producing simulations where the reasoning is consistent with the observed code edits.

The authors highlight two main implications:

Evaluation and Optimization: Student simulators with internal dialogue allow for evaluating whether AI tutor feedback resolves misconceptions and support counterfactual analysis of interventions before deployment.
Learner-Facing Tools: Externalizing internal dialogue can support student reflection and metacognition, and enable clustering students based on reasoning patterns rather than just surface-level behavior.

The paper acknowledges limitations, primarily that the internal dialogue is reconstructed rather than observed, potentially reflecting more coherent reasoning than actual novice students. However, it asserts that this approach provides a useful proxy for unobservable cognitive signals, advancing the state of human-centered AI in education.