Technical Summary: AgentSchool

1. Problem Statement

The rapid deployment of Large Language Models (LLMs) into education presents a unique validation challenge. Unlike standard software, educational AI intervenes in the development of learners whose cognitive, social, and emotional trajectories are irreversible. Real-world trials are often slow, ethically constrained, and institutionally locked, making it difficult to evaluate whether an intervention improves learning or merely optimizes for short-term efficiency at the cost of long-term capacity (e.g., critical thinking or curiosity).

Existing LLM-based educational simulators often suffer from two structural limitations:

1. Fidelity Gap: They rely on prompt-engineered role-play that treats learning as static persona-conditioned behavior rather than a dynamic state transition process. This fails to capture the internal mechanisms of learning (e.g., knowledge restructuring, misconception repair).
2. Reliability Gap: They lack long-term data collection, human-in-the-loop inspection, and multi-level evaluation, often optimizing for the reproduction of existing classroom behaviors rather than exploring novel institutional configurations.

Consequently, there is a need for a "computational wind tunnel" that can rehearse the consequences of educational AI in a plausible, inspectable, and ethically safer environment before real-world deployment.

2. Methodology: The AgentSchool System

AgentSchool is an LLM-driven multi-agent simulation platform that models education as a partially observable, multi-agent state transition process. It moves beyond static role-play by coupling cognitively growable agents with adaptive pedagogical logic and a configurable environment.

2.1 System Architecture

The system is defined as a state transition process $X_{t+1} \sim P_\theta(X_t, a_{1,t}, \dots, a_{n,t})$, where the state $X_t$ includes student states, teacher states, scenery configuration, interaction graphs, and event history. The architecture consists of four core modules:

• Student Agents (Cognitively Growable):

  • State Representation: Unlike static personas, student agents maintain explicit internal states: episodic memory ($M$), a weighted subject knowledge graph ($K$) with mastery scores, a pool of thinking workflows ($W$), explicit misconceptions ($Z$), and stable attributes ($P$).
  • Growth Mechanism: Learning is modeled as state transitions. Mastery updates are bounded transitions based on instructional quality, learner uptake, decay, and scaffolding. Misconceptions are structured objects ($v, \hat{p}, p^*, c, \rho$) representing competing beliefs with persistence scores, allowing for targeted intervention.
  • Observation: Internal states are inferred from observable performance (utterances, task outputs) via an evidence function, creating a gap between internal state and inferred state that mirrors real-world pedagogical uncertainty.

• Teacher Agents (Adaptive Scaffolding):

  • Pedagogical Logic: Agents operate within the Zone of Proximal Development (ZPD), planning, scaffolding, and reflecting. They maintain a declarative knowledge base and an experiential knowledge base updated after each lesson.
  • Adaptation: Teachers select instructional actions (explanation, questioning, grouping, hints) based on estimated learner readiness. A ZPD compatibility score evaluates whether tasks are too easy, too difficult, or appropriately scaffolded.
  • Reflection: Post-lesson, agents update their experiential knowledge based on observed student state changes and misconception persistence.

• Scenery Generator (Learning Fields):

  • Defines the relational field of learning, including spatial structure, resources, norms, and temporal rhythms.
  • Configurability: Supports predefined templates (e.g., lecture vs. open discussion) and informal scenes (e.g., recess, peer gossip).
  • Evolution: Future iterations plan to use an Alpha-Evolve-like process to mutate and stress-test candidate educational scenarios (e.g., new AI tools, grouping policies) against criteria like equity, safety, and novelty.

• Simulator Architecture:

  • Coordinates interaction across scales (turn-by-turn to semester-long) and granularities.
  • Efficiency: Uses adaptive resolution, allocating high-fidelity LLM calls to consequential moments (e.g., misconception diagnosis) while compressing routine transitions.
  • Interaction Patterns: Supports one-to-one, one-to-many, and unstructured group chat, enabling the emergence of learning communities and social networks.

2.2 Calibration and Validation

AgentSchool employs a dual calibration strategy:

1. Empirical Calibration: Minimizing the distance between simulated and real-world distributions where data exists.
2. Theory-Driven Calibration: Constraining trajectories to satisfy theoretical constraints (e.g., gradual mastery growth, ZPD alignment) to prevent surface-level plausibility from masking structural invalidity.

3. Key Contributions

1. Cognitively Growable Student Agents: A shift from static persona prompts to stateful learners with mutable knowledge graphs, reasoning workflows, and explicit misconception objects.
2. Adaptive Teacher Agents: A design where teachers plan, scaffold, and reflect based on student states, operationalizing ZPD alignment as an evaluable metric rather than a static script.
3. Scenery-Driven Multi-Scale Simulator: A platform that decouples interaction scale, temporal granularity, and simulation duration, allowing for the simulation of both formal instruction and informal social dynamics within a unified state-transition framework.
4. Methodological Shift: A move from "behavioral believability" (does it sound like a student?) to "educational mechanism fidelity" (does the internal process match learning theory?) and "institutional counterfactual usefulness" (can it test policies that don't exist yet?).

4. Preliminary Results

The authors conducted a $2 \times 3$ controlled lesson study across five backbone LLMs (Claude, DeepSeek, Qwen, GPT, Gemini) and a social simulation study.

4.1 Lesson Simulation

• Student Effect: Structured student agents ($S_{ours}$) produced significantly more differentiated learning traces than baseline role-play simulators ($S_{base}$). $S_{ours}$ achieved higher average mastery and generated a richer distribution of high-mastery nodes and structured misconceptions. $S_{base}$ often failed to represent meaningful knowledge acquisition or produced uniform, low-variance error patterns.
• Teacher Effect: Adaptive teacher agents ($T_{ours}$) showed patterns consistent with ZPD-informed adaptation, particularly when paired with $S_{ours}$. However, performance was backbone-dependent; in some cases, scripted teachers ($T_{scripted}$) performed competitively when content paths were stable, suggesting that adaptivity is most valuable when student states are sufficiently complex to require dynamic response.
• Backbone Sensitivity: The underlying LLM significantly shaped emergent behaviors (e.g., graph size, node counts), highlighting the need to evaluate agent architectures across multiple models rather than assuming model independence.

4.2 Social Simulation

• Network Dynamics: In informal social scenes, AgentSchool generated plausible traces of peripheral participation, clique formation, and opinion-leader emergence.
• Aggressive Behavior: The simulation captured second-order group effects: aggressive agents became isolated, while non-aggressive agents formed denser cohesive bonds, mirroring real-world classroom social dynamics.
• Emergent Structures: The system successfully modeled the evolution of social networks, including the formation of central hubs and the impact of social exclusion on group cohesion.

5. Significance and Claims

The paper positions AgentSchool as a computational wind tunnel for education, serving two primary roles:

1. For Education Research and Policy: It offers a safer space to prototype future educational models, evaluate institutional reforms, and identify failure modes (e.g., algorithmic bias, social stratification) before they impact real learners. It allows for "what-if" reasoning about policies and AI-mediated teaching arrangements that do not yet exist.
2. For AI Research: It provides a socially meaningful testbed for studying long-horizon memory, heterogeneous multi-agent coordination, and agentic reasoning under organizational pressure.

Modesty of Claims:
The authors explicitly state that these results are preliminary validation studies rather than claims of ecological equivalence. They do not claim that AgentSchool perfectly replicates real-world classrooms. Instead, they claim that structured, stateful agents produce more inspectable and educationally interpretable traces than prompt-only role-play under controlled conditions. The system is designed to generate hypotheses and compare plausible mechanisms, requiring future empirical calibration and expert review before high-stakes decisions. The current work focuses on formal lessons and informal social scenes, with institutional and policy-level simulations positioned as planned extensions.