Personality-Driven Student Agent-Based Modeling in Mathematics Education: How Well Do Student Agents Align with Human Learners?

1. Problem Statement

Educational research relies heavily on understanding how different teaching methods impact student learning. However, conducting real-world experiments faces significant ethical constraints (e.g., inability to repeatedly test the same student with different interventions) and logistical limitations. While Large Language Model (LLM)-based generative agents offer a promising alternative for simulation, a critical gap exists: there is no established validation of whether student agents can faithfully simulate human learning behaviors and personality-driven outcomes.

Current research often uses agents as teachers with humans as students, or uses agents without memory systems. There is a lack of full-pipeline simulations where agents act as students with memory, personality traits, and the ability to self-regulate learning over time. This paper addresses the fundamental question: Are Big Five personality-based student agents credible enough to simulate human learning trajectories in mathematics?

2. Methodology

The authors constructed a comprehensive Agent-Based Modeling (ABM) framework to simulate student-teacher interactions, self-study, and examinations.

A. System Architecture & Data Pipeline

• Dataset: The system utilizes the NuminaMath-CoT dataset (860k problems with Chain-of-Thought solutions).
• Question Classification: A bert-finetuned-math-prob-classification model classifies questions into four domains: Algebra, Number Theory, Counting & Probability, and Geometry.
• Retrieval System: Questions are vectorized using inf-retriever-v1-1.5b. The system maintains a memory bank where acquired knowledge is encoded and retrieved during exams.
• Base Model: All agents run on gpt-oss-120b (OpenAI, 2025).

B. Agent Design

• Personality Foundation: Agents are grounded in the Big Five Personality Traits (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism), derived from Goldberg's (1990) research. Specific prompts define how each trait influences decision-making (e.g., High Neuroticism agents feel anxious and seek reassurance; High Conscientiousness agents plan carefully).
• Interaction Loop:
  1. Learning Rounds: Agents autonomously choose one of three actions:
     • Ask Teacher: Selects a topic; the teacher (adapted to the student's personality) explains and provides a related question.
     • Self-Study: Agent selects a topic and retrieves a question independently.
     • Rest: Skips the round.
  2. Memory Update: New knowledge is vectorized and merged into the agent's memory.
  3. Exam Round: Agents retrieve relevant memories to answer 100 questions per round.
• Experimental Setup: Simulations ran for 10, 20, and 50 learning rounds across three independent runs. A control group with no learning was also established.

C. Evaluation Framework

To validate the agents, the authors distilled 13 empirical studies on Big Five traits and human learning into 14 scoring criteria across five dimensions:

1. General Meta-Analyses
2. Mathematics-Specific Studies
3. Teacher Interaction
4. Learning Behavior
5. Self-Regulation

Each criterion was scored as Accurate (1), Partial (0.5), or Not Met (0) based on whether the agent's behavior matched human literature.

3. Key Contributions

1. Full-Pipeline Student Agent: Developed a novel ABM where agents possess a memory system, make autonomous decisions on learning strategies (ask vs. self-study vs. rest), and adapt to personality traits, bridging the gap between static LLMs and dynamic learners.
2. Quantitative Validation Framework: Created a rigorous evaluation protocol comparing agent behavior against 13 prior human studies, establishing a baseline for "fidelity" in educational simulation.
3. Discovery of "Over-Learning" Phenomenon: Identified that excessive learning rounds (50) can degrade performance due to memory saturation, causing retrieval systems to fail in finding useful information.
4. Personality-Performance Correlation: Provided empirical data on how specific personality traits correlate with math performance and interaction styles in a simulated environment.

4. Key Results

A. Alignment with Human Learners

• Overall Fidelity: The student agents aligned with human learning characteristics on 10 out of 14 criteria (71.4%).
• Strongest Alignments:
  • Self-Regulation: High-Neuroticism agents exhibited the most procrastination; High-Conscientiousness agents exhibited the least.
  • Learning Outcomes: Most agents improved after learning, though performance dropped after over-learning.
• Discrepancies:
  • Teacher Interaction: High-Neuroticism agents interacted with the teacher in every round (highest rate), whereas human literature suggests anxiety can lead to help-avoidance. The agents successfully simulated "anxiety-driven help-seeking" but failed to simulate the "help-avoidance" mechanism.
  • Math Domain Performance: Agents performed significantly worse in Geometry compared to Algebra (approx. 30% score difference), despite the data coming from the same source.

B. Performance by Personality Trait

• Best Performers: High-Extraversion agents achieved the highest average scores.
  • Reasoning: They engaged more frequently with the teacher and received longer, more detailed explanations.
• Worst Performers: High-Neuroticism agents performed the worst.
  • Reasoning: While they interacted frequently, the quality of interaction was lower (driven by anxiety/reassurance seeking), and they consumed the most time (procrastination).
• Conscientiousness Paradox: Contrary to some literature suggesting Conscientiousness is the top predictor of success, these agents performed moderately.
  • Reasoning: They interacted the least with the teacher, preferring self-study, which may have limited their exposure to diverse explanations.

C. Behavioral Metrics

• Time Cost: High-Neuroticism agents consumed the most timestamps (time), while High-Conscientiousness agents were the most efficient.
• Blank Answers: Learning significantly reduced blank answers, except for High-Neuroticism agents, who remained prone to leaving questions unanswered.

5. Significance and Future Directions

• Research Impact: This study validates that LLM-based agents can serve as credible proxies for human students in educational research, specifically for testing hypotheses about personality-driven learning that are impossible to run with human participants (e.g., crossover designs of teacher-student personality matches).
• Practical Application: The framework offers a cost-effective, ethical alternative for testing new pedagogical strategies, curriculum designs, and AI tutor interactions before deploying them in real classrooms.
• Limitations & Future Work:
  • Static vs. Dynamic Personality: Current prompts are static; future work should explore dynamic personality shifts during learning.
  • Evaluation Granularity: The study relies on manual scoring against literature. Future work needs automated, fine-grained metrics for interaction quality.
  • Mechanism Analysis: Further investigation is needed to understand how personality prompts influence token probabilities and decision-making mechanisms within the model.

In conclusion, the paper demonstrates that while not perfect, Big Five-based student agents provide a high-fidelity simulation of human learning behaviors, particularly in self-regulation and general performance trends, making them a viable tool for the next generation of educational research.