Personalizing Mathematical Game-based Learning for Children: A Preliminary Study

1. Problem Statement

Game-based learning (GBL) is effective for enhancing critical thinking and engagement in mathematics, but current systems face two significant bottlenecks:

• Content Scarcity: Creating high-quality, intrinsically integrated math game levels requires significant labor from dedicated designers.
• Scalability of User-Generated Content (UGC): While tools like "Creative Mode" allow advanced players to design levels, manually filtering thousands of user-generated levels to ensure they are mathematically valid and suitable for other players is unfeasible.
• Lack of Personalization: Existing systems struggle to deliver adaptive game levels that match individual learners' abilities, often relying on low-resolution adaptivity rather than fine-grained, dynamic adjustments.

The core challenge is to automate the identification of valid, high-quality player-generated levels and use them to provide personalized learning experiences without relying on exhaustive manual review.

2. Methodology

The authors propose a framework guided by adaptive learning theory that utilizes Machine Learning (ML) to classify player-generated levels.

Dataset

• Source: Data was collected from a math GBL app (ages 5–10) between 2024 and 2025.
• Volume: 206 distinct game levels.
  • 86 expert-designed levels (serving as a baseline).
  • 120 player-generated levels (created via "Creative Mode").
• Labels: Among the 120 player levels, 44 were validated by experts as "selectable" for the public pool, while the rest were excluded.
• Feature Extraction: Each level was encoded into a JSON file. The authors extracted 61 variables, categorized into four groups:
  1. Player Character: Character type and value.
  2. Goal: The completion objective.
  3. Physics Objects: Interactive elements (e.g., ice blocks, lava, bubbles, platforms) with associated values for arithmetic operations.
  4. Obstacles: Barriers (e.g., clouds, doors, breakable walls).

Data Processing

• Preprocessing: Missing values were recoded.
• Feature Selection: Lasso regression was applied to reduce dimensionality and select the most informative features for prediction.
• Modeling: Four classification algorithms were trained and compared:
  1. k-Nearest Neighbors (KNN)
  2. Decision Trees (DT)
  3. Support Vector Machines (SVM)
  4. Random Forest (RF)
• Validation: Nested cross-validation was used to minimize evaluation bias and assess model generalization.

3. Key Contributions

• Novel Screening Framework: The study introduces a pipeline that automates the filtering of player-generated levels, moving away from labor-intensive manual curation.
• AI Integration in GBL: It demonstrates how AI can be integrated into the game design loop to facilitate "intrinsic" educational games where the gameplay mechanics themselves are the math practice.
• Feature Importance Analysis: The study identifies that structural gameplay mechanics (player characters, goals, and physics objects) are more predictive of level quality than decorative elements or obstacles.
• Personalization Pathway: The framework lays the groundwork for delivering adaptive levels to players based on their specific learning needs and abilities.

4. Results

The study compared the performance of the four ML models using metrics including Accuracy, Precision, Recall, F1-Score, and ROC-AUC.

• Optimal Model: Random Forest (RF) emerged as the superior classifier.
  • Outer Loop Performance: RF achieved an Accuracy of 82.42%, Precision of 74.53%, Recall of 72.69%, and an F1-Score of 72.70%.
  • ROC-AUC: RF scored 86.57%, slightly outperforming SVM (86.22%) and significantly outperforming KNN and DT.
• Critical Advantage: The Confusion Matrix analysis revealed that RF was the least likely to discard expert-validated levels (high recall). This is crucial for a screening pipeline, as the goal is to preserve high-quality user content rather than falsely rejecting it.
• Feature Insights: Lasso regression indicated that expert selection decisions rely heavily on:
  • Number of player characters and goals.
  • Presence of specific physics objects (one-way platforms, platform bubbles, poppable bubbles).
  • Interpretation: Experts prioritize levels that allow for constructive manipulation (calculation/logic) over passive barriers.

5. Significance and Limitations

Significance:

• Scalability: The framework offers a scalable solution for GBL systems to handle large volumes of UGC, ensuring a continuous supply of fresh, mathematically valid content.
• Quality Assurance: By automating the initial screening, the system ensures consistency and standardization, reserving human expert time for final reviews rather than initial filtering.
• Educational Impact: It supports the creation of "intrinsic" math games where learning is embedded in the mechanics, potentially improving student motivation and critical thinking.

Limitations:

• Dataset Size: The study is preliminary with only 206 levels, which limits the robustness and generalizability of the model comparisons.
• Feature Scope: The current analysis focuses on specific modules; other game modules with different features were not fully explored.
• Future Work: The authors plan to collect more data to improve model robustness and explore how this framework can drive fully adaptive, personalized learning paths.

Conclusion

The paper successfully demonstrates that a Random Forest classifier can effectively predict the quality of player-generated math game levels. By automating the selection process, this approach bridges the gap between the need for vast amounts of practice content and the limitations of manual design, paving the way for more personalized and engaging mathematical game-based learning environments.