💻 computer science

Towards Generalizable Representations of Mathematical Strategies

This paper introduces a novel method for learning generalizable, problem-invariant representations of entire algebraic solution pathways by combining transition embeddings with contrastive learning, enabling scalable analysis of student strategies and their correlation with learning outcomes without relying on manual labeling or platform-specific data.

Original authors: Siddhartha Pradhan, Ethan Prihar, Erin Ottmar

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Siddhartha Pradhan, Ethan Prihar, Erin Ottmar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a student solve a math problem. Traditionally, teachers and computers only care about the final answer: "Did they get it right? How fast did they do it?"

But this paper argues that the journey is just as important as the destination. It's like judging a hiker not just by whether they reached the summit, but by how they got there. Did they take the scenic route? Did they try three different paths before finding the trail? Did they get lost and backtrack?

This research team from Worcester Polytechnic Institute built a new "smart camera" that doesn't just look at the final answer, but analyzes the entire path a student takes to solve an algebra problem. Here is how they did it, explained simply:

1. The Problem: The "GPS" Was Too Specific

Previous computer programs tried to understand student strategies by looking at the specific numbers in the problem.

The Analogy: Imagine a GPS that only recognizes "Main Street." If you drive down "Main Street" in New York, it knows you. But if you drive down "Main Street" in Chicago, the GPS gets confused because the buildings look different.
The Issue: Old systems were too focused on the specific numbers (the "Main Street") rather than the type of move the student made (the "turn"). This meant they couldn't compare how a student solved Problem A with how they solved Problem B.

2. The Solution: Focus on the "Moves," Not the "Map"

The researchers decided to ignore the specific numbers and focus on the transitions (the moves).

The Analogy: Instead of recording "I walked past the bakery," they recorded "I took a step forward." Whether you are walking past a bakery in New York or a bakery in Chicago, the action "step forward" is the same.
The Tech: They used powerful AI models (pre-trained on math) to look at two steps in a solution and calculate the difference between them. This "difference vector" represents the strategy (e.g., "combining numbers" or "moving a variable") rather than the specific math problem.

3. The "Magic Lens" (SimCSE)

Once they had these "move" descriptions, they needed to bundle the whole journey into a single summary.

The Analogy: Imagine you have a long video of a student solving a problem. You want to turn that hour-long video into a single 30-second highlight reel that captures the style of the player.
The Method: They used a technique called SimCSE. Think of this as a "twin detector." It takes a student's solution path, creates a slightly "noisy" copy of it (like a blurry photo), and teaches the AI that these two should look very similar. Then, it shows it a different student's path and teaches the AI that those should look very different.
The Result: The AI learns to create a unique "fingerprint" for every solution path. If two students used the same clever strategy (even on different problems), their fingerprints match. If one was efficient and the other was messy, their fingerprints are far apart.

4. What Did They Discover?

They tested these "fingerprints" in three ways:

Can it read the moves? Yes. The system could guess what kind of math moves were used just by looking at the fingerprint.
Can it spot efficiency? Yes. It could tell if a student took the "express lane" or the "scenic route."
Can it predict future success? This is the big one. They linked these fingerprints to test scores.

The Surprising Findings on Creativity:
They defined three types of "math personality" based on these fingerprints:

Conformity: Following the most common, efficient path. (Like taking the highway).
Uniqueness: Taking a path no one else took. (Like taking a secret dirt trail).
Diversity: Trying many different paths for the same problem. (Like exploring every alley in the city).

The Result:

Conformity was the strongest predictor of high test scores. Students who could identify and stick to the "best" path did very well.
BUT, Uniqueness and Diversity also mattered! Students who explored different, unique paths also did better in the long run.
The Takeaway: The best math students aren't just robots who follow one rule. They are like explorers who can try many different routes (divergent thinking) but eventually know how to pick the best one (convergent thinking).

Why Does This Matter?

Currently, if a student gets the right answer, we say "Good job!" If they get it wrong, we say "Try again." We rarely know why they got it wrong or if they were being creative.

This new method gives teachers a scalable, automatic way to see:

"This student is stuck in a loop of inefficient moves."
"This student is being very creative but missing the efficient solution."
"This student is ready for a harder challenge because they can handle complex paths."

It turns the messy, invisible process of "thinking" into a clear, measurable map, helping educators nurture not just math skills, but mathematical creativity.

1. Problem Statement

Educational Data Mining (EDM) has traditionally struggled to represent and analyze student problem-solving strategies at the pathway level (the entire sequence of steps) in a way that is platform-agnostic and generalizable across problems.

Limitations of Current Methods:
- Manual Labeling: Relies on labor-intensive expert coding, which does not scale.
- Platform-Specific Actions: Existing models often learn representations tied to specific system actions (e.g., "click button X"), limiting transferability to other learning environments.
- State-Centric Embeddings: Previous work focused on embedding individual algebraic states or formulas. These representations often capture problem-specific surface features (e.g., specific numbers) rather than the underlying strategic transitions (e.g., "combining like terms"), making cross-problem comparison difficult.
The Gap: There is a need for a scalable method to learn invariant representations of entire algebraic solution pathways that capture the strategy rather than just the content, enabling the analysis of creativity, flexibility, and efficiency without manual intervention.

2. Methodology

The authors propose a novel framework for learning problem-invariant sequence embeddings using a two-stage approach:

A. Data Source

Platform: "From Here To There!" (FH2T), a gamified algebra learning platform where students manipulate expressions to transform a "Start State" into a "Goal State."
Dataset: Data from a large-scale Randomized Controlled Trial (RCT) involving 4,092 seventh-grade students. The study utilized 81,306 filtered solution sequences (max 15 steps) derived from 1,649 students in the treatment group.

B. Representation Learning Architecture

State Encoding (Fixed Encoders):
- The raw algebraic states ( $s_t$ ) in a solution pathway are first encoded using high-capacity, pretrained mathematical encoders (specifically MathBERT, MathBERT-mamut, and math_pretrained_bert).
- These models are kept frozen to leverage their pre-trained semantic understanding of mathematical expressions.
Transition Embedding Construction:
- Instead of aggregating state embeddings directly, the authors compute transition embeddings ( $\delta_t$ ) by taking the vector difference between consecutive state embeddings: $\delta_t = h_t - h_{t-1}$ .
- Rationale: This emphasizes the transformation (the strategy) rather than the specific algebraic content, promoting generalizability across different problems.
Sequence-Level Learning (SimCSE):
- The sequence of transition embeddings is fed into a Transformer-based encoder trained using SimCSE (Simple Contrastive Learning of Sentence Embeddings).
- Contrastive Objective: The model learns to pull embeddings of semantically similar solution pathways closer together and push dissimilar ones apart.
- Positive Pairs: Created by applying independent dropout masks to the same sequence.
- Negative Pairs: Other sequences within the mini-batch.
- Architecture: A stack of Transformer layers (tested with $L=1, 2, 4$ ) followed by global average pooling and a projection layer.

C. Evaluation Strategy

The authors evaluated the learned embeddings using three probe tasks (without using ground-truth action labels during training):

Action Type Content: Multi-label classification to predict which of the 75 possible algebraic action types (e.g., "AddNumbers", "Distribute") are present in the sequence.
Solution Efficiency: Classification to determine if a pathway is optimal, suboptimal, or incomplete.
Action Sequence Reconstruction: An autoregressive task (using an LSTM probe) to reconstruct the original sequence of actions from the fixed-size embedding, testing if order is preserved.

D. Creativity Metrics

Using the final sequence embeddings, the authors derived three metrics to measure mathematical creativity:

Uniqueness: Mahalanobis distance from the cohort's distribution (measuring originality).
Diversity: Average pairwise cosine dissimilarity of a student's multiple attempts (measuring fluency/exploration).
Conformity: Cosine similarity to the most common optimal solution (measuring convergence).

3. Key Contributions

Transition-Based Embeddings: Demonstrated that computing vector differences between states (transitions) yields representations that are invariant to specific problem content, unlike state-based embeddings which cluster by problem ID.
Platform-Agnostic Strategy Modeling: Developed a method to encode entire solution pathways without relying on system-specific action labels, enabling cross-platform analysis.
Scalable Creativity Proxies: Established that embedding-based distances can serve as scalable proxies for complex cognitive constructs like divergent thinking (uniqueness/diversity) and convergent thinking (conformity).
Empirical Validation: Showed that these metrics correlate significantly with both short-term learning gains and distal standardized test scores.

4. Results

RQ1: State vs. Transition Embeddings

t-SNE Visualization: State-based embeddings formed tight clusters based on Problem ID (surface features), whereas transition-based embeddings showed no clustering by problem ID.
Finding: Transition embeddings clustered by Action Type (e.g., addition vs. multiplication), confirming they capture the strategy rather than the problem content.

RQ2: Encoding Strategy Information

Performance: The SimCSE-based models (specifically using MathBERT-mamut with $L=1$ $L = 1$ layer and $k=768$ $k = 768$ dimension) outperformed baseline mean-pooled embeddings.
- Action Content: Achieved 86.1% Micro-F1 (vs. ~81% for baselines).
- Sequence Reconstruction: Achieved 77.0% token-level accuracy and 2.06 perplexity (vs. ~71% and 2.34 for baselines).
Efficiency Prediction: Interestingly, the SimCSE models performed slightly worse than baselines on efficiency prediction. The authors suggest the contrastive objective prioritizes capturing the composition and order of actions over the length/efficiency of the path.
Architecture Insight: A single Transformer layer ( $L=1$ ) performed best; deeper layers ( $L>1$ ) degraded performance, likely because deeper layers optimized for retrieval tasks washed out the specific sequential distinctions needed for strategy analysis.

RQ3: Relation to Learning Outcomes

Regression analysis revealed that the embedding-derived metrics were statistically significant predictors ( $p < .001$ ) of learning outcomes:

Short-term Outcomes: Uniqueness, diversity, and conformity all positively predicted post-test scores in conceptual knowledge, procedural skills, and flexibility.
Distal Outcomes: All three metrics positively predicted 7th-grade state test scores.
Strongest Predictor: Strategy Conformity showed the largest effect size, suggesting that converging on common optimal strategies is highly beneficial. However, Uniqueness and Diversity also showed positive associations, indicating that exploring varied strategies contributes to learning.

5. Significance and Implications

Educational Data Mining: This work provides a scalable, automated alternative to manual coding for analyzing student strategies, allowing researchers to study problem-solving behaviors across different platforms and problems.
Mathematical Creativity: It operationalizes "mathematical creativity" (divergent and convergent thinking) using vector space distances, offering a way to measure these traits at scale.
Instructional Design: The findings suggest that effective instruction should balance encouraging divergent thinking (exploring unique strategies) with convergent thinking (identifying optimal paths).
Future Applications: The embeddings can power intelligent tutoring systems that suggest alternative strategies to students, helping them explore the solution space more broadly or guiding them toward more efficient paths.

Conclusion

The paper successfully demonstrates that transition-based sequence embeddings, learned via contrastive objectives on top of pretrained mathematical encoders, can effectively capture the structural and strategic essence of student algebraic problem-solving. These representations are not only generalizable across problems but also serve as powerful, scalable proxies for measuring mathematical creativity and predicting learning outcomes.