🤖 machine learning

Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing

This paper proposes Phase-Aware Knowledge Tracing (PAKT), a novel framework that disentangles student learning into ability-building and proficiency-oriented phases using a multi-branch Transformer to address the limitations of phase-agnostic models and achieve superior performance across multiple benchmarks.

Original authors: Duantengchuan Li, Yingqian Bi, Jinsong Chen, Rui Zhang, Mingwen Tong

Published 2026-07-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Duantengchuan Li, Yingqian Bi, Jinsong Chen, Rui Zhang, Mingwen Tong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where a computer tutor could watch you study and instantly know not just what you know, but how you learned it. This is the heart of "Knowledge Tracing," a field of educational technology that tries to predict how well a student will do on their next test based on their past homework. Think of it like a GPS for learning: instead of just showing you where you are, it tries to guess where you're going next. For years, these computer tutors treated every single question you answered as just another step on the same long road. They assumed that the way you struggled with a concept on day one was exactly the same as the way you breezed through it on day ten. But anyone who has ever learned a new skill knows that's not true. At first, you are clumsy and figuring things out; later, you are smooth and automatic. The big question researchers are asking is: Can we build a smarter tutor that understands the difference between the "figuring it out" phase and the "mastering it" phase?

This is exactly what the paper "Disentangling Knowledge States with Ability and Proficiency Modeling for Knowledge Tracing" tackles. The authors, a team of researchers, argue that current computer tutors are missing a crucial detail: they are mixing up two very different types of learning behaviors. They propose a new system called PAKT (Phase-Aware Knowledge Tracing) that splits a student's history into two distinct chapters: the Ability Phase and the Proficiency Phase.

To understand the magic of PAKT, imagine you are learning to juggle. In the beginning, you are in the Ability Phase. You are dropping balls, your arms are flailing, and every time you catch one, it feels like a huge victory. You are building the raw skill from scratch. Your success here depends entirely on your natural "ability" to grasp the concept. But once you've practiced enough and finally stop dropping the balls, you enter the Proficiency Phase. Now, you aren't just catching balls; you are juggling them with rhythm and ease. Your success here isn't about whether you can do it anymore; it's about how smoothly and consistently you can do it after all that practice.

The paper suggests that old computer tutors were like a coach who watched your whole juggling career and gave you one single grade, mixing up your early flailing with your later smoothness. This creates a "confusing signal." The new PAKT model, however, acts like a coach with two different clipboards. It separates your history: one clipboard tracks your early struggle (Ability), and the other tracks your later consistency (Proficiency).

Here is how they built it and what they found:

The Split: The researchers created a clever rule to split the data. They counted how many times a student got a specific concept right. Once a student reached a certain number of correct answers (a threshold they call $k$ ), the model stopped treating that concept as "new" and started treating it as "mastered." Everything before that count went into the Ability bucket; everything after went into the Proficiency bucket.
The Engine: They fed these two separate buckets, plus the original full history, into a special AI brain called a "Multi-branch Transformer." Think of this as having three different experts looking at the same student: one expert focuses only on the early struggles, one focuses only on the later smoothness, and a third looks at the whole picture.
The Result: When they tested this new system on six different real-world datasets (ranging from algebra to geography and Spanish), it consistently outperformed the best existing models. The paper reports that PAKT improved the prediction accuracy (measured by a metric called AUC) by an average of 0.82%, with the biggest jump being 1.33% on the Spanish dataset. While these numbers might sound small, in the world of high-stakes testing and AI, a consistent gain like this is a significant victory.

The authors also used a method called "causal analysis" to explain why the old way was failing. They showed that by mixing ability and proficiency together, the old models were creating a "confounding bias." It's like trying to guess how fast a car will go by mixing up the engine's horsepower (ability) with how well the tires grip the road (proficiency). If you don't separate them, you can't tell which one is actually helping the car move. By untangling them, PAKT gives a clearer, more accurate picture of what a student actually knows.

However, the paper is careful not to claim this is a perfect, solved problem. The authors note that their method relies on a specific number ( $k$ ) to decide when to switch from "ability" to "proficiency," and this number has to be tuned carefully for different subjects. If a student doesn't practice enough to reach that number, the model can't see the "proficiency" part of their story. But overall, the study suggests that by recognizing that learning has different stages, we can build tutors that understand students much better than before.

Technical Summary: Phase-Aware Knowledge Tracing (PAKT)

Problem Statement

Knowledge Tracing (KT) aims to predict a student's future performance by modeling their evolving knowledge states based on historical interaction sequences. While existing deep learning-based KT methods (e.g., DKT, SAINT, AKT) have achieved significant progress, they predominantly treat the raw interaction sequence as a unified behavioral process. This unified paradigm implicitly assumes that all interactions contribute to knowledge state evolution in the same manner, overlooking the phase-specific nature of learning.

The authors observe that student learning behaviors differ significantly across stages: early interactions primarily reflect ability building (initial acquisition and understanding of concepts), while later interactions reflect proficiency (skill automation, fluency, and consistency after mastery). Treating these distinct phases as a single homogeneous sequence can entangle complex learning signals, leading to confounding bias and limiting the model's ability to precisely characterize the dynamic evolution of knowledge states.

Methodology: Phase-Aware Knowledge Tracing (PAKT)

To address these limitations, the authors propose PAKT, a framework that decomposes student interaction sequences into distinct Ability and Proficiency phases for phase-specific modeling. The methodology consists of three core components:

1. Learning Behavior-Aware Sequence Decomposition

The core innovation is a decomposition mechanism that partitions a student's response history for each knowledge concept (KC) based on the cumulative count of correct responses.

Mechanism: A binary phase indicator function $\delta(s, c, t)$ is defined using a threshold $k$ . If the cumulative correct count for a concept $c$ by student $s$ at step $t$ is less than $k$ , the interaction is assigned to the Ability Phase. Once the count reaches or exceeds $k$ , subsequent interactions are assigned to the Proficiency Phase.
Output: This process generates two decomposition mask matrices ( $M^{A}$ and $M^{F}$ ) that split the original sequence into an Ability subsequence and a Proficiency subsequence, while retaining the original complete sequence.

2. Multi-Branch Transformer-Based Backbone

PAKT employs a multi-branch Transformer architecture to process the decomposed sequences:

Three Input Branches:
1. Ability Branch: Processes interactions masked by $M^{A}$ , capturing intrinsic cognitive capacity and initial acquisition.
2. Proficiency Branch: Processes interactions masked by $M^{F}$ , capturing practice-induced automation and consistency.
3. Complete Branch: Processes the original, non-decomposed sequence to maintain a holistic view of the learning trajectory.
Two-Stage Training: The backbone utilizes a two-stage decoder mechanism. The first stage uses a shared Transformer decoder across all branches to learn universal sequential features efficiently. The second stage employs independent decoders for each branch to enable phase-specific specialization.

3. Type-Aware Readout Module

To integrate the representations from the three branches, a type-aware readout module is designed:

Adaptive Fusion: The Ability ( $H_A$ ) and Proficiency ( $H_F$ ) representations are fused via a learnable coefficient $\alpha$ : $H_{AF} = \alpha \cdot H_A + (1-\alpha) \cdot H_F$ .
Differential Gain: A gap vector $H_{\Delta} = H_F - H_A$ is computed to explicitly capture the performance improvement driven by practice.
Final Representation: The final student representation $H_S$ is constructed by concatenating the fused ability-proficiency vector, the complete branch representation ( $H_C$ ), and the differential gain vector: $H_S = [H_{AF} \parallel H_C \parallel H_{\Delta}]$ .
Prediction: This representation is combined with the target question features and passed through an MLP to predict the probability of a correct response.

Causal Analysis

The paper provides a causal analysis using a Structural Causal Model (SCM). It posits that in conventional KT models, the student's intrinsic ability ( $A$ ) and acquired proficiency ( $F$ ) are entangled through the observed interaction history ( $D$ ), creating a confounding bias that affects the latent student representation ( $S$ ). By explicitly disentangling and separately modeling $A$ and $F$ , PAKT breaks this spurious correlation, allowing for a more accurate estimation of each factor's true contribution to the prediction outcome.

Key Contributions

Decomposition Mechanism: Development of a new learning behavior-aware mechanism that partitions interaction sequences into ability and proficiency phases based on cumulative correct responses.
Model Architecture: Proposal of a multi-branch Transformer backbone with a type-aware readout module to jointly capture phase-specific and holistic knowledge states.
Empirical Validation: Extensive experiments demonstrating consistent improvements over representative baselines across six public datasets.
Theoretical Insight: A causal analysis revealing the confounding bias in phase-agnostic models and justifying the necessity of disentangling learning behaviors.

Experimental Results

The authors evaluated PAKT on six public benchmarks: Algebra05, ASSISTments09, ASSISTments17, Ed-Net, Slepemapy, and Spanish.

Performance: PAKT achieved the best performance across all six datasets on both AUC (Area Under the ROC Curve) and ACC (Accuracy) metrics.
Improvement: The method showed consistent gains over the strongest baselines, with an average AUC gain of 0.82% and a maximum AUC gain of 1.33% (on the Spanish dataset).
Comparison: While DisKT (a previous decomposition method) showed competitive results, its performance was unstable. PAKT's fine-grained, phase-based decomposition proved more robust and universally effective.
Sensitivity: Sensitivity analysis on the threshold parameter $k$ indicated that optimal performance generally occurs at low values ( $k \le 5$ ), with specific optimal values varying by dataset characteristics (e.g., concept density and interaction volume).
Ablation: Removing the "Complete" branch caused the largest performance degradation, confirming the necessity of the holistic view, while both phase-specific branches provided complementary refinements.

Significance and Limitations

The paper claims that PAKT advances the field by moving beyond the "homogeneous modeling paradigm" to explicitly model the temporal transition from ability acquisition to skill consolidation. This approach better aligns with the behavioral semantics of learning and mitigates confounding bias.

However, the authors acknowledge specific limitations:

Data Sparsity: If a student has too few interactions with a concept to reach the threshold $k$ , the proficiency branch receives little to no signal, potentially weakening its effectiveness.
Hyperparameter Tuning: The optimal threshold $k$ must be manually tuned and varies across datasets, suggesting a need for future automatic decomposition strategies.
Short Sequences: The method may struggle with very short interaction sequences where the phase transition is not observable.

The work is supported by the National Natural Science Foundation of China and various university reform projects.