A Transparent Fuzzy-Inference Layer for Explainable Knowledge Tracing: Why Prerequisite Graphs Must Be Expert-Supplied, and Which T-Norm to Use

In the modern classroom, technology has become a constant companion, offering systems that track how students learn and adapt lessons in real time. These systems, known as knowledge tracing, act like a digital tutor that watches a student's sequence of answers to math problems or reading questions. They are remarkably good at predicting whether a student will get the next question right or wrong. However, these digital tutors often operate as black boxes. They can signal that a student is struggling with a specific concept, but they cannot explain why. They might flag that a student is failing at "dividing fractions," but they cannot tell the teacher which earlier concept the student missed that caused this failure. For a human teacher, knowing the root cause is essential; it is the difference between re-teaching the current topic and going back to fix the foundation. Without this clarity, the most powerful predictions remain useless for actual intervention.

Researchers have long hoped to solve this by teaching computers to automatically discover the hidden map of how concepts connect. The idea was that by analyzing millions of student answers, an algorithm could figure out that "multiplying fractions" is a necessary step before "dividing fractions," without any human needing to draw the map. If this were possible, the system could scale effortlessly to any subject, generating its own explanations from the data alone. A new study, however, challenges this optimistic premise. By rigorously testing whether these automatic maps can be built from student logs, the researchers found that the data simply does not contain the necessary clues. Instead of a computer discovering the rules, the study concludes that these maps must be provided by human experts, and the real breakthrough lies in building a transparent system that uses those expert maps to explain the computer's predictions.

The researchers began by testing three different methods that a computer might use to learn these connections on its own. They fed the algorithms vast amounts of interaction logs from a popular online math platform, looking for patterns where mastering one skill seemed to lead to success in another. The first method looked for information flow, the second tried to filter out the natural tendency of smart students to get everything right, and the third used a few known examples to guess the rest. Despite using different mathematical approaches, all three methods failed. When the researchers compared the computer's guesses against a known, expert-verified map of sixteen math concepts, the results were indistinguishable from random chance. The algorithms could not tell the difference between a true prerequisite and two unrelated skills that just happened to be answered by the same students.

The reason for this failure is a subtle but powerful trap in the data. In any classroom, students vary in their overall ability; some are naturally quick learners, while others struggle with everything. This creates a confusing signal where almost every skill appears to be related to every other skill. A strong student will get both "adding fractions" and "subtracting fractions" right, not because one causes the other, but simply because the student is good at math. A weak student will get both wrong for the same reason. This "ability trap" drowns out the true causal links, making it impossible for a computer to learn the map from the logs alone. The researchers confirmed this by testing the same problem on two other massive datasets, one with nearly a million interactions and another with over twenty-five million. In every case, the trap held firm, proving that the limitation is not a flaw in the specific algorithms but a fundamental property of the data itself.

Since the computer cannot build the map, the researchers shifted their focus to a different question: if we give the computer an expert-verified map, can it use that map to explain its predictions in a way that is clear and trustworthy? They built a new layer on top of the standard prediction engine, a system that acts as a translator. This translator takes the computer's internal estimate of a student's skill level and checks it against the expert map. If the computer predicts a student will fail at a difficult concept, the translator looks at the map to see which earlier concepts are required. It then generates a simple, readable statement: "The student is at risk because their understanding of the prerequisite skill is weak." Crucially, this translator does not learn or change; it is a fixed set of rules that anyone can inspect. This design ensures that the explanation is always tied to a specific, auditable rule, rather than being a mysterious output from a complex neural network.

The team then tested whether this approach actually worked better than the standard methods used to explain AI decisions today. They compared their new translator against popular techniques that try to reverse-engineer a computer's thinking by highlighting which past interactions were most important. The results were striking. The standard methods were unreliable; they often pointed to the wrong past interactions or scattered their attention across many irrelevant details, failing to identify the true cause even when the computer's prediction was correct. In contrast, the new translator, because it was built on the expert map, consistently identified the correct prerequisite. It did this regardless of the underlying computer architecture, working just as well on different types of models. The study showed that while the computer's internal logic was often opaque to standard explanation tools, the expert map provided a reliable guide that the translator could follow without fail.

The researchers also examined how to best combine the different pieces of information to create these explanations. They tested four different mathematical ways of deciding when a prerequisite is truly the cause of a problem. One method was very strict, another was very loose, and two were in between. By comparing their choices against the actual behavior of the computer model, they found that a specific method, known as the product rule, aligned best with reality. This method only flagged a problem when both the prerequisite was weak and the current skill was at risk, striking the right balance between being too sensitive and too strict. This finding suggests that the way we combine these logical steps matters, and that a careful choice can make the explanation more accurate.

Ultimately, the study offers a clear path forward for educational technology. It demonstrates that trying to automatically discover the structure of knowledge from student logs is a dead end; the data is too noisy and the connections too obscured by general student ability. Instead, the most effective approach is to accept that human experts must provide the map of how concepts connect. Once that map is in place, a transparent, rule-based system can use it to turn opaque predictions into clear, actionable advice for teachers. This does not mean the computer is less powerful; it means the computer is now working with a human guide to provide explanations that are not just statistically probable, but logically sound and trustworthy. The value of this work is not in a new algorithm that learns faster, but in a new design principle that prioritizes clarity and correctness over the illusion of automatic discovery.