🤖 AI

LearnOpt: Recovering the Latent Cognitive Structure of Standardized Examinations via Knowledge Graphs and Constrained Optimization

The paper introduces LearnOpt, a framework that utilizes knowledge graphs and constrained optimization to uncover the stable yet piecewise-shifted latent cognitive structures of standardized exams like NEET and JEE Advanced, demonstrating that these hidden skill distributions are more strongly shaped by exam tier and curricular changes than by time or subject, and can be leveraged to generate personalized, optimized study plans.

Original authors: Joy Bose, Om Thomas

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joy Bose, Om Thomas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Stop Studying Blindly

Imagine you are preparing for a massive, high-stakes exam (like the NEET medical entrance exam in India). The standard advice is: "Cover the whole syllabus evenly. Read every chapter. Solve old papers. Repeat."

The authors of this paper argue that this is like trying to fill a bucket with water by pouring it from a hose that sprays everywhere equally. It's wasteful. They believe exams aren't just random collections of facts; they are adversarial systems with hidden, stable patterns.

LearnOpt is a new tool that acts like a detective and a personal trainer combined. It looks at thousands of past exam questions to figure out the "secret recipe" of the test, then builds a custom study plan that fits your specific strengths and weaknesses within your limited time.

1. The Detective Work: Cracking the "Cognitive Code"

The researchers didn't just look at what topics were on the test (like "Genetics" or "Physics"). They asked a smarter question: "How does the exam actually ask these questions?"

They used an AI (a Large Language Model) to read 1,496 past questions and sort them into five "thinking styles" (like sorting tools in a toolbox):

The Fact Checker (Recall): "What is the capital of France?" (Just remembering a fact).
The Scenario Solver (Application): "If it rains here, what happens there?" (Using a known rule in a new situation).
The Connector (Integration): "How does the heart's rhythm affect the blood pressure?" (Mixing two different topics).
The Calculator (Quantitative): "Do the math." (Using formulas).
The Eliminator (Negation): "Which of these is NOT true?" (Finding the wrong answer among many right ones).

The Big Discovery:
The exam's "personality" is surprisingly stable, but it can change overnight.

The Stable Era (2016–2021): The mix of these five thinking styles was consistent year after year. If you studied the 2016 papers, you were well-prepared for 2021.
The Shift (2023): Suddenly, the exam changed. The "Fact Checker" questions dropped, and the "Eliminator" questions (the tricky "Which is NOT correct?" type) skyrocketed.
Why? This shift happened right after the official textbook (NCERT) was shortened in 2023. The authors suggest that because there were fewer topics to cover, the exam writers started testing the remaining topics more deeply and in trickier ways.

The Lesson: If you are studying using old papers (pre-2022), you might be practicing for a game that no longer exists. The "rules of the game" have changed.

2. The Personal Trainer: The Optimization Engine

Once the "rules" are understood, LearnOpt acts as a smart GPS for your study time.

Imagine you have a backpack with a strict weight limit (your available study time). You want to pack the most valuable items (topics) to get the highest score.

Old Way: Pack everything equally, or just pack the topics that appear most often.
LearnOpt Way: It looks at your backpack (your current knowledge).
- Are you already great at Genetics? Don't waste time there.
- Are you weak at Cell Biology? Pack that in.
- But wait! The exam loves "Eliminator" questions. So, even if a topic appears less often, if it usually comes in that tricky "Eliminator" format, LearnOpt tells you to prioritize it because that's what the exam is currently rewarding.

It uses a mathematical formula (like a puzzle solver) to figure out the perfect order to study topics so you get the maximum score boost for every hour you spend.

3. The "Regime" Concept: Why Time Matters Less Than You Think

The paper found something surprising: The subject matters more than the year.

Physics questions are almost always "Connectors" (mixing concepts).
Biology questions are a mix, but recently have become heavy on "Eliminators."
Chemistry sits somewhere in between.

The authors call the exam's structure a "Regime." As long as the textbook syllabus stays the same, the exam's style is stable. But when the syllabus changes (like in 2023), the regime shifts, and the exam's "flavor" changes permanently.

The Takeaway: You don't need to worry about the exam changing randomly every year. It only changes when the official curriculum changes. Once you know which "Regime" you are in, you can trust the data from the last few years to predict the future.

4. What This Means for Students (In Plain English)

The paper concludes with a practical shift in strategy:

Don't just memorize facts. The exam is moving away from simple "What is X?" questions toward "Which of these is wrong?" questions. You need to practice spotting errors, not just recalling facts.
Check your dates. If you are using study materials from before 2022, they might be training you for the wrong style of question.
Personalize your plan. Studying "hard" isn't enough; studying "smart" means focusing on the specific type of thinking the exam demands right now, adjusted for what you already know.

In summary: LearnOpt proves that exams have a hidden, stable structure that can be mapped. By understanding this structure and your own weaknesses, you can stop studying everything and start studying the right things.

Technical Summary: LearnOpt – Recovering the Latent Cognitive Structure of Standardized Examinations

1. Problem Statement

Standardized examination preparation in India, particularly for the National Eligibility cum Entrance Test (NEET), is traditionally approached as a uniform syllabus coverage problem. Students and coaching centers typically allocate time evenly across chapters or prioritize based on historical topic frequency. The authors argue this approach suffers from two structural flaws:

Non-Uniform Sampling: Exams do not sample the syllabus uniformly; certain cognitive operations appear with systematically higher frequencies than others.
Conflation of Topic and Skill: Questions within the same syllabus chapter may demand entirely different cognitive operations (e.g., direct fact retrieval vs. multi-concept integration), yet standard strategies treat them as equivalent.

The core challenge is reframing exam preparation as a constrained optimization problem: maximizing expected score given a fixed time budget and a student's current knowledge state. This requires the "stationarity assumption"—that the exam's latent cognitive structure is stable enough over time to make historical data predictive. The paper investigates whether this assumption holds for NEET and, if not, how to detect and adapt to regime shifts.

2. Methodology: The LearnOpt Framework

LearnOpt is a four-component pipeline designed to recover the exam's latent structure and generate personalized study plans.

2.1 Data Ingestion and LLM Tagging

Input: 1,496 English-language NEET questions from 2016–2024 (Biology, Chemistry, Physics).
Process: Raw PDFs are extracted via PyMuPDF. Each question is processed by a Large Language Model (LLM) using a chain-of-evidence prompt.
Output: Structured JSON records containing the question, options, answer, and a classification into one of five latent skill categories (S1–S5). The LLM must provide textual evidence for the classification to enable human review.
Reliability: Inter-model agreement (Cohen's $\kappa$ ) between two different LLMs on a subset of 74 questions was 0.51 (moderate). Disagreements were concentrated in the boundary between "Conceptual Application" (S2), "Multi-concept Integration" (S3), and "Quantitative Reasoning" (S4), while the "Elimination/Negation" category (S5) showed high agreement.

2.2 Exam Knowledge Graph

Structure: A graph $G=(V, E)$ stored in FalkorDB.
Nodes: Questions, Topics, Chapters, Subjects, Skills, and NCERT Sections.
Edges: Relationships include TESTS, PART_OF, REQUIRES_SKILL, TAUGHT_IN, and PREREQUISITE_OF.
Construction: Prerequisite edges are inferred from NCERT chapter sequencing and LLM-inferred concept dependencies.

2.3 Latent Skill Taxonomy

The authors define five skill categories based on surface structural discriminability (avoiding the low inter-rater reliability of 6-level Bloom's Taxonomy):

S1 (Direct Recall): Single fact retrieval.
S2 (Conceptual Application): Applying a known principle to a new context.
S3 (Multi-concept Integration): Combining knowledge from distinct chapters.
S4 (Quantitative Reasoning): Numerical calculation or formula application.
S5 (Elimination/Negation): Questions structured as "which is NOT correct" or "all EXCEPT," requiring exhaustive verification of claims.

2.4 Study Optimization Engine

The study plan is formulated as a variant of the Knapsack Problem solved via Google OR-Tools CP-SAT solver.

Objective: Maximize expected score gain.
Constraints: Time budget and prerequisite ordering (a student cannot master a concept without prerequisites).
Value Function: Incorporates historical frequency ( $R_i$ ), current mastery ( $m_i$ ), and a skill-weighted term ( $W_s$ ) based on the empirical frequency of the skill category in the exam.
Learning Model: Uses Bayesian Knowledge Tracing (BKT) with prerequisite-aware constraints to update mastery states after each study session.

3. Key Results

3.1 Regime Stability and Structural Break

Analysis of the 2016–2024 dataset reveals that NEET's latent skill distribution is piecewise stationary:

Pre-2023 Regime (2016–2021): The distribution is stable. Consecutive-year Kullback-Leibler (KL) divergence ranges from 0.004 to 0.032, with no significant shifts under permutation testing ( $p > 0.05$ ).
The 2023 Shift: A significant structural break occurs coinciding with the NCERT 2023 syllabus rationalization.
- Comparison: Pooling 2016–2021 ( $n=1,072$ ) vs. 2023–2024 ( $n=392$ ) yields a KL divergence of 0.040 ( $p=0.0005$ ).
- Nature of Shift: Direct Recall (S1) questions dropped from ~4–6% to ~2%. Elimination/Negation (S5) questions rose from ~20–29% to ~31–35%.
- Attribution: The shift is attributed to the removal of entire chapters from the NCERT syllabus, forcing question setters to test remaining material more rigorously via multi-statement verification (S5) rather than isolated fact retrieval (S1).

3.2 Subject vs. Time Predictors

Within a syllabus regime, the subject predicts the skill profile more strongly than the year.

Physics: Dominated by Multi-concept Integration (S3, 51.9%) and nearly devoid of S5.
Biology: Rich in S5 (34.6%) and S1 (7.8%).
Chemistry: Balanced between S2 and S3.
Divergence: The KL divergence between subjects (e.g., Physics vs. Biology = 0.420) far exceeds the year-to-year divergence within a regime (max 0.032).

3.3 Optimization Evaluation

The skill-weighted objective was tested against two baselines (Uniform coverage and Topic-frequency only) across three student profiles (one real, two synthetic).

Findings: The skill-weighted approach produced a modest but real reordering of recommended topics. For example, it prioritized topics dominated by S5 (high frequency in the post-2023 regime) over topics with similar historical frequency but lower-weighted skill profiles.
Limitation: The personalization signal was diluted because only 28.2% of questions could be mapped to the specific ~18 topics used in student self-assessment; the remainder defaulted to neutral values.

3.4 Cross-Exam Comparison (JEE Advanced)

Applying the pipeline to JEE Advanced revealed a fundamentally different cognitive structure:

JEE Advanced: Dominated by Multi-concept Integration (80.9%) with almost no pure recall.
Divergence: The KL divergence between JEE Advanced and NEET (0.505) exceeds the largest cross-subject divergence within NEET, indicating that exam tier shapes latent structure more than subject.

4. Significance and Claims

The paper claims the following contributions:

Empirical Evidence of Piecewise Stationarity: It demonstrates that while exam structures are not permanently static, they are stable within identifiable syllabus regimes. This validates the use of historical data for optimization provided that regime shifts (like the 2023 NCERT changes) are detected and accounted for.
Latent Skill Extraction: It introduces a five-category taxonomy that captures structural features of MCQs (specifically S5: Elimination/Negation) often overlooked by standard taxonomies, showing that question format is a distinct predictor of cognitive demand.
Optimization Framework: It presents LearnOpt, a system that moves beyond "high-weightage chapter" advice to a constrained optimization model that weights study time by both topic frequency and the empirical cognitive demand of the exam.
Open Data: The authors release the code, the knowledge graph, and the annotated dataset (1,496 questions with skill tags) to allow independent verification of these patterns.

The authors remain modest regarding the current state of the tool, noting that the personalization signal is currently limited by the granularity of topic mapping and that the 2025 data could not be reliably analyzed due to the lack of full question text in available sources. The primary contribution is the analysis of the exam's latent structure rather than a fully mature, production-grade tutoring system.