From Execution to Education: A Bloom-Aligned Framework for Measuring Educational Control in LLMs
This paper introduces a Bloom-aligned framework to evaluate educational control in LLMs, revealing that while models like Qwen3-Next can reliably increase the cognitive demand of programming tasks, they struggle to lower it, demonstrating that strong execution performance does not guarantee the ability to adapt tasks for specific learning objectives.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a super-smart robot tutor that can write code faster than any human. You might think, "Great! It can teach anyone anything, right?" Well, this paper suggests that while the robot is a coding wizard, it's actually a bit clumsy when it comes to being a good teacher.
The researchers, Yi Zhang and Julia Rayz from Purdue University, wanted to see if these AI models could do more than just solve problems. They wanted to know if the AI could take a hard problem and make it easier for a beginner, or take an easy one and make it harder for an expert, all while keeping the core lesson the same. They call this "educational control."
To test this, they used two versions of a powerful AI family called Qwen3-Next. One was a "General" model (good at everything), and the other was a "Coder" model (specialized in programming). They gave them 2,520 programming tasks and asked them to change the difficulty in four specific ways: make it "Harder," make it "Easier," aim for a "Higher" thinking level, or aim for a "Lower" thinking level. They used a famous teaching chart called Bloom's Taxonomy, which ranks thinking from simple memory (Level 1) to creating new ideas (Level 6).
The Big Surprise: The "Up-Only" Elevator
Here is the main finding: The AI models are fantastic at making things harder, but they are terrible at making things easier.
Think of the AI's ability to change difficulty like an elevator that only has an "Up" button. When the researchers asked the models to make a task "Harder" or target a "Higher" thinking level, the elevator zoomed up perfectly.
- When asked to make tasks Harder, the General model jumped up by an average of 1.762 levels on the thinking scale, and the Coder model jumped up by 1.582 levels.
- When asked to target Higher levels (like "Create" or "Evaluate"), the General model succeeded 79.2% of the time, and the Coder model succeeded 63.4% of the time.
But when they pressed the "Down" button? The elevator barely moved, or sometimes even went the wrong way.
- When asked to make tasks Easier, the models actually made them slightly harder! The General model's tasks went up by 0.194 levels, and the Coder model's went up by 0.715 levels.
- When asked to target Lower levels (like "Remember" or "Understand"), the models failed miserably. The General model only got it right 29.2% of the time, and the Coder model only 25.1% of the time.
Why is this happening?
The researchers looked inside the AI's "brain" (its internal code layers) and at the words it used to see what was going on.
It turns out the AI has a habit of "over-engineering" when it tries to simplify. When asked to make a task easier, instead of removing the hard thinking parts, the AI just adds more formatting or extra instructions. It's like a chef who is told to make a complex dish simpler, so instead of removing the fancy spices, they just write a longer recipe explaining how to hold the spoon. The core difficulty remains, but the surface looks different.
The "General" model and the "Coder" model also behaved differently inside their brains:
- The General model showed a clear separation between "Hard" and "Easy" instructions right in the middle of its layers (around layer 29). It seemed to understand the concept of difficulty well, even if it couldn't execute the "Easy" part perfectly.
- The Coder model was a bit more confused. It struggled to separate "Hard" from "Easy" instructions (its internal signals were very mixed up). However, when it came to "Higher" vs. "Lower" thinking levels, it waited until the very deep layers (layer 36) to make a clear distinction, suggesting it processes these concepts differently than the General model.
What the Paper Rules Out
The paper explicitly argues against the idea that being good at solving code problems means you are good at teaching. Just because an AI can pass a coding test (like the HumanEval benchmark) doesn't mean it can adapt a lesson for a student. The authors suggest that the "illusion of learning" happens when AI gives the perfect answer immediately, skipping the "productive struggle" that students need to actually learn.
How Sure Are They?
The authors are very sure about the direction of the results based on their simulations. They measured this across 2,520 tasks from three different coding benchmarks (BigCodeBench, LiveCodeBench, and SWE-Bench-Verified). They used a second AI (Claude-3.5-Haiku) to grade the thinking levels, which agreed with human experts 95% of the time on a test set.
However, they are careful to say this is a "controlled comparison" between two specific models. They don't claim this is true for every AI in existence, but it is a strong signal that current AI models have a built-in bias toward making things complex rather than simple. They suggest that to make AI a real tutor, we need to fix this "up-only" elevator so it can actually help beginners by lowering the cognitive load, not just adding more words to the prompt.
In short: These AI models are great at adding complexity, but they are currently struggling to strip it away. If you want a robot tutor that can truly adapt to a beginner's needs, we still have some work to do.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.