Faster Completion, Less Learning: Generative AI Reduced Study Time on Math Problems and the Knowledge They Build
This large-scale study utilizing a decade of ALEKS data reveals that generative AI has significantly reduced students' study time on susceptible math problems while simultaneously degrading their long-term knowledge retention, a phenomenon the authors identify as "cognitive surrender."
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Picture: The "Cheat Code" Effect
Imagine a massive library where millions of students go every day to solve math problems. For years, they had to do the work themselves: reading the questions, thinking through the steps, and writing down the answers.
Then, a new tool arrived (ChatGPT) that acts like a super-fast "cheat code." If a student types a word problem into this tool, it spits out the answer instantly. But if the problem involves drawing a graph or moving pieces around on a screen, the tool can't help much.
This paper asks: Did students start using this cheat code? And if they did, did it hurt their actual learning?
To find out, the researchers didn't just ask students, "Do you use AI?" (because people often lie or forget). Instead, they looked at the digital footprints of 3.2 million students over ten years. They watched exactly how long it took students to solve different types of problems before and after the "cheat code" became popular.
The Experiment: The "Text" vs. "Graph" Test
The researchers set up a clever comparison, like a race between two types of runners:
- The "Text" Runners (AI-Susceptible): These are word problems. You can copy and paste them into an AI chatbot, and it solves them in seconds.
- The "Graph" Runners (AI-Resistant): These are problems with charts, graphs, or interactive buttons. You can't just copy-paste these; you have to actually look at the screen and click things. AI struggles with these.
The Theory: If students are using AI to cheat, they should finish the "Text" problems much faster than before, but their speed on "Graph" problems should stay the same (because AI can't help them there).
What They Found
1. Students Are "Speed-Running" the Easy Stuff
After ChatGPT was released, students started finishing the "Text" problems incredibly fast.
- College students spent about 27% less time on these problems over the following year.
- High schoolers spent even less time (31% less).
- Middle schoolers sped up a little (9% less).
- 5th graders didn't change at all.
The Analogy: Imagine a student who usually takes 10 minutes to write an essay. Suddenly, they start doing it in 2 minutes. But when they have to build a model out of clay (which the AI can't do), they still take the same amount of time. This suggests they aren't getting smarter; they are just outsourcing the work.
2. The "Proctor" Test: The Magic Disappears
The researchers then looked at tests where students were watched by a proctor (a human or software watching them to ensure they don't use phones or AI).
- Result: In these supervised tests, the speed-up vanished completely. Students took the same amount of time on "Text" problems as they did before ChatGPT existed.
The Analogy: It's like a student who runs fast when no one is watching, but slows down to a normal jog when a referee is standing right next to them. This proves the speed-up wasn't because the problems got easier or the students got naturally faster; it was because they were using the "cheat code" when no one was looking.
3. The Cost: "Cognitive Surrender"
Here is the most important part. The researchers checked if this speed-up helped students remember the math later. They gave students a surprise test (with a proctor) on the material they had studied earlier.
- The Result: Students who had "speed-ran" the text problems using AI were 25% less likely to get the answers right on the supervised test later.
- The Twist: When the researchers looked at the unsupervised tests (where students could use AI again), the students looked like geniuses—they got almost everything right.
The Analogy: Think of it like a video game.
- Using AI is like using a "god mode" cheat code. You beat the level instantly and get all the points (high scores on unsupervised tests).
- But when you turn the cheat code off for the final boss battle (the proctored test), you realize you never actually learned how to play the game. You lose.
The paper calls this "Cognitive Surrender." Instead of using the AI as a calculator (which helps you do math but still lets you think), students used it to surrender the thinking process entirely. They let the AI do the work, so their brains didn't build the "muscle memory" needed to solve the problem later.
Why Age Matters
The study found a clear pattern based on age:
- College & High School: Big changes. These students have more freedom and privacy to use AI without being watched.
- 5th Grade: No change. These students likely don't have their own devices or the freedom to use AI independently.
The Bottom Line
This paper provides the first massive, real-world evidence that generative AI has changed how students study.
- They are studying less: They are spending significantly less time on problems they can copy-paste.
- They are learning less: Because they aren't doing the mental work, they forget the material faster.
- It's not a glitch: The fact that the speed-up disappears when a proctor is watching proves it's a behavioral choice, not a change in the curriculum.
The authors conclude that while AI might make students look like they are learning faster (by getting quick answers), it is actually causing them to "surrender" their thinking, leading to a significant drop in what they actually know and remember.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.