Ensuring Computer Science Learning in the AI Era: Open Generative AI Policies and Assignment-Driven Written Quizzes
This paper proposes and preliminarily validates an assessment model for upper-level computer science courses that permits the use of generative AI for take-home programming assignments while ensuring student mastery through heavily weighted, assignment-driven written quizzes, finding no significant correlation between AI usage and academic performance under this framework.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a computer science classroom where students are learning to build complex machines. Traditionally, the teacher would say, "Go home, build this machine from scratch, and bring it back." The fear with new AI tools is that students might just ask the AI to build the machine for them, hand it in, and get an 'A' without ever learning how the gears actually turn. This is called "cognitive offloading"—letting a robot do the thinking so your brain doesn't have to.
This paper describes a clever new way to handle this problem, which the author calls "Open and Verify."
The New Strategy: The "Cooking Class" Analogy
Think of the class like a cooking school.
- The Old Way: The teacher says, "Go home and bake a cake. Bring it in tomorrow." If a student buys a cake from a bakery and claims they baked it, the teacher might never know.
- The "Ban" Way: The teacher says, "No store-bought cakes allowed!" But in the age of AI, this is like trying to ban ovens; it's impossible to stop students from using the tools available to them.
- The "Open and Verify" Way (This Paper's Solution): The teacher says, "You can use a recipe book, a sous-chef, or even a robot to help you bake the cake at home. BUT, when you come to class, you must immediately take a test where you have to explain exactly how the cake was made, why you used those ingredients, and fix a broken part of the recipe on the spot."
In this study, the "cake" is a programming assignment, and the "robot" is Generative AI (GenAI).
How It Worked in the Study
The professor at Lawrence Technological University tried this with a small group of 14 advanced students. Here is the recipe they used:
- The Assignment (The Take-Home Cake): Students were allowed to use AI to write code for their homework. They even had to fill out a "disclaimer" saying how much help they got from the AI (e.g., "I used AI for 50% of this code").
- The Twist (The Weight): The homework itself was worth very little points (only 2%).
- The Real Test (The In-Class Quiz): Immediately after the homework was due, the students took a written, closed-book quiz in class. This quiz was worth much more (5% or 10%).
- The quiz questions were directly about the homework they just turned in.
- They had to explain the code, fill in missing lines, or predict what the code would do.
- The Goal: If you used AI to write the code but didn't understand it, you would fail the quiz. If you understood it, you would ace it.
What Did They Find?
The researchers wanted to know: Does using AI make students dumber, or does it help them?
They looked at the data and found some surprising results:
- No Connection: There was almost zero link between how much AI a student used and how well they did in the class. Whether a student used AI for 10% of their work or 90%, their final grade was about the same.
- The "Middle" Trap: Interestingly, students who used AI a moderate amount (25–50%) actually scored slightly lower than those who used very little or a lot. The author suggests these "middle" students might have been too lazy to write the code themselves but not skilled enough at "prompting" the AI to get good results. They were stuck in the middle.
- Student Feedback: The students loved it. They said the quizzes forced them to actually read and understand the code the AI wrote, rather than just blindly submitting it. They felt less stressed because they could use AI to help, but the quizzes ensured they still learned the material.
The Bottom Line
The paper concludes that you don't have to ban AI to keep students learning.
If you let students use AI for the "heavy lifting" of writing code, but then immediately test them on their own understanding of that code, they still learn the concepts. It's like letting a student use a calculator for math homework, but then making them solve the problem on the board without the calculator to prove they know the math.
The Catch: The author admits this study was small (only 14 students), so it's more of a "proof of concept" than a final answer. Also, it creates more work for the teacher, who has to design and grade these extra quizzes. But for now, it suggests that with the right "Open and Verify" rules, AI can be a helpful tool rather than a cheating shortcut.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.