Analyzing Curricular Pattern Complexity Using AI to Improve On-Time Graduation Rates
This paper proposes using Large Language Models to automatically analyze and revise Software Engineering curricula, thereby reducing the time required for updates and mitigating bottlenecks that delay student graduation.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Great Course Maze
Imagine you are trying to solve a massive, three-dimensional puzzle where every piece is a class you have to take to get your degree. In the world of higher education, this puzzle is called a curriculum. Sometimes, the pieces are arranged in a long, winding line where you can't pick up piece B until you finish piece A, and you can't touch piece C until piece B is done. If you trip and drop piece A, the whole line behind it collapses, and you might be stuck waiting a whole year to try again. This is what researchers call "curricular complexity." It's like a traffic jam in a city where one broken traffic light stops every car for miles.
To fix these jams, universities usually have a team of human experts (professors) who stare at the puzzle for hours, trying to figure out which lines to shorten or which pieces to move. But this is slow, boring, and hard work. Recently, a new kind of digital brain has arrived: Artificial Intelligence, specifically Large Language Models (LLMs). Think of these as super-smart robots that have read almost every book and website on the internet. They are great at understanding language and spotting patterns. The big question for scientists is: Can these digital brains look at a university's course puzzle, spot the traffic jams, and suggest a better way to arrange the pieces so students can graduate on time? This paper dives right into that question, treating the university curriculum like a complex game board that needs a fresh strategy.
The Digital Detective and the Course Maze
In this study, a team of researchers from Embry-Riddle Aeronautical University decided to put five different AI "detectives" to the test. Their mission was to analyze the Software Engineering degree program, which is known for having some very long, tricky chains of classes. If a student fails one class in a long chain, they might be delayed by a semester or even a whole year. The researchers wanted to see if an AI could look at the course list, figure out which classes were causing the biggest bottlenecks (the traffic jams), and suggest changes to fix them.
The researchers gave each AI detective the same job: read the rules of the game (the course descriptions and flow charts), calculate how "complex" the maze was, and then suggest how to rearrange the classes to make it easier for students to finish in four years. They tested five different models, including some from OpenAI, Google, and Amazon, and even tried one special version that had a "cheat sheet" (a technique called Retrieval-Augmented Generation, or RAG) to help it remember specific facts.
The results were a mix of "wow" and "oops." Most of the AI detectives got the basic idea of what a bottleneck was, but they struggled with the math. It's like asking a smart robot to do long division; it knows what the numbers mean, but it sometimes trips over the actual calculation. One detective, however, stood out. Claude Sonnet 4.6 was the only model to correctly calculate the "blocking" and "delay" factors—the specific numbers that tell you how many students get stuck and for how long. It was so good that it even built a little interactive tool (a JavaScript app) that let people visualize the changes. However, even this top performer noted a limitation: it could not calculate "instructional complexity" (a measure of difficulty based on pass rates) without access to the university's private historical data.
Based on its analysis, Sonnet 4.6 suggested some bold moves to untangle the maze. It suggested:
- Cutting the chains: Removing some classes as strict requirements for the final senior project, so failing one earlier class doesn't ruin the whole year.
- Moving things around: Shifting a class called "Software Quality Assurance" to an earlier semester so students aren't stuck waiting until their very last year to take it.
- Opening more lanes: Suggesting that certain difficult classes be offered in both the fall and spring semesters, rather than just once a year, so students don't have to wait a whole year to retake them if they fail.
However, the story doesn't end with the robot winning. The researchers took Sonnet's suggestions to the real-life experts: the university professors. The professors nodded at some ideas but shook their heads at others. They pointed out that the AI didn't know about some hidden rules, like how some classes are shared with graduate students or that the department simply doesn't have enough teachers to offer every class twice a year. In fact, the AI suggested moving a class that was already offered twice a year, showing it had missed a detail in the real-world data.
The paper concludes that while AI isn't ready to take the wheel and redesign the curriculum all by itself, it is an incredibly powerful co-pilot. Sonnet 4.6 proved to be the best tool for spotting the traffic jams and brainstorming ideas, though the researchers noted that when they tried to give it a "cheat sheet" (RAG) to help it remember facts, the system actually became confused by the visual flow charts. Just like a GPS that suggests a route without knowing about road construction, the AI's suggestions need to be checked by human experts. The researchers found that AI can speed up the process of finding problems and suggesting fixes, but the final decision on how to rearrange the puzzle still belongs to the humans who know the school best. The study suggests that in the future, if we can teach AI to "see" the course flow charts better (perhaps by using special vision tools), these digital detectives might become even sharper at keeping students on the fast track to graduation.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.