AI Assisted Video Analysis Reveals Divergent Laboratory Competence Profiles Despite Equivalent Practical Scores in Undergraduate Titration
This study demonstrates that AI-assisted video analysis of undergraduate titration tasks reveals significant divergences in specific competence domains despite equivalent aggregate practical scores, highlighting the need for granular, evidence-based diagnostic profiles over traditional summary grades in STEM assessment.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the chemistry laboratory, learning is a physical act. It is not enough to know the theory behind a reaction; a student must also learn to handle glassware, control the flow of liquids, watch for subtle color changes, and record results with precision. This is the domain of practical skill, where knowledge meets the hand. Traditionally, when instructors assess these skills, they often reduce a complex performance into a single number. A student might receive a grade of eighty-five for a titration experiment, a task that involves preparing solutions, operating a buret, spotting a color change, and repeating the process. That single score, however, acts like a heavy blanket, hiding the specific details of how the student actually performed. It tells us the result, but it often fails to reveal whether the student struggled with the steady hand needed to stop the liquid, or perhaps with the eyes needed to see the color change, or simply with the patience to repeat the task consistently.
This limitation creates a blind spot in education. If two students receive the exact same grade, a teacher might assume they have the same strengths and weaknesses. But what if one student is excellent at spotting the color change but poor at controlling the flow, while the other is the reverse? They would have the same total score, yet they would need completely different help to improve. Researchers at Universitas Negeri Semarang in Indonesia set out to see if they could peel back this single grade to reveal the true, detailed picture of student performance. They turned to video analysis, using artificial intelligence not to replace human judgment, but to help organize the hours of footage into a structured map of what happened. Their goal was to see if the technology could uncover the hidden differences between students who appeared to be equally skilled based on a traditional score.
The researchers recorded twenty-six undergraduate students performing a standard acid-base titration. In this experiment, a student slowly adds a liquid from a tall glass tube, called a buret, into a flask until a specific color change signals the reaction is complete. The team used a computer system to scan the videos and locate specific moments of action, such as when the student held the tube, when they watched the liquid, or when they stopped the flow. This system did not grade the students on its own. Instead, it acted as a powerful search engine, finding the evidence in the video and organizing it into twenty-five tiny categories, or micro-skills, which were then grouped into seven broader areas like "pipetting," "buret operation," and "endpoint observation." Human reviewers then checked these findings to ensure the computer had correctly identified the actions.
The study revealed a striking reality: the single grade given to these students was a poor reflection of their actual abilities. When the researchers looked at the detailed profiles of the students, they found that many of them had received the exact same overall score, yet their performance patterns were completely different. In fact, among the pairs of students who shared the same final score, nearly ninety percent of them had different strengths and weaknesses across the various skills. One student might have been very good at recognizing the color change but struggled to stop the liquid at the right moment, while another might have controlled the liquid perfectly but missed the color change. The single grade had compressed these two very different stories into one identical number, effectively erasing the specific details that a teacher would need to provide useful feedback.
The analysis also highlighted a specific bottleneck in how students learn this skill. The data showed that most students were quite good at seeing the color change that signals the end of the experiment. However, there was a sharp drop in performance when it came to the physical act of stopping the liquid flow at that exact moment. While eighty percent of the students successfully recognized the color, only about forty percent managed to stop the flow in a timely and controlled manner. This suggests that seeing the event and physically reacting to it are two separate challenges. Furthermore, the study found that students who managed to complete one successful trial often failed to show the same consistency when asked to repeat the task. The ability to perform a procedure once did not guarantee the ability to perform it reliably again, a crucial distinction for scientific work that the single grade had completely missed.
The researchers were careful to note that their automated system was not perfect and required significant human oversight. In this specific study, the vast majority of the data points used to build the student profiles were flagged for manual review because the computer could not confidently interpret the video evidence on its own. This means the findings are not a result of a machine grading students in isolation, but rather a demonstration of how video evidence, when carefully organized and checked by humans, can reveal patterns that a simple score hides. The study does not claim that the computer is ready to take over the grading process, but rather that it can serve as a powerful tool to help educators see the full picture of a student's performance.
Ultimately, the work suggests that in practical science education, the whole is not always equal to the sum of its parts, and sometimes the sum hides the parts entirely. A single grade can tell an administrator that a student passed, but it cannot tell a teacher what that student actually needs to learn next. By using video to break down a performance into its individual components, educators can move beyond ranking students and start diagnosing their specific needs. The study concludes that for digital assessment to be truly useful, it must preserve the detailed profile of a student's skills alongside the final score. Only then can the feedback given to a student be as precise and helpful as the action they are trying to master.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.