Associations Between Multi-Source Formative Assessment and Summative Examination Performance in Undergraduate Diagnostic Medicine: Evidence Supporting Programmatic Assessment
This study demonstrates that multi-source formative assessments across cognitive, procedural, and communication domains provide significant incremental validity in predicting undergraduate diagnostic medicine examination performance, thereby supporting the design of multidomain programmatic assessment systems while highlighting the limitations of theory assignments due to ceiling effects.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to figure out who is the best baker in a town. You could just ask them to bake one giant cake on a single Saturday and judge them by that one result. That's like a "summative" test: a big, final exam that happens once. But what if you wanted to see how they actually learn? What if you watched them practice making bread every week, saw how they handled a tricky dough, and listened to how they talked to their customers? That's "formative" assessment: a series of smaller, low-stakes check-ins that happen over time to help students grow.
In the world of medical school, teachers are trying to figure out the best way to mix these two approaches. They use a system called "programmatic assessment," which is like building a giant mosaic out of hundreds of tiny tiles. Each tile is a small assignment or a quick check-up. The idea is that if you look at all the tiles together, you get a much clearer picture of a student's skills than you would from just one big picture. But here's the tricky part: do all those little tiles actually predict how well a student will do on the big final exam? And do different types of tiles—like ones that test brainpower, ones that test hand skills, and ones that test talking skills—each add something unique to the puzzle?
This study from Guangdong Medical University dives right into that question. The researchers looked at 382 medical students taking a course on diagnostic medicine (learning how to figure out what's wrong with a patient). They tracked five different types of homework and practice tasks that happened throughout the semester, all recorded on a single online platform. They wanted to see if these "practice tiles" could predict the students' scores on the final written test.
Here is what they found, and it's a bit like a surprise party where some guests are much more helpful than others. First, they discovered that almost every single type of practice task was linked to the final exam score. If a student did well on the practice, they tended to do well on the final. However, when the researchers looked closer to see which tasks were the real independent predictors (meaning, which ones added new information that the others didn't already cover), the results got interesting.
The "brainy" tasks were a mixed bag. One type of homework, where students answered multiple-choice questions right after a lecture, turned out to be a bit of a trick. The scores were so high for everyone (an average of 89.04 out of 100) that it was like a ceiling effect; almost everyone hit the top, so it couldn't tell the difference between a good student and a great one. It was too easy and too soon after class to be a useful predictor. But the other cognitive task, called Problem-Based Learning (PBL), was a superstar. In PBL, students work in groups to solve complex medical mysteries. This task was a strong predictor, suggesting that the ability to reason through a problem is a huge key to success.
Then there were the "hands-on" tasks. The students had to record videos of themselves performing physical exams (like checking a heartbeat or listening to lungs) and get feedback from teachers. This video assignment was another major winner. It predicted final exam scores even after accounting for how smart the students were or how well they did on the group puzzles. This suggests that the act of recording yourself, getting feedback, and fixing your technique actually helps you learn the material deeply, not just how to move your hands.
Finally, there was the "people" task. Students practiced talking to actors (called standardized patients) to learn how to take a medical history. This task added a small but real boost to the prediction. It wasn't the biggest factor, but it did add a unique piece of information that the other tests missed.
When you put all these pieces together, the study found that these different types of practice tasks explained about 50.5% of the differences in how students performed on the final exam. The researchers concluded that a good assessment system needs all three types of "tiles": cognitive (thinking), procedural (doing), and communication (talking). They also learned that if you want a homework assignment to be a good predictor of future success, it needs to be challenging enough to separate the students, not just a quick, easy quiz right after class. By using one online platform to gather all this data, schools can build a much richer, more accurate picture of a student's journey, proving that learning is about more than just passing a single big test.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.