Supporting physics instructors to use a variety of evidence-based approaches to improve student learning: An example from quantum mechanics

This paper illustrates how supporting a quantum mechanics instructor through an online community enabled him to persist with evidence-based active engagement strategies by adapting his approach to include productive struggle via graded corrections, ultimately improving student learning outcomes when the initial clicker-based method failed.

The Gist

Imagine teaching a complex subject like Quantum Mechanics is like trying to teach a group of people how to navigate a dense, foggy forest using a new, high-tech map. The map is supposed to be "evidence-based," meaning it was proven to work by the people who made it. But when you try to use it with your specific group of hikers, they get lost anyway.

This paper tells the story of two physics professors (let's call them Professor A and Professor B) who faced this exact problem. It's a story about how support, patience, and having a "toolbox" of different teaching strategies can save the day when one method fails.

Here is the story broken down into simple parts:

1. The Problem: The "Magic Map" Didn't Work

The professors were teaching a tricky concept called "Adding Angular Momentum." Think of this like trying to figure out the total spin of two spinning tops combined. It's abstract and confusing.

To help students, they used a method called Clicker Questions (CQS).

• How it works: The teacher asks a multiple-choice question. Students vote with a handheld device (a "clicker"). Then, they turn to their neighbor and argue about the answer. Finally, they vote again.
• The Goal: This "peer discussion" is supposed to clear up confusion, like a group of hikers helping each other find the path.

Professor A's Experience:
Professor A tried this map. It worked well for some parts of the forest (students learned how to list the possible directions), but it failed for the tricky parts (calculating the actual forces). The students were still confused about why the map worked the way it did.

Professor B's Experience:
Professor B tried the same map the next year. But this time, it failed almost completely. The students were even more lost. Why? Because the students didn't have enough basic knowledge to start with. The "peer discussion" didn't work because half the group didn't know the rules of the game.

2. The Crisis: The "I Give Up" Moment

Usually, when a teacher tries a new method and it fails, they feel discouraged. They might think, "This evidence-based method is a scam. I'll just go back to lecturing from a chalkboard."

This is where the paper's main message comes in: You need a support system.

The authors argue that teaching is a process, not a switch. If a tool doesn't work immediately, you don't throw it away; you tweak it, or you grab a different tool from your toolbox. But you need other teachers to tell you, "Hey, don't quit! Try this other thing."

3. The Solution: The "Do-Over" Incentive

Professor B didn't give up. With encouragement from other physics educators, he tried a different strategy called "Incentives for Learning from Mistakes" (ILM).

Think of this like a video game "Continue" button.

• The Old Way: You take a test, get it wrong, and the teacher gives you the answer key. You look at it, nod, and move on. You never really learned why you were wrong.
• The New Way (ILM): Professor B gave the students their graded tests back. He marked the wrong answers but didn't give the solutions yet. He said: "If you figure out your own mistakes and fix them, I will give you back half the points you lost."

This forced the students to struggle productively. They had to open their notes, re-read the textbook, and think hard about why they got it wrong before they could see the right answer. It was like making them re-run the level in the video game until they beat it, rather than just watching someone else play.

4. The Result: The "Struggle" Paid Off

The results were amazing.

• The students who took the time to fix their mistakes didn't just get their points back; they actually learned the material better.
• On the final exam (the "boss battle" of the course), the students who had fixed their mistakes scored significantly higher than those who didn't bother to fix them.
• Even the students who didn't fix their mistakes did better than before, just by seeing the correct answers later, but the ones who did the "hard work" of fixing them were the true winners.

The Big Takeaway

The paper concludes with a simple, powerful lesson for all teachers (and anyone learning anything new):

1. Don't Panic: If a new teaching method fails, it doesn't mean you are a bad teacher or the method is bad. It just means it needs adjustment.
2. Have a Toolbox: Don't rely on just one trick. If "Clicker Questions" don't work, try "Mistake Correction." If that doesn't work, try something else.
3. Find Your Tribe: Teachers need a community (online or in-person) to talk to. When you are stuck, a friend can say, "I had that happen too! Try this instead." This support keeps teachers from quitting and helps students learn better.

In short: Teaching is a journey of trial and error. When you hit a wall, don't turn around and go home. Ask a friend for a different ladder, climb over the wall, and keep going. That's how students learn physics.

Technical Summary

1. Problem Statement

The paper addresses a critical barrier in Physics Education Research (PER): the low sustained adoption of Evidence-Based Active Engagement (EBAE) pedagogies by instructors.

• The Issue: Instructors often abandon EBAE tools (like clicker questions) after initial implementations fail to replicate the high learning gains observed by the tools' developers.
• The Cause: Early adaptations often fail because the pedagogy requires refinement to match the specific instructor's style and the students' prior knowledge. When immediate success is not achieved, instructors may feel discouraged and revert to traditional lecturing.
• The Gap: There is a lack of support structures to help instructors persist through the iterative process of refining EBAE approaches or switching to alternative strategies when one fails.

2. Methodology

The study utilizes a multi-year, multi-instructor case study within an upper-level undergraduate Quantum Mechanics (QM) course focusing on the addition of angular momentum.

• Participants: Two instructors (Instructor A and Instructor B) taught the same QM course in consecutive years at a large research university.
• Pedagogical Tools:
  1. Clicker Question Sequences (CQS): A validated sequence of 13 conceptual multiple-choice questions designed to guide students through uncoupled and coupled representations of angular momentum, including basis state construction and matrix element calculation.
  2. Incentives for Learning from Mistakes (ILM): An EBAE strategy where students are given grade incentives (partial credit recovery) to correct their errors on assessments before being shown the correct solutions, fostering "productive struggle."
• Procedure:
  • Year 1 (Class A): Instructor A implemented the CQS after traditional lectures. Pre- and post-tests were administered to measure learning gains.
  • Year 2 (Class B): Instructor B implemented a slightly modified CQS (adding two questions based on Year 1 data). However, post-test results were poor.
  • Intervention: Upon seeing poor results in Class B, Instructor B received support from a community of physics educators. Instead of abandoning the topic, he implemented the ILM pedagogy as a follow-up homework assignment, allowing students to correct their post-test errors for partial credit.
• Assessment:
  • Pre/Post-tests: Focused on constructing basis states, evaluating matrix elements for spin-spin ($H_1$) and magnetic field ($H_2$) Hamiltonians, and determining diagonalization properties.
  • Final Exam: Included a problem analogous to the post-test to measure long-term retention and transfer.
  • Metrics: Normalized gain ($g$) and Cohen's $d$ effect size were calculated.

3. Key Contributions

• Instructor Support Framework: The paper argues that the persistence of instructors in using EBAE is contingent on support systems (local or online communities) that normalize the "process" of teaching improvement rather than expecting immediate perfection.
• Pedagogical Flexibility: It demonstrates the efficacy of combining multiple EBAE strategies. When the CQS failed to address specific misconceptions in Class B, the instructor successfully pivoted to the ILM strategy without requiring additional class time.
• Validation of "Productive Struggle": The study provides empirical evidence that explicitly incentivizing students to diagnose and correct their own mistakes leads to superior learning outcomes compared to simply reviewing correct solutions.

4. Results

Class A (Instructor A - CQS Only):

• Performance: Students showed significant gains in constructing basis states (Question a, $g=0.88$) and calculating matrix elements for the magnetic field Hamiltonian (Question c, $g=0.66$).
• Limitations: Performance remained low on the spin-spin interaction Hamiltonian (Question b, $g=0.31$) and determining diagonalization (Question d, $g=0.33$).
• Analysis: Students successfully chose the uncoupled representation for basis construction but struggled to calculate matrix elements in a representation where the operator was not diagonal (the spin-spin Hamiltonian is diagonal only in the coupled representation).

Class B (Instructor B - CQS + ILM):

• Initial CQS Results: The CQS implementation was less effective than in Class A (Total post-test mean 57% vs. 69%). Pre-test scores were lower, suggesting insufficient prior knowledge.
• ILM Intervention: 12 of 19 students took advantage of the incentive to correct their mistakes.
  • Correction Gains: The "Correctors" improved their post-test scores significantly (e.g., Question b improved from 47% to 94%).
  • Final Exam Transfer: On a final exam question covering the same concepts:
    • Correctors: Average score 85%.
    • Non-Correctors: Average score 71%.
    • Note: Even non-correctors performed better than their initial post-test scores (71% vs. 53% initial), likely due to seeing the solutions, but the active correction group performed significantly better.

5. Significance

• Sustainability of EBAE: The study proves that when instructors are supported to view teaching as an iterative process, they are less likely to abandon effective tools after a single failure.
• Scalable Support: The authors highlight that online communities (e.g., Discord, Zoom) can effectively replace or supplement local support, allowing instructors to share strategies and maintain optimism.
• Student Learning Mechanism: The results reinforce that active error correction is a superior learning mechanism to passive review. By forcing students to engage with their misconceptions before seeing the answer, the ILM pedagogy facilitated deeper conceptual understanding of complex QM topics like angular momentum addition.
• Practical Application: The paper offers a concrete workflow for instructors: Implement a tool $\rightarrow$ Assess results $\rightarrow$ If results are suboptimal, do not quit; instead, refine the tool or introduce a complementary strategy (like ILM) to target specific gaps.