Asynchronous Video Feedback and Early Performance Variability in Resident Laparoscopic Cholecystectomy: A Multicenter Observational Study

Technical Summary: Asynchronous Video Feedback and Early Performance Variability in Resident Laparoscopic Cholecystectomy

Problem Statement
Competency-based surgical education requires longitudinal assessment and feedback to guide trainees toward entrustment. While simulation-based training has been shown to improve operating room performance, the specific impact of asynchronous video-based feedback during the early phases of real operative learning remains uncertain. A critical educational challenge is that mean performance scores can mask significant variability among trainees; a training intervention might not raise average scores but could still be valuable if it reduces performance dispersion, attenuates low outliers, and helps residents converge toward a minimum acceptable standard. This study investigates whether asynchronous video feedback is associated with higher technical performance or reduced early performance variability in residents performing laparoscopic cholecystectomies.

Methodology
The authors conducted a multicenter observational study involving general surgery residents from seven residency programs across five cities in Chile.

• Cohort: The de-duplicated analytic cohort included 66 residents and 531 unique laparoscopic cholecystectomy videos.
• Exposure: Residents were classified into a "Feedback" group (41 residents, 281 videos) or a "No-Feedback" group (25 residents, 250 videos) based on their participation in a remote, asynchronous video-review workflow. Feedback was defined at the resident level; residents did not contribute to both groups.
• Assessment: Videos were assessed by blinded raters using a five-item Objective Structured Assessment of Technical Skills (OSATS) rubric (range 5–25).
• Milestones: Early performance was evaluated at the 3rd, 5th, and 10th recorded cases. Longitudinal analysis also examined cumulative case-volume bins.
• Statistical Analysis: The study analyzed both central tendency (means/medians) and dispersion (variance). Methods included Welch's t-tests, Mann–Whitney U tests, and Levene's test for variance. Crucially, a linear mixed-effects model was employed with resident as a random intercept to account for repeated observations by the same trainee, treating feedback exposure, case number, and simulation exposure as fixed effects.

Key Results

• Central Performance: Baseline OSATS scores were similar between groups. Across the learning curve, OSATS scores increased with accumulated experience (coefficient 0.93 per 10 cases; p < 0.001). However, at the 3rd, 5th, and 10th case milestones, there were no statistically significant differences in central OSATS performance between the Feedback and No-Feedback groups. The mixed-effects model confirmed that feedback exposure was not independently associated with higher OSATS scores (p = 0.905).
• Performance Variability: At the 10th case, the Feedback group showed a non-significant trend toward lower dispersion compared to the No-Feedback group (Levene p = 0.070).
• Intra-Resident Change: An exploratory complete-case analysis of intra-resident change (ΔOSATS = mean cases 8–10 minus mean cases 1–3) showed a mean decrease of -2.02 in the Feedback group versus an increase of +2.27 in the No-Feedback group. The authors explicitly note this was treated as exploratory and likely influenced by small sample sizes and non-randomized allocation, rather than evidence of a causal negative effect.
• Simulation: Documented simulation exposure showed a favorable descriptive trajectory but was not independently significant in the mixed-effects model (p = 0.151).

Key Contributions

1. Reframing Feedback Evaluation: The study challenges the assumption that the primary metric for feedback success is an increase in mean technical scores. It proposes that in competency-based education, reducing performance variability and standardizing early behaviors may be as educationally significant as raising average scores.
2. Methodological Rigor: By utilizing mixed-effects modeling and analyzing both central tendency and dispersion (variance), the study provides a more nuanced statistical framework for evaluating surgical education interventions than simple pre-post mean comparisons.
3. Contextualizing Variability: The paper highlights that similar median scores can coexist with different spreads of performance, suggesting that a narrower spread may indicate fewer residents performing at the low end, which is a critical safety consideration.

Significance and Claims
The authors conclude that asynchronous video feedback was not associated with sustained superiority in central OSATS performance. However, the exploratory analyses suggest a hypothesis that video feedback may function as an early standardizing intervention that reduces performance variability rather than acting solely as a direct performance booster.

The paper asserts that future studies should be powered to detect differences in both central performance and variability. It argues that educational success metrics must be defined a priori: if the goal is standardization, dispersion and threshold-based outcomes are more informative than average OSATS scores. The findings support a staged educational approach where early asynchronous video review is used to identify inconsistency and low-end performance, serving as part of a longitudinal system rather than a standalone intervention. The study acknowledges limitations, including the observational nature of the exposure, the inability to model center-level effects due to missing identifiers, and the small sample size for specific milestone analyses.