Effects of Audit and Feedback on the Clinical Performance of Physicians Providing Direct-to-Consumer Teleconsultation in China: A 2×2×2 Factorial Randomised Controlled Trial

Technical Summary: Effects of Audit and Feedback on DTCT Physician Performance

Problem Statement
Direct-to-consumer teleconsultation (DTCT) in China has expanded rapidly but suffers from suboptimal clinical quality, with diagnostic accuracy and guideline adherence rates significantly below public expectations. While Audit and Feedback (AnF) is a proven quality improvement strategy in traditional healthcare, its application in DTCT is hindered by structural differences: the absence of institutional hierarchies, the supplementary nature of online work for most physicians, and the lack of physical examination data which complicates quality assessment. Furthermore, conventional AnF mechanisms (e.g., expert peer review) are often absent in DTCT, and existing patient feedback mechanisms are frequently viewed by physicians as lacking clinical authority or containing insufficient detail. There is a critical gap in understanding which specific AnF components are effective in this unique, market-driven environment and whether combining them yields synergistic or antagonistic effects.

Methodology
The study employed a two-phase approach culminating in a 2×2×2 factorial randomised controlled trial (RCT).

• Phase 1 (Framework Development): The researchers conducted a structured literature review and a modified Delphi consultation with nine multidisciplinary experts. This process utilized grounded theory coding to identify core tensions in DTCT feedback design. The outcome was a refined AnF framework that selected three experimental factors for testing:

  1. Peer Comparison Ranking (Factor A): Displaying the physician's adherence percentile relative to peers.
  2. Clinical Guideline-Based Feedback (Factor B): Specific, itemized performance data against clinical guidelines.
  3. Patient-Centred Ratings (Factor C): Scores derived from the Patient Perception of Patient-Centredness scale (PPPC-R).
     All groups received a universal baseline report containing case vignettes and general quality ratings.

• Phase 2 (The Trial):

  • Participants: 152 physicians from three major Chinese platforms (Haodf Online, WeDoctor, Ping An Health) who had previously been assessed by Unannounced Standardised Patients (USPs).
  • Design: A fully randomised 2×2×2 factorial design created eight experimental arms (presence/absence of each of the three factors).
  • Intervention: Physicians received electronic feedback reports via platform messaging. The reports were optimised through iterative testing to use a "mixed framing" (affirming competence before identifying gaps) and were delivered following telephone consent.
  • Outcomes:
    • Primary: Guideline adherence intention (0–100% visual analogue scale).
    • Secondary: Intervention acceptability (Digital Health Acceptability Questionnaire, DHAQ).
  • Analysis: Multiple linear regression was used to estimate main effects and interaction effects, adjusting for physician title and baseline diagnostic quality.

Key Results
Of the 152 randomised physicians, 74 (48.7%) completed the outcome questionnaire. The study found that no single feedback component produced a statistically significant main effect on either guideline adherence intention or intervention acceptability. Instead, the effects were driven by interactions between components:

1. Synergistic Effect on Adherence Intention: A significant positive interaction was observed between Peer Ranking and Patient Feedback (β=5.23, p=0.034). When both social comparison sources were presented together, they significantly increased the intention to adhere to guidelines compared to either alone.
2. Antagonistic Effect on Acceptability: A significant negative interaction was found between Guideline-Based Feedback and Patient Feedback (β=−1.10, p=0.027). Combining these two elements reduced the acceptability of the intervention, suggesting a conflict between professional standards and consumer evaluation.
3. Baseline Quality: Higher baseline diagnostic quality was positively associated with intervention acceptability (p=0.001).
4. Attrition: The study noted a high physician attrition rate (23.7% over two years), significantly exceeding the pre-specified 5% estimate, highlighting the fluid nature of the DTCT workforce.

Key Contributions

• First USP-Based AnF Trial in DTCT: This is the first study to apply Unannounced Standardised Patient (USP) methodology to generate AnF data specifically for direct-to-consumer teleconsultation.
• Factorial Design Efficiency: The study demonstrates the utility of a 2×2×2 factorial design in implementation science, allowing for the simultaneous estimation of main and interaction effects with greater statistical efficiency than parallel-arm trials.
• Mechanism Identification: The research identifies that in the DTCT context, where institutional accountability is weak, multi-source social comparison (combining peer and patient data) may be necessary to generate motivational pressure, whereas combining clinical authority with consumer ratings may create cognitive dissonance.

Significance and Claims
The paper claims that its findings provide component-level evidence to guide the design of future, fully-powered trials and the development of scalable AnF strategies for DTCT. The authors assert that:

• AnF in DTCT operates through context-dependent interactions rather than independent component effects.
• Optimising AnF for DTCT requires careful consideration of how feedback sources are combined; specifically, peer ranking and patient feedback should be combined to boost adherence intentions, while guideline-based and patient feedback should likely be separated to maintain acceptability.
• The study validates the operational feasibility of delivering USP-based feedback via platform-native messaging, achieving a 64% participation rate among contactable physicians through a multi-modal (telephone + digital) approach.
• Future implementations must account for high workforce turnover and gender disparities in participation (male physicians were significantly more likely to complete the questionnaire).

The authors maintain a modest tone, noting that the significant interaction effects observed warrant confirmation in a fully-powered trial and that the primary outcome (intention) is a proximal measure requiring future validation through objective behavioural observation.