Perceived usability and usefulness of a clinical decision-support application among newly graduated physicians in rural areas: a mixed-methods study

Technical Summary: Perceived Usability and Usefulness of a Clinical Decision-Support Application Among Newly Graduated Physicians in Rural Peru

Problem Statement
Newly graduated physicians in Peru often serve in rural and urban-marginal health facilities through the Rural and Urban-Marginal Health Service (SERUMS) program. In these settings, physicians frequently operate with considerable autonomy but face significant constraints, including limited clinical experience, restricted access to diagnostic tests and supplies, and a lack of immediate specialist support. While mobile clinical-support applications offer point-of-care access to evidence-based information, there is a scarcity of evidence regarding their usability and perceived usefulness specifically among early-career physicians working in resource-limited rural environments. This study addresses the gap in understanding how these tools function within the specific context of SERUMS, where connectivity and local service capacity vary widely.

Methodology
The authors conducted a convergent mixed-methods study involving 81 eligible newly graduated physicians from the National University of San Marcos assigned to rural facilities during 2023–2024.

• Intervention: Participants were provided with access to BMJ Best Practice, a clinical-support application offering evidence-based information, guidelines, and tools. The interface was primarily in English, with limited Spanish content.
• Sampling: All 81 eligible physicians were invited; 32 enrolled, received credentials and training, and were granted three months of access.
• Quantitative Component: A self-reported online survey assessed participant demographics, application usage patterns, and perceived usability using a 17-item Spanish adaptation of the Computer System Usability Questionnaire (CSUQ).
• Qualitative Component: Semi-structured interviews were conducted with 10 physicians purposively selected for having the highest self-reported consultation frequency. These interviews explored barriers, facilitators, and the application's role in clinical decision-making.
• Analysis: Quantitative data were analyzed descriptively. Qualitative data underwent thematic analysis. The two datasets were integrated during the interpretation phase to provide a comprehensive view of the user experience.

Key Results

• Adoption and Usage: Of the 32 enrolled physicians, 23 (71.9%) reported using the application. Usage was split between the mobile app (60.9%) and the web version (39.1%). Most users accessed the tool once or twice per week, often after patient encounters.
• Perceived Usability: The quantitative survey indicated a predominantly favorable perception of usability, with 70%–90% of participants agreeing or strongly agreeing with statements regarding ease of use, speed, clarity, and organization.
• Qualitative Findings (Facilitators): Users valued the application for its rapid search capabilities, organized structure, and evidence-based content. It was particularly useful for supporting diagnostic reasoning, determining referral needs, and increasing clinical confidence. The mobile version was preferred for its portability and practicality during the workday.
• Qualitative Findings (Barriers): Significant barriers hindered effective use:
  • Linguistic: Difficulties searching in Spanish and the necessity of using English terms or synonyms limited rapid information retrieval.
  • Technical: Limited internet connectivity, automatic updates that interrupted workflow, and difficulties locating specific clinical calculators were reported.
  • Contextual Mismatch: A critical finding was the gap between application recommendations and local reality. Physicians frequently found that suggested medicines, diagnostic tests, or resources were unavailable in their rural facilities, rendering some evidence-based recommendations unactionable.
  • Content Specificity: Treatment information was sometimes too general or lacked specific dosing instructions relevant to the local primary care setting.

Significance and Claims
The paper claims that mobile clinical-support applications can complement decision-making and learning for early-career physicians in rural primary care, primarily by providing rapid access to up-to-date, evidence-based information that bolsters diagnostic confidence. However, the authors modestly assert that the practical value of these tools is not inherent but contingent on adaptation.

The study highlights that usability and evidence quality alone are insufficient for successful implementation in low-resource settings. The authors conclude that for such applications to be truly useful, they must be adapted to local language (specifically Spanish search optimization), workflow, connectivity constraints (offline access), and, crucially, the actual service capacity of local facilities. The paper emphasizes that while these tools support decision-making, they cannot compensate for structural limitations in the health system, such as the lack of essential medicines or diagnostic infrastructure.

Limitations Noted by Authors
The authors acknowledge several limitations: the small, non-probabilistic sample limited to one university; the reliance on self-reported usage rather than objective logs; the lack of formal psychometric validation for the Spanish usability instrument; and the absence of data on clinical outcomes or patient safety. Furthermore, the study did not deeply explore the reasons for non-use among the one-third of participants who did not engage with the tool.