📄 medicine

Safety performance indicators for prehospital care services in Malaysia: a Fuzzy Delphi consensus study

Using the Fuzzy Delphi Method, a panel of 21 Malaysian prehospital experts reached consensus on a 16-item safety performance indicator framework covering structure, process, and outcome domains, with process indicators receiving the strongest agreement while resource and uniform target measures required further context-sensitive refinement.

Original authors: Abdul Karim Mustafa, Nik Hishamuddin Nik Ab Rahman, Tuan Hairulnizam Tuan Kamauzaman, Zaharah Sulaiman, Rosnah Ismail, Mohd Ridhuan Mohd Jamil

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abdul Karim Mustafa, Nik Hishamuddin Nik Ab Rahman, Tuan Hairulnizam Tuan Kamauzaman, Zaharah Sulaiman, Rosnah Ismail, Mohd Ridhuan Mohd Jamil

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where the most critical moments of medical care happen not in a sterile hospital room, but in the back of a speeding vehicle, navigating traffic, weather, and unpredictable emergencies. This is the reality for prehospital care teams, the first responders who bridge the gap between a crisis and a hospital. While these teams are trained to save lives, they work in one of the most dangerous environments in healthcare, facing risks ranging from traffic accidents and infectious diseases to violence and extreme fatigue. For years, the success of these services has been measured by how quickly they arrive, a simple number that tells us little about whether the care provided was actually safe or effective. To truly protect both patients and the staff who serve them, experts need a better way to measure safety, one that looks at the entire system rather than just a stopwatch. This requires understanding three distinct layers: the resources available to the team, the specific actions they take during a call, and the final results for both the patient and the worker.

In Malaysia, where emergency medical services are primarily run by hospital-based teams, there was no validated tool to measure this safety performance until a recent study brought together a group of seasoned experts to build one. The researchers, drawing on the experience of twenty-one specialists including emergency physicians and assistant medical officers, sought to create a standardized list of indicators that could be used to audit and improve safety across the country. They did not simply ask for opinions; they used a structured method to find where the experts truly agreed, filtering out vague suggestions to keep only the most robust and actionable measures. The goal was to move beyond guesswork and establish a clear, shared understanding of what makes prehospital care safe, creating a framework that could be used to track progress and identify weaknesses before they lead to harm.

The process began with a list of twenty potential safety measures, organized into the three layers of resources, actions, and results. The experts evaluated each item, and the study revealed a striking pattern in what they considered essential. The strongest agreement, nearly universal among the panel, focused on the actions taken during care. Every single one of the five process indicators was accepted, with the highest level of consensus given to emergency response protocols. This suggests that the experts believe safety is most directly ensured by having clear, reliable procedures for how to assess a patient, what medical interventions to perform, how to document the care, and how to supervise the team. These operational steps form the core of the new safety instrument, representing the daily work where human performance meets system design.

While the experts were united on what happens during a call, they were less certain about how to measure the resources that make that care possible. Although everyone agreed that having enough money and staff is important, the group could not reach a consensus on using financial budgets and human resource counts as direct safety indicators. The experts seemed to view these as background conditions that enable safety rather than measurable outcomes themselves. Instead, they preferred to measure the tangible results of those resources, such as whether equipment is compliant with regulations and whether vehicles and medical devices are properly maintained. These concrete measures of equipment readiness and policy adherence were accepted with high confidence, indicating that safety is best judged by what can be seen and checked, rather than by the size of a budget or a staffing roster.

The study also looked at the final outcomes of the care provided, and here the experts took a broad view that included the well-being of the workers alongside the health of the patients. All five outcome indicators were accepted, covering everything from patient satisfaction and the rate of medical errors to the job satisfaction of the staff and the frequency of injuries. This inclusion of worker well-being is significant, as it recognizes that a supported and safe team is more likely to provide safe care. However, the experts drew a firm line when it came to setting uniform targets for the entire country. They rejected the idea of a single, national performance target that would apply to every service, arguing that a standard that works in a busy city center might be impossible to achieve in a rural or semi-urban area. This finding highlights the need for safety goals to be flexible and sensitive to local conditions, rather than rigid and one-size-fits-all.

Ultimately, the study produced a validated set of sixteen safety indicators that Malaysian prehospital services can use to measure and improve their performance. The experts confirmed that this framework is comprehensive and applicable to real-world settings, providing a solid foundation for systematic safety monitoring. By focusing on the processes of care, the maintenance of equipment, and the well-being of both patients and staff, the new instrument offers a way to move beyond simple response times and address the complex reality of emergency care. While the study was limited to experts in the Klang Valley and Putrajaya, the consensus reached provides a clear path forward for measuring safety in a way that is both rigorous and relevant to the unique challenges of the Malaysian context.

Technical Summary: Safety Performance Indicators for Prehospital Care Services in Malaysia

Problem Statement
Prehospital care personnel operate in high-risk environments characterized by ambulance crashes, workplace violence, infectious exposure, and adverse clinical events. Despite these hazards, Emergency Medical Services (EMS) in Malaysia, and many other middle-income systems, lack a validated, context-specific instrument to measure safety performance. Existing metrics often rely on surrogate endpoints like response times, which have weak connections to actual patient safety. Furthermore, current indicator sets frequently lack a theoretical foundation integrating human factors, systems theory, and psychosocial safety climate (PSC) within the standard Donabedian structure–process–outcome framework. There is a critical gap in measuring safety not just as an outcome of care, but as a systemic property involving workforce wellbeing and organizational resources.

Methodology
This study represents the third phase of a three-phase Design and Development Research (DDR) program, utilizing the Fuzzy Delphi Method (FDM) to achieve expert consensus on a Prehospital Safety Performance Indicator (PSPI) framework.

Study Setting and Panel: The study was conducted in the Klang Valley and Putrajaya, Malaysia. A purposive panel of 21 experts (8 emergency physicians and 13 assistant medical officers) was recruited. All panelists possessed at least five years of prehospital experience and held relevant degrees.
Instrument Development: Based on a prior needs analysis and literature synthesis, 20 candidate items were developed, categorized into four domains:
1. Structure (6 items): Financial resources, human resources, safety policies, safety training, equipment compliance, equipment maintenance.
2. Process (5 items): Emergency response protocols, patient assessment, medical interventions, documentation quality, monitoring and supervision.
3. Outcome (5 items): Injury incidence, error/adverse events, patient satisfaction, safety-compromising behavior, job satisfaction.
4. Overall Usability (4 items): Comprehensiveness, target-setting, real-world applicability, capacity to improve safety.
Data Collection and Analysis: Experts rated items on a seven-point linguistic scale (1 = strongly disagree to 7 = strongly agree). Responses were converted into Triangular Fuzzy Numbers (TFN). Consensus was determined by three sequential conditions:
1. Threshold Value ( $d$ ): The distance between the mean TFN and individual expert TFNs must be $\le 0.2$ (vertex method).
2. Group Consensus: At least 75% of experts must have an individual $d \le 0.2$ .
3. Defuzzification: The defuzzified score ( $A$ ) must be $\ge 0.5$ .

Key Results
Of the 20 candidate items, 17 reached consensus and were accepted into the final PSPI framework.

Process Domain (Strongest Agreement): All five process indicators were accepted with a mean fuzzy score of 0.860 and a mean consensus of 98.1%.
- Emergency response protocols achieved the highest score in the entire study ( $A=0.887$ , 100% consensus, $d=0.070$ ).
- Patient assessment and monitoring/supervision also achieved unanimous consensus.
Structure Domain: Four of six items were accepted (mean $A=0.806$ $A = 0.806$ ).
- Accepted: Equipment maintenance ( $A=0.881$ ), equipment compliance ( $A=0.868$ ), safety policies, and safety training.
- Rejected: Financial resources ( $A=0.670$ , 23.8% consensus) and human resources ( $A=0.721$ , 57.1% consensus) failed the threshold and consensus conditions. Experts viewed these as enabling infrastructure rather than direct, measurable safety performance indicators.
Outcome Domain: All five indicators were accepted (mean $A=0.825$ $A = 0.825$ ).
- The framework successfully integrated both patient-centered measures (patient satisfaction, adverse events) and workforce-centered measures (job satisfaction, injury incidence, safety-compromising behavior).
Overall Usability: Three of four usability items were accepted. However, the item regarding comprehensive performance targets was rejected ( $A=0.786$ , $d=0.217$ ). While 81% agreed, the threshold condition failed, indicating expert disagreement on the feasibility of uniform national targets across heterogeneous settings (e.g., urban vs. semi-urban/rural).

Significance and Claims
The paper claims to provide the first validated, context-specific safety measurement instrument for Malaysian prehospital services, organized within the Donabedian model. Its significance lies in three areas:

Operationalizing Safety: The study shifts the focus from simple response times to a 16-indicator framework that captures the "what is done" (process) and the systemic conditions (structure/outcome) of prehospital care.
Theoretical Integration: By incorporating Human Factors, Systems Theory, and Psychosocial Safety Climate (PSC) theory, the PSPI explicitly links workforce wellbeing (e.g., job satisfaction, safety-compromising behavior) to patient safety outcomes, arguing that a supported workforce is a prerequisite for safe care.
Policy Alignment: The accepted indicators map directly onto existing Malaysian regulations (Occupational Safety and Health Act 1994, Road Transport Act 1987, Medical Device Act 2012), providing a structured, auditable system to turn legal requirements into actionable data.

The authors conclude that while the PSPI is a usable instrument for systematic safety monitoring, the rejection of uniform performance targets suggests that future implementation must utilize context-sensitive benchmarks rather than rigid national thresholds. The study acknowledges limitations regarding generalizability to rural areas or East Malaysia and notes that the predictive validity of the PSPI against actual safety events remains to be tested in future work.

Technical Summary: Safety Performance Indicators for Prehospital Care Services in Malaysia

More like this