Implementation strategy modifications: An applied multi-site comparison using ERIC and FRAME-IS for the "Fluoroquinolone Restriction for the Prevention of Clostridioides difficile infection Trial" (FIRST)

This study utilizes the ERIC and FRAME-IS frameworks to systematically document and analyze 458 context-driven modifications to implementation strategies across four diverse hospital sites in the FIRST trial, revealing that organizational characteristics significantly influence the frequency and type of adjustments needed to maintain evidence-based practice fidelity.

The Gist

Imagine you have a perfect recipe for a delicious cake that works great in one specific kitchen. Now, imagine you want to bake that same cake in four very different kitchens: a high-tech professional bakery, a small rural diner, a busy community cafe, and a university cafeteria with constantly changing staff.

This paper is about the "Fluoroquinolone Restriction" trial (FIRST), which tried to bake a specific "cake": a computer alert system designed to stop doctors from overusing a certain type of antibiotic (fluoroquinolones) that can cause dangerous infections. The goal was to see how to make this recipe work in those four different "kitchens" (hospitals).

Here is the story of what they found, explained simply:

The Problem: The "Black Box" of Change

Usually, when researchers try to introduce a new rule in a hospital, they say, "We did this, and it worked." But they rarely explain how they had to tweak the plan along the way. It's like saying, "We baked the cake," without mentioning that you had to swap the oven, add extra eggs, or change the mixing time because the kitchen was different. This paper wanted to open that "black box" and write down every single change they made.

The Tools: A Map and a Diary

To track these changes, the researchers used two special tools:

1. The Map (ERIC): A giant list of 73 different "tactics" (like training staff, changing workflows, or holding meetings) that can be used to introduce new rules.
2. The Diary (FRAME-IS): A structured way to write down exactly what was changed, why, when, and who decided to change it.

They tracked every single tweak made over two years at four different hospitals.

The Big Findings: One Size Does Not Fit All

The researchers found 458 different changes made to the original plan across the four sites. Here is what that looked like in everyday terms:

1. The "Planned" vs. "Unplanned" Dance

• Planned Changes (The Anticipators): Most changes (about 72%) were planned ahead. These are like a chef saying, "I know this rural kitchen has only one oven, so I'll plan to bake the cake in batches."
• Unplanned Changes (The Reactors): About 28% were surprises. These are like the oven breaking down mid-bake, forcing the chef to improvise.
• The Ratio: Hospitals that had done similar things before (like the big academic centers) had a high ratio of planned-to-unplanned changes. They were like experienced chefs who knew exactly what could go wrong. Hospitals that were new to this or had fewer resources had more "surprise" changes, meaning they were reacting to problems as they popped up.

2. The Four Kitchens (Hospitals)

• The Big Academic Center (Site A): This place had a huge, rotating staff of trainees (like students who change every few months). The biggest challenge here was training. They had to constantly re-teach the new rule to new people, like a school that has to teach the same safety drill to a new class every semester.
• The Rural Hospital (Site B): This was the hardest kitchen. They had very few staff and limited computer experts. They had to spend a lot of time building relationships and fixing things as they went. When the pandemic hit, they had to stop everything for six months, which was a huge shock. They had to rebuild trust and re-teach everyone when they started again.
• The Community Hospital (Site C): They were already good at managing antibiotics. They didn't need many changes, but when they decided to expand the rule from just the ICU to the whole hospital, they hit some unexpected snags with how patients moved between rooms. They had to tweak the process on the fly.
• The Academic Center with a New System (Site D): This hospital had just switched to a new computer system and didn't like computer "pop-up" alerts. They had to work hard to convince doctors that the alerts were helpful, not annoying. They had to change their culture to accept the new rule.

3. What Was Actually Changed?
The most common changes weren't about the medical rule itself (the "recipe" stayed the same). Instead, they changed how the rule was delivered:

• Training: How often and to whom they taught the rule.
• Relationships: Who they talked to and how often.
• Feedback: How they shared data about whether the rule was working.

The Takeaway

The main lesson is that you cannot just copy-paste a successful plan from one hospital to another. Even if the core rule (the antibiotic restriction) stays exactly the same, the support system around it must be customized.

• Rural hospitals need more time and relationship-building.
• Academic hospitals need endless training cycles for new staff.
• Hospitals with new computer systems need help changing their culture.

The paper concludes that successful implementation isn't about following a rigid script. It's about having a flexible plan that knows exactly how to adapt to the specific "kitchen" it is being used in. By tracking these changes, future projects can be smarter about anticipating what will need to be tweaked before they even start.

Technical Summary

1. Problem Statement

Implementation science faces a critical gap in understanding how to effectively adapt evidence-based practices (EBPs) across diverse healthcare contexts. While strategies are often described as a "black box," there is insufficient standardized documentation of how and why implementation strategies are modified during real-world deployment.

• The Challenge: Standardized implementation approaches often fail to account for critical organizational differences (e.g., rural vs. academic, resource-rich vs. resource-constrained), leading to variable success rates.
• The Specific Context: The study focuses on the FIRST trial, a hybrid type-2 implementation-effectiveness trial designed to reduce fluoroquinolone (FQ) prescribing to prevent Clostridioides difficile infection (CDI) via an Electronic Health Record (EHR) best practice alert (BPA).
• The Knowledge Gap: There is a lack of systematic frameworks to document the dynamic nature of strategy modifications, specifically distinguishing between planned (anticipatory) and unplanned (reactive) changes, and linking these modifications to specific organizational characteristics.

2. Methodology

The study employed a qualitative, multi-site comparative case study design across four distinct hospital sites in the US.

• Study Sites:
  • Site A: Large academic research center (urban, established restrictions).
  • Site B: Rural community hospital (resource-constrained, limited IT).
  • Site C: Community hospital with established stewardship (urban).
  • Site D: Flagship academic research center (limited prior BPA culture).
• Intervention: An EHR-integrated FQ restriction requiring pre-prescription authorization for non-approved indications.
• Data Collection (24 months):
  • Implementation diaries (daily logs by site coordinators).
  • Facilitated meeting notes (58 total meetings).
  • Semi-structured interviews with key informants (n=19).
  • Site context questionnaires.
• Analytical Frameworks:
  • ERIC (Expert Recommendations for Implementing Change): Strategies were coded into 9 conceptual clusters (e.g., "Train and educate stakeholders," "Adapt and tailor to context").
  • FRAME-IS (Framework for Reporting Adaptations and Modifications to Evidence-Based Implementation Strategies): Used to document 7 domains of every modification, including what was modified, who decided, when it occurred, and whether it was planned or unplanned.
• Analysis: Two researchers independently coded data using MaxQDA, achieving consensus through iterative discussion. Modifications were categorized as Planned (anticipated/contingency) or Unplanned (reactive to unforeseen barriers).

3. Key Contributions

This paper advances implementation science methodology by:

1. Integrating ERIC and FRAME-IS: Demonstrating a replicable method to systematically document strategy modifications across heterogeneous settings.
2. Defining the "Planned vs. Unplanned" Ratio: Proposing this ratio as a novel metric for assessing organizational absorptive capacity and implementation readiness.
3. Contextualizing Modification Intensity: Showing that the volume of modifications does not equate to implementation failure; rather, it reflects contextual complexity and iterative refinement.
4. Fidelity vs. Adaptation: Providing evidence that implementation processes can be heavily modified while maintaining the core fidelity of the evidence-based intervention (the FQ restriction mechanism remained unchanged).

4. Key Results

Across the four sites, 458 total strategy modifications were documented over two years.

• Overall Distribution:

  • Planned Modifications: 330 (72%)
  • Unplanned Modifications: 157 (34%)
  • Note: The total exceeds 100% due to some modifications being counted in both categories or specific coding nuances, but the text explicitly states 72% planned and 34% unplanned, with a total count of 458.

• Site-Specific Variations (Planned:Unplanned Ratios):

  • Site A (Academic, experienced): 3.5:1 ratio. High anticipatory planning due to prior restriction experience.
  • Site B (Rural): 2.3:1 ratio. Highest total modifications (174). Driven by resource constraints, extended timelines, and severe COVID-19 disruption (6-month suspension).
  • Site C (Community, established stewardship): 1.6:1 ratio. Lowest total modifications (94). High reliance on evaluative strategies and iterative refinement rather than upfront planning.
  • Site D (Academic, new to BPA): 1.8:1 ratio. Moderate planning; struggled with cultural resistance to alerts and IT resource competition.

• Dominant ERIC Strategy Clusters Modified:

  1. Train and educate stakeholders (18.1%): Heavily modified in academic sites (A & D) due to rotating resident/trainee populations requiring perpetual onboarding.
  2. Adapt and tailor to context (17.6%): Critical for rural sites (B) and sites with unique workflow constraints.
  3. Develop stakeholder interrelationships (16.2%): Essential for building trust, particularly in rural settings.
  4. Use evaluative and iterative strategies (16.2%): Dominant in Site C, reflecting a data-driven refinement approach.

• Triggers for Modifications:

  • Workforce Dynamics: Resident rotations (academic) and staff turnover (rural).
  • Technological Challenges: EHR build issues, alert fatigue, and integration with other decision support tools.
  • External Shocks: The COVID-19 pandemic caused significant unplanned modifications, particularly in Site B (suspension) and Site D (IT staff shortages).

• Timing:

  • Pre-implementation: Focused on stakeholder engagement and education (62% of planned mods).
  • Early Implementation (<4 months): Focused on troubleshooting and workflow adjustments (71% of unplanned mods).
  • Late Implementation: Focused on sustainability and institutionalization.

5. Significance and Implications

• Tailored Implementation Support: The study proves that "one-size-fits-all" implementation strategies are ineffective. Support must be calibrated based on organizational characteristics:
  • Rural Hospitals: Require extended pre-implementation phases (12+ months), intensive relationship building, and contingency planning for resource constraints.
  • Academic Centers: Require perpetual education systems to handle rotating trainees and cultural change management regarding physician autonomy.
  • Community Hospitals: Can leverage existing stewardship relationships but must plan for scale-related challenges when moving from unit-specific to hospital-wide interventions.
• Metric for Readiness: The Planned:Unplanned ratio serves as a diagnostic tool. A high ratio suggests robust anticipatory planning and strong infrastructure, while a low ratio indicates a reactive implementation style requiring more support.
• Methodological Advancement: The integration of ERIC and FRAME-IS provides a rigorous, standardized way to "open the black box" of implementation, allowing researchers to distinguish between necessary adaptations and protocol deviations.
• Resilience: Sites with mature stewardship cultures (Site C) and strong interprofessional bonds demonstrated greater resilience to systemic shocks (like the pandemic) compared to resource-constrained sites.

Conclusion: Successful implementation of complex interventions like antibiotic stewardship requires dynamic, context-sensitive strategy selection. Systematic documentation of modifications reveals that organizational context dictates not just which strategies are needed, but when and how intensively they must be deployed.