📄 medicine

The association between fidelity enactment and intervention effectiveness of the Support Healthcare Improvement through Facilitated Training (SHIFT) intervention: A pragmatic convergent mixed methods study

This pragmatic convergent mixed methods study demonstrates that higher fidelity enactment of the SHIFT intervention in nursing homes is strongly associated with improved resident outcomes, highlighting the critical role of leadership support and facilitation in achieving successful implementation within complex care environments.

Original authors: Matthias Hoben, Liane R. Ginsburg, Lonni R. Kehler, Narges Khodabandehloo, Jennifer Pietracci, Adam Easterbrook, Danielle Saj, Carole A. Estabrooks, Adrian S. Wagg, Whitney Berta, Malcolm B. Doupe

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Matthias Hoben, Liane R. Ginsburg, Lonni R. Kehler, Narges Khodabandehloo, Jennifer Pietracci, Adam Easterbrook, Danielle Saj, Carole A. Estabrooks, Adrian S. Wagg, Whitney Berta, Malcolm B. Doupe

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Fixing a Leaky Boat in a Storm

Imagine nursing homes as old, leaky boats sailing through a storm. The water (problems like pain, falls, or bad nutrition) is constantly coming in, and the crew (care aides) is often tired, short-staffed, and underpaid. For years, experts have tried to hand these crews new "bailing buckets" (improvement plans) to stop the leaks. But often, the buckets get lost, the crew doesn't know how to use them, or the storm is just too strong, and the boat keeps sinking.

This study, called SHIFT (Support Healthcare Improvement through Facilitated Training), asked a simple question: If we give the crew a really good map, a coach, and a sturdy bucket, can they actually fix the boat while still sailing it?

The Experiment: The "Coach and Crew" Setup

The researchers didn't just hand out a manual and walk away. They set up a training program for 10 different nursing home teams in Winnipeg, Canada.

The Crew: Each team consisted of the people who actually do the work—the care aides (the frontline workers who feed, bathe, and comfort residents), plus their managers (the "Team Sponsors" and "Senior Sponsors").
The Map (The Intervention): The team was taught a specific method called PDSA (Plan, Do, Study, Act). Think of this as a recipe for trying small changes. You try a new way to help a resident (Plan), you do it (Do), you check if it worked (Study), and then you tweak the recipe for next time (Act).
The Coach: A "Quality Advisor" from the research team acted like a personal trainer. They visited the teams, watched them practice, and helped them figure out why a recipe wasn't working.
The Checkpoints: The teams met at "Learning Congresses" (like a big report card day) to show off their progress.

The Two Big Questions

The researchers wanted to measure two things to see how they were connected:

Fidelity Enactment (Did they follow the rules?): This is like asking, "Did the team actually use the map and the recipe correctly?" Did they track their data? Did they try the changes? Did they adjust when things went wrong?
Intervention Effectiveness (Did the boat stop leaking?): This is asking, "Did the residents actually get better?" Did they have less pain? Did they fall less? Did they eat better?

What They Found: The "Follow-Through" Connection

The results were clear and linked together like a chain:

The Scoreboard: The researchers gave every team a score from 0 to 4 for both "following the rules" and "getting results."
The Strong Link: They found a very strong connection between the two scores.
- The High Flyers: The teams that followed the recipe closely (high fidelity) were the ones that actually saw their residents improve (high effectiveness).
- The Strugglers: The teams that skipped steps, forgot to write things down, or didn't really try the new methods (low fidelity) didn't see much improvement in the residents.

The Analogy: Imagine a cooking competition.

Team A followed the recipe exactly, measured the ingredients, and tasted the food as they cooked. Their cake turned out delicious.
Team B guessed the amounts, didn't check the oven, and forgot to taste the batter. Their cake was dry and burnt.
The study showed that you can't get a great cake (good resident care) if you don't follow the recipe (high fidelity).

Why Some Teams Succeeded and Others Didn't

The researchers looked closely at three specific teams to understand why the scores were different.

The Team That Stalled (Low Score): This team wanted to help, but their "Team Sponsor" (the manager) was too busy to help them. They were also short on staff. The care aides said, "We are just trying to survive; we don't have time to write down our data." Because they couldn't track their progress, they couldn't prove they were getting better, even if they tried.
The Team That Tried Hard (Medium Score): This team did the work but struggled with the "math" part. They filled out their charts, but the charts were messy or didn't match the real data. They made progress, but they couldn't fully prove it or scale it up because they got stuck on the paperwork.
The Team That Soared (High Score): This team had a manager who was right there with them, cheering them on. The care aides felt empowered. They tracked their data perfectly, saw that their changes were working, and felt proud. The manager helped them solve problems when the evening shift said, "This is too hard." Because they had support, they followed the rules and got great results.

The "Secret Sauce": Support is Everything

The study concluded that the "recipe" (SHIFT) works, but only if the crew has support.

Leadership Support: The managers needed to be involved, not just sign a form. They needed to make sure the staff had time to do the work.
Research Team Support: The "Quality Advisors" were crucial. They helped the teams learn how to measure things correctly. Without this help, even good teams got lost in the paperwork.

The Bottom Line

You can give a nursing home team a perfect plan to improve care, but if they don't have the time, the tools, or the manager's backing to actually do the plan, the residents won't get better.

The study shows that when nursing home teams are given the right training, the right tools, and—most importantly—the active support of their leaders, they can successfully follow the plan and make real, positive changes for the people they care for. However, this requires a system that provides resources; you can't expect overworked staff to fix a broken system without help.

Technical Summary: The Association Between Fidelity Enactment and Intervention Effectiveness in the SHIFT Study

Problem Statement
Nursing home (NH) care operates within complex, under-resourced environments characterized by systemic barriers such as market-driven policies, workforce instability, and inadequate training for care aides. These conditions have historically prevented evidence-based improvement interventions from achieving sustainable, system-level change. While previous interventions like the Safer Care for Older Persons in Residential Care Environments (SCOPE) showed promise, they faced challenges regarding role clarity, staff buy-in, and the ability to scale. There is a critical gap in implementation science literature regarding the simultaneous measurement of fidelity enactment (the extent to which an intervention is implemented as intended) and intervention effectiveness (the achievement of intended clinical outcomes) in real-world NH settings. Existing studies often report one without the other, or fail to quantify their association, limiting the understanding of how implementation processes drive outcomes in pragmatic contexts.

Methodology
This study employed a pragmatic convergent mixed methods design as part of the Support Healthcare Improvement through Facilitated Training (SHIFT) intervention. The study involved 10 NH care teams in Winnipeg, Manitoba, Canada, representing a mix of for-profit and not-for-profit facilities.

Intervention: SHIFT is a five-month quality improvement intervention designed to accelerate change. It builds on the SCOPE framework but incorporates a Readiness for Change (R4C) assessment, a stronger focus on leadership support, and a reduced timeline (from 12 to 5 months) with three Learning Congresses (LCs) instead of four. Teams utilized the Plan-Do-Study-Act (PDSA) model to test and scale change ideas focused on resident outcomes (e.g., responsive behaviors, falls, nutrition).
Fidelity Enactment Measurement: Fidelity was assessed using the Overall Fidelity Enactment Scale for Complex Interventions (OFES-CI), a validated tool rated on a 0–4 scale. Two independent research team members rated each team's fidelity at LC 2 and LC 3 based on presentations, field notes, quality advisor diaries, and focus group transcripts. Discrepancies were resolved by consensus.
Intervention Effectiveness Measurement: Effectiveness was rated on a parallel 0–4 scale based on standardized criteria applied to resident outcome tracking data (safety crosses and run charts). Ratings considered the completeness of data collection (percentage of residents tracked) and the demonstrated improvement in outcomes over time.
Analysis: Quantitative analysis included calculating inter-rater reliability using Intraclass Correlation Coefficients (ICC) and assessing the correlation between fidelity and effectiveness scores using Kendall's Tau. Qualitative data were analyzed using content analysis to contextualize scores and identify facilitators or barriers. Three teams (low, medium, and high effectiveness) were selected for detailed case studies.

Key Results

Variability in Performance: Fidelity enactment and effectiveness scores varied significantly across teams. At LC 2, 6 of 10 teams achieved fidelity scores $\ge$ 2. Effectiveness scores ranged from 0 to 4, with 6 teams showing substantial improvement (scores > 2).
Reliability: Inter-rater agreement was high for all measures:
- Fidelity at LC 2: ICC = 0.856 [0.846; 0.881]
- Fidelity at LC 3: ICC = 0.926 [0.923; 0.936]
- Effectiveness: ICC = 0.745 [0.718; 0.810]
Correlation: There was a strong positive association between fidelity enactment and intervention effectiveness.
- At LC 2: Kendall's Tau = 0.546 [0.198; 0.894]
- At LC 3: Kendall's Tau = 0.750 [0.530; 0.971]
Qualitative Drivers: Case studies revealed that leadership support (from both senior and team sponsors) and facilitation by the research team were the primary contributors to high fidelity and effectiveness. Conversely, teams with low scores struggled with inconsistent staff engagement, staffing shortages, and a lack of dedicated time. A recurring challenge across all teams, regardless of leadership support, was the difficulty in measuring and documenting resident outcomes (e.g., creating accurate run charts from safety cross data).

Key Contributions

Empirical Evidence of Association: The study provides robust evidence that in pragmatic NH settings, higher fidelity enactment is strongly associated with better intervention effectiveness. This addresses a gap in literature where these two metrics are rarely reported simultaneously.
Validation of Measurement Tools: The study demonstrates the utility of the OFES-CI and a newly developed effectiveness scale in a real-world, multi-team implementation context, showing high inter-rater reliability.
Identification of Implementation Barriers: The research highlights that while leadership is crucial, specific technical skills regarding data measurement and documentation are often lacking among care aides and even team sponsors, acting as a bottleneck for effective implementation.
Refinement of the SHIFT Model: The study confirms that a condensed, facilitated training model can be successfully enacted by NH care teams, provided adequate support structures are in place.

Significance and Claims
The paper claims that with the right support, NH care teams can enact complex improvement interventions with high fidelity and effectively improve resident outcomes under real-world conditions. The authors assert that the strong correlation between fidelity and effectiveness suggests that early detection of teams with low fidelity enactment could allow for targeted support to improve performance.

However, the authors maintain a modest tone regarding scalability. They conclude that for SHIFT to be applied at a health system level, systems must create internal resources to support care teams and leaders, rather than relying on external research teams for facilitation. The study acknowledges limitations, including a small sample size (10 teams), the lack of a control group (though previous SCOPE trials included controls), and the reliance on team-generated documentation which may underestimate true effects due to under-documentation. The findings suggest that while the intervention is feasible and effective, its sustainability depends on systemic investment in leadership and measurement capacity within the care homes themselves.