Lessons in Implementing Complex Interventions in a Public Health Emergency: A Process Evaluation of the California Contact Tracing Support Initiative

This paper evaluates a pilot program that integrated Kaiser Permanente’s clinical network with contact tracing efforts in California, finding that while the model showed promise, its implementation was hindered by data-sharing complexities, misaligned organizational visions, and the challenges of adapting to an evolving public health emergency.

The Gist

The Big Idea: The "Emergency Kitchen" Experiment

Imagine you are trying to run a massive, high-speed soup kitchen to feed a city during a sudden flood. You have two main groups: The Professional Chefs (a large, established hospital system called Kaiser Permanente) and The Rapid Response Volunteers (a public health organization called the Public Health Institute).

The goal is to find people who have been exposed to the "flood" (COVID-19) and get them help immediately. This paper is a "post-game report" looking at what happened when these two groups tried to work together in the middle of the chaos.

---

The Plan: The "Smart" Soup Kitchen

Usually, when a city tries to find people to help, it’s like sending out flyers and hoping someone calls. It’s slow and often fails.

The CCTSI program tried something different: The Integrated Approach. Instead of waiting for people to call the city, they used the hospital's own records. It was like having a VIP list of everyone who walked into the kitchen, so the moment someone got sick, a team was already standing by to help them.

---

What Went Wrong? (The Growing Pains)

Even though the idea was brilliant, the execution hit three major "speed bumps":

1. The "Clash of Cultures" (The Chef vs. The Volunteer)

Imagine the Professional Chefs are used to a very strict, high-end kitchen where every garnish must be perfect and the owner is always watching. The Rapid Response Volunteers are used to a "pop-up" style—they are fast, flexible, and used to making their own rules on the fly.

• The Conflict: The Chefs (Kaiser) wanted to be in the kitchen every second, checking every spoon. The Volunteers (PHI) felt like they were being micromanaged and couldn't do their jobs. It was like two different captains trying to steer the same ship in two different directions.

2. The "Broken Recipe Book" (The Data Disaster)

To run a kitchen, you need to know exactly how many carrots you have. In this program, "carrots" were the data (who is sick, who was contacted).

• The Conflict: The two groups didn't agree on how to write the reports. One group thought they’d send a report once a week; the other group wanted a live scoreboard updated every minute. Because they weren't on the same page, the "scoreboard" ended up showing wrong numbers, which made the Chefs lose trust in the Volunteers. It’s hard to cook a meal if you don't know if you have enough salt!

3. The "Changing Menu" (The Scope Creep)

In the middle of the flood, people started asking for more than just soup. They wanted bread, water, and blankets.

• The Conflict: The Volunteers wanted to change the menu to meet the new needs (like helping with vaccines). The Chefs wanted to stick to the original recipe (contact tracing) to make sure they perfected it first. They were arguing over whether to "fix the soup" or "start making sandwiches."

---

The Lessons Learned (The "Recipe for Success")

The researchers concluded that if we ever face another "flood," we shouldn't wait until the water is rising to figure out how to work together. They suggest three rules:

1. Agree on the Menu Early: Even in an emergency, sit down and decide exactly what you are making and who is in charge of the stove. Don't assume everyone is reading the same cookbook.
2. Build a Live Scoreboard: Don't wait until the end of the week to see if the food is good. You need real-time data so you can fix mistakes before the whole meal is ruined.
3. Prepare the Kitchen Before the Storm: We need better "pipes" (data systems) between hospitals and the government before a pandemic hits, so we aren't trying to build the plumbing while the house is underwater.

Summary in one sentence:

The program was a great idea that showed how powerful hospitals and public health groups can be together, but it struggled because the two groups didn't speak the same "operational language" or use the same "data scoreboard."

Technical Summary

Technical Summary: Lessons in Implementing Complex Interventions in a Public Health Emergency

Problem Statement

During the SARS-CoV-2 pandemic, traditional public health contact tracing efforts were overwhelmed by rapid case surges, logistical delays, and behavioral challenges (such as mistrust and low response rates). While contact tracing is a proven non-pharmaceutical intervention, its effectiveness is highly time-sensitive. The core problem addressed by this study is how to implement a large-scale, "clinically integrated" contact tracing model—linking a private health system (Kaiser Permanente) with a public health implementation organization (Public Health Institute)—to reduce time lags and improve trust, and how such a complex partnership functions under the extreme pressure of a public health emergency.

Methodology

The researchers conducted a retrospective, mixed-methods process evaluation of the California Contact Tracing Support Initiative (CCTSI) from August 2020 to June 2021. The study focused on implementation in Fresno and San Bernardino Counties. The methodology included:

• Document Review: Analysis of 474 program documents (training materials, meeting minutes, contracts, and data dictionaries).
• Qualitative Interviews: Semi-structured interviews with 47 participants, including stakeholders from Kaiser Permanente (KP), the Public Health Institute (PHI), and local health departments, as well as frontline staff (contact tracers, resource coordinators, and supervisors).
• Data Analysis: Interviews were transcribed and analyzed using NVivo software, employing both inductive and deductive coding via the Framework Approach.
• Modeling: The team developed a Theory of Change (Impact Model) to map the causal chain from inputs to outcomes and constructed an epidemiological timeline to contextualize program milestones against COVID-19 case surges.

Key Contributions

The paper contributes to the field of implementation science and public health emergency management by:

1. Evaluating a Novel Model: It provides one of the first evaluations of a clinically integrated contact tracing model that leverages a private HMO's existing infrastructure to bypass traditional public health delays.
2. Identifying Implementation Barriers in Partnerships: It identifies specific "clashes of culture" and operational friction points that occur when clinical-oriented organizations (KP) partner with research/implementation-oriented organizations (PHI).
3. Developing Frameworks for Emergency Response: It offers three formal recommendations for implementing complex interventions during crises, focusing on vision articulation, real-time monitoring, and contextual influence.

Results

The evaluation revealed three primary areas of implementation difficulty:

• Inter-Organizational Dynamics: A "clash of cultures" emerged between KP and PHI. KP operated with a "contractual" mindset, seeking high-touch involvement and real-time clinical-style feedback. PHI operated with a "grant-funded" mindset, seeking autonomy and periodic reporting. This led to conflicting expectations regarding roles, decision-making authority, and program scope.
• Data System Challenges: There was a significant misalignment regarding data demands. KP expected on-demand, real-time performance data, whereas PHI was prepared for periodic reporting. This led to unanticipated workloads, poor initial data quality (due to lack of quality control), and a subsequent loss of stakeholder confidence, which delayed the scaling of the program to new sites. Furthermore, the inability to access KP’s Electronic Health Records (EHRs) forced the development of a custom, labor-intensive database.
• Competing Program Visions: Stakeholders disagreed on "program fidelity" versus "adaptability." PHI viewed the program as a dynamic response that should evolve (e.g., shifting to vaccine outreach), while KP sought to maintain the original, strict focus on the integrated contact tracing innovation to avoid "scope creep."

Significance

This study is significant for public health leaders and policymakers because it highlights that technical innovation (like clinical integration) is insufficient without operational and relational alignment. It demonstrates that in an emergency, the "urgency to act" often leads to the neglect of essential project management tasks—such as defining a shared Theory of Change and establishing data standards—which ultimately creates bottlenecks. The findings underscore the potential of public-private partnerships to provide rapid surge capacity, provided that the boundaries, data-sharing protocols, and decision-making frameworks are formalized before the crisis peaks.