Pathways from AI Literacy to Sustained Engagement with AI-Powered Cognitive Behavioural Therapy: A Structural Equation Model with Moderated Mediation in a National UK Sample

This study of 1,247 UK adults demonstrates that AI literacy fosters sustained engagement with AI-powered cognitive behavioural therapy primarily by enhancing trust and perceived therapeutic alliance, though these positive pathways are significantly weakened by mental health stigma.

The Gist

Imagine you've just downloaded a new, high-tech app designed to help you feel better when you're anxious or down. It's like having a friendly robot therapist in your pocket. But here's the problem: most people download it, try it for a few days, and then delete it. They stop using it before they ever get the full benefit.

This research paper asks a simple but crucial question: Why do some people stick with these AI therapy tools while others quit? And how can we get more people to keep using them?

The authors, researchers from King's College London and the University of Manchester, looked at over 1,200 people in the UK who had tried these AI tools. They built a complex map (called a "Structural Equation Model") to understand the hidden psychological gears turning behind the scenes.

Here is the story of their findings, explained with some everyday analogies.

1. The Starting Point: "AI Literacy" (Knowing How to Drive)

Think of AI Literacy as knowing how to drive a car.

• If you don't know how a car works, you might be scared to get in. You might worry the engine will explode or that you'll crash.
• If you do know how to drive, you feel confident. You understand the rules, you know what the dashboard lights mean, and you trust that you can handle the vehicle.

The study found that people who understood how AI works (how it learns, what it can do, and what it can't do) were much more likely to keep using the therapy app. But why? That's where the middle part of the journey comes in.

2. The Two Bridges: Trust and Connection

The researchers found that "knowing how to drive" (AI Literacy) didn't just magically make people stay. Instead, it built two important bridges that led to long-term use:

• Bridge A: Trust (The Safety Belt)
  When you understand the AI, you trust it. You believe it's reliable and won't say something weird or hurtful. It's like putting on a seatbelt; once you know the car is safe, you feel comfortable riding in it for the long haul.
• Bridge B: The Therapeutic Alliance (The Handshake)
  This is the most surprising part. Even though the "therapist" is a robot, people can still feel a sense of connection, like a handshake or a friendly nod. When people understand the AI, they feel they can work with it to solve their problems. They feel like the robot is on their team.

The Big Discovery: The study showed that AI Literacy leads to Trust and Connection, and those two things are what actually make people stick with the therapy. In fact, about 58% of the reason people kept using the app was because they trusted it and felt connected to it.

3. The Roadblock: Stigma (The Heavy Backpack)

Now, imagine you are trying to cross those two bridges (Trust and Connection), but you are carrying a giant, heavy backpack labeled "Stigma."

• Stigma is the shame or fear people feel about having mental health issues or using a "robot doctor." They might think, "I'm not strong enough to use this," or "People will think I'm crazy for talking to a machine."

The study found that this heavy backpack slows you down.

• If you have low stigma (a light backpack), the bridges of Trust and Connection work perfectly. You cross them easily and keep using the app.
• If you have high stigma (a heavy backpack), the bridges are still there, but they are much harder to cross. The connection between understanding the AI and actually trusting it gets weaker. The shame makes it harder to believe the robot can help you.

4. What This Means for the Future

The researchers are basically saying: "We can't just build better robots; we have to teach people how to drive them and help them take off their heavy backpacks."

Here are the three main takeaways for anyone building or using these tools:

1. Teach the Users: Before someone starts therapy, we should give them a quick, friendly guide on how the AI works. If they understand the "engine," they will trust the "car" more.
2. Design for Connection: The AI needs to be programmed to act like a good friend. It should listen, remember goals, and make the user feel understood. That "human-like" connection is the strongest glue keeping people engaged.
3. Fight the Shame: We need to make it clear that using an AI tool is normal and brave, not a sign of weakness. If we can help people put down their "stigma backpack," the bridges of trust and connection will work much better for everyone.

The Bottom Line

AI therapy tools have huge potential to help millions of people, but they only work if people keep using them. This study proves that understanding the technology and feeling a human-like connection are the keys to keeping people engaged. However, if we don't also tackle the shame surrounding mental health, even the best tools might fail to reach the people who need them most.

It's not just about the software; it's about the human mind behind the screen.

Technical Summary

1. Problem Statement

Despite the proven clinical efficacy of AI-powered Cognitive Behavioural Therapy (AI-CBT) tools (e.g., Woebot, Wysa) in reducing symptoms of anxiety and depression, a critical implementation barrier exists: sustained user engagement is critically low. Users often drop out after the first two weeks. While qualitative studies have identified barriers like stigma and lack of trust, the psychological mechanisms explaining how AI literacy (the ability to understand and interact with AI) translates into long-term engagement remain unquantified. Furthermore, it is unclear how mental health stigma moderates these pathways. There is a lack of large-scale quantitative models that simultaneously test trust, therapeutic alliance, and stigma as mediators and moderators in a national sample.

2. Methodology

Study Design & Sample:

• Design: Cross-sectional national online survey conducted in the UK (Jan–Mar 2024).
• Sample: $N = 1,247$ adults (18+) with a history of anxiety/depression who used an AI-CBT tool in the past 12 months.
• Sampling: Stratified random sampling via Kantar Public to ensure national representativeness.
• Exclusion: Primary psychosis, bipolar I, or substance use disorders.

Measures:

• Predictor: AI Literacy (AILS scale; 12 items covering Knowing, Using, Evaluating, Creating).
• Mediators:
  1. Trust in AI (Adapted Trust in Automation Scale; 14 items).
  2. Perceived Therapeutic Alliance (Adapted Working Alliance Inventory–Short Revised; Bond, Goals, Tasks).
• Moderator: Mental Health Stigma (Composite index of Self-Stigma [ISMI-C] and Public Stigma).
• Outcome: Sustained Engagement (Composite of session count, intention to continue, and depth of feature use).
• Covariates: Age, gender, ethnicity, education, employment, symptom severity (PHQ-9, GAD-7), and tool type.

Statistical Analysis:

• Framework: Structural Equation Modelling (SEM) using lavaan in R.
• Estimation: Full-Information Maximum Likelihood (FIML) to handle missing data.
• Model Type: Moderated Mediation model.
  • AI Literacy $\rightarrow$ (Trust + Alliance) $\rightarrow$ Engagement.
  • Stigma moderates the paths from mediators to engagement.
• Techniques:
  • Latent Moderated Structural Equations (LMS) for interaction terms.
  • Bootstrapping (5,000 replications) for bias-corrected confidence intervals (BC-CIs).
  • Measurement invariance testing across gender and ethnicity.
• Fit Indices: CFI, TLI, RMSEA, SRMR.

3. Key Contributions

1. First Quantitative Mechanism Model: This is the first study to formally model the psychological pathway from AI literacy to sustained AI-CBT engagement using SEM, moving beyond simple correlations to test mediation and moderation.
2. Dual-Pathway Validation: It empirically validates two parallel mediators: Trust (cognitive) and Therapeutic Alliance (affective/relational), demonstrating that literacy works by improving both the user's belief in the system's competence and their emotional bond with it.
3. Stigma as a Boundary Condition: It identifies mental health stigma as a significant moderator that attenuates (weakens) the effectiveness of trust and alliance in driving engagement, providing a nuanced view of implementation barriers.
4. Generalizability: The findings are robust across different AI tool types (chatbots vs. apps) and demographic groups (gender, ethnicity), supported by measurement invariance testing.

4. Key Results

Model Fit:
The final SEM demonstrated excellent fit:

• CFI = 0.967, TLI = 0.959
• RMSEA = 0.043 (90% CI: 0.036–0.051)
• SRMR = 0.052

Path Coefficients & Mediation:

• AI Literacy significantly predicted both Trust ($\beta = 0.441, p < .001$) and Therapeutic Alliance ($\beta = 0.372, p < .001$).
• Both mediators significantly predicted Sustained Engagement:
  • Trust $\rightarrow$ Engagement ($\beta = 0.483$)
  • Alliance $\rightarrow$ Engagement ($\beta = 0.527$)
• Indirect Effects:
  • Via Trust: $\beta = 0.213$
  • Via Alliance: $\beta = 0.187$
  • Total Indirect Effect: $\beta = 0.400$ (Accounting for 58.4% of the total effect of AI literacy on engagement).
• Direct Effect: Remained significant but reduced ($\beta = 0.148$), indicating partial mediation.

Moderated Mediation:

• Stigma significantly moderated the Trust $\rightarrow$ Engagement pathway ($\Delta R^2 = 0.042, p = .003$) and the Alliance $\rightarrow$ Engagement pathway.
• Conditional Effects: The indirect effect of AI literacy on engagement was strongest for individuals with low stigma ($\beta = 0.306$) and significantly attenuated for those with high stigma ($\beta = 0.120$), though still statistically significant.
• Index of Moderated Mediation (IMM): Significant for both pathways (Trust: $-0.094$; Alliance: $-0.078$), confirming that stigma disrupts the translation of literacy into engagement.

5. Significance and Implications

Theoretical Significance:
The study extends Technology Acceptance Models (TAM) and therapeutic alliance theory to the AI domain. It confirms that AI literacy is a modifiable upstream determinant that fosters the "human-like" factors (trust and alliance) necessary for sustained use, even with non-human agents. It also highlights that parasocial relationships with AI are measurable and critical for retention.

Practical & Policy Implications:

1. Implementation Strategy: To scale AI-CBT in the NHS and globally, interventions must go beyond tool deployment. AI literacy training (explaining how AI works, its limits, and data usage) should be embedded in onboarding to boost trust and alliance.
2. Design Optimization: Developers should prioritize features that mimic alliance-building behaviors (reflective listening, goal alignment) as these are the strongest predictors of retention.
3. Stigma Reduction: For populations with high mental health stigma, AI literacy alone is insufficient. Implementation must include stigma-reduction interventions (e.g., peer testimonials, normalizing messaging) to prevent the attenuation of engagement pathways.
4. Equity: The model's invariance across demographics suggests these mechanisms are universal, but the stigma moderation effect suggests targeted support is needed for stigmatized groups to achieve equitable engagement levels.

Limitations:
The cross-sectional design limits causal inference (though SEM strengthens the argument), and self-reported data may introduce bias. Future research requires longitudinal designs to confirm causal chains.