Early Working Alliance Mediates the Association between Engagement and Outcomes among Users of an AI Mental Health Conversational Agent: A Secondary Hierarchical Clustering Analysis of a Randomized Controlled Trial
This study demonstrates that early working alliance, rather than sustained behavioral engagement volume, fully mediates the relationship between user engagement and clinical outcomes in an AI mental health conversational agent, suggesting that onboarding strategies fostering alliance formation are more critical for symptom improvement than merely increasing usage frequency.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the expanding world of digital mental health, a persistent puzzle has long challenged researchers and developers alike: why do some people find deep relief in a smartphone app while others download the same tool, use it for a few days, and then abandon it without feeling better? For years, the industry operated on a simple assumption—that the more a person uses an app, the better their results will be. This logic suggests that if a user opens the application frequently, stays for long sessions, and completes every exercise, they are on a direct path to healing. However, this view overlooks a crucial human element that has been central to traditional therapy for decades: the relationship between the person seeking help and the source of that help. In face-to-face counseling, this connection is known as the working alliance, a bond built on trust, shared goals, and agreement on the methods used to achieve change. It is the quiet understanding that the therapist and the client are on the same team. As artificial intelligence enters the mental health space, offering support through chatbots and voice assistants, a critical question arises: can a machine form this kind of alliance, and if it does, does that relationship matter more than the sheer number of times a user clicks a button?
A team of researchers set out to answer these questions by studying a specific AI emotional support application called Sonia. They recruited two hundred adults in the United States who were experiencing significant anxiety and assigned them to use the app for four weeks. Instead of simply counting how many times each person opened the app or how long they stayed logged in, the researchers looked for patterns. They gathered a vast amount of data, tracking daily behaviors like journaling, reflecting on the day, and having conversations with the AI, while also asking users how much they agreed with the app's goals, how much they trusted its methods, and how connected they felt to it. By analyzing these behaviors and feelings together, the researchers discovered that users did not fall into a single group of "active" or "inactive" people. Instead, four distinct profiles emerged. There were "Sporadic Users" who tried the app briefly and then stopped; "Moderate Session Users" who engaged regularly but only with a few features; "Engaged Multi-Feature Users" who used the app consistently across many different tools; and "Power Users" who maintained high levels of activity throughout the entire month.
The researchers then examined what these different patterns meant for the users' mental health. They looked at whether the people who used the app the most saw the biggest drops in their anxiety and depression scores. Surprisingly, the data showed that the volume of use did not predict clinical improvement. A person who used the app heavily was not necessarily more likely to feel better than someone who used it only a little. In fact, the groups that used the app the most did report higher satisfaction and were more likely to recommend the app to others, but their actual symptoms of anxiety and depression improved at the same rate as those who used it less. This finding suggests that for this type of tool, simply logging more hours does not drive healing. The key to improvement lay elsewhere.
When the researchers dug deeper, they found a potential mechanism at work: the early working alliance. This is the sense of connection and agreement a user feels with the AI very early in the process. The study revealed that this relationship, established within the first week of use, may act as a pathway through which behavioral engagement leads to symptom improvement. However, the researchers noted that the study was not large enough to definitively confirm this link, and these mediation results should be viewed as exploratory and hypothesis-generating rather than settled fact. In other words, the data suggests that using the app helped people feel better primarily through the formation of a strong, trusting connection with the AI, but this conclusion requires further confirmation. If a user opened the app many times but never felt that the AI understood them or that its methods were right for them, those extra clicks did not translate into healing. The researchers found that the quality of this initial connection was far more important than how much the relationship changed over time. Once a user felt a sense of trust and shared purpose in the first week, that feeling remained stable and drove their progress for the rest of the month.
The study also looked at who was likely to fall into each of the four user groups. It turned out that the severity of a person's anxiety at the start did not predict how they would engage with the app. Instead, demographic factors played a larger role. Older adults and those with higher levels of education were more likely to become the consistent, multi-feature users who formed strong alliances. This suggests that the risk of dropping out of a digital mental health program might be identified through simple background information rather than clinical symptoms. The researchers noted that while their findings are promising, the study was not large enough to be considered a final, definitive proof, and the results should be viewed as a strong suggestion rather than a settled fact. However, the pattern was clear enough to shift the focus of how these tools are built.
The implications of these findings are significant for the future of digital mental health. For a long time, product developers have focused on keeping users on the app for as long as possible, believing that more time equals better health. This study suggests that the most critical moment is not the middle of the month or the end of the program, but the very beginning. The first week of interaction is where the relationship is forged. If an app can help a user feel understood and convinced that its methods are credible right from the start, that initial connection may carry them through to a better outcome. If that connection is missing, no amount of extra usage will compensate for it. The path to healing in an AI-driven world appears to depend less on the quantity of interaction and more on the quality of the first handshake.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.