Scoping review of governance, privacy, and design in AI- supported conversational agents for perinatal mental health
This scoping review of 35 studies reveals that while AI-supported conversational agents for perinatal mental health are rapidly evolving toward advanced architectures, they suffer from a critical "AI perinatal paradox" characterized by inadequate governance, privacy safeguards, and inclusivity, prompting the proposal of eight minimum standards to prioritize safety and equity in their development.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Mental health challenges during pregnancy and the first year after birth are common, affecting a significant portion of parents worldwide, yet many struggle to find timely help. In recent years, technology has offered a potential solution through conversational agents: computer programs designed to chat with users via text or voice, providing information, emotional support, and guidance. These digital tools promise to be available at any hour, offering a private space for parents to discuss sensitive fears about their mental state or their baby's well-being. However, because these conversations can involve deeply personal and sometimes dangerous topics, such as thoughts of self-harm or violence, the safety and privacy of these systems are critical. The question is no longer just whether these chatbots can talk, but whether they are built and governed well enough to be trusted with the most vulnerable moments of a parent's life.
A team of researchers at James Cook University in Australia set out to examine the current state of these artificial intelligence tools specifically for perinatal mental health. They conducted a wide-ranging review of scientific studies published between 2015 and 2025, searching through thousands of records to find reports on how these chatbots are designed, how they handle user data, and how they are tested. The team analyzed 35 studies that described 30 unique chatbot systems used in 14 different countries. Their goal was to map the landscape of these tools, looking beyond just whether they work to understand who they serve, how they are protected, and what happens when a user is in crisis.
The researchers found that the technology behind these chatbots is changing rapidly. For a long time, most systems relied on simple, rule-based instructions, where the computer followed a fixed set of "if this, then that" commands to generate responses. However, in the last few years, newer systems have begun using large language models, which are advanced computer programs trained on vast amounts of text to generate more natural and flexible conversations. While these newer tools can sound more human and adapt better to a user's specific situation, the review revealed a troubling disconnect. The systems using the most advanced technology often had the weakest safety and privacy protections.
When the researchers looked at how these systems handle private information, the picture was stark. None of the studies reported using advanced methods to keep data secure, such as processing information directly on the user's phone without sending it to a central server, or using mathematical techniques to hide individual details while still allowing the computer to learn. Instead, nearly half of the systems relied on commercial services provided by outside companies to process the conversations. This means that sensitive details about a parent's mental state were often sent over the internet to third-party servers, with little transparency about where that data went, who could see it, or how long it was kept. Furthermore, most studies did not clearly explain how they obtained permission from users to use their data, and very few described how they would update the system's behavior after it was launched.
The review also highlighted a significant gap in who these tools are designed to help. The vast majority of the chatbots were built exclusively for mothers, often ignoring fathers, same-sex partners, or non-birthing parents. Only a handful of systems attempted to include fathers, and even fewer considered the complex relationship between partners. Additionally, most tools operated only in English and required a strong internet connection, leaving out parents in remote areas or those with limited access to technology. This narrow focus means that the benefits of these digital tools are not reaching the diverse families that need them most.
Perhaps the most critical finding concerned how these systems handle emergencies. When a parent expresses thoughts of suicide or harm to their baby, the response of the chatbot is a matter of life and safety. The researchers found that while some systems could automatically detect these crisis signals, almost none had a reliable plan to connect the user to a human helper. In many cases, the system might flag a crisis but then do nothing, or it might provide a generic list of phone numbers without ensuring the user actually reached a person. Only one study described a system that guaranteed a human would follow up with a user within a specific time frame after a crisis was detected. The others operated without this safety net, leaving users in danger with no clear path to professional help.
The researchers described this situation as a paradox: as the technology becomes more sophisticated and capable of complex conversation, the safeguards that make it safe to use are becoming less consistent. The older, simpler systems often had clearer rules for privacy and human oversight, while the newer, more flexible systems were often built with less attention to these critical protections. The review concluded that the rapid development of these tools has outpaced the establishment of responsible standards. The authors propose a set of minimum safety rules for the future, suggesting that developers, doctors, and policymakers must prioritize safety, privacy, and inclusivity from the very beginning of the design process. They argue that for these tools to be truly helpful, they must be proven safe and effective for real people in real crises before they are widely trusted, ensuring that the promise of digital support does not come at the cost of parental safety.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.