Reporting of AI and ML Studies in Elderly Care Settings: A Rapid Systematic Scoping Review

Imagine a world where technology quietly watches over the elderly, not to replace human care, but to support it. In nursing homes, assisted living facilities, and private residences, computers are increasingly being asked to do things like spot a fall before a caregiver arrives, predict when a resident might need more help, or monitor daily routines for signs of trouble. These systems rely on artificial intelligence and machine learning, two terms that often sound like science fiction but are, in practice, simply ways for computers to learn from data to make predictions or spot patterns. The hope is that these tools can make care safer and more efficient. However, for these tools to be trusted and used effectively, the people building them must explain exactly how they work, what data they learned from, and where they might fail. Without clear explanations, a system that looks perfect on paper might be useless or even dangerous in the real world.

A team of researchers from Kempten University of Applied Sciences recently decided to look closely at how scientists are currently reporting on these artificial intelligence projects for the elderly. They did not build a new system or test a specific device. Instead, they acted as auditors of the literature, gathering and examining 69 recent studies published since 2020. Their goal was to see if the authors of these studies were providing enough information for others to understand the systems they created. They wanted to know if the reports included details about who would use the technology, what kind of data trained the computer, and whether the system had been tested outside of a controlled lab environment.

What they found was a landscape of uneven reporting. While the researchers were generally good at describing the basic purpose of their systems and where they got their data, they often left out the crucial details that determine whether a system can actually work in a busy nursing home or a quiet living room. For instance, almost every study described what the computer was supposed to do, but fewer than half clearly explained who the human decision-maker would be. In many cases, it was unclear whether the system was meant to alert a nurse, a family member, or the resident themselves, or how that person was supposed to react to the alert. This lack of clarity makes it difficult to know how the technology fits into the complex, human-centered flow of daily care.

The gap widened when the researchers looked at how these systems were tested. Most of the studies relied on internal validation, meaning the computer was tested on the same data it had already seen or on data that was very similar to what it learned from. This is like a student taking a practice test using the exact same questions they studied. While this shows the computer can recognize patterns, it does not prove it can handle the messy, unpredictable reality of a new environment. Only a small fraction of the studies, just 16 percent, reported testing their systems on completely new, independent data. Even fewer, about 20 percent, described testing the system in a real-world care setting where it interacted with actual patients and staff. Without these real-world tests, it remains uncertain whether the systems will perform as well when faced with the noise, movement, and variety of a real home or facility.

Perhaps the most concerning omission was the lack of discussion regarding uncertainty and bias. When a computer makes a prediction, it is rarely 100 percent certain, and it can sometimes be wrong in ways that affect specific groups of people more than others. The researchers found that only 9 percent of the studies explained how the system handled uncertainty or how confident it was in its answers. Similarly, only 23 percent of the studies checked if the system performed differently for different subgroups, such as men versus women or people with different types of mobility issues. Without this information, there is no way to know if a system is fair or reliable for all the people who might use it. The studies also rarely discussed what happens when data is missing, a common occurrence in real life, leaving a significant blind spot in how these systems would behave when sensors fail or records are incomplete.

The authors of the review suggest that the current way these studies are written focuses too much on the technical construction of the computer model and not enough on how the system fits into the broader human environment. Existing guidelines for reporting medical research, which are designed for diagnostic tools like X-rays or blood tests, do not seem to fully cover the unique challenges of elderly care, where the "patient" is often a whole community of people with varying needs, and the "diagnosis" is often a continuous stream of behavioral data. The review indicates that while the technology is advancing rapidly, the way researchers communicate its readiness and limitations has not kept pace. Until the reports include clear details about who uses the system, how it was tested in the real world, and where it might fail, it will be difficult for caregivers and institutions to know if these tools are truly ready to be trusted with the safety and well-being of the elderly.