Premarket transparency and postmarket observability in FDA-authorized AI-enabled medical devices: a source-linked cross-sectional study
This cross-sectional study of FDA-authorized AI-enabled medical devices reveals that while premarket reporting completeness has significantly improved in recent years, prospective evidence remains infrequently documented and postmarket observability is uneven, underscoring the need for transparency measures to support evidence monitoring rather than device rankings or unsupported safety conclusions.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of medical technology as a massive, bustling library where every new invention gets a special ID card. For years, the most exciting new books in this library have been written by "smart" computers—artificial intelligence (AI) that can help doctors spot diseases, read X-rays, or monitor heartbeats. But here's the tricky part: just because a book has a shiny new cover doesn't mean we know exactly what's written inside, or if the story holds up when the lights go down and the real world gets messy.
To understand if these AI doctors are ready for the job, we need to look at two different things. First, there's the "Premarket" check. This is like the author showing their homework to a strict librarian (the FDA) before the book hits the shelves. We want to know: Did they test it on different kinds of people? Did they try it in many different hospitals? Did they show the math behind their answers? Second, there's the "Postmarket" check. This is what happens after the book is sold. It's like watching a library's "Reader Feedback" box. If a book has a typo or a confusing chapter, do people write in to complain? If the library doesn't have enough feedback forms, or if the forms are missing, we can't really tell if the book is safe for everyone. The big question is: Are the new AI medical devices getting better at showing us their homework, and can we actually see their feedback forms once they start working?
The Homework Check: Are the Authors Showing Their Work?
In this study, a researcher named Olga Lavinda decided to play the role of a very thorough librarian. She went through the FDA's public list of AI medical devices, looking at 1,491 different decisions made between 2011 and March 2026. She wasn't checking if the devices were perfect; she was just checking if the public summaries (the "homework" handed to the librarian) actually said the important things.
She created a "report card" with seven key items she wanted to see:
- Did they mention using real patient data?
- Did they give at least one clear score on how well the AI worked?
- Did they describe who the patients were (like age or gender)?
- Did they test the AI in more than one hospital?
- Did they test it on data the AI had never seen before?
- Did they use different types of scanners or machines?
- Did they show how the AI performed on specific groups of people?
The results were a mix of good news and "still working on it" news. In the early days (2011–2020), only about 16.5% of these devices had summaries that checked off five or more of these seven boxes. But by the time we got to 2024–2026, that number jumped to 59.5%. It's like the authors suddenly started writing much more detailed introductions to their stories. They were much more likely to say, "We tested this in three hospitals," or "We made sure it worked on people of different ages."
However, there was one thing that didn't get much better: the "Prospective or Reader Study." This is a fancy way of saying, "Did we test this on real patients as they walked in, or did we have a group of doctors look at the AI's answers to see if they agreed?" This remained rare, hovering around 16–18% across all the years. It's as if the authors are still mostly showing us their practice tests rather than the final exam taken in real-time.
The Feedback Box: Can We See What Happens Next?
Once these devices are out in the wild, the researcher looked at the "Postmarket" side. She tried to link every device to its public feedback history, specifically looking at the MAUDE database (a system where doctors and hospitals report problems).
Here, the story gets a bit more complicated. Out of the 1,477 devices she could link to a feedback record, only about 54% had enough history to even calculate a trend. Think of it like trying to judge a new restaurant. If the restaurant opened last week, you can't really tell if the food is good or bad yet because not enough people have eaten there to leave reviews. The study found that for the newest devices (2024–2026), only about 45% had enough public feedback history to be analyzed. For the older devices (2011–2020), that number was higher at 71.7%.
The researcher also looked at "recalls" (when a device is pulled off the shelf). She found that about 47.7% of the devices had a recall notice somewhere in their product category in the last two years. But she was very careful to say this doesn't mean that specific device was recalled. It's like seeing a "Warning: Check your tires" notice for a whole brand of cars; it doesn't mean your specific car has a flat tire, just that the category has some issues.
The Big Picture
So, what does this all mean? The study suggests that the public "homework" for AI medical devices is getting much more complete. Manufacturers are doing a better job of telling us where they tested their tools and who they tested them on. However, the "final exam" where they test on real patients as they happen is still rare.
Furthermore, even though the homework is getting better, the "feedback box" for the newest devices is often empty simply because they haven't been out long enough to gather a crowd of reviewers. The researcher emphasizes that just because a summary is detailed doesn't mean the device is perfect, and just because we don't see a problem in the public feedback box yet doesn't mean there isn't one—it might just be too early to tell.
In short, the library is getting better at organizing the books and writing clearer summaries, but we still need to wait a bit longer to see if the stories hold up when everyone starts reading them in the real world.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.