Comparison of Influenza Vaccine Effectiveness Estimates Using Test-Negative Prospective Enrollment and Electronic Health Record Data, United States, 2024--2025
This study found that influenza vaccine effectiveness estimates derived from retrospective electronic health record data were generally consistent with those from prospective test-negative enrollment in the United States during the 2024–2025 season, although notable discrepancies emerged specifically for adults aged 65 and older.
Original paper dedicated to the public domain under CC0 1.0 (https://creativecommons.org/publicdomain/zero/1.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
Imagine the flu season as a massive, chaotic dance party where a sneaky virus tries to crash the party and make everyone sick. Every year, scientists act like the bouncers and the party planners, trying to figure out if the "vaccine shield" people wear actually stops the virus from getting in. To do this, they usually play a game of "detective." They find people who came to the doctor because they felt terrible (coughing, fever, the works) and ask, "Did you get the flu shot?" Then, they test those people to see if they actually have the flu. If the people who don't have the flu are more likely to have had the shot than the people who do have the flu, the shield is working!
But there's a catch: finding these sick people and testing them one by one is like trying to count every single grain of sand on a beach by picking them up with tweezers. It takes forever and requires a huge team of people running around with swabs. Recently, a new idea popped up: what if we just looked at the giant digital "diary" that doctors keep for every patient? These electronic health records (EHR) are like massive, automated libraries containing millions of entries about who got sick, who got tested, and who got vaccinated. The big question scientists have been asking is: Can we trust this giant digital library to tell us the same story as the hard-working detectives with the tweezers? If the library says the shield works, does the detective agree?
This paper is the story of a massive showdown between those two methods during the 2024–2025 flu season in the United States. The researchers decided to pit the "Old School Detectives" (the US Flu VE Network, who went out and personally enrolled thousands of patients) against the "Digital Librarians" (who scanned millions of electronic health records). They wanted to see if both groups would come up with the same answer about how well the flu vaccine worked.
The results were surprisingly harmonious. When the "Old School Detectives" looked at 6,793 patients they personally interviewed and swabbed, they found that the flu vaccine was about 36% effective at preventing people from getting sick enough to visit the doctor. When the "Digital Librarians" scanned a whopping 282,444 patient visits from their electronic records, they found the vaccine was 38% effective. That's a very small difference! It's like two different weather forecasters looking at the same storm and both agreeing it's going to rain, even if one says "heavy rain" and the other says "moderate rain."
However, the story gets a little twisty when you look at the older crowd. For most age groups, from kids to adults under 65, both methods agreed perfectly. But when it came to adults aged 65 and older, the two methods started to sing different tunes. The "Digital Librarians" still saw a solid 35% protection. But the "Old School Detectives," who had a much smaller group of older people to study, couldn't say for sure if the vaccine was working or not; their numbers were so wobbly that the effectiveness looked like it could be anywhere from -3% (which would mean the vaccine did nothing or maybe even made things slightly worse, though that's likely just a statistical fluke) to 30%.
The paper suggests that while the digital library method is a powerful, fast, and accurate tool that matches the traditional detective work for most people, we need to be extra careful when looking at the oldest adults. The digital records gave a clear, confident answer for them, but the traditional method was too small to be sure. The authors aren't saying one method is "wrong" and the other is "right," but rather that the digital approach seems to be a reliable, high-speed alternative that can handle huge numbers of people. They suggest that as we get more comfortable with these digital records, we might be able to use them to track the flu shield's strength much faster than before, as long as we keep an eye on those tricky older age groups where the numbers can get a bit fuzzy.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.