Quality of Olfactory Dysfunction Information Across Digital Health Platforms: A Comparative Study of Google Search and Artificial Intelligence Platforms
This study reveals that while Google Search occasionally provides high-quality olfactory dysfunction information, current AI platforms and digital health sources generally offer only moderate-to-low quality content with poor readability, necessitating clinician guidance toward validated specialty resources.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine your nose is like a high-tech security system for your brain. When it works, it doesn't just tell you that pizza smells delicious; it warns you about gas leaks, smoke, and spoiled milk. But for about one in five people, this alarm system is broken or completely silent. This condition, called olfactory dysfunction, can make life feel a bit like living in a foggy, flavorless world, affecting how you eat, socialize, and even feel safe.
When people with this problem get confused or worried, they often turn to the internet for answers, just like you might look up a video game strategy or a recipe. In the past, they would type a question into a search engine like Google and hope to find a trustworthy doctor's advice hidden among the ads and random blogs. But recently, a new kind of helper has arrived: Artificial Intelligence (AI) chatbots. These are like super-smart, talking robots that can read millions of books and then write a custom answer for you in seconds. The big question is: if you ask these robots about your lost sense of smell, will they give you the truth, or will they just make things up? This study dives into that exact mystery, comparing the old-school search engine against the new AI giants to see who actually knows what they are talking about.
The Great Internet Smell-Off: Humans vs. Robots
The researchers, a team of curious scientists from the University of Sheffield, decided to put five different digital platforms to the test. They wanted to see which one could give the best, safest, and easiest-to-understand advice about losing your sense of smell. The lineup included the classic Google Search (the king of finding links) and four AI chatbots: ChatGPT, Gemini, Claude, and Perplexity.
To make it a fair fight, they asked all five platforms the same eight questions. Some questions were fancy and medical, like "What causes parosmia?" (a fancy word for when smells get twisted), while others were plain and simple, like "I can't smell anything, what's wrong?" They then acted like strict judges, scoring the answers on two main things: Quality (Is the advice safe and accurate?) and Readability (Is it written in a way a normal person can understand?).
The Results: The Robots Stumble, Google Wins (Sort Of)
Here is the twist: None of the AI robots produced a single "high-quality" answer.
When the judges scored the answers using a strict checklist called DISCERN, the results were a bit of a shock.
- Perplexity (the AI that looks up facts in real-time) came in first place among the robots with a score of 50.56 ± 5.74. That sounds good, but it's only "moderate" quality.
- ChatGPT and Gemini were close behind, scoring around 43 and 45 respectively.
- Claude was the lowest scorer at 38.13 ± 4.57, with a whopping 75% of its answers rated as "low quality."
The only platform that managed to find a few "high-quality" sources was Google Search. Out of all the answers Google gave, only 3.75% (three tiny slices of the pie) were considered truly excellent. The rest were mostly "moderate" or "low." So, while Google didn't win a gold medal, it was the only one to even touch the podium.
The Missing Pieces: What the Robots Forgot
The study found a huge, gaping hole in almost every answer, whether it came from a robot or a website. The platforms were terrible at answering the most important questions for a patient:
- What are the risks of the treatment? (Score: 1.22 ± 0.61 out of 5)
- What happens if I do nothing? (Score: 1.57 ± 0.79 out of 5)
- How will this change my life? (Score: 1.29 ± 0.71 out of 5)
It's as if a mechanic told you how to fix your car but refused to tell you if the fix was dangerous, what would happen if you didn't fix it, or how the car would drive afterward. The robots were happy to list symptoms, but they completely skipped the part about making life better or safer.
The Reading Level Problem: Too Hard to Read
There was another problem: the answers were too complicated. The researchers measured how hard the text was to read, using a scale where 6–8 is the recommended level for patient information (think middle school reading).
- The average reading level across all platforms was 11.01 (high school level).
- Only 27.7% of the sources met the easy-to-read target.
- Claude and Perplexity were the worst offenders, with reading levels of 14.66 and 14.30 respectively. That's college-level reading!
The study found a funny, sad pattern: the better the information was (higher quality), the harder it was to read. It's like the smartest doctors were writing in a secret code that only other doctors could understand, leaving the patients in the dark.
The Verdict: Don't Trust the Robot Just Yet
So, what's the takeaway? If you lose your sense of smell, you can't just ask an AI chatbot and expect a perfect, safe answer. The robots are currently consistently failing to meet the standards needed for medical advice, especially when it comes to telling you the risks or how treatment affects your daily life.
The study suggests that doctors shouldn't just tell patients to "look it up online." Instead, they should point patients toward specific, trusted resources created by real medical experts and patient charities. While AI might be a cool starting point to get a general idea, it's not ready to be the final word on your health. Until these robots learn to write clearly and tell the whole story (including the scary parts), it's best to let a human doctor be the one to guide you through the fog.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.