Comparative performance of DeepSeek-R1 and GPT-4o on standardized ophthalmic diagnostic and treatment tasks

In the modern clinic, a doctor's mind is a powerful tool, trained over years to connect a patient's story, physical signs, and test results into a clear picture of what is wrong. This process of clinical reasoning is the backbone of medicine, yet it is also a skill that takes a long time to master. Recently, a new kind of digital mind has emerged: large language models. These are artificial intelligence systems trained on vast amounts of text, capable of reading, understanding, and generating human language with remarkable fluency. They have shown promise in helping with everything from writing letters to answering general questions. But in a field as precise and visual as eye care, where a diagnosis can depend on subtle details in an image or a specific pattern of symptoms, it remains unclear how well these digital assistants truly understand the complexities of disease. The question is no longer just whether they can speak the language of medicine, but whether they can think like a specialist when faced with a difficult case.

A team of researchers set out to test this question using two of the most advanced artificial intelligence systems available: DeepSeek-R1 and GPT-4o. They chose to focus on ophthalmology, the branch of medicine dedicated to the eye, because it requires a high level of specialized knowledge and the ability to weigh many different pieces of information. To create a fair and rigorous test, the team did not ask the computers to look at real patients or actual eye scans. Instead, they used thirty-three standardized case studies drawn from a national database used to train and examine eye doctors in China. These cases covered a wide range of conditions, from retinal diseases and glaucoma to injuries and tumors, representing the kinds of scenarios a specialist might encounter in their daily work. The researchers asked both artificial intelligence models to act as senior eye doctors. They provided the text of each case and requested a diagnosis, a list of other possible conditions that could explain the symptoms, and a plan for treatment.

To ensure the results were objective, three experienced, board-certified eye doctors reviewed the answers provided by the two artificial intelligence systems. The experts did not know which model had written which answer. They scored each response based on two main criteria: accuracy and completeness. Accuracy measured how correct the medical advice was, while completeness measured whether the answer covered all the necessary details without leaving anything out. The experts used a simple scoring system, giving points for how well the answers matched the standard, correct solutions found in the training database. This process allowed the researchers to compare the performance of the two digital minds directly against the gold standard of human expertise.

The results showed a clear difference between the two systems. Across all the cases they reviewed, DeepSeek-R1 consistently provided answers that were more accurate and more complete than those from GPT-4o. When the experts looked at the specific task of identifying the correct disease, DeepSeek-R1 received an average score of 8.04 out of 10, while GPT-4o scored 6.46. The gap was even wider when the models were asked to list other possible conditions, a task known as differential diagnosis. Here, DeepSeek-R1 achieved a score of 7.52 compared to 5.50 for GPT-4o. In terms of treatment plans, DeepSeek-R1 again outperformed the other model, scoring 7.62 versus 6.65. The difference was not just in the final numbers; the experts also noted that DeepSeek-R1's answers were more thorough, covering more of the required elements in its explanations.

The study found that the advantage of DeepSeek-R1 was most pronounced in areas of eye care that require complex, multi-step thinking. Conditions such as diseases of the retina, glaucoma, disorders of the optic nerve, and issues with eye alignment and vision development showed the largest performance gap. These are the types of cases where a doctor must synthesize many different clues to reach a conclusion. In contrast, for conditions with more obvious or distinct signs, such as certain eye injuries or surface diseases, the difference between the two models was smaller. This suggests that the specific way DeepSeek-R1 was trained, with a heavy focus on advanced reasoning, may give it an edge in situations where the answer is not immediately obvious and requires careful deduction.

Despite these promising results, the researchers were careful to frame their findings as a preliminary step rather than a final verdict. The study was limited by the fact that it only used text-based cases, meaning the artificial intelligence systems never saw actual images of eyes, which are critical for real-world diagnosis. The sample size of thirty-three cases was also relatively small, and the evaluation relied on the subjective judgments of three human experts. The authors emphasized that these results do not mean that either artificial intelligence system is ready to replace a human doctor or to be used directly in a hospital to make decisions for patients. The performance of these models can change rapidly as they are updated, and the specific versions tested in early 2025 may not reflect the capabilities of future systems.

Ultimately, this work serves as an important benchmark for understanding where artificial intelligence stands in the specialized field of eye care. It demonstrates that one of the newer models, DeepSeek-R1, has shown a stronger ability to reason through complex medical cases than the established GPT-4o in this specific context. However, the study concludes that much more work is needed. Future research must test these systems with real images and in live clinical settings to see if they can truly support doctors in the messy, complex reality of patient care. For now, the findings suggest that while artificial intelligence is becoming a powerful tool for medical reasoning, it is still a work in progress, and its role in the clinic will depend on continued validation and careful integration with human expertise.