OpenAI’s demonstrated near-expert levels in assessing eye problems

Recent advancements in artificial intelligence have once again spotlighted its potential in the medical field, particularly in ophthalmology. OpenAI's GPT-4 model, backed by Microsoft, has demonstrated near-expert levels of performance in diagnosing and suggesting treatments for eye conditions '' according to a new study published in the PLOS Digital Health journal.

How does it work?

The GPT-4 model was rigorously tested using 87 different patient scenarios, ranging from non-specialist junior doctors to both trainee and expert ophthalmologists. This research uniquely pits the AI's capabilities directly against practicing medics rather than merely comparing it to examination results.

GPT-4's performance was commendable, matching or even surpassing all but the top expert doctors.

Notably, it excelled in both basic recall tasks and complex reasoning challenges, demonstrating an impressive ability to interpolate, interpret, and process information.

Why does it matter?

The implications of such technology are profound.

"What this work shows is that the knowledge and reasoning ability of these large language models in an eye health context is now almost indistinguishable from experts," remarked Arun Thirunavukarasu, the study's lead author.

As AI continues to evolve, it could potentially assist in triaging patients and extending access to specialist healthcare in areas where it is limited.

The study emphasizes AI's broad generative capabilities, which could revolutionize medical diagnostics and treatment plans.

The context

Ophthalmology is at the forefront of integrating AI into clinical practice, aiming to overcome historical barriers such as the tendency of AI models to generate inaccurate data, a phenomenon known as "hallucinating." This study is part of a broader movement within the medical community to leverage AI for diagnostics, exemplified by AI's role in detecting early-stage diseases that might be overlooked by human doctors. However, as researchers like Pearse Keane of University College London caution, there remains a delicate balance between harnessing AI's potential benefits and managing the risks associated with false diagnoses.

Keane finds the study's approach "super-interesting," yet he underscores the need for caution. He shares an anecdote from his own research: when he queried an AI about macular degeneration, it returned "made-up" references. This underscores the critical need for ongoing refinement of AI models to ensure their reliability in clinical settings.