ChatGPT (still) can’t diagnose you

We've all done it — typed our symptoms into a search bar, hoping the internet will tell us what's wrong. From "Dr. Google" to the rise of ChatGPT, the web is teeming with health advice. But here's the rub: while artificial intelligence can talk the talk, it's still no doctor. Experts warn that when it comes to your health, algorithms are a shaky substitute for medical judgment.

Ahmed Abdeen Hamed, a research fellow at Binghamton University, recently led a study to see just how good ChatGPT really is at diagnosing disease. "People talk to ChatGPT all the time these days," he said. "'I have these symptoms. Do I have cancer? Do I have cardiac arrest?'" His findings? The chatbot's bedside manner might charm, but its diagnostic accuracy doesn't yet make the cut.

Dr. Andrew Thornton, a Wellstar physician in Atlanta, put it bluntly: "That is the time to call 911, or get someone to take you to a hospital immediately." In short — AI might be clever, but it can't save your life.

How does it work?

The study, published in iScience, tested how well ChatGPT could identify diseases, drugs, and genes. When it came to factual recall, the model shone — scoring between 88% and 97% accuracy. It was sharp at spotting terms like diabetes or aspirin, but less so when faced with real-world complexity.

"The diseases were really very easy to identify," Hamed said. "Same story with the drugs, same story with genes, but not the same story with symptoms."

Here's where things went south. When users phrased symptoms casually — like "my chest feels tight" or "I've had this weird headache all week" — ChatGPT stumbled. It often failed to connect vague or conversational language to the right medical causes.

Hamed explained why: "ChatGPT uses more of a friendly and social language... it started to minimize the formalities of medical language to appeal to those users." In trying to sound human, the AI simplified too much — and accuracy took a hit.

Worse still, the chatbot doesn't show uncertainty. "It's going to present information in a way that sounds very confident," Dr. Thornton said. "And it will do so the same way with inaccurate information as it does with accurate information." Confidence without clarity — dangerous territory when your health's on the line.

Why does it matter?

That unearned confidence can mislead users into believing false information. In a world where people already self-diagnose at the first sniffle, an overconfident chatbot can quietly amplify risk.

Surveys show how common this behavior has become:

• About 34% of U.S. adults have used ChatGPT — double the rate from 2023.
• 17% of adults use AI chatbots monthly for health information.
• Among adults under 30, that number jumps to 25%.

Thornton says patients now freely admit to googling symptoms before visiting him. "I find that patients are more forthcoming now than 10 years ago," he noted. "Because they know we expect it."

So the question isn't if people will use AI for health — it's how they'll do it responsibly.

The context

AI isn't the villain here. It's just not ready for the white coat. Used wisely, it can be an educational companion — helping users understand medical terms, treatment options, or the basics of a condition.

"I think that the internet and AI platforms can be used to add information," said Thornton, "to give more context about different disease states." But he was clear about its limits: "It really needs to be used for general knowledge... not to narrow down possible diagnoses."

The takeaway? Treat ChatGPT like an eager intern — great at explaining things, terrible at making the final call. The future might bring smarter, medically verified AI assistants. But for now, when your health's on the line, trust your doctor, not your chatbot.