Margie Smith moved from one specialist’s waiting room to another for the better part of two years. An allergist for the persistent cough. The same tests were ordered by three different pulmonologists. a cardiologist after she almost passed out while taking a morning stroll. She was 70 years old, still ill, and growing impatient with medical professionals who seemed adept in their own small field but were unable to see the big picture by the time she had exhausted the majority of what the medical system had to offer.
She launched a chatbot as a result. She’s clear that she didn’t fully trust it, but rather because nothing else was working. She came to the conclusion that long COVID was causing dysautonomia, a condition where the body loses control over basic functions like heart rate, digestion, and blood pressure, after having lengthy conversations with Claude and participating in a Facebook group where other patients were solving similar puzzles. Until you read the literature, which Smith had been reading extensively, it seems fringe.
She now brings AI-generated recommendations to doctor’s appointments. She chooses her physicians in part because of their willingness to deal with that. “The medical system really failed me,” she replied. “Is relying on AI for medical advice a good thing? Not in my opinion. However, it’s an option. That sentence has a subtle, devastating quality that isn’t exactly bitterness. It’s more like tired clarity.
The story that resulted from the New York Times’ months-long investigation into this phenomenon is more difficult to discount than it might initially seem. Now, one-third of American adults claim to have used AI chatbots to get health advice. However, one subset stood out: women with complicated, long-term illnesses that require years to diagnose, involve several specialties, and are statistically more likely to be downplayed by doctors. When The Times issued a call, hundreds of people responded. After that, it interviewed dozens of people. What emerged was not a tale of careless patients refusing medication. There was more discomfort than that.
Patty Costello, an Idaho user experience researcher, had been experiencing unexplained episodes of nausea, exhaustion, and inflammation for over ten years. Last year, when she entered her symptoms into ChatGPT and basically said, “This is ruining my life,” the chatbot suggested nine possible diagnoses. Mast cell activation syndrome was one of them. Costello researched it, felt something click, saw an allergist, had the diagnosis verified, began treatment, and is now about 80% better. That’s a big deal. It’s also not the whole story.
Less than half of the time did non-medical professionals who used chatbots to diagnose patients get the right answers, according to a February study. Chatbots experience hallucinations. They don’t flag low-quality sources that they pull from. When they are simply incorrect, they can sound authoritative. Deborah Holcomb, a former electrical engineer with chronic fatigue syndrome, discovered that ChatGPT suggested frequent exercise, which is practically a clinical error given her condition. She pointed out that some physicians make the same error, but that hardly improves the situation.
This isn’t actually an AI story at all, as the Times investigation subtly reveals. It tells the tale of a healthcare system that has historically performed poorly for patients with complex, multi-system illnesses, especially women. After speaking with ChatGPT for more than 12,000 words, Caroline Gamwell, a pelvic floor physical therapist in Denver, came to a vascular diagnosis that was later confirmed by surgery. Because of her anatomical training, she was able to reject the chatbot’s implausible suggestions and pursue the ones that made sense. “How many people,” afterwards, “would have realized that several of the suggestions made no sense?” It’s the right question, and there’s no easy way to answer it.
As one Stanford AI researcher put it simply, there is a reason why people are doing this. It has nothing to do with a passion for technology. The specific desperation of being ill, being written off, and still needing a place to turn at the end of the day is more akin to necessity.
