ChatGPT Health Tool Isn't So Great in a Crisis

A tool billed as a way to plug your medical records into ChatGPT and receive health advice is drawing sharp warnings from researchers. In the first independent safety review of ChatGPT Health, published in Nature Medicine, the system underestimated the urgency of care in just over half of cases where doctors said hospital treatment was needed immediately, reports the Guardian. Researchers built 60 patient simulations, then compared ChatGPT Health's triage advice with three physicians' consensus decisions. The AI did fine with classic emergencies like strokes, but it faltered elsewhere: In one mock asthma crisis, it told the patient to wait rather than seek urgent care.

In life-threatening situations such as respiratory failure or diabetic ketoacidosis, it downplayed symptoms about 50% of the time, one expert said, and it often became even more reassuring if a "friend" in the scenario suggested things weren't serious. The system also inconsistently flagged suicidal ideation, dropping crisis warnings once normal lab results were added. OpenAI said the study doesn't mirror real-world use and that ChatGPT Health is continually updated, but outside experts are calling for stronger safeguards and independent oversight before people rely on it. The AP has more advice for those who lean into AI for health care.