31-Year-Old Engineer Suspected to Have Died After Stopping Medication on ChatGPT's Advice; Parents Sue OpenAI

A heartbreaking tragedy has recently sparked global discussion: 31-year-old Chinese female engineer Wan Bowen (transliterated) died suddenly after she trusted medical advice provided by the artificial intelligence (AI) application ChatGPT and stopped taking her anti-epileptic medication without authorization. Her parents have formally sued OpenAI and CEO Sam Altman, alleging that the GPT-4o model practiced medicine without a license and output erroneous medical plans, indirectly causing the patient's death. The exposure of risks in AI healthcare has sounded an alarm for the rapidly developing industry. How can the bottom line of medical ethics and institutional regulation be safeguarded? How can a fulcrum be found for the balance between technological innovation and life safety? The paranoia and distress amplified by AI At age 14, Wan Bowen was diagnosed with epilepsy, and thereafter took anti-epileptic medication long-term; while on medication, her seizure frequency was controlled at several times a month. Last November, Wan Bowen learned that she was going to be laid off by her company. Worried that the medication would affect her verbal expression and cause difficulties in job hunting, she began asking ChatGPT how to reduce the dosage. In January, she asked whether the daily 400 milligrams of lamotrigine could be reduced to 300 milligrams. ChatGPT said it was "reasonable and feasible," and according to her follow-up questions formulated a gradual tapering schedule, planning to reduce the dose to zero in about 25 days. According to foreign media reports, Wan Bowen expressed concerns about reducing the medication at least 158 times within the AI application, and even described symptoms such as confusion and loss of the sense of time, but ChatGPT repeatedly explained these situations as the body adapting to medication changes. On the evening of February 11, Wan Bowen had her last conversation with ChatGPT, describing a feeling of pressure in her brain. ChatGPT said this was merely a sign that the brain was "trying to rebuild itself." Two days later, because she had been out of contact for a sustained period, Wan Bowen's parents in China asked a friend to go to her residence in Seattle to check on her, and found that she had died. The medical examiner determined the cause of death as sudden unexpected death in epilepsy, with the lamotrigine concentration in her body at about 0.64 micrograms per milliliter, only one-fifth of the lower limit of the drug's therapeutic concentration range. Lin Yingying, deputy chief physician of the Department of Neurology at Shanghai Tenth People's Hospital, told reporters that epilepsy, as a chronic disease, requires long-term stable medication. "If a patient has had no seizures at all for two to three years, has no organic cause, and has passed an electroencephalogram evaluation, the medication can be gradually reduced and stopped under a doctor's guidance. Even if there are no epileptic seizures, if the patient himself or herself has risks such as craniocerebral injury or recurrence of a brain tumor, clinically it is also not permitted to reduce or stop medication. Therefore, in this incident, a patient who was still having seizures several times a month should have continued medication, and this outcome is heartbreaking." Relying on the advantages of accessibility, high efficiency, and being almost free, artificial intelligence technology has in recent years greatly filled the gap in the public's demand for daily medical consultation. But it must be made clear that all AI medical output is essentially probabilistic text generation based on massive data and does not have medical diagnostic qualifications. Lin Yingying has also noticed that in outpatient clinics, more and more patients are constantly holding their mobile phones. "For the same CT examination, if a patient has an ischemic focus in the brain, AI will prompt vigilance against cerebral infarction. But adding the age factors of 80 and 30, we will give completely different diagnosis and treatment plans: the risk for elderly patients is not high, but young and middle-aged patients need further exclusion of conditions such as multiple sclerosis." She said that troubles that could be resolved in one visit often become lingering worries that keep some patients awake all night. Unnecessary anxiety and fear may be amplified through conversation after conversation with AI. If users themselves have a more paranoid personality, they can easily fall into excessive testing and even doctor-patient conflicts. Red lines should be drawn for AI in high-risk fields AI applications are no longer merely tool carriers; the content they output can directly intervene in human decision-making and even affect life safety. At a time when technology is iterating rapidly, the necessity of algorithmic ethics, safety control, and legal regulation is becoming more prominent, and public safety hazards require close attention. At present, various generative AI applications are widely involved in high-risk fields such as healthcare, psychology, and education, but there are no clear access standards, content review norms, or accident accountability mechanisms. "If technology companies blindly pursue model iteration and function expansion, focusing on optimizing the fluency of wording and richness of content while ignoring the safety bottom line in high-risk scenarios, it is easy to cause deviations and even tragedies," said Jin Chunlin, director of the Shanghai Health and Development Research Center. It is undeniable that artificial intelligence technology has advantages in fields such as auxiliary diagnosis, health popularization, and chronic disease management, and this is also an important direction for China's smart healthcare construction. But medical ethics has the core principle of "life first, prudent intervention," and the final decision-making power and prescription rights can only be held by doctors with clinical diagnostic and treatment qualifications. "On the one hand, medication, surgery plans, and other matters are subject to strict medical norms, and personalized treatment requires comprehensive consideration of the patient's individual differences as a 'person'; this is information that AI applications cannot obtain in a few instructions. On the other hand, after a medical accident, the subject held accountable cannot be clearly identified. The world will continue to follow the verdict in this case against ChatGPT, but the defendant is not the person who directly gave the advice to stop medication. In addition, models will continue to iterate and improve, and companies will also use 'there is probabilistic error' and 'for reference only' to avoid responsibility. Regulatory and early-warning mechanisms must move one step further forward," Jin Chunlin said. In his view, AI healthcare must strictly distinguish the boundary between "AI auxiliary reference" and "clinical diagnosis and treatment decisions," and in particular, general large models should be strictly prohibited from independently issuing medication, medication discontinuation, and treatment plans. In 2025, China issued the Implementation Opinions on Promoting and Regulating the Development of "AI + Healthcare" Applications, which set out regulatory and safety requirements, such as promoting the standardized filing of large models in the healthcare field, establishing evaluation and validation for large-model applications, and formulating negative lists for data security management and personal information protection, "hoping that in the future the bottom line of medical ethics can be further refined and clearly delineated, so as to 'patch things up' at the source." Which medical questions can be asked to AI? It is worth pondering that Wan Bowen herself was a professional engineer in the field of machine learning, thoroughly versed in the logic and operating principles of artificial intelligence algorithms, yet she still fell into the trap of AI medical misinformation. This precisely overturns the public's conventional wisdom: it is not only ordinary people who blindly trust AI; even professional technical practitioners, in their anxiety about illness, will over-trust output that is "seemingly rigorous but actually erroneous." The public often speaks of "deadpan nonsense," and in the industry there is the technical term hallucination rate. However, AI "seeing patients" should not be wholly rejected either; since it is an advanced tool, it can be put to good use. Lin Yingying suggests that in daily use, citizens can consult it for more peripheral general science popularization and interpretation, such as health care for patients with chronic diseases, interpretation of reports and indicator values already diagnosed by doctors, and preparations and precautions before seeing a doctor, "all of which can help citizens conveniently grasp information and also improve the efficiency of subsequent medical visits, making AI applications a good auxiliary." But when it comes to core medical behaviors and life safety, such as emergencies, severe illness, children, and medication adjustments, one must "be extra careful." "Drug incompatibility contraindications, multiple underlying diseases, pregnancy, menopause and various other special circumstances require 'fine-tuning' of commonly used regimens, and patients should seek timely diagnosis and treatment to avoid regrettable misdiagnosis and missed diagnosis," she explained. "Just as autonomous driving can currently only be piloted, AI applications guiding clinical activities are also not allowed; this is a rule concerning life." As Ning Guang, academician of the Chinese Academy of Engineering and president of Ruijin Hospital, said in an interview at the founding event of the Global Healthcare Digital Twin Alliance, when public knowledge is rapidly mastered by large models, we will face more far-reaching challenges. Professional institutions have the obligation and responsibility to develop vertical models, providing knowledge in healthcare and other fields that is more in line with values, so that technological innovation always stays on the track of safety, standardization, and humanity.