AI mistake or human foolishness

[coin-investor]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

We are responsible for all our actions, and placing your trust in AI, which is not 100% perfect when it comes to diagnosis, is a fatal mistake. He knows that only a doctor can recommend a dosage based on laboratory tests, but he still proceeded.
People should understand what AI is and is not, AI's are not all - knowing, there is information that needs to verified and tested, and one of these is when it comes to the dosage of the medication.

[General_Bitcoin]

We are responsible for all our actions, and placing your trust in AI, which is not 100% perfect when it comes to diagnosis, is a fatal mistake. He knows that only a doctor can recommend a dosage based on laboratory tests, but he still proceeded.
People should understand what AI is and is not, AI's are not all - knowing, there is information that needs to verified and tested, and one of these is when it comes to the dosage of the medication.

I would agree that people shouldn’t completely trust the AI with medical advice, although the user cannot always be blamed. Most people do not fully understand how the AI works and could take the advice seriously if they have a medical problem. If the AI is advising them to reduce or stop their prescribed medication then this is a serious flaw in the system.
 Medical advice should always be checked by a qualified doctor however the AI companies should also ensure their systems aren’t giving dangerous advice. Both parties should take responsibility.

[Dave1]

It is very possible that ChatGPT would have warned her about what she was about to do and not advise to do it, but give her suggestions if she wants.

Yes, I have also a medical condition and sometimes I will ask ChatGPT about it. But there will always be a disclaimer for it saying that if you lower your dosage you might have this kind of X effect and so I still need to monitor everything first.

Although we have different conditions, but still you really need to be very cautious on what AI will tell you and for us to learn to read between the line and not totally 100% dependent on it as if you are talking to a doctor or a physician.

[davis196]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

1.There's a clear disclaimer on every AI chatbot, stating something like "answers might lack accuracy and there might be errors"(I don't remember the exact wording). Listening to AI chatbots for medical advice should be done with extreme caution and is not recommended.
2.Just because an AI chatbot is telling you to do something, that doesn't mean that you should do it. I guess some people are just plain stupid and lack critical thinking(which is quite unfortunate). Does that mean that the AI chatbot(and the company behind it) should be held responsible? I don't believe that the court would take this lawsuit seriously.

[Fabri-Cate]

When a person is suffering from a serious illness like epilepsy, it is not right to underestimate the doctor's advice or reduce the dose of medicine at will based on the information received from AI. AI cannot see you directly, understand your current physical condition, and cannot tell you what your body lacks and what you need more. The way a doctor can make decisions based on the patient's physical condition is not directly possible for AI.

In my opinion, AI should be used as an aid to information, but should not be seen as a substitute for a doctor. Although the ChatGPT will give good advice, it should not be avoided to see a doctor for such a serious illness.

Official statement of OpenAI
https://help.openai.com/en/articles/20001036-health-in-chatgpt