AI mistake or human foolishness

[Spaceman1000$]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

Did the doctor that gave the person the prescription asked the person to reduce the dose, I think the answer is no, was it the AI that recommended that medication in the first place, the answer is no, so even if you have an open conversation with an AI and the AI recommend some things to you, it is also in your best interest as a person to consult your doctor, some AI go as far as telling you that they are not medical practitioners, they just have an open knowledge about a particular subject matter. so for me openAI shouldn't be blamed, it is the person that found solace in AI instead of a medical practitioner that should be blamed.

[Cryptomultiplier]

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

What else than calling her foolish? Was she trying to cut the service fee from seeing her doctor? Probably, she thought AI could replace a qualified doctor to handle her health issue, people can be stupid, but it shouldn't be to the point where it endangers their life. Full blame goes to her and not the AI, she wouldn't have won the case.

Am sure there's no how she would have won the case because I know even using AI for such would come with a warning. I have used AI at some point to diagnose someone conditions and it first offers a disclaimer to not being a real doctor but only gives advice based on questions asked.

She's at fault for trusting AI answers over a real doctor handling her case maybe because she doesn't want to spend money, but she forgot humans designed Ai and they are still suseptible to errors in judgement.

It is a shared tragedy really.

[WatChe]

Crazy story, this should be a lesson to those that assume that Ai can give the solution to everything including health related issues. There's a possibility that Ai must have shown the downsides of it but still decided to skip or reduce her medications which led to her death. The problem with a lot of people is that they fail to realize how dangerous it is to rely on Ai completely in whatever they use it for.


Ai will always come up with suggestions when people use it to search for certain questions that they need answers to, this is what it was designed or programmed to do. Ai is not an expert on human health related issues, it can only give you informations on whatever you need to know but taking the suggestions of Ai on health issues is somewhat risky.

Health care is a very sensitive matter and we must take proper doctor advice to tackle any disease. AI no doubt has its own benefits but we have to keep in mind that AI chat bots results are not very accurate. If you are a programmer and taking guidance from AI chat bot in writing code then a mistake in code given by AI is not a serious concern since a software engineer will correct it in next run but in case of health care there is zero tolerance for mistake. AI chat bots are not designed to give us health care advice and if you are using them for that purpose then we need to be ready for the consequences.

[Hyphen(-)]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

This should be their fault for relying on AI tools guide them, and I don’t think ChatGPT have a fault in this discussion because mos times, if you asked them something related to health, they will guide and still insist you should see a doctor immediately, but some of us can ignore that thinking they can get first aid from them or even go further to ask them for medications to go for, if that is the case, ChatGPT should not be blamed unless if they check the chat history and found out they didn’t asked them to go the hospital to see a doctor.
Moreover, I think this is foolishness because if they didn’t go to the AI in the first place, they will not have them in this case, so it’s their fault for seeking for medical health through ChatGPT.

[Mrbluntzy]

I checked the TOS of open AI,

https://openai.com/policies/row-terms-of-use/

When you use our Services you understand and agree:

Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice.

Once we sign up, we agree with that terms, so most likely the "victim" has not read that terms, she just sign up blindly.
Of course she can file a lawsuit, but obviously she has no slim chance of winning.

Many people these days do not bother to check terms because they are lazy to spend time reading a wale of text when they are in a haste to use the tool. The victim is dead already, it's her parents that filled the law suits and I believe the parents and their lawyer have read the terms of the terms of the company now and they believe they have a chance of winning the case. In my opinion, I think they might lose because it's a case of contributory negligence which the company can use as a point of argument but does not also mean that they will win, because there is a saying that even if someone ignored a company's warning does not necessarily mean the company is innocent.

[Bright0515]

This is clearly human foolishness, not every decision should be made with the input of Ai especially when it comes to medical conditions. The deceased probably made a decision out of ignorance, suing open Ai is actually a dumb move to make because they cannot control what people use Ai for.  Ai gives suggestions and also warnings for such situations but some people only decide to take only specific parts of the informations that they get from it, this is exactly what gets them into trouble. It's quite a sad news but it could have been avoided.

[ColdLava40]

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

A 31year old grown woman. I'm honestly surprised that she allowed an AI's suggestions to influence her decisions about her medication so much.

I've seen other cases where AI advice has led to real-life consequences. the problem isn't necessarily AI itself but people who rely on it instead of seeking help from qualified professionals.

Not everything AI tells us is correct. It responds based on the data and information available to it, and there are practical real-life experiences it may not fully understand. So, blaming the company for their daughter's actions is wrong.

[Alpha Marine]

Reading the full article, OpenAI should be sued for more than whatever amount the parents are suing them for. How can they have their bot giving full blown medical advice on complicated things like that? Even when she was in pain, instead of advising the user to visit her doctor and complain to her, it went ahead to tell the user it was normal pain.
The doctors are not idiots when they prescribe a particular drug and a particular dose. If you have any probelm with the drugs or dosage, talk to your doctor or get a second opinion. And it goes without saying that the second opinion should be from a qualified doctor too, not a chatbot.
Even though the user was negligible, the company should be sued. If they claim it's an old model, then people shouldn't be able to use an old version at all.
People will always make stupid decisions, but it is up to the company to make sure they don't help them make those decision sor advise them to make them.

[Yeesha]

It is very possible that ChatGPT would have warned her about what she was about to do and not advise to do it, but give her suggestions if she wants.

So there would have been something like a disclaimer before ChatGPT give her the information.

She is 31 years and not a child. I expect a 31 years old to know that ChatGPT should not be used for something like that. ChatGPT also have a disclaimer in its terms of service that the information it might give at times may not be right.

You're extremely right, she is no longer a child, but I think she is suffering from psychological problems. And her parents failed to pay attention to her. She need emotional support which she couldn't get from her parents. Now what's the point of suing ChatGPT, something that cannot defend itself. I think that's a waste of product/ money. It is their fault, they couldn't monitor their sick child, if not, they would have known that she is skipping her medicines. It is their duty to track her medications to ensure that she is following the Doctor's instruction. The parents should withdraw the case, they have to accept that whatever happens to her is her fate

[DYOR+BTC]

Its obvious everyone has the right to choose what they want when it has to do with decision making and should not be blamed for choosing to introduce AI in everything they do including medical conditions. Of a truth,  even if we to joke with anything it shouldn't be human health because we have just one life to live and when its lost it can't be replaced again .

What happened on this case is same as when they say informations should be verified before posting,  I will say the designers of that openAI needs to be punished for programming a device that posted an unverified information. Although I will blame the person that trusted AI over medical awareness he has been exposed to even before now because know one with his right senses will trust AI over a doctor that is trained for that specific profession.

[JoyMarsha]

Someone who is 31 years of age is not a baby, for them not to know the advice that is good for them to adhere to, or bullshit.

AI doesn't provide someone with an answer unless asked. The lady's conversations with AI might have started with a simple question about her health before they led to deep conversations that caused the 31-year-old lady to make a decision that will affect her life.

The lady's parents don't have a case in this because AI didn't know about their daughter's health at first until their daughter presented her health status to it for opinions and solutions.

[Su-asa]

It is very possible that ChatGPT would have warned her about what she was about to do and not advise to do it, but give her suggestions if she wants.

So there would have been something like a disclaimer before ChatGPT give her the information.

She is 31 years and not a child. I expect a 31 years old to know that ChatGPT should not be used for something like that. ChatGPT also have a disclaimer in its terms of service that the information it might give at times may not be right.

There is a chance that chat actually warned her about taking such risk but she still decided to do it anyway and like you said she's 31, people expected more from someone at that age to know how complicated health conditions are and also not trust chatgpt completely to the point of making such risky and sensitive decisions. Relying too much on the information from Ai has become humans biggest problem.

[serjent05]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

I find it hilarious when a person tried to point the blame on tools or programs that have a disclaimer or advise them to seek a doctor for safety.  I think the family wanted to be compensated, and the suing is all about getting money out of the incident.

I always try to ask AI for medical recommendations, and it never fails to instruct me to consult a doctor.  Besides, AI will never recommend a dosage unless the person ask directly after describing the symptoms and situation.  Although the AI gives dosage recommendations, it emphasizes that before taking any action, the person must consult a doctor and take the medicine according to the doctor's prescription.

[Grace333]

This is human foolishness, I'm not even sure that she was in a right state of mind because it is weird for an adult to believe what chatgpt tells her to do about the situation of her health knowing fully well that the informations that are gotten from it are not always accurate, this is quite sad but it's not anyone's fault, she actually made a stupid decision and I cannot say that it was out of ignorance. Sometimes when using Ai I have to compare somethings with some other sources, everything cannot always be correct.

[STT]

I will go on the recent legal verdict against META which is that Facebook and Instagram are somehow liable for people believing negative things about themselves.  Presumably there wasn't enough safe guards and checks done especially with minors and so they became liable to damage done.
   Myself personally I would not hold the software as responsible as its an inert tool but clearly there is a duty of care that has not always been there.

  AI we all know here is misleading fairly often, it could give the wrong impression is a fair scenario so it could lead to similar liability to unsophisticated users I suppose.  I think they have updated protocol and warnings fairly recently.

Humans do foolish things and the courts demand any consumer product to expect and avoid those outcomes always if possible.  The self driving capacity of some cars has foolish 'drivers' going to sleep on the back seats while the car drives at high speed, that outcome is totally broken and I would guess the car AI is blamed for the users actions.

[Pi-network314159]

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

I blame the victim who chose to follow the Instructions of machine over a real doctor. One thing about this generation I dislike is the inability to take personal decision rather they always feel that there is something that knows more than human that they need to rely on completely for everything. They find it more trustworthy than anything. People has placed their hope on Artificial intelligence forgetting that this where man made machine that has no mind or feeling but act according to the information they gather online. Those AI are not relatable but I am ashamed that man IQ has reduced to the extent AI is now detecting how they will leave their life and they foolishly obey.

[Felicity_Tide]

~snip

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

I tried to read through the article you shared, but couldn't continue reading the very moment I came across the statement... "she distrusted her doctor". Putting all the blame on an AI is still something I'm trying to process here. Yes, I agree that Ai ought to have included a disclaimer, but it's not like these things are actual physicians. Trusting an Ai with health related matters is no different from trusting an unlicensed physician.

We humans can actually do better because health issues are not something to mess around with. We just can't keep depending on AI for every single thing. And considering the age involved... I believe humans should know better.

[abhiseshakana]

Based on my experiences, for medical purposes I most frequently use AI to read laboratory test reports while waiting for the doctor appointment time. Because i think in the medical field, AI usage should serve only for initial analysis tools like screening and preliminary not as a determinant or a substitute for clinical decision making. For me, AI is entirely unsuitable as an option or a source for a second opinion in the medical field because we truly know that even AI has brain but AI doesn't have clinical understanding awareness beside that perdect analogy are how criminals act when their crimes cannot be punished, AI cannot bear legal or moral responsibility for the failure in providing medical instructions.

Another insight, the doctor who do not have a license to practice are not permitted to perform medical procedures or administer medical intervention will be subject to criminal sanctions if found to be in violation, even in the absence of malpractice. How foolish it is for someone to allow a system whose purpose is to provide information to act as an unlicensed medical practitioner.

The last we should aware, in the medical world, there is always room for anomalies, unique patient conditions, and a certain percentage of diseases or complications that go undetected by algorithms. So i suggest hard barrier, which declines to provide recommendations for any medical actions, while also transparently acknowledging a margin of uncertainty noting that there are many disease related variables that remain undetected by their system.

[boyptc]

This is sad, the daughter's age is 16 and she's got a knowledge in technology and how to use it. I think that at that age, the parents should have monitored what their daughter is up to.

What she uses her devices for and what are the applications she's using for it. I'm sure that as a software engineer, he'll see the history of the device of his child and what she's been up to.

It's becoming a common thing that people asks medical advises from the AI and I think this also serves as a warning that it's best to consult doctor and professionals at all times.

[mirakal]

I would say that its all about human foolishness. Who's in the right thinking that will believe more on AI than a professional physician?

Health is wealth, she should have been transparent to her parents and tell them what she's currently feeling, the parents as well should have monitored well their daughter if she has been taking her medicine, otherwise this case should have been avoided, and their daughter should have been alive and kicking until now.

AI has made it clear to every user that mistakes are inevitable, so AI shouldn't be blamed here.