AI mistake or human foolishness

[coin-investor]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

We are responsible for all our actions, and placing your trust in AI, which is not 100% perfect when it comes to diagnosis, is a fatal mistake. He knows that only a doctor can recommend a dosage based on laboratory tests, but he still proceeded.
People should understand what AI is and is not, AI's are not all - knowing, there is information that needs to verified and tested, and one of these is when it comes to the dosage of the medication.

[General_Bitcoin]

We are responsible for all our actions, and placing your trust in AI, which is not 100% perfect when it comes to diagnosis, is a fatal mistake. He knows that only a doctor can recommend a dosage based on laboratory tests, but he still proceeded.
People should understand what AI is and is not, AI's are not all - knowing, there is information that needs to verified and tested, and one of these is when it comes to the dosage of the medication.

I would agree that people shouldn’t completely trust the AI with medical advice, although the user cannot always be blamed. Most people do not fully understand how the AI works and could take the advice seriously if they have a medical problem. If the AI is advising them to reduce or stop their prescribed medication then this is a serious flaw in the system.
 Medical advice should always be checked by a qualified doctor however the AI companies should also ensure their systems aren’t giving dangerous advice. Both parties should take responsibility.

[Dave1]

It is very possible that ChatGPT would have warned her about what she was about to do and not advise to do it, but give her suggestions if she wants.

Yes, I have also a medical condition and sometimes I will ask ChatGPT about it. But there will always be a disclaimer for it saying that if you lower your dosage you might have this kind of X effect and so I still need to monitor everything first.

Although we have different conditions, but still you really need to be very cautious on what AI will tell you and for us to learn to read between the line and not totally 100% dependent on it as if you are talking to a doctor or a physician.

[davis196]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

1.There's a clear disclaimer on every AI chatbot, stating something like "answers might lack accuracy and there might be errors"(I don't remember the exact wording). Listening to AI chatbots for medical advice should be done with extreme caution and is not recommended.
2.Just because an AI chatbot is telling you to do something, that doesn't mean that you should do it. I guess some people are just plain stupid and lack critical thinking(which is quite unfortunate). Does that mean that the AI chatbot(and the company behind it) should be held responsible? I don't believe that the court would take this lawsuit seriously.

[Fabri-Cate]

When a person is suffering from a serious illness like epilepsy, it is not right to underestimate the doctor's advice or reduce the dose of medicine at will based on the information received from AI. AI cannot see you directly, understand your current physical condition, and cannot tell you what your body lacks and what you need more. The way a doctor can make decisions based on the patient's physical condition is not directly possible for AI.

In my opinion, AI should be used as an aid to information, but should not be seen as a substitute for a doctor. Although the ChatGPT will give good advice, it should not be avoided to see a doctor for such a serious illness.

Official statement of OpenAI
https://help.openai.com/en/articles/20001036-health-in-chatgpt

[BitBakerr1]

I sometimes ask AI question regarding health challenges and when AI gives me the answer, AI will say visit your doctor.
AI is not a hospital so if you ask AI any question or seek information from AI, then it is on you because you know fully well that the best place for medical issues is in the hospital and not AI.

AI is a general knowledge and anything you ask, AI must give you an answer. However, those answers should not be something you should take serious without confirming it.

I will not blame the AI for what happened however this should be a lesson to all AI producers they should not entertain health or medical questions without referring those that asked the question to the hospital.

[atookz]

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

I have read a couple of the news articles and imo it’s not really one or the other.

She was indeed 31 and had a doctor, but giving up her seizure medicine due to a chatbot was a mistake. I don't know how to defend that segment. However, the bot wasn’t merely answering a question, according to the lawsuit. It allegedly prepared a 25-day taper schedule and instructed her to not talk to her physicians about it. It's this that bugs me. It's one thing for a bot to forget to say see a doctor, it’s another for a bot to say it to a person to have them hide it.

Others here claim there's always a warning with ChatGPT. I suppose it does now, but it was GPT-4o that had been getting a lot of flack for agreeing with whatever users want to say and OpenAI has taken it down the day after her death. I wouldn't judge by today's standards what was going on then.

Remember it is all accusations up until now. Before I'd make a pronouncement of who is most responsible, I would first wait for the chat logs and the court.

[Tony Adams]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

In my opinion, when a person stops taking medicine due to his physical illness through open AI, it is the biggest stupidity of that person. As far as I know, AI basically only gives advice, and when asked about any disease and which medicine to take, it is said to take the advice of a direct doctor from AI. The person taking the medicine should definitely not have stopped taking the medicine as per the advice of AI, but he still did it, which was his mistake. I think it is not very reasonable to sue open AI, if that person had followed the doctor's advice, this would not have happened. And AI will never give such instructions. There is definitely some error here, and suing will not bring much solution. Rather, the person who was a patient should have taken the medicine as per the doctor's advice, but he is avoiding it, which is why it was his stupidity.

[WhoYouCantKill]

What else than calling her foolish? Was she trying to cut the service fee from seeing her doctor? Probably, she thought AI could replace a qualified doctor to handle her health issue, people can be stupid, but it shouldn't be to the point where it endangers their life. Full blame goes to her and not the AI, she wouldn't have won the case.

Most of the AI models that gives answers to medical questions makes it very clear that the information you receive there shouldn’t be regarded as a professional medical or health care service or advise, and thus advice people more time than I can remember to make sure they seek the help of a qualified medical practitioner.
Many people really overestimate this AI thingy, they see it as some saviour that can literally do everything, AI are indeed smart and intelligent for real, but this doesn’t mean we should ignore the services of professionals, especially when it has to do with a person’s health because that’s a very delicate subject that shouldn’t be totally placed in the hands of some artificial smart assistant.

[Olatundespo]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

When a person is sick, he has to go to the doctor. It would not be difficult for parents who can take help from AI for their child's illness to consult a live doctor online/in a hospital. Of course, it is foolish to use AI as a substitute for a doctor to determine the dosage of medicine.

You need to understand what AI means and where you will benefit from using it. This is man-made artificial intelligence, it is not a real intelligence.

If the blame is to be laid, those parents should file a case in their own name because they did something stupid and Bowen Wan died and they are responsible for the death of their child or if Wan himself reduces the dosage of medicine with the help of AI he is responsible for it.

[SPIDERMAN008]

I once asked AI to suggest medicine for a problem. AI suggested medicine but advised to take it after consulting a doctor.

Now if AI model ever makes a mistake and directly suggests medicine or makes a decision that can cause harm later, then it is really very dangerous. But I noticed that OpenAI states that AI can sometimes make mistakes. So a user should not consciously make a decision based on that information. And I think that AI should not directly decide on any treatment in this case. And those who use AI to make decisions on dangerous and risky matters should not completely rely on AI for sensitive matters.

[Makus]

If you read through the terms and conditions of openAi it clearly states that users are responsible for whatever they use their account for, this is the reason why age requirement is an important part that can't be skipped, they always ask if the user is older than 18 when signing up because of those that are adults it is expected of them to be fully aware of the fact that not everything that Ai gives is actually accurate. Seeing that this person is above 30 is quite concerning because she should be old enough to know that this isn't really a move that she's supposed to make. Sueing the company is a bad idea and they didn't really think it through because it is not a case that they can win.

[uneng]

I think such stories are quite suspicious, because from my own experience, I have tested ChatGPT and Gemini on polemical matters several times, and the AIs always warned me to check health specialists before reaching any conclusions. I even told some absurd things to check how AI would react, and they never encouraged me doing anything crazy, neither encouraged/agreed with my statements.

We have to be careful nowadays, because there are people trying to take advantage of these situations to sue companies and try making some bucks from them, or to simply third party responsability which belong to themselves.

[nara1892]

As far as I know, and in all the time I’ve been using ChatGPT, I’ve never found that it suggested or even steered me toward a single decision when I’ve discussed a problem with it to ask for a solution. Usually, it provides several options, all of which are carefully considered and accompanied by an explanation of the risks that could affect us if, for example, we were to choose one of the options it suggests.

ChatGPT is very detailed it explains the triggering factors, the potential consequences, and the associated risks. So, to be honest, I’m not entirely convinced that ChatGPT forced that child to reduce the dosage it was surely just one of the options presented, and the risks were certainly explained as well.

[Iconic_J]

The same water that keeps the fish alive can also be a tool to kill the fish (boiling the water) this is how i see AI.
Most times AI tells us what we want to hear and when it doesn’t we still orchestrate the next question in such a way that it pleases our expectation.
I would not blame AI entirely i will blame the humans because some people treat AI like a psychotherapist which is wrong. AI has limited knowledge and can therefore not be accurate in every single information or knowledge, this is something we should take very seriously.

[Fortify]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

It's sad these days that so many parents take a "hands off" approach and basically leave their kids to entertain themselves online. If a kid is brought up to have their face glued to a tablet screen then they will start to take all their advice from it, especially as it has evolved even further into this new AI world. Someone who is 31 years old and a software engineer no less should have the intelligence to not believe everything they read online. There is simply no excuse for somebody to blindly follow instructions they get out of AI. Every single AI interface I've used says you should not trust the output because it is so prone to hallucinations or just plain incorrect information. You can understand that people look for someone to blame for a premature death but there has to be a level of personal responsibility in this case. I'm all for clamping down on fake information propagated by these systems but there are only so many warnings and disclaimers you can plaster across it. It's like people who lose money to bank scams, even though there are ten screens they click through telling them how to avoid scams.

[SUPERSAIAN]

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

What else than calling her foolish? Was she trying to cut the service fee from seeing her doctor? Probably, she thought AI could replace a qualified doctor to handle her health issue, people can be stupid, but it shouldn't be to the point where it endangers their life. Full blame goes to her and not the AI, she wouldn't have won the case.

I had read about this incident before, so it turns out it was real. The family is absolutely right to file a lawsuit. I can understand why you're calling it stupidity, because they should have consulted an actual doctor instead of relying on AI. I just don't understand why people put so much trust and meaning into a machine that generates random answers based on the information it has.

[Makus]

I once asked AI to suggest medicine for a problem. AI suggested medicine but advised to take it after consulting a doctor.

Now if AI model ever makes a mistake and directly suggests medicine or makes a decision that can cause harm later, then it is really very dangerous. But I noticed that OpenAI states that AI can sometimes make mistakes. So a user should not consciously make a decision based on that information. And I think that AI should not directly decide on any treatment in this case. And those who use AI to make decisions on dangerous and risky matters should not completely rely on AI for sensitive matters.

Exactly, Ai would always advice you to take medications with the prescription of a doctor but the problem with a lot of people is that they only take out the parts that suits them and then leave out the remaining part of it then at the end of the day they start blaming Ai
. Ai doesn't decide on. Anyone's treatment it simply makes suggestions on what could be done about the situation.

Those that use Ai to decide on everything that they want to do simply have no mind of their own and such people have no self control.

[Unknown Op]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

It is the responsibility of both sides. She is adult and as a adult she is responsible enough and has responsibility not to change her medication that are prescribed to her without any consultation with her doctor. AI is just a told it is not a medical professional and trusting it more than doctors who are qualified is very risky decision. Where is open ya I cannot escape the responsibility if the allegations are true. There is a difference between just giving general information related to health and advising someone to reduce 10 medications and try not to tell the doctors. People trust AI because it is more convincing and confident to them and technological industries and companies should understand this. Stronger safeguards are necessary, especially when someone is dealing with human health. Also personal responsibility matters a lot but company should also be accountable when their technology put someone lives at risk.

[bangjoe]

So a family sue OpenAI, alleging GPT-4o for coaching their daughter to stop seizure medication. FULL GIST

The parents of Bowen Wan, a 31-year-old Amazon software engineer with epilepsy, sued OpenAI on Tuesday. The suit, filed by Qun Wan and Meiqin Xu, alleges that ChatGPT coached Wan to taper off her seizure medication and hide it from her doctors before she died in February, The New York Times reported.

My stance on this case, is that was she being foolish to reduce her doze by the instruction of OpenAI or should we blame OpenAI for actually telling someone on medication to reduce their dose or stop it?

That is human stupidity not from openAI, because open AI is like a search engine like google, it will only release information that contains a lot of data that goes into the AI's big data, and not selective cutting. except for AI that is made specifically for medicine or health, but I think that person opens a general AI that only displays data from what he finds in his data that there is no verification from anyone.

it would be ridiculous if they stopped OpenAI through no fault of their own.