[Warning]ChatGPT provided a phishing link, approximately $2.1M token was stolen

[robelneo]

P.S. ChatGPT is an AI. It might make mistakes!

That's true, and on many instances they posted wrong sources. AIs are prompt-based; if you're asking for recommendations or sources and you don't add "only trusted sources, no phishing links allowed," the AI could return dubious sources.
Even if you've been given sources, you have to check them and validate them; don't trust that they will give you trusted links or correct information- you still need to validate.

[tabas]

That's true, and on many instances they posted wrong sources. AIs are prompt-based; if you're asking for recommendations or sources and you don't add "only trusted sources, no phishing links allowed," the AI could return dubious sources.
Even if you've been given sources, you have to check them and validate them; don't trust that they will give you trusted links or correct information- you still need to validate.

Right. There's no guarantee if those words will be added. What if a newly launched website is a part of it and AI doesn't know whether it's a phishing link or not. These scammers are not clever that they can create new websites that won't be detected as a phishing link/website. Anyone who uses chatgpt or other AI tools need to be vigilant. As always, we're reminded to verify these sources.

[PrivacyG]

That's true, and on many instances they posted wrong sources. AIs are prompt-based; if you're asking for recommendations or sources and you don't add "only trusted sources, no phishing links allowed," the AI could return dubious sources.

What makes a source 'Trusted'?  I know what I trust, you know what you do, but how can it objectively pick only legitimate sources?  I do not believe there is a way to do it, when it is about sensitive information no body should trust it with that in my opinion.  'AI' is like someone who has a mental episode.  It can be the best at giving relationship advice until it all of a sudden picks the wrong website to pull information from and tells you to push your partner out of a window.

[Renampun]

The fact that he trusts AI completely for something that important only shows that he is really lazy and just wants everything instantly.. and what's even funnier is that there are quite a lot of people like this, who trust that AI can do things really well and believe it completely, yet they ignore the fact that AI is just a tool.. even AI platforms usually put disclaimers that their services can make mistakes and users are asked to verify things themselves.. this means that the AI platforms themselves might not be confident about the information they provide because their services are also limited and can make errors.. if people neglect something like this, then they can only blame themselves for being careless and trusting a tool that can make mistakes too.

[PolarCrypto]

In my opinion, everything generated by AI should be double checked. I once received a script with a backdoor in it. People are dangerously too trusting of AI these days.

I also read somewhere that a farmer ended up killing all of his plants because he trusted a recipe that was supposed to do the opposite.

[robelneo]

That's true, and on many instances they posted wrong sources. AIs are prompt-based; if you're asking for recommendations or sources and you don't add "only trusted sources, no phishing links allowed," the AI could return dubious sources.

What makes a source 'Trusted'?  I know what I trust, you know what you do, but how can it objectively pick only legitimate sources?  I do not believe there is a way to do it, when it is about sensitive information no body should trust it with that in my opinion.  'AI' is like someone who has a mental episode.  It can be the best at giving relationship advice until it all of a sudden picks the wrong website to pull information from and tells you to push your partner out of a window.

Brother, I added that

the AI could return dubious sources.
Even if you've been given sources, you have to check them and validate them; don't trust that they will give you trusted links or correct information- you still need to validate.

The prompt to provide trusted, reliable sources is a precaution; you still need to verify they are trusted and note the flaw in the AI's output. AI is just an assistant; you are still responsible for what you choose to pick or believe.

[Synchronice]

That's true, and on many instances they posted wrong sources. AIs are prompt-based; if you're asking for recommendations or sources and you don't add "only trusted sources, no phishing links allowed," the AI could return dubious sources.

What makes a source 'Trusted'?  I know what I trust, you know what you do, but how can it objectively pick only legitimate sources?  I do not believe there is a way to do it, when it is about sensitive information no body should trust it with that in my opinion.  'AI' is like someone who has a mental episode.  It can be the best at giving relationship advice until it all of a sudden picks the wrong website to pull information from and tells you to push your partner out of a window.

This is a very interesting question, what makes a source Trusted? First of all, when too many things that source says is true, i.e. success rate is very high, they become trusted but honestly, it's not like how people perceive a source trusted.
Today, yes, source has to say many true things to be considered trusted but popularity is another very important factor too. Someone might have no idea how many true or false information BBC has given but this name is so popular worldwide that the information posted by them might be automatically considered trustworthy by lots of people.

By the way, when it comes to URL suggestions, it's like using Google with "I'm feeling lucky today" option. Domain authority, backlinks and SEO in general can seriously affect the result of ChatGPT when you ask it question like suggest me X website for Y service. It's not a human level intellect, it's simply amazingly super algorithm but not on a human level when it comes to certain things.

[davis196]

Another story of someone suffering a loss after interacting with ChatGPT⁽¹⁾, which provided a response containing a Web3 phishing link. It is unclear whether the man's wallet anti-phishing feature failed to work or if he simply ignored the danger warnings. The victim attempted to tag the relevant token team's account to draw attention to the issue, but it appeared to be too late; another user had already tracked the funds from the address to Tornado Cash and another mixer⁽²⁾.

A strong warning: do not seek advice on fintech platforms from AI chatbots.


1) https://x.com/vesnuhin/status/2065626112084836845
2) https://x.com/osint_based/status/2095523994996707526

There are disclaimers about Chatbots lacking accuracy and that Chatbots should not be trusted. The guy, who got scammed simply neglected those disclaimers. I wonder if the victim of this scam would hire a lawyer and start a lawsuit against OpenAI. Maybe the judge would decide the case in his favor and order OpenAI to financially compensate the victim, but I have doubts. AI is about to become soo good at hacking and scamming people, that things are starting to look scary.

[RGBTC]

I wonder if the victim of this scam would hire a lawyer and start a lawsuit against OpenAI. Maybe the judge would decide the case in his favor and order OpenAI to financially compensate the victim, but I have doubts. AI is about to become soo good at hacking and scamming people, that things are starting to look scary.

OpenAI limits its legal liability for such cases. See Section 5 of the Terms of Service: https://openai.com/policies/service-terms/.
That does not mean they are immune to liability for violations, especially when dealing with paying subscribers. Let's saya if ChatGPT repeatedly recommends the same fake websites or explicitly persuades users to utilize a specific service.

[tabas]

That does not mean they are immune to liability for violations, especially when dealing with paying subscribers. Let's saya if ChatGPT repeatedly recommends the same fake websites or explicitly persuades users to utilize a specific service.

That's true, I've seen other known tech companies which have been sued for mistakes and misleading information they've given to their users. They are liable for what they do and they are not untouchable for that. And I hope that this step[1] will materialize in making them accountable. Although this is another case from the phishing it provides. This shows that it can make mistakes and harm us still.

[1] OpenAI is accused of taking dangerous cyber risks for ‘private gain’ in lawsuit over rogue AI agents

[aioc]

The fact that he trusts AI completely for something that important only shows that he is really lazy and just wants everything instantly.

AI developers want us to be like that, to be lazy, and to trust everything in its results, because if AIs have our full confidence, it's more money flowing into the pockets of AI developers.
AIs show disclaimers that they can make mistakes, but because of accurate results after verification, many have come to conclude that AIs are good and perfect; the one in OP's post is obviously part of this kind of user.

[examplens]

A strong warning: do not seek advice on fintech platforms from AI chatbots.


1) https://x.com/vesnuhin/status/2065626112084836845

The interesting thing is that the same user used AI to write a post on X about how AI (ChatGPT) gave him the wrong information, which cost him money.
He's obviously learned his lesson. 

[PrivacyG]

The interesting thing is that the same user used AI to write a post on X about how AI (ChatGPT) gave him the wrong information, which cost him money.
He's obviously learned his lesson. 

The most obvious part is that they have not been using their own brain both before and after the incident!