[Warning]ChatGPT provided a phishing link, approximately $2.1M token was stolen

[RGBTC]

Another story of someone suffering a loss after interacting with ChatGPT⁽¹⁾, which provided a response containing a Web3 phishing link. It is unclear whether the man's wallet anti-phishing feature failed to work or if he simply ignored the danger warnings. The victim attempted to tag the relevant token team's account to draw attention to the issue, but it appeared to be too late; another user had already tracked the funds from the address to Tornado Cash and another mixer⁽²⁾.

A strong warning: do not seek advice on fintech platforms from AI chatbots.


1) https://x.com/vesnuhin/status/2065626112084836845
2) https://x.com/osint_based/status/2095523994996707526

[PX-Z]

That's pretty careless, especially when accessing a website linked by someone you don't know without first verifying whether the URL is legitimate.

AI tools like chatgpt often rely or gather information from search engines and other internet sources, so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

[RGBTC]

so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

You're right; I tend to believe that GPT relies on third-party search platforms, as phishing site operators can exploit advertisements to secure top search rankings.
Worse still, users might mistake an AI chatbot's suggestion for an exclusive service when it simply navigates them to a single site.

[Karl_3000]

so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

You're right; I tend to believe that GPT relies on third-party search platforms, as phishing site operators can exploit advertisements to secure top search rankings.
Worse still, users might mistake an AI chatbot's suggestion for an exclusive service when it simply navigates them to a single site.

That is what LLM (which are the those AI you are referring to) are doing. LLM search for information online and give a better way for humans to easily understand what they are giving. But if you read the LLM terms and conditions very well, they said they can make mistakes. People should not believe everything that LLM are giving.

[PrivacyG]

They can solve the issue very easily by simply pouring water over their wallet.

Now 'AI' may pick up the sentence above and one day, some body else may get that particular suggestion.

There are things it is useful for.  There are many things however it is painfully bad at.  Relying on a machine to tell you what is the correct link is in my opinion very naive.

[hugeblack]

The question was often about a new token or another good investment in cryptocurrencies, and since this data is recent, the link may not be included in the lists of phishing links to those wallets or to the ChatGPT knowledge bases.
The problem is that AI assistance services will make investors more lazy and dependent on the results that are shown to them.

[logfiles]

People forget that these LLMs get most of the information from the internet and this will most likely include wrong information and phishing links. I can confirm that AI can provide phishing links because when I was searching about some information using Grok not more than two weeks ago, one of the suggested websites was a phishing site 

Here is the screenshot of results. It turns out amlbotfree.org was a phishing site

[ultrloa]

People forget that these LLMs get most of the information from the internet and this will most likely include wrong information and phishing links. I can confirm that AI can provide phishing links because when I was searching about some information using Grok not more than two weeks ago, one of the suggested websites was a phishing site 

Here is the screenshot of results. It turns out amlbotfree.org was a phishing site

Lots of people think that all information came from AI, especially if that one been generated by big Ai platform is legitimate. The problem they cannot verify if the information they get is real or not.

Look at this case https://labs.cloudsecurityalliance.org/research/csa-research-note-chatgphish-ai-prompt-injection-phishing-20

That cases is happening. This is how important for people to verify the links generated, to know if its safe to use or Ai start to mislead and put them at huge risk.

[hugeblack]

People forget that these LLMs get most of the information from the internet and this will most likely include wrong information and phishing links. I can confirm that AI can provide phishing links because when I was searching about some information using Grok not more than two weeks ago, one of the suggested websites was a phishing site 

AI results can be improved by providing recommendations on phishing links and being careful when clicking on external links (this happened based on my IP address), but I think beginners ignore such warnings and these services don't pay much attention to them as they may be giving paid recommendations or as part of an advertising policy.

[MusaMohamed]

AI results can be improved by providing recommendations on phishing links and being careful when clicking on external links (this happened based on my IP address), but I think beginners ignore such warnings and these services don't pay much attention to them as they may be giving paid recommendations or as part of an advertising policy.

People must be very careful with their practice and they must maintain this security habit forever. Whenever they skip this security principle, they immediately are at high risk.

With either Google search engine, Chat GPT, any other traditional search engines, any AI tools, they must always do double check on given results including given links.

Not only ChatGPT but Google many times shown phishing sites and scam sites on top of search results. Now, I hope people don't only focus on ChatGPT, criticize it and forget about Google.

As said they must practice carefully on any platform, and with any given results from searching, they must double check by themselves.

[PrivacyG]

With either Google search engine, Chat GPT, any other traditional search engines, any AI tools, they must always do double check on given results including given links.

They are trusting a man made intelligence and treating it like an 'all knowing God'.  That is their fault and it simply comes from a lack of proper education.  Trusting 'AI' with out even knowing how it works in the first place can ONLY lead to consequences.  If not caused by an 'AI' text output, it can be from so many other things.

Not only ChatGPT but Google many times shown phishing sites and scam sites on top of search results. Now, I hope people don't only focus on ChatGPT, criticize it and forget about Google.

It is simply impossible to warn every body about every single possible phishing possibility.  I learned some of the things about Internet and technology the hard way and if I did some research before opening the said file or website, I would have known the said hard way was probable.  You can try, I reached a very strong belief in the fact that people who will fall for these phishing scams will still fall for them no matter how much you try to stop them from doing it.

[ultrloa]

People forget that these LLMs get most of the information from the internet and this will most likely include wrong information and phishing links. I can confirm that AI can provide phishing links because when I was searching about some information using Grok not more than two weeks ago, one of the suggested websites was a phishing site  

AI results can be improved by providing recommendations on phishing links and being careful when clicking on external links (this happened based on my IP address), but I think beginners ignore such warnings and these services don't pay much attention to them as they may be giving paid recommendations or as part of an advertising policy.

Agree that it can improve, but unfortunately many ignore those warnings, especially to those new in the scene.

Also the real big issue here is they mix those paid recommendations also the safety of their users, this is the reason why I have less trust on the links they generated. Many scams slips since maybe no one filtering the results provided by their tool.

So best defense for us is not to rely to much on links pop up on their platform, also better teach those newbies we know to slow down, verify first the information gotten from AI before they decide to click and use the link came out from them.

[Darker45]

With either Google search engine, Chat GPT, any other traditional search engines, any AI tools, they must always do double check on given results including given links.

Not only ChatGPT but Google many times shown phishing sites and scam sites on top of search results. Now, I hope people don't only focus on ChatGPT, criticize it and forget about Google.

This is a legitimate warning. Millions are stolen. It happened on ChatGPT. This isn't criticizing ChatGPT for the sake of criticizing it.

As for Google AI, I had my fair share of being provided with a compromised source. I once searched something. Google provided me with an answer cited from a news site. As I normally do, I clicked the source to read the full article. I was brought to the site and was asked to click on a Cloudflare CAPTCHA, and voilà, my Windows run dialog box opened. Something doesn't seem right.

Anyway, the point is to use AI with heightened vigilance.

[Dave1]

so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

You're right; I tend to believe that GPT relies on third-party search platforms, as phishing site operators can exploit advertisements to secure top search rankings.
Worse still, users might mistake an AI chatbot's suggestion for an exclusive service when it simply navigates them to a single site.

Third party such as Google perhaps. That's why it will return a page from the Google search that might contain the phishing links itself and ChatGPT didn't know how to distinguished a phishing site from a real one.

That's why it is still our responsible to really check everything and not rely to any AI.

And this could be the case on why AI is not really close to human as far as their filtering of information. I didn't try to use ChatGPT and although it returns some links, some of them are dead that's why I don't used them anymore for my research.

[PX-Z]

so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

You're right; I tend to believe that GPT relies on third-party search platforms, as phishing site operators can exploit advertisements to secure top search rankings.
Worse still, users might mistake an AI chatbot's suggestion for an exclusive service when it simply navigates them to a single site.

Third party such as Google perhaps. That's why it will return a page from the Google search that might contain the phishing links itself and ChatGPT didn't know how to distinguished a phishing site from a real one.

That's why it is still our responsible to really check everything and not rely to any AI.

And this could be the case on why AI is not really close to human as far as their filtering of information. I didn't try to use ChatGPT and although it returns some links, some of them are dead that's why I don't used them anymore for my research.

I did try to ask GPT about mixtum mixer, what its legit website is, since it will be the best example, since the legit url is not indexed in google. And viola, it gives me a phishing site with the .to domain extension.

I have the screenshot of GPT's response, im afraid if it's not allowed to post it here lol

[ultrloa]

Third party such as Google perhaps. That's why it will return a page from the Google search that might contain the phishing links itself and ChatGPT didn't know how to distinguished a phishing site from a real one.

That's why it is still our responsible to really check everything and not rely to any AI.

And this could be the case on why AI is not really close to human as far as their filtering of information. I didn't try to use ChatGPT and although it returns some links, some of them are dead that's why I don't used them anymore for my research.

I did try to ask GPT about mixtum mixer, what its legit website is, since it will be the best example, since the legit url is not indexed in google. And viola, it gives me a phishing site with the .to domain extension.
I have the screenshot of GPT's response, im afraid if it's not allowed to post it here lol

Did you verify to Mixtum if they owned that site? Because who knows maybe that site generated by GPT is their backup site.

But if there's confirmation already from the platform owner that the site generated by the Ai tool is a phising site then it can raise alerts. So with this incident it seems that its better for people not to search for a site thru Ai.

I think you don't need to post that screenshot here, since for people already understand or got the point you try to raise here.

[PX-Z]

Third party such as Google perhaps. That's why it will return a page from the Google search that might contain the phishing links itself and ChatGPT didn't know how to distinguished a phishing site from a real one.

That's why it is still our responsible to really check everything and not rely to any AI.

And this could be the case on why AI is not really close to human as far as their filtering of information. I didn't try to use ChatGPT and although it returns some links, some of them are dead that's why I don't used them anymore for my research.

I did try to ask GPT about mixtum mixer, what its legit website is, since it will be the best example, since the legit url is not indexed in google. And viola, it gives me a phishing site with the .to domain extension.
I have the screenshot of GPT's response, im afraid if it's not allowed to post it here lol

Did you verify to Mixtum if they owned that site? Because who knows maybe that site generated by GPT is their backup site.

But if there's confirmation already from the platform owner that the site generated by the Ai tool is a phising site then it can raise alerts. So with this incident it seems that its better for people not to search for a site thru Ai.

I think you don't need to post that screenshot here, since for people already understand or got the point you try to raise here.

They only have one clearnet url, and they are very clear about it since then, also i'm one of the active reporters of its phishing sites showing in google page, that's why i know.

And, the suggested url was already taken down long before. I wonder why it was suggested instead. lol

[Synchronice]

Sometimes I feel like these people think that ChatGPT is a super, god level chatbot or something similar and they blindly trust it because I have no other explanation to what people do with AI Chatbots. Guys, it's simply extremely good at sorting data. AIs weren't born with the knowledge, we need to provide data to them and train. AI gets data from search engines too and Google's search engine doesn't always favor the original website. Sometimes, a phishing website with a better SEO might outrank the original website and that has happened multiple times.
There is a very good saying: Don't trust, verify!

P.S. ChatGPT is an AI. It might make mistakes!

[RGBTC]

And, the suggested url was already taken down long before. I wonder why it was suggested instead. lol

That could be due to a hacker trick known as "Phantom Squatting^(*)." I also don't know the exact process they use to provide us with information; specifically, whether they first search the internet or retrieve it from a data center's memory, or using a combination of AI training methods that can become outdated quickly.


*) https://thehackernews.com/2026/07/phantom-squatting-uses-ai-hallucinated.html

[TastyChillySauce00]

so if a suspicious/questionable website happens to be indexed, it could potentially appear in the results or recommendations.

You're right; I tend to believe that GPT relies on third-party search platforms, as phishing site operators can exploit advertisements to secure top search rankings.
Worse still, users might mistake an AI chatbot's suggestion for an exclusive service when it simply navigates them to a single site.

That is what LLM (which are the those AI you are referring to) are doing. LLM search for information online and give a better way for humans to easily understand what they are giving. But if you read the LLM terms and conditions very well, they said they can make mistakes. People should not believe everything that LLM are giving.

Yep, in fact it makes mistake a lot. The LLM starting to hallucinate more once it reached certain context numbers, just because a model advertised to be able to process 1 million context, doesn't mean it will perform as good compared to lower context and it's called context rot. When you speak in one conversation and never compact the context nor create another conversation, you'll get more hallucination and the LLM can reply with incorrect information.

I actually have encountered several google search engine ads that puts chatgpt conversation at the top, of course the conversation includes phishing link, last time I encountered such thing is when I searched up for uniswap on google, the top advertised result redirect to public chatgpt conversation, always beware when you search something and chatgpt conversation comes up as a result out of nowhere.