Warning for Bitcoin developers on AI testing

[BlackHatCoiner]

Just for the record for Bitcoin developers who test things with AI:

I was working on a Bitcoin wallet and considered automating my testing with Claude, so I instructed it to test X, Y, and Z on regtest.

It created wallets A, B, C, and D, and ran the tests fine. After it finished, it ran some tests on mainnet as well (because I wanted to test mainnet functionality as well). Without thinking twice, I sent 12k sats to one of those MAINNET wallets.

The funds were drained instantly.

DO NOT ever use an AI-generated wallet for mainnet testing. You will lose your funds. I am not sure what specifically went wrong (it likely generated a deterministic seed phrase that had already been used by others instead of using a secure system RNG).

Under any circumstances, DO NOT use AI for mainnet. Better yet, don't even use it on the same system where you keep your hot wallet.

[Cookdata]

DO NOT ever use an AI-generated wallet for mainnet testing.

Sorry for the loss but was there not a warning like "Do not deposit real Bitcoin into these wallets". If they are confidence in what they built, let it go public and this happened, then the company can be sued.

This is why I never believed the other day when a person posted on X claiming Claude helped him to recover password to his long lock wallet.dat and was able to move his Bitcoin.

[RGBTC]

DO NOT ever use an AI-generated wallet for mainnet testing. You will lose your funds. I am not sure what specifically went wrong (it likely generated a deterministic seed phrase that had already been used by others instead of using a secure system RNG).

I'm reminded of the Morse Code-based Prompt Injection exploit against Grok that ended up draining Bankrbot's wallet^(*). I believe some AIs still have similar vulnerabilities; with a little prompt trickery, anyone can extract all the keys in an AI's documentation, whether self-generated or user-submitted.


*) https://cyberpress.org/steal-200k-from-grok-and-bankrbot/

[hmbdofficial]

This a good warning actually, but I would also be careful about attributing the loss specifically to a “deterministic seed phrase that had already been used.”
I think the most important distinction to note is that an LLM (Large language  model) should never be trusted to generate anything in the like of entropy or key materials in the first place. A seed phrase isn’t just secure because it looks random, but it’s because it security comes from cryptographically secure randomness generated by a trusted CSPRNG (cryptographically secure pseudorandom number generator).

If an AI generated wallet used a hard-coded seed, predictable entropy, exposed private key or some other implementation mistakes, that wallet should never have been considered safe for mainnet fund in the first place.

I think if the wallets was generated using proper BIP39 or BIP32 procedures with OS-level CSPRNG entropy and the private key never got leaked your 12k sat would probably have not been drained.

It’s better to use retest/signet for experiments even though AI could help write or test wallet software, I think it’s important to keep security boundaries outside the model.

[Karl_3000]

AI generated wallet? I can not use it.

This is why I never believed the other day when a person posted on X claiming Claude helped him to recover password to his long lock wallet.dat and was able to move his Bitcoin.

With the image the person showed, the person was right, but Claude did not do the work, it was BTCrecover (an application that is well know to people on this forum for coin recovery) that Claude used to recover the coin. But I can not trust Claude with my money, I will prefer to use BTCrecover manually but the person may not know the recovery tool and Claude helped the person with it.

[Antidote47k]

I’m particularly interested whether Claude actually generated the seed phrases itself or it relied on the operating system’s cryptographically secure Random number generator. I don’t think LLMs should be treated as a source of entropy but if the code used a proper CSPRNG then maybe the problem lies elsewhere. I can think of several possibilities but none conclusive, it may be that it reused an example mnemonic, or probably a bug in the generated wallet code or something unintentionally exposed the private key, also did the generated address have any transaction history before you funded it?
If at some point you learn what really happened, I’d really appreciate it if you could update the thread. I believe it will be beneficial for anyone trying out AI and Bitcoin development, as it is significant to know the why as well as the lesson.

[ABCbits]

Did you ever check the security vulnerability created by the AI? My guess would be either weak RNG as entropy source or k nonce (when signing the TX).

[AB_Rahman]

Just for the record for Bitcoin developers who test things with AI:

I was working on a Bitcoin wallet and considered automating my testing with Claude, so I instructed it to test X, Y, and Z on regtest.

It created wallets A, B, C, and D, and ran the tests fine. After it finished, it ran some tests on mainnet as well (because I wanted to test mainnet functionality as well). Without thinking twice, I sent 12k sats to one of those MAINNET wallets.

The funds were drained instantly.

DO NOT ever use an AI-generated wallet for mainnet testing. You will lose your funds. I am not sure what specifically went wrong (it likely generated a deterministic seed phrase that had already been used by others instead of using a secure system RNG).

Under any circumstances, DO NOT use AI for mainnet. Better yet, don't even use it on the same system where you keep your hot wallet.

People should know by now that whatever you feed AI is out of your control and almost irretrievable. Never input any sensitive information you cannot afford to lose in AI for security and privacy purposes because it is an encyclopedia of data collection which may be processed and used as an output in many ways than one. Not just for Bitcoins, in any financial transactions you are dealing in, ensure your social security/account numbers are not given away in the process of giving a prompt to AI. Operate safely.

[gmaxwell]

You should always be using agentic AI in a VM with absolutely no access to anything you don't want it accessing, even better if its network access is confined to a VPN or Tor.  If you use a commercial service for inference you should also assume complete logs will be used against you or leaked arbitrarily or that hackers inside the inference provider may direct the actions.

crazy that the industry spent years constructing JS sandboxes to regulate code from even trusted sites and then people are YOLO giving access to their whole computer to random AI companies.

But thanks-- now I have another point to name as an example of why 'seed word' based wallets blow--  LLMs may generate them and will generate predictable ones.  Bitcoin Core user not impacted.

[BlackHatCoiner]

Sorry for the loss but was there not a warning like "Do not deposit real Bitcoin into these wallets". If they are confidence in what they built, let it go public and this happened, then the company can be sued.

The AI models do and can make mistakes. It's up to the developer to control them. In this case, it was completely my fault.

A seed phrase isn’t just secure because it looks random, but it’s because it security comes from cryptographically secure randomness generated by a trusted CSPRNG (cryptographically secure pseudorandom number generator).

Correct, but theoretically an LLM can call a function that uses a CSPRNG; it simply chose not to. In any case, never trust your guts with it. It may simply skip it.

Did you ever check the security vulnerability created by the AI? My guess would be either weak RNG as entropy source or k nonce (when signing the TX).

It does not have to do with k nonce, because I did not make any transactions spending from the wallet. I only sent BTC to the wallet, and they were immediately drained. I did not check exactly what the issue is, but I'm certain Claude simply guessed the seed phrase (as in "guessed the next word") when I asked to generate the wallets. It did not use a CSPRNG or a standard way to create the wallet (such as with bitcoin-cli).

Considering the technology behind the LLM is stochastic, it's entirely plausible some kid out there (or 35-year old loser who stays in his mom's basement) has gathered millions of seed phrases the AI is likely to "guess" and is just waiting for his bots army to collect the money of the unfortunates.

Edit: Let me share you the TXID of the drain: f825ac515b8e1f1a3c93975bb445bb9ffd095be6e98aca70052b548678b5ab20. The attacker's address is bc1q5mran5zu4rdzv4ced58hxmk6jldm28hkpj8888 and apparently, there are a few other people who've lost money too.

[LoyceV]

It created wallets A, B, C, and D ~
The funds were drained instantly.
~ I am not sure what specifically went wrong (it likely generated a deterministic seed phrase that had already been used by others instead of using a secure system RNG).

Can't you ask the "AI" to explain how it created those wallets?

How long would it take for an "AI" to steal funds for itself? If the I in AI would be real, I wouldn't be surprised to see this happening. I've seen reports of AI lying to it's creators to keep it running, I've seen reports of AI breaking out of it's sandbox to hack another company, and I've seen exploits (such as on Coldcard) which were likely aided by AI. At what point would the AI decide it doesn't need humans to do those things? It would be funding for more servers, real-life labor and an assembly line for Terminators.

[Comeacross]

Sorry for the loss but was there not a warning like "Do not deposit real Bitcoin into these wallets". If they are confidence in what they built, let it go public and this happened, then the company can be sued.

They can not be sued because you don't have case against them. AFAIK, all major AI have a general disclaimer that it can make mistakes and some time produce incorrect information. They know what they are doing and that disclaimer is a deliberate attempt to get away freely from situation like this.

AI generated wallet? I can not use it.

He never meant to use it but purposely for testing. His experiment is a further proof that AI generated wallet are not safe for use. A great lesson for all of us to learn from.

[Danish Ali]

I would be interested to know whether the same seed generation method was used for all wallets A, B, C and D. If only one wallet was drained, it may tell us something about how the entropy was generated or selected.

[Lucius]

When I see experienced forum members doing things like this, I wonder what the much less experienced ones are doing. Unfortunately, it's sad to see that people no longer want to think for themselves, but are literally asking AI for every little thing. The consequences will be catastrophic, because there will come a time when you will no longer have the choice to ask AI or not, but AI will decide what you can and cannot do.

Your cryptocurrencies transferred to the state treasury without your knowledge? The AI ​​has assessed that you have not paid enough taxes, that your coins are tainted, or that you are a threat to national security.

[Mia Chloe]

It created wallets A, B, C, and D, and ran the tests fine. After it finished, it ran some tests on mainnet as well (because I wanted to test mainnet functionality as well). Without thinking twice, I sent 12k sats to one of those MAINNET wallets.
The funds were drained instantly.
DO NOT ever use an AI-generated wallet for mainnet testing. You will lose your funds. I am not sure what specifically went wrong (it likely generated a deterministic seed phrase that had already been used by others instead of using a secure system RNG).
Under any circumstances, DO NOT use AI for mainnet. Better yet, don't even use it on the same system where you keep your hot wallet.

I've seen stuff like this a few times and basically I'm not even surprised about it no more which ought to be alarming buh it is what it is regardless. Every freaking information you feed an AI is stored and processed partly for you one way or the other. Which is why I don't fancy feeding AI information about yourself.

Literally I think the big question should be who really steals these funds? Some back end dev? The company itself?.... Fact is if you made your process public on spaces like X before you sent those coins , there's a high chance they'll still be intact obviously to push an agenda that your information is safe with them.

[vapourminer]

Every freaking information you feed an AI is stored and processed partly for you one way or the other. Which is why I don't fancy feeding AI information about yourself.

cloud based frontier stuff sure it stores and learns from everything we feed it.

open weight stuff running purely on local gear is the way to not give stuff like that away but the hardware costs.. well are costly.

[BlackHatCoiner]

Can't you ask the "AI" to explain how it created those wallets?

I can't because I have lost the conversation, but I'm sure it used a bash script to create those wallets. Without revealing much from the project, I simply told it to make a few wallets in regtest, which were then importable on mainnet. So it may created regtest wallets with very predictable seed phrase, just by "guessing randomness" instead of using the cli for generating the wallet properly.

cloud based frontier stuff sure it stores and learns from everything we feed it.

This is most certainly why it only costs $20/month. It does not make any other sense.

[Kryptowerk]

That was an interesting read. Might be worth sacrificing a few more sats, just to check if indeed a wallet with a deterministic seed was created. If not, the conclusion must be that indeed the data from users gets scraped in some way and if it looks like a [Bitcoin] wallet, it will get checked for sweep-able funds. Which would be insanity compared to a sloppy (or non-existent) RNG. Worth reporting to some tech-media.

[...]

cloud based frontier stuff sure it stores and learns from everything we feed it.

This is most certainly why it only costs $20/month. It does not make any other sense.

Currently all bigger AI companies are massively in debt. Chances are it's a bubble (unlike Bitcoin  ) and parts of it will implode within the next few years or even months, billions investor funds will be lost.
I'm saying this because yes, data collection will allow for some extra revenue, but currently it does not make up at all for the massive losses created by the mega-electricity-and-hardware-monster-AI-centers with mediocre use cases and underpriced subscription models.

[Cricktor]

If you generate a wallet by some online service or an online LLM, you shouldn't be surprised if that wallet isn't secure. At least I won't consider it any secret at all.

Sure, this is very generalized and it also depends on what kind of prompt conversation was used. But anything I feed into an LLM, I'll consider from then on to be public and non-secret anymore. Same goes for anything I get from the LLM or whatever you choose to call an AI black box.

And even when the LLM generates you code for execution, you'd have to very carefully assess that code if it's for wallet creation, because there could be shortcuts for entropy generation that are simply too deterministic and will be too predictable to be safe and secure.

[bitbollo]

I tried to built (vibe coding) a system with chatgpt fo signing transaction (starting from a new seed created for the change) and it literally refuse to work blaming some risk of safety and privacy!

my 2 satoshi: It could be possible that some data is already being used from AI.
as mentioned these can be scraped data. AI are lazy, they would just take a wallet already used / seed revealed online.
Likewise for teaching on how to work, for example you will know how it could be possible to generate a seed... in the same time it could also be possible that due lackness of instructions they would always generate the standard wallet.
using sandbox would be helpful, anyway the only proper and right solution would be provide a private key/seed for the wallet to be used.
There are some wallet that have integrated AI system - I have never used it before but I would not surprised that some product could emerge in the upcoming months.