Why the wallet.dat file isn't fully encrypted?

[cellard]

At the beginning, there was no encryption. I think in the original client anyone was able to open the Bitcoin-qt client and just see your funds and use them which is insane.

Then the encryption we know nowadays was added.

But why was it choose to not fully encrypt it? I mean, you can see your funds, your sending and receiving addresses etc... by just opening the client. I don't like this.

Why not add full encryption like Electrum? maybe with 2 different password, one to open it, and then the current one to confirm transactions, so you don't have to enter the pass that actually allows you to transact each time you want to open the wallet... even tho it shouldn't matter if it's cold storage.

[1714475201]

1714475201

[aleksej996]

A good question.

I guess that you answered your own question quite well by pointing out that you would need to decrypt this data everytime you start your client and it would stay unencrypted like that for the whole time the program is running. Since Bitcoin Core is usually not a wallet that you just open up for a short period of time and close again, I guess devs just figured that it would add too much complication to add two passwords for no good reason.

Adding full wallet encryption with one password would be very dangerous though, as users would unlock it every time they want to check their balance.

[achow101]

If the wallet.dat file were fully encrypted, you would not be able to see your funds, get addresses, etc. without entering the password. This means that for the wallet to be useful, you have to prompt the user to unlock the wallet at start. Unfortunately this is not really easily done with bitcoind which does not have a GUI. Furthermore, it is not necessary for the entire wallet.dat to be encrypted because all that really matters are the private keys, and not anything else in the wallet file.

[cellard]

If the wallet.dat file were fully encrypted, you would not be able to see your funds, get addresses, etc. without entering the password. This means that for the wallet to be useful, you have to prompt the user to unlock the wallet at start. Unfortunately this is not really easily done with bitcoind which does not have a GUI. Furthermore, it is not necessary for the entire wallet.dat to be encrypted because all that really matters are the private keys, and not anything else in the wallet file.

Yup but that's what why I proposed a double lock system:

-one password to enter the wallet.dat file
-a secondary password, ideally stronger, that is the one that allows you to move funds

If someone manages to obtain your wallet.dat file, this attacker would be able to look at your funds. Depending on how much you have, the attacker will try more or less to get access to your funds. If they see you have 0.001, they may not bother. If you have 1000 BTC, you should be worried, because now you are a target for a huge bounty.

An attacker can also inspect your transactions on the blockchain.

I think it is insane that we don't have a way to do this. Ideally, we would have a triple password system, for plausible deniability, so you open a wallet that has little funds in case someone is menacing you with a gun or some other extortion.

[achow101]

Yup but that's what why I proposed a double lock system:

A double lock system does not get around the fact that it is still a UI/UX issue.

Furthermore, you are asking users to remember yet another password. It seems that it is far more common for people to lose their coins by forgetting their password rather than having someone steal them. Even cases of stolen encrypted wallets seem to be quite low. By adding another password, you introduce yet another way for people to lose their coins. Forget one of two passwords and you are screwed.

[formatme]

You could just encrypt it yourself with something

[andrew1carlssin]

Yup but that's what why I proposed a double lock system:

A double lock system does not get around the fact that it is still a UI/UX issue.

Furthermore, you are asking users to remember yet another password. It seems that it is far more common for people to lose their coins by forgetting their password rather than having someone steal them. Even cases of stolen encrypted wallets seem to be quite low. By adding another password, you introduce yet another way for people to lose their coins. Forget one of two passwords and you are screwed.

Hi Achow,

just as an analogy .. can you tell more about the possibleUI/UX issue related ..
I think I can make an analogy to the problems I faced on openpgp  ..

Can an OpenPGP public key without UID exist?
https://stackoverflow.com/questions/38771572/can-an-openpgp-public-key-without-uid-exist

[DannyHamilton]

Hi Achow,

just as an analogy .. can you tell more about the possibleUI/UX issue related ..

He already did:

If the wallet.dat file were fully encrypted, you would not be able to see your funds, get addresses, etc. without entering the password. This means that for the wallet to be useful, you have to prompt the user to unlock the wallet at start. Unfortunately this is not really easily done with bitcoind which does not have a GUI.

At the beginning, there was no encryption. I think in the original client anyone was able to open the Bitcoin-qt client and just see your funds and use them which is insane.

Then the encryption we know nowadays was added.

But why was it choose to not fully encrypt it? I mean, you can see your funds, your sending and receiving addresses etc... by just opening the client. I don't like this.

I guess that you answered your own question quite well by pointing out that you would need to decrypt this data everytime you start your client and it would stay unencrypted like that for the whole time the program is running. Since Bitcoin Core is usually not a wallet that you just open up for a short period of time and close again

I suspect this is a big part of why that was never a thought.

Bitcoin Core has always focussed MUCH more on being a node than a wallet.  Wallet features exist, and are occasionally updated, but consensus rules, reliability, stability, and performance have always been more important.  If you want other features, use another wallet, or add the features yourself.  It's open source.  If you can get together a team of developers that will write, review, and test the code, then there's a pretty good chance that it could be pulled into Core.

[andrew1carlssin]

Hi Achow,

just as an analogy .. can you tell more about the possibleUI/UX issue related ..

He already did:

If the wallet.dat file were fully encrypted, you would not be able to see your funds, get addresses, etc. without entering the password. This means that for the wallet to be useful, you have to prompt the user to unlock the wallet at start. Unfortunately this is not really easily done with bitcoind which does not have a GUI.

well,

GUI reminds me Graphical Unit Interface ... witch reminds unix X window (X11, people used to say terrible security flaws  ) ...    

https://www.x.org/releases/X11R7.6/doc/xextproto/security.html

[HCP]

GUI reminds me Graphical Unit Interface ... witch reminds unix X window (X11, people used to say terrible security flaws  ) ...   

I think you're missing the point... Bitcoin Core is capable of being run WITHOUT a GUI (and many people use it like this). How is one supposed to prompt a user for a password to unlock the wallet at startup, if it can't actually display a prompt or accept input of any kind?

As someone else already pointed out... the wallet functionality of Bitcoin Core is more of an "add-on", offering basic features... rather than the main focus of the software. If you find that it isn't offering the features you want, your options are:

A. Add them in yourself (it's open source afterall)
or
B. Use a different wallet. There are plenty of wallets that provide for fully encrypted wallet files.

If you want/need to run a Node, but also want full wallet file encryption, there is no reason you can't simply run the node without using the wallet component of Bitcoin Core and then use another wallet application offering the full encryption.


ps. It's Graphical User Interface

[cellard]

Yup but that's what why I proposed a double lock system:

A double lock system does not get around the fact that it is still a UI/UX issue.

Furthermore, you are asking users to remember yet another password. It seems that it is far more common for people to lose their coins by forgetting their password rather than having someone steal them. Even cases of stolen encrypted wallets seem to be quite low. By adding another password, you introduce yet another way for people to lose their coins. Forget one of two passwords and you are screwed.

Fair enough, I have had problems in the past remembering password. I did full disk encryption with Veracrypt and I forgot the passwords I used, and now i can't access a couple of hard disks, but I didn't really bother remembering them that much since there wasn't anything of value inside. It would be good to have it optional.

GUI reminds me Graphical Unit Interface ... witch reminds unix X window (X11, people used to say terrible security flaws  ) ...    

I think you're missing the point... Bitcoin Core is capable of being run WITHOUT a GUI (and many people use it like this). How is one supposed to prompt a user for a password to unlock the wallet at startup, if it can't actually display a prompt or accept input of any kind?

As someone else already pointed out... the wallet functionality of Bitcoin Core is more of an "add-on", offering basic features... rather than the main focus of the software. If you find that it isn't offering the features you want, your options are:

A. Add them in yourself (it's open source afterall)
or
B. Use a different wallet. There are plenty of wallets that provide for fully encrypted wallet files.

If you want/need to run a Node, but also want full wallet file encryption, there is no reason you can't simply run the node without using the wallet component of Bitcoin Core and then use another wallet application offering the full encryption.


ps. It's Graphical User Interface

So how can you make a transaction with bitcoind in an encrypted wallet if it doesn't ask for a password?

Bitcoin Core has always focussed MUCH more on being a node than a wallet.  Wallet features exist, and are occasionally updated, but consensus rules, reliability, stability, and performance have always been more important.  If you want other features, use another wallet, or add the features yourself.  It's open source.  If you can get together a team of developers that will write, review, and test the code, then there's a pretty good chance that it could be pulled into Core.

Well, you can rename your wallet.dat file so it doesn't get opened everytime you open the client. So if you want to just have the client opened 24/7 to help the network, you can rename it, and when you need to make a transaction, you can name it back to wallet.dat. This way your wallet.dat would remain encrypted.

You aren't supposed to have your wallet.dat file in an online computer anyway, so that is not really an issue. You just want to ideally block access to wallet.dat as much as possible.

Also about separaing full node from a wallet, I agree with Luke-jr here:

It isn't secure to have a wallet without a node. Might as well be using PayPal.

60 GB is (as @jonnyb42 points out) practically nothing these days, certainly not a barrier to adoption. There are many far more important things needed before Bitcoin is ready for mainstream use.

Third-party services are, as already mentioned, not much better than PayPal (even if you hold the private keys).

https://github.com/bitcoin/bitcoin/issues/7525

So ideally, you want to have the full client to transact, and just don't have to settle to other software to hold the keys.

[ranochigo]

So how can you make a transaction with bitcoind in an encrypted wallet if it doesn't ask for a password?

Of course not. It would be a terrible design flaw if that is possible.

You need to unlock your wallet first by putting in the command walletpassphrase *PASSPHRASE* *TIMEUNLOCKED*. Some commands doesn't need it to be unlocked, some do.

You aren't supposed to have your wallet.dat file in an online computer anyway, so that is not really an issue. You just want to ideally block access to wallet.dat as much as possible.

No one said you shouldn't have it on an online computer. Everyone did say you shouldn't use it on an infected computer.

So ideally, you want to have the full client to transact, and just don't have to settle to other software to hold the keys.

If you'd like, run Bitcoin Core as a server for your whatever SPV client to connect to. Your trust lies with your Bitcoin Core instance. A bit more annoying to do but if you want it that way, there's that.

[HCP]

So how can you make a transaction with bitcoind in an encrypted wallet if it doesn't ask for a password?

Using the walletpassphrase RPC call... which isn't possible on startup.

Well, you can rename your wallet.dat file so it doesn't get opened everytime you open the client. So if you want to just have the client opened 24/7 to help the network, you can rename it, and when you need to make a transaction, you can name it back to wallet.dat. This way your wallet.dat would remain encrypted.

You can't simply rename the wallet file and Bitcoind will magically be able to read it...

You'd have to shut the node down, then rename the file, then restart the node.

Then, after doing whatever you want with wallet... You'd need to stop the node, rename wallet again... Then restart the node.

Not exactly a user friendly, nor quick method.

So ideally, you want to have the full client to transact, and just don't have to settle to other software to hold the keys.

Walletless Full Node + ElectrumX + Electrum... Problem solved.

[LoyceV]

You could just encrypt it yourself with something

This would be my suggestion too. If you want to enter a password to open your wallet in watch-only mode, you can just as well keep Bitcoin Core closed until you want to look at it. Without knowing your addresses, Bitcoin Core would require to rescan all new blocks when you look at your wallet, which makes it slower to use.
A separate encrypted partition to run Bitcoin Core would give you the double password system you're looking for: mount the encrypted partition to read your wallet, and use your wallet password to send your coins.

Personally, I think you're too paranoid And with that, I mean you're more paranoid than I am, which I consider the maximum level to keep my own sanity

[cellard]

You could just encrypt it yourself with something

This would be my suggestion too. If you want to enter a password to open your wallet in watch-only mode, you can just as well keep Bitcoin Core closed until you want to look at it. Without knowing your addresses, Bitcoin Core would require to rescan all new blocks when you look at your wallet, which makes it slower to use.
A separate encrypted partition to run Bitcoin Core would give you the double password system you're looking for: mount the encrypted partition to read your wallet, and use your wallet password to send your coins.

Personally, I think you're too paranoid And with that, I mean you're more paranoid than I am, which I consider the maximum level to keep my own sanity

Well, I think it's a must... let's say someone gets your laptop somehow, looks at your watch only wallet, sees you have 1000 BTC or something... you are fucked at this point. They will come looking for you.

If they just get prompted with a password, they may not bother, there may or not be a nice bounty in the laptop, this is a gamble in a thief's mindset. With the other scenario there is no gamble, there is a guaranteed bounty. So yes, you should not be having how much BTC you have on the clear.

Now when it comes to encryption, for Linux you have dm-crypt which is easier to use or so I've heard, and then LUKS with many other options, harder to use. Personally I haven't used any of these. I tested with Veracrypt and I screwed up because the password I set doesn't work and I can't open the disks, but it was just a test, so yeah take notes, specially when you are using special characters, if the language is changed, you may be setting a different character but you don't see it because it just shows * as you type.

[bob123]

Well, I think it's a must... let's say someone gets your laptop somehow, looks at your watch only wallet, sees you have 1000 BTC or something... you are fucked at this point. They will come looking for you.

If you are storing 1000 BTC on a laptop, you are insane.  Don't do that, and you won't have to worry.

He didn't say a word about storing them on a laptop. 

The other question is whether it is necessary to frequently monitor a cold wallet containing 1000 BTC's along the way.

[Wipro]

Well, I think it's a must... let's say someone gets your laptop somehow, looks at your watch only wallet, sees you have 1000 BTC or something... you are fucked at this point. They will come looking for you.

If you are storing 1000 BTC on a laptop, you are insane.  Don't do that, and you won't have to worry.

He didn't say a word about storing them on a laptop. 

The other question is whether it is necessary to frequently monitor a cold wallet containing 1000 BTC's along the way.

LOL, by the block explorer is allow you to find the transaction with the bitcoin public address or tx id of concern transaction mate.
Since wallet.dat file is been used to decrypt the wallet. I am not sure why he is asking this for more encryption with the .dat file.

I do not know whether op saw the richest bitcoin address or not. Still they are safe right.

[LoyceV]

Well, I think it's a must... let's say someone gets your laptop somehow, looks at your watch only wallet, sees you have 1000 BTC or something... you are fucked at this point. They will come looking for you.

If you have 1000 BTC there are better options to store it than a wallet.dat.

If they just get prompted with a password, they may not bother

Like I said: encrypt your laptop. Then make sure you always lock the screen when you leave it, or put your screen locker to a very low minimum time, and no thief will ever be able to use it.

Now when it comes to encryption, for Linux you have dm-crypt which is easier to use or so I've heard

In many (most?) Linux distributions, you can just click "Encrypt home folder to protect sensitive data" when you create a new user account. I didn't like the increased disk activity as it creates encrypted files on a normal file system.
I'd prefer to encrypt the whole partition, but that's a bit more work to setup.

[bob123]

LOL, by the block explorer is allow you to find the transaction with the bitcoin public address or tx id of concern transaction mate.

How is this related to the topic?

Since wallet.dat file is been used to decrypt the wallet. I am not sure why he is asking this for more encryption with the .dat file.

Thats not true.
The wallet.dat file is the 'storage' of information (private keys, addresses, labels, ..) for the actual wallet (bitcoin core client).
A wallet is a piece of software (or a printed paper, hardware-) which manages private-/public keys.

The client itself is not being encrypted. Sensitive information are being encrypted (e.g. private keys / seed inside wallet.dat or wallet.dat itself).
Either with your password or a randomly generated key (which is tied to your password).

I do not know whether op saw the richest bitcoin address or not. Still they are safe right.

This has nothing to do with people at your physical location realizing you own massive amounts of money.
You might read the thread again.

[rozenkreitzer]

At the beginning, there was no encryption. I think in the original client anyone was able to open the Bitcoin-qt client and just see your funds and use them which is insane.

Then the encryption we know nowadays was added.

But why was it choose to not fully encrypt it? I mean, you can see your funds, your sending and receiving addresses etc... by just opening the client. I don't like this.

Why not add full encryption like Electrum? maybe with 2 different password, one to open it, and then the current one to confirm transactions, so you don't have to enter the pass that actually allows you to transact each time you want to open the wallet... even tho it shouldn't matter if it's cold storage.

To be honest, I'm a little embarrassed by what I learned about the fact that the file wallet.dat is not completely encrypted. Now when you said this - i ask myself the same question. In fact, it's not so difficult, Why did not the developers do this?