Bitcoin Forum
September 22, 2018, 10:08:30 PM *
News: ♦♦ New info! Bitcoin Core users absolutely must upgrade to previously-announced 0.16.3 [Torrent]. All Bitcoin users should temporarily trust confirmations slightly less. More info.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: secure private key handling  (Read 2580 times)
doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
July 16, 2014, 09:45:43 PM
 #1

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
1537654110
Hero Member
*
Offline Offline

Posts: 1537654110

View Profile Personal Message (Offline)

Ignore
1537654110
Reply with quote  #2

1537654110
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1537654110
Hero Member
*
Offline Offline

Posts: 1537654110

View Profile Personal Message (Offline)

Ignore
1537654110
Reply with quote  #2

1537654110
Report to moderator
1537654110
Hero Member
*
Offline Offline

Posts: 1537654110

View Profile Personal Message (Offline)

Ignore
1537654110
Reply with quote  #2

1537654110
Report to moderator
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 17, 2014, 03:31:35 AM
 #2

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

This answer is sort of cheating, but any deterministic wallet (Armory, Electrum, MultiBit HD) should qualify (even if it's not because they do so explicitly), assuming you do a reboot or a sync after initial wallet creation.
shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1470
Merit: 1270


No I dont escrow anymore.


View Profile WWW
July 17, 2014, 06:58:13 AM
 #3

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

This answer is sort of cheating, but any deterministic wallet (Armory, Electrum, MultiBit HD) should qualify (even if it's not because they do so explicitly), assuming you do a reboot or a sync after initial wallet creation.

Electrum and Multibit are written in Java IIRC, thus AFAIK you have no control over the memory. So while a reboot is a solution to wipe the key off memory, OP was (if I got that right) talking about the moment before you send a TX. You need the private key in memory in order to sign the TX. Then the signed TX gets broadcasted over the network. OP wants a wallet that makes sure the private key is off of memory when the TX gets broadcasted. A reboot would maybe work here, not sure how e.g. bitcoin core handles a reboot just before the final confirmation ("are you sure you want to send x btc to y?), but I doubt it will just continue like nothing happened.

@OP any Java is not an option. I might be mistaken in my Java knowledge, but AFAIK as a Java dev you do not have sufficient control over memory to ensure this. Apparenty [1] you have to go out of your way and rewrite the data that previously stored your key. Since most wallets are open source you can check if they do that.




[1] https://stackoverflow.com/questions/6473352/java-security-how-to-clear-zero-out-memory-associated-with-an-object-and-or-e
Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
July 17, 2014, 07:49:50 AM
 #4

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

This answer is sort of cheating, but any deterministic wallet (Armory, Electrum, MultiBit HD) should qualify (even if it's not because they do so explicitly), assuming you do a reboot or a sync after initial wallet creation.

Electrum and Multibit are written in Java IIRC, thus AFAIK you have no control over the memory. So while a reboot is a solution to wipe the key off memory, OP was (if I got that right) talking about the moment before you send a TX. You need the private key in memory in order to sign the TX. Then the signed TX gets broadcasted over the network. OP wants a wallet that makes sure the private key is off of memory when the TX gets broadcasted. A reboot would maybe work here, not sure how e.g. bitcoin core handles a reboot just before the final confirmation ("are you sure you want to send x btc to y?), but I doubt it will just continue like nothing happened.

@OP any Java is not an option. I might be mistaken in my Java knowledge, but AFAIK as a Java dev you do not have sufficient control over memory to ensure this. Apparenty [1] you have to go out of your way and rewrite the data that previously stored your key. Since most wallets are open source you can check if they do that.




[1] https://stackoverflow.com/questions/6473352/java-security-how-to-clear-zero-out-memory-associated-with-an-object-and-or-e

I'm reading the OP's question in a different way. He wants to ensure the private key has been written to disk so that if there is sudden power loss you don't loose bitcoins. Sort of an atomic operation. I don't think he's worried about some malware reading the keys from memory. If you have malware it could just as easily read the keys from disk.

(Also, electrum is coded in python not java. Your point about lack of low level hardware access stands, though.)

shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1470
Merit: 1270


No I dont escrow anymore.


View Profile WWW
July 17, 2014, 08:05:02 AM
 #5

-snip-
I'm reading the OP's question in a different way. He wants to ensure the private key has been written to disk so that if there is sudden power loss you don't loose bitcoins. Sort of an atomic operation. I don't think he's worried about some malware reading the keys from memory. If you have malware it could just as easily read the keys from disk.

(Also, electrum is coded in python not java. Your point about lack of low level hardware access stands, though.)

You are probably right, now that I read it again "write behind cache" makes more sense if we are talking about writing to HDD.

But wouldnt this only be relevant after creating a new private key? So If I want to be paranoid about that a reboot would indeed be a solution. Open wallet -> create new private key and address -> close wallet -> reboot system -> open wallet and use the newly generated address. Repeat if you need a new address or generate a sufficient number of addresses the first time.
Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
July 17, 2014, 08:28:47 AM
 #6

-snip-
I'm reading the OP's question in a different way. He wants to ensure the private key has been written to disk so that if there is sudden power loss you don't loose bitcoins. Sort of an atomic operation. I don't think he's worried about some malware reading the keys from memory. If you have malware it could just as easily read the keys from disk.

(Also, electrum is coded in python not java. Your point about lack of low level hardware access stands, though.)

You are probably right, now that I read it again "write behind cache" makes more sense if we are talking about writing to HDD.

But wouldnt this only be relevant after creating a new private key? So If I want to be paranoid about that a reboot would indeed be a solution. Open wallet -> create new private key and address -> close wallet -> reboot system -> open wallet and use the newly generated address. Repeat if you need a new address or generate a sufficient number of addresses the first time.

I suspect he's been reading this thread:

https://bitcointalk.org/index.php?topic=678910.0

shorena
Copper Member
Legendary
*
Offline Offline

Activity: 1470
Merit: 1270


No I dont escrow anymore.


View Profile WWW
July 17, 2014, 09:28:16 AM
 #7

-snip-
I'm reading the OP's question in a different way. He wants to ensure the private key has been written to disk so that if there is sudden power loss you don't loose bitcoins. Sort of an atomic operation. I don't think he's worried about some malware reading the keys from memory. If you have malware it could just as easily read the keys from disk.

(Also, electrum is coded in python not java. Your point about lack of low level hardware access stands, though.)

You are probably right, now that I read it again "write behind cache" makes more sense if we are talking about writing to HDD.

But wouldnt this only be relevant after creating a new private key? So If I want to be paranoid about that a reboot would indeed be a solution. Open wallet -> create new private key and address -> close wallet -> reboot system -> open wallet and use the newly generated address. Repeat if you need a new address or generate a sufficient number of addresses the first time.

I suspect he's been reading this thread:

https://bitcointalk.org/index.php?topic=678910.0

Well my interpretation of the lack of response form OP in that thread is that OP actually found the private key in said folder but is ashamed to admin that the solution was that obvious Wink Either that or losing this many coins drove him/her away from bitcoin forever.

Quote
in June i created another receiving address

Yet posted it in July. I would be very surprised if MultiBit would not write the private key to disk after days (at least 6 if the time OP posted is any indictator).

Anyway. I just realized a way to make sure you have all the private keys on (at least one) disk. Make a backup on a USB drive after creating a new address. This way when you (properly) remove the drive from the machine you know that the OS commited the write to (at least) the USB drive. In case anything goes wrong you can restore from that backup.
This is probably the only 100% way anyway, since writing to disk is ultimatly a system call which could be faulty. A backup on USB (or any other removeable media) would allow you to verifiy the backup with another machine, preferably running a different OS. I think we are back in the tin foil hat area again Wink

Id recomment regular, redudant backups anyway, depending which wallet you use. The data on your disk could be corrupted any time so after creating new private key(s) a new backup should be common sense.

This is void if you use a wallet like electrum or dark wallet where you only need to backup the initial seed once.

Bitcoin core has several private keys allready pregenerated thus - in theory - you only have to backup every 100 new addresses.
doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
July 17, 2014, 10:36:49 AM
 #8

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

This answer is sort of cheating, but any deterministic wallet (Armory, Electrum, MultiBit HD) should qualify (even if it's not because they do so explicitly), assuming you do a reboot or a sync after initial wallet creation.

I wouldn't regard the usage of a deterministic wallet as "cheating", but of course a deterministic wallet comes with pros and cons of its own. But assuming that a reboot or sync is done by the user could be considered as "cheating". Basically, this would be the same as using a "regular" wallet software, and doing the private sync manually. I'm looking for something foolproof, where other components in the software stack can rely on every private key in use being persistently stored. If there is a deterministic wallet which enforces the sync in software, this would qualify (assuming that a deterministic wallet is applicable).

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
July 17, 2014, 11:18:10 AM
 #9

Hi!

Is there any wallet software which guarantees that private keys are always committed to disk (using appropriate kernel primitives - there must be no chance that the key is still waiting in a write-behind cache) before being used on the network?

Thanks!

This answer is sort of cheating, but any deterministic wallet (Armory, Electrum, MultiBit HD) should qualify (even if it's not because they do so explicitly), assuming you do a reboot or a sync after initial wallet creation.

Electrum and Multibit are written in Java IIRC, thus AFAIK you have no control over the memory. So while a reboot is a solution to wipe the key off memory, OP was (if I got that right) talking about the moment before you send a TX. You need the private key in memory in order to sign the TX. Then the signed TX gets broadcasted over the network. OP wants a wallet that makes sure the private key is off of memory when the TX gets broadcasted. A reboot would maybe work here, not sure how e.g. bitcoin core handles a reboot just before the final confirmation ("are you sure you want to send x btc to y?), but I doubt it will just continue like nothing happened.

@OP any Java is not an option. I might be mistaken in my Java knowledge, but AFAIK as a Java dev you do not have sufficient control over memory to ensure this. Apparenty [1] you have to go out of your way and rewrite the data that previously stored your key. Since most wallets are open source you can check if they do that.




[1] https://stackoverflow.com/questions/6473352/java-security-how-to-clear-zero-out-memory-associated-with-an-object-and-or-e

I'm reading the OP's question in a different way. He wants to ensure the private key has been written to disk so that if there is sudden power loss you don't loose bitcoins. Sort of an atomic operation. I don't think he's worried about some malware reading the keys from memory. If you have malware it could just as easily read the keys from disk.

(Also, electrum is coded in python not java. Your point about lack of low level hardware access stands, though.)

Yes. I am currently more concerned about reducing the risk of key loss. I think the client should not run in an environment which makes it easy to sneak in malware anyway.

I also don't think the client I'm looking for would necessarily degrade security. The keys could still be written to disk encrypted.

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
July 17, 2014, 11:34:36 AM
 #10

I wouldn't regard the usage of a deterministic wallet as "cheating", but of course a deterministic wallet comes with pros and cons of its own. But assuming that a reboot or sync is done by the user could be considered as "cheating". Basically, this would be the same as using a "regular" wallet software, and doing the private sync manually. I'm looking for something foolproof, where other components in the software stack can rely on every private key in use being persistently stored. If there is a deterministic wallet which enforces the sync in software, this would qualify (assuming that a deterministic wallet is applicable).

The sync only needs to be done once with a deterministic wallet because all private keys are generated from the seed. Actually with electrum you don't even need to sync. You have to write down the seed on a piece of paper as part of the wallet creation process so there's your backup. It is all you need to generate any and all private keys in that wallet.

doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
July 18, 2014, 01:57:36 PM
 #11

I wouldn't regard the usage of a deterministic wallet as "cheating", but of course a deterministic wallet comes with pros and cons of its own. But assuming that a reboot or sync is done by the user could be considered as "cheating". Basically, this would be the same as using a "regular" wallet software, and doing the private sync manually. I'm looking for something foolproof, where other components in the software stack can rely on every private key in use being persistently stored. If there is a deterministic wallet which enforces the sync in software, this would qualify (assuming that a deterministic wallet is applicable).

The sync only needs to be done once with a deterministic wallet because all private keys are generated from the seed. Actually with electrum you don't even need to sync. You have to write down the seed on a piece of paper as part of the wallet creation process so there's your backup. It is all you need to generate any and all private keys in that wallet.

Yes, I'm aware of deterministic wallets and how they work cryptography-wise. It would work in cases where the determinism is permissible. However, a setup process where you have to write something down would be bad for automating. Electrum, supplemented with a printer interface which blocks until the key has been printed out could work. But I understand that there is no ready-to-use solution yet which automates the process of having keys securely synced before being used on the network.

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
dabura667
Sr. Member
****
Offline Offline

Activity: 478
Merit: 250


View Profile
July 18, 2014, 04:06:02 PM
 #12

Yes, I'm aware of deterministic wallets and how they work cryptography-wise. It would work in cases where the determinism is permissible. However, a setup process where you have to write something down would be bad for automating. Electrum, supplemented with a printer interface which blocks until the key has been printed out could work. But I understand that there is no ready-to-use solution yet which automates the process of having keys securely synced before being used on the network.

The writing down seed step is merely in place for human backupability. If you were to code a separate project that would automate wallet creation for some kind of script... a la bitpay's merchant address generation script etc... you wouldn't need to write down anything and you could store the hex seed encrypted and not even bother with making it into a mnemonic.

I am not quite understanding what you are trying to do.

I also don't really understand your main concern either... but I think if you tell us what you are trying to do, we will be better equipped to assist you.

My Tip Address:
1DXcHTJS2DJ3xDoxw22wCt11FeAsgfzdBU
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 18, 2014, 11:30:52 PM
 #13

I also don't really understand your main concern either... but I think if you tell us what you are trying to do, we will be better equipped to assist you.

Agreed... not that I'm claiming I'm knowledgeable enough to help, but I'll bet someone here is. Smiley

Electrum, supplemented with a printer interface which blocks until the key has been printed out could work. But I understand that there is no ready-to-use solution yet which automates the process of having keys securely synced before being used on the network.

This might be harder than you think, if you really want 100% foolproofness. Printers can run out of ink, hardware RAID can lie to the OS and claim that data has been written if it thinks its backup batteries are sufficient... there are a lot of what-ifs if you don't go into more details.

What I'm trying to say is that even if there is a client that makes the appropriate sync call, things can still go wrong. At some point, it becomes more about risk management than about 100% guarantees...

Also, for what it's worth, I can say that neither Bitcoin Core nor Armory (my two personal favorites...) force a sync after modifying a wallet file.
trasla
Hero Member
*****
Offline Offline

Activity: 713
Merit: 500



View Profile
July 22, 2014, 02:28:46 PM
 #14

I dont get the point...
Where does the key come from?
If I generated a new address, how am I supposed to be able to immediately send from it?
There is nothing on it... and until I received funds there, every app will have had plenty of time to persist the key, wont it?
If I scanned the key, well, i have a persistent copy already.

Is your concern something like:
I generate address, hand it to someone, then some crash occurs, I dont have the priv key, but receive money on the address?
Really, as well as the key being trapped in some buffer, your smartphone could catch fire or whatever.
Just use keys with a backup Smiley
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 22, 2014, 10:23:24 PM
 #15

There is nothing on it... and until I received funds there, every app will have had plenty of time to persist the key, wont it?

Probably true if it's a human generating the key, but what if it's a website application that's generating the key? I could easily see a case where the key is generated, persisted only to the disk cache, then a page with the address is sent to some user, and then a crash occurs...

Of course servers shouldn't be generating private keys at all, they should either be using a cache of public addresses or generating addresses deterministically from a master public key. 
trasla
Hero Member
*****
Offline Offline

Activity: 713
Merit: 500



View Profile
July 23, 2014, 09:24:17 AM
 #16

Yes, servers shouldnt do on-the-fly key generation without master key, and the questions seemsto be pointed at user wallets, not online shop software.
doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
July 29, 2014, 10:39:03 AM
 #17

I dont get the point...
Where does the key come from?
If I generated a new address, how am I supposed to be able to immediately send from it?
There is nothing on it... and until I received funds there, every app will have had plenty of time to persist the key, wont it?
If I scanned the key, well, i have a persistent copy already.

Is your concern something like:
I generate address, hand it to someone, then some crash occurs, I dont have the priv key, but receive money on the address?
Really, as well as the key being trapped in some buffer, your smartphone could catch fire or whatever.
Just use keys with a backup Smiley

That's the point. If I have a wallet software which does predictable syncing of private keys prior to their usage, it can be plugged to a filesystem abstraction offering additional security like redundant storage (thereby handling your flaming smartphone concern).

Secure software isn't normally designed using the "we don't need security because things could catch fire" paradigm.

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
August 01, 2014, 12:01:16 AM
 #18

Electrum, supplemented with a printer interface which blocks until the key has been printed out could work. But I understand that there is no ready-to-use solution yet which automates the process of having keys securely synced before being used on the network.

Printers are even slower than hard disks and they have their own buffers too. The OS maintains a print spool file as well. So there are many levels of buffering. Electrum would be told it's "all good, a-ok, go ahead" but in reality it's still sitting in one of those intermediary places and hasn't yet been printed onto paper.


doldgigger
Full Member
***
Offline Offline

Activity: 170
Merit: 100


View Profile
August 12, 2014, 10:24:22 AM
 #19

Electrum, supplemented with a printer interface which blocks until the key has been printed out could work. But I understand that there is no ready-to-use solution yet which automates the process of having keys securely synced before being used on the network.

Printers are even slower than hard disks and they have their own buffers too. The OS maintains a print spool file as well. So there are many levels of buffering. Electrum would be told it's "all good, a-ok, go ahead" but in reality it's still sitting in one of those intermediary places and hasn't yet been printed onto paper.



It's not so different from write-behind caching. Would be the responsibility of the printer driver supplier to properly implement a synchronous printing mode. Anyway, the best hard disk or printer driver won't help if the Bitcoin client doesn't enforce synchronous writing of private keys on its side.

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!