Help with key imports and finding the best client

[TerraHasher]

What is the best way to import a few million private keys into a single wallet? And, is there one client that is better for that and handling that kind of load than another?

We just had 3 servers crash collectively with a few million addresses on them and after trying to run the backup .dat files, they are all corrupt. Luckily we have a dump file in csv that has all the private keys backed up individually as it stores them after each address is used.

Just in case your scratching your head thinking what the hell would someone need or use a few million addresses for, a few of us have been running a private tumbling service for a few private clients on a now and then basis.

Thanks in advance guys 

[The Ferox]

Thats a rough spot to be in, lucky you had the key dump file  I am in a similar spot myself, only just a thousand or so addresses from years back. I'll be keeping my eye out though to see what the community has to say about this, i too would be interested in a good answer.

[dabura667]

For that number of addresses you really need a full-node client. (Armory or Core)

If you don't, then querying the lite-wallet servers will probably get you soft-banned from querying them (they'll view you as DOSing them)

I would use Core and write a script that will run the following RPC call for each private key

Code:

importprivkey <privatekey> "" False

(The private keys should be in WIF format and without the <> or any "" around them...)

The [ "" False ] part is a must. only remove that false (or change it to True) on the LAST private key you import...

If you don't, it will rescan the blockchain after every import and take a REAAAAALLLY long time, if it doesn't crash.




Maybe Armory has a better solution, but afaik bitcoind would be the best way to go.

[btchris]

For that number of addresses you really need a full-node client. (Armory or Core)

If you don't, then querying the lite-wallet servers will probably get you soft-banned from querying them (they'll view you as DOSing them)

I would use Core and write a script that will run the following RPC call for each private key

Code:

importprivkey <privatekey> "" False

(The private keys should be in WIF format and without the <> or any "" around them...)

The [ "" False ] part is a must. only remove that false (or change it to True) on the LAST private key you import...

If you don't, it will rescan the blockchain after every import and take a REAAAAALLLY long time, if it doesn't crash.




Maybe Armory has a better solution, but afaik bitcoind would be the best way to go.

I'd tend to agree with using Bitcoin Core, even though it looks like that's what you were using when the problem occurred. Bitcoin Core uses Berkeley DB as its wallet.dat format. Berkeley DB is a bit bloated, but it's mature, very scalable, and has some reasonably decent recovery tools. I'm not aware of any other client that would be equally scalable (not that I'm an expert on alternative clients though...).

As much as I like Armory, I don't think it would scale as well (although it's wallet format is nice and simple and more error-resistant than any other wallet format I'm aware of).

For importing, as an alternative you could use the RPC command:

Code:

importwallet "filename"

The wallet import format is a bunch of lines like so:

Code:

Kx2GEDGhepMhkUnymSFtrU2Q59KMPY75oYjBVofYV9032p5L4a2b 2014-06-19T00:58:01Z change=1 reserve=1 label=label%20with%20spaces

The first two fields are required. If you don't have the address creation time, you can replace that field with something meaningless (e.g. "DATE-UNKNOWN") but something needs to be there. The last three fields are each optional. Only a single rescan is done at the end of the file import, and if you do have all of the creation times, the rescan will intelligently start at the right time instead of starting at the genesis block.

[The Ferox]

Sorry for semi-hijacking the thread Terra


BTCchirs, Totally noob when it comes to the advance stuff. To do what you said i would basically open Bitcoind, and in the command line just type importwallet C:\myfilenamehere then hit enter? and from there it would import the entire file? or is there some step i am missing? i am not too familiar with the backend stuff or RPC

[btchris]

Sorry for semi-hijacking the thread Terra


BTCchirs, Totally noob when it comes to the advance stuff. To do what you said i would basically open Bitcoind, and in the command line just type importwallet C:\myfilenamehere then hit enter? and from there it would import the entire file? or is there some step i am missing? i am not too familiar with the backend stuff or RPC

First off, this is only for importing wallets that were exported using the "dumpwallet" command or in the same format, it won't import an actual wallet.dat file (which you probably already know, but I'm just making sure...)

Are you using the full bitcoin-qt.exe program, with the graphical interface for sending/receiving? Or the command-line bitcoind.exe?

If you're using the full bitcoin-qt.exe, you'd start it up, click the Help menu, choose Debug window, and then click the Console tab. At the bottom of this window, there's a text entry box, and in this box you'd type exactly this, including the quotes, and then hit enter:

Code:

importwallet "C:\myfilenamehere"

If you're using bitcoind.exe (I'm guessing you're not), you'd start it up first, and then type this at a Command Prompt, including the quotes:

Code:

cd \Program Files\Bitcoin\daemon
bitcoin-cli importwallet "C:\myfilenamehere"

[The Ferox]

Thanks Chris, i am going to give it a try as soon as i get home. If this works, i owe you one big time.

[The Ferox]

Just one quick question in case your not on when i get home does the file path have to be in quotes "C:\myfilenamehere" or are the quotes not needed?

[The Ferox]

Ok, a few questions here crhis,

1: what format (.txt, .bat, .dat etc.) does the address file have to be in. I cannot seem to get it to work. it ran as a .txt but then i did not see any addresses in any fields in the wallet so i assume they did not import.

2: if they did import where do i go to look as i cannot find anything as of now.

Thanks

[btchris]

Just one quick question in case your not on when i get home does the file path have to be in quotes "C:\myfilenamehere" or are the quotes not needed?

Include the quotes.

Ok, a few questions here crhis,

1: what format (.txt, .bat, .dat etc.) does the address file have to be in. I cannot seem to get it to work. it ran as a .txt but then i did not see any addresses in any fields in the wallet so i assume they did not import.

The import file is a normal ASCII .txt file, the kind you'd create with Notepad.

2: if they did import where do i go to look as i cannot find anything as of now.

You could check the debug log: Help menu -> Debug window, then click the Open button, or look for debug.log in the %AppData%\Bitcoin folder. It should have "Importing" lines if the imports succeeded.

You didn't include "change=1" or "reserve=1" did you? If you did, you've effectively hidden them all.... If you were importing into an empty wallet, I'd create a new empty wallet and try again. If you were importing into an existing wallet, do you have any backups of the wallet that you can restore and then try the import again?

Whatever you do, make sure to keep lots of backups of the wallet.dat before you start mucking around