Bitcoin Forum
September 21, 2018, 03:22:21 AM *
News: ♦♦ New info! Bitcoin Core users absolutely must upgrade to previously-announced 0.16.3 [Torrent]. All Bitcoin users should temporarily trust confirmations slightly less. More info.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Help with key imports and finding the best client  (Read 1491 times)
TerraHasher
Full Member
***
Offline Offline

Activity: 140
Merit: 100


View Profile
July 27, 2014, 01:22:47 AM
 #1

What is the best way to import a few million private keys into a single wallet? And, is there one client that is better for that and handling that kind of load than another?

We just had 3 servers crash collectively with a few million addresses on them and after trying to run the backup .dat files, they are all corrupt. Luckily we have a dump file in csv that has all the private keys backed up individually as it stores them after each address is used.

Just in case your scratching your head thinking what the hell would someone need or use a few million addresses for, a few of us have been running a private tumbling service for a few private clients on a now and then basis.

Thanks in advance guys  Wink

Donations: 1A8anGihV3pvHeuhqzGsAjt2S71WY2qea4
1537500141
Hero Member
*
Offline Offline

Posts: 1537500141

View Profile Personal Message (Offline)

Ignore
1537500141
Reply with quote  #2

1537500141
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1537500141
Hero Member
*
Offline Offline

Posts: 1537500141

View Profile Personal Message (Offline)

Ignore
1537500141
Reply with quote  #2

1537500141
Report to moderator
1537500141
Hero Member
*
Offline Offline

Posts: 1537500141

View Profile Personal Message (Offline)

Ignore
1537500141
Reply with quote  #2

1537500141
Report to moderator
The Ferox
Member
**
Offline Offline

Activity: 72
Merit: 10


View Profile
July 27, 2014, 01:30:48 AM
 #2

Thats a rough spot to be in, lucky you had the key dump file  Grin I am in a similar spot myself, only just a thousand or so addresses from years back. I'll be keeping my eye out though to see what the community has to say about this, i too would be interested in a good answer.
dabura667
Sr. Member
****
Offline Offline

Activity: 478
Merit: 250


View Profile
July 27, 2014, 07:58:30 AM
 #3

For that number of addresses you really need a full-node client. (Armory or Core)

If you don't, then querying the lite-wallet servers will probably get you soft-banned from querying them (they'll view you as DOSing them)

I would use Core and write a script that will run the following RPC call for each private key
Code:
importprivkey <privatekey> "" False
(The private keys should be in WIF format and without the <> or any "" around them...)

The [ "" False ] part is a must. only remove that false (or change it to True) on the LAST private key you import...

If you don't, it will rescan the blockchain after every import and take a REAAAAALLLY long time, if it doesn't crash.




Maybe Armory has a better solution, but afaik bitcoind would be the best way to go.

My Tip Address:
1DXcHTJS2DJ3xDoxw22wCt11FeAsgfzdBU
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 27, 2014, 04:45:22 PM
 #4

For that number of addresses you really need a full-node client. (Armory or Core)

If you don't, then querying the lite-wallet servers will probably get you soft-banned from querying them (they'll view you as DOSing them)

I would use Core and write a script that will run the following RPC call for each private key
Code:
importprivkey <privatekey> "" False
(The private keys should be in WIF format and without the <> or any "" around them...)

The [ "" False ] part is a must. only remove that false (or change it to True) on the LAST private key you import...

If you don't, it will rescan the blockchain after every import and take a REAAAAALLLY long time, if it doesn't crash.




Maybe Armory has a better solution, but afaik bitcoind would be the best way to go.

I'd tend to agree with using Bitcoin Core, even though it looks like that's what you were using when the problem occurred. Bitcoin Core uses Berkeley DB as its wallet.dat format. Berkeley DB is a bit bloated, but it's mature, very scalable, and has some reasonably decent recovery tools. I'm not aware of any other client that would be equally scalable (not that I'm an expert on alternative clients though...).

As much as I like Armory, I don't think it would scale as well (although it's wallet format is nice and simple and more error-resistant than any other wallet format I'm aware of).

For importing, as an alternative you could use the RPC command:
Code:
importwallet "filename"

The wallet import format is a bunch of lines like so:

Code:
Kx2GEDGhepMhkUnymSFtrU2Q59KMPY75oYjBVofYV9032p5L4a2b 2014-06-19T00:58:01Z change=1 reserve=1 label=label%20with%20spaces

The first two fields are required. If you don't have the address creation time, you can replace that field with something meaningless (e.g. "DATE-UNKNOWN") but something needs to be there. The last three fields are each optional. Only a single rescan is done at the end of the file import, and if you do have all of the creation times, the rescan will intelligently start at the right time instead of starting at the genesis block.
The Ferox
Member
**
Offline Offline

Activity: 72
Merit: 10


View Profile
July 27, 2014, 07:55:13 PM
 #5

Sorry for semi-hijacking the thread Terra


BTCchirs, Totally noob when it comes to the advance stuff. To do what you said i would basically open Bitcoind, and in the command line just type importwallet C:\myfilenamehere then hit enter? and from there it would import the entire file? or is there some step i am missing? i am not too familiar with the backend stuff or RPC
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 27, 2014, 09:14:11 PM
 #6

Sorry for semi-hijacking the thread Terra


BTCchirs, Totally noob when it comes to the advance stuff. To do what you said i would basically open Bitcoind, and in the command line just type importwallet C:\myfilenamehere then hit enter? and from there it would import the entire file? or is there some step i am missing? i am not too familiar with the backend stuff or RPC

First off, this is only for importing wallets that were exported using the "dumpwallet" command or in the same format, it won't import an actual wallet.dat file (which you probably already know, but I'm just making sure...)

Are you using the full bitcoin-qt.exe program, with the graphical interface for sending/receiving? Or the command-line bitcoind.exe?

If you're using the full bitcoin-qt.exe, you'd start it up, click the Help menu, choose Debug window, and then click the Console tab. At the bottom of this window, there's a text entry box, and in this box you'd type exactly this, including the quotes, and then hit enter:

Code:
importwallet "C:\myfilenamehere"

If you're using bitcoind.exe (I'm guessing you're not), you'd start it up first, and then type this at a Command Prompt, including the quotes:

Code:
cd \Program Files\Bitcoin\daemon
bitcoin-cli importwallet "C:\myfilenamehere"
The Ferox
Member
**
Offline Offline

Activity: 72
Merit: 10


View Profile
July 27, 2014, 11:16:20 PM
 #7

Thanks Chris, i am going to give it a try as soon as i get home. If this works, i owe you one big time.
The Ferox
Member
**
Offline Offline

Activity: 72
Merit: 10


View Profile
July 27, 2014, 11:23:25 PM
 #8

Just one quick question in case your not on when i get home does the file path have to be in quotes "C:\myfilenamehere" or are the quotes not needed?
The Ferox
Member
**
Offline Offline

Activity: 72
Merit: 10


View Profile
July 28, 2014, 03:10:17 AM
 #9

Ok, a few questions here crhis,

1: what format (.txt, .bat, .dat etc.) does the address file have to be in. I cannot seem to get it to work. it ran as a .txt but then i did not see any addresses in any fields in the wallet so i assume they did not import.

2: if they did import where do i go to look as i cannot find anything as of now.

Thanks
btchris
Hero Member
*****
Offline Offline

Activity: 672
Merit: 501

a.k.a. gurnec on GitHub


View Profile WWW
July 28, 2014, 12:15:17 PM
 #10

Just one quick question in case your not on when i get home does the file path have to be in quotes "C:\myfilenamehere" or are the quotes not needed?

Include the quotes.

Ok, a few questions here crhis,

1: what format (.txt, .bat, .dat etc.) does the address file have to be in. I cannot seem to get it to work. it ran as a .txt but then i did not see any addresses in any fields in the wallet so i assume they did not import.

The import file is a normal ASCII .txt file, the kind you'd create with Notepad.

2: if they did import where do i go to look as i cannot find anything as of now.

You could check the debug log: Help menu -> Debug window, then click the Open button, or look for debug.log in the %AppData%\Bitcoin folder. It should have "Importing" lines if the imports succeeded.

You didn't include "change=1" or "reserve=1" did you? If you did, you've effectively hidden them all.... If you were importing into an empty wallet, I'd create a new empty wallet and try again. If you were importing into an existing wallet, do you have any backups of the wallet that you can restore and then try the import again?

Whatever you do, make sure to keep lots of backups of the wallet.dat before you start mucking around Wink
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!