pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 04:01:23 PM |
|
Hello guys!
Is it possible to dump all private keys from a huge wallet in Bitcoin Core (or outside) that had any movement of BTC on them?
How could I do that?
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17342
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 04:09:10 PM |
|
Is it possible to dump all private keys from a huge wallet in Bitcoin Core (or outside) that had any movement of BTC on them? I think you're looking for the dumpwallet command: click Window > Console and type " dumpwallet": dumpwallet "filename"
Dumps all wallet keys in a human-readable format to a server-side file. This does not allow overwriting existing files. Imported scripts are included in the dumpfile, but corresponding BIP173 addresses, etc. may not be added automatically by importwallet. Note that if your wallet contains keys which are not derived from your HD seed (e.g. imported keys), these are not covered by only backing up the seed itself, and must be backed up too (e.g. ensure you back up the whole dumpfile).
Arguments: 1. filename (string, required) The filename with path (absolute path recommended)
Result: { (json object) "filename" : "str" (string) The filename with full absolute path }
Examples: > bitcoin-cli dumpwallet "test" > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "dumpwallet", "params": ["test"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/ (code -1) I don't think this is limited to keys with "movement" though, it dumps all keys. But for a huge wallet, I expect most of the keys to have been used.
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 04:35:29 PM |
|
Dumpwallet can help you narrow down your range of addresses. You might want to use listreceivedbyaddress RPC and specify your own parameters to dump the appropriate addresses.
|
|
|
|
NotATether
Legendary
Offline
Activity: 1722
Merit: 7203
In memory of o_e_l_e_o
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 05:43:45 PM |
|
You will probably want to watch by address instead of by private key as there is already software you can run on your box that can do just that (electrum server, in particular). It would require you to keep a mapping between addresses and their corresponding private keys so I think it's only feasible in the scale of thousands.
|
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 06:51:22 PM |
|
I surely know dumpwallet command, but it is not what I am looking for. Dumpwallet would be ok if it could only dump private keys of used addresses/keys.
I don't want to go to other app for that, so electrum and it's 1000 limit for one wallet is not enough for me. Simply it is blocking IP addresses with larger wallets.
Eventually I could use dump wallet but how to search it then for used addresses? I can make csv consisting of private and public keys/addresses.
I am talking here about 7.5M addresses/keys per one wallet.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
mynonce
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 07:13:11 PM |
|
... I am talking here about 7.5M addresses/keys per one wallet.
7.5M addresses/keys per one wallet? SATOSHI IS HERE! I'm just joking ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Happy new year!
|
|
|
|
jackg
Copper Member
Legendary
Offline
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 08:04:05 PM |
|
Can you program this as a shell script or externally in any way?
I'd look at something in python and going: dump wallet (to dump the addresses) Iterate address over addresses If getreceivedbyaddress(address) not zero Open a file editor and file file add new line dumpprivkey(address) close file
Why you'd want to iterate over 7M addresses is another question entirely but I'm assuming there's a chance if someone owned a faucet or something. Or if someone's testing it's possibility before steganographically trying to secure something.
|
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 08:24:39 PM |
|
Yes, I can do it in Python. Bash no because it is too slow. Your idea for script is good. Script should do the work.
I have csv files: transactions, balances. Only have to connect transactions and balances with public addresses/private keys.
Will think about it.
Thanks!
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17342
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 08:32:10 PM |
|
I am talking here about 7.5M addresses/keys per one wallet. How is Bitcoin Core handling that? Can't you just let it sync and see what it shows? Dumpwallet would be ok if it could only dump private keys of used addresses/keys. I haven't used it (and don't want to test on my wallet): what does the output look like? Bash no because it is too slow. A few million entries isn't a problem in bash. Getting only addresses from a long list is very fast with grep. I have csv files: transactions, balances. Only have to connect transactions and balances with public addresses/private keys. What are you looking for? Addresses with a balance?
|
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 08:40:07 PM |
|
I am talking here about 7.5M addresses/keys per one wallet. How is Bitcoin Core handling that? Can't you just let it sync and see what it shows? Dumpwallet would be ok if it could only dump private keys of used addresses/keys. I haven't used it (and don't want to test on my wallet): what does the output look like? Bash no because it is too slow. A few million entries isn't a problem in bash. Getting only addresses from a long list is very fast with grep. I have csv files: transactions, balances. Only have to connect transactions and balances with public addresses/private keys. What are you looking for? Addresses with a balance? Bitcoin Core in current version is handling such a wallet very badly. Import takes over 24 hours and "Add to wallet" takes another several hours. Rescan is additional few hours. Yes, in the end it is going to sync, but I am looking for a higher performance solution. Dumpwallet gives private key, date and public address. I am only not sure if imported are only addresses with balances. I would rather use Python than Bash because it is much faster to do things inside script than using external tools (like grep). Grep at this point will be too slow, in effect it needs to search whole data for each call. I think I cannot run grep for hundreds of thousands parameters. Am I right? Yes, I am looking for addresses with balances, which I have now, but looking further I need to link addresses with private keys I have. If some of them exist with balances.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17342
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 08:48:01 PM |
|
Grep at this point will be too slow, in effect it needs to search whole data for each call. I think I cannot run grep for hundreds of thousands parameters. Am I right? Why would you need so many parameters? One grep regex can grep Bitcoin addresses. You can Google the regex, or I'll look it up later (I mean next year, cause I'm off for now). Yes, I am looking for addresses with balances, which I have now, but looking further I need to link addresses with private keys I have. If some of them exist with balances. I have some of my favourite bash searches collected in this topic. You may also want to look for "bitcoin-tool" to get addresses out of (many) private keys using command line. Good luck ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 09:27:24 PM |
|
Why would you need so many parameters? One grep regex can grep Bitcoin addresses. You can Google the regex, or I'll look it up later (I mean next year, cause I'm off for now). I have some of my favourite bash searches collected in this topic. You may also want to look for "bitcoin-tool" to get addresses out of (many) private keys using command line. Good luck ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Thanks for the info about bitcoin-tool! I have it but didn't used it till now. Looks like it may be handy. This grep regex would be very useful later if you don't mind sharing it. As I understand I could make one file with addresses that have balance, second file with private keys and then compare these files.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
jackg
Copper Member
Legendary
Offline
Activity: 2856
Merit: 3071
https://bit.ly/387FXHi lightning theory
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 09:50:09 PM |
|
Yes, I can do it in Python. Bash no because it is too slow.
I may be the only fan of using python as something to run simple commands and using it as a docker for running other programs in different languages. You might want to discover using os to run things in bash from a python script. import os os.system("echo 'hello'")
For example will run "echo 'hello'" in bash if you're using a bash shell on your system. You can then use that same module to access the command line for running different programs. Bash scripts are slow to be interpreted afaik, but the actual commands you run in bash are fast enough (using the os.system command and using python to handle the interactions is much faster). Your idea for script is good. Script should do the work.
I have csv files: transactions, balances. Only have to connect transactions and balances with public addresses/private keys.
Will think about it.
You can just read columns and rows in CSVs though and import them all into a data structure (probably a dictionary for what you're after - of type {key,value} ={address, balance} or = {address, list[balance,txs,etc]).
|
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 09:53:57 PM |
|
You can just read columns and rows in CSVs though and import them all into a data structure (probably a dictionary for what you're after - of type {key,value} ={address, balance} or = {address, list[balance,txs,etc]).
Exactly! I could make two csv files and just compare them with WinMerge. Then I can get duplicates on both sides and get private keys for that list, in lookup table having pub/priv keys. I am using Linux also.
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
pbies (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 31, 2021, 11:59:13 PM Last edit: January 01, 2022, 12:15:51 AM by pbies |
|
Ok, guys, just saying what I have done.
Found nothing.
But made it technically.
Well, after many sort, uniq, dos2unix and file operations I have come with:
1. 6 GB private WIFs/public addresses as a source data 2. 23M private real WIFs after selection (many of 1. were not private keys) 1.1 GB 3. 50MB of hex keys which I will convert to private keys later (hex to bin to sha256 and so on...) 4. 2.7 GB of private key - public address pairs
The result was:
1. public addresses to which I have private key - 588 MB 2. public addresses which have >0 balance - 1.1 GB 3. some Segwit "bc1q" beginning addresses/keys - will take care of them later
WinMerge didn't seen any duplicates in these two files (1 & 2).
As I don't trust 100% in WinMerge I used Linux:
sort $1 $2 | uniq -d > non-uniq.txt
|
BTC: bc1qmrexlspd24kevspp42uvjg7sjwm8xcf9w86h5k
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17342
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 01, 2022, 09:49:35 AM |
|
This grep regex would be very useful later if you don't mind sharing it. This is the most complete list I've seen so far: I actually can ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) I found this regexp on Stackoverflow: egrep --regexp="^[13][a-km-zA-HJ-NP-Z1-9]{25,34}$" filename With some slight changes it stops matching parts of Eth-addresses: egrep -w --regexp="[13][a-km-zA-HJ-NP-Z1-9]{25,34}" * I have compiled these from various sources and use them to automatically set my blockchain explorer options based on user input, and also keep them at my .zshrc : #cryptocurrency greps
#btc1 and btc2 combined alias btcgrep="grep -Ee '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b' -e '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'"
#legacy addresses only alias btcgrep1="grep -E '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b'" #http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html
#bech32 v1 and v0 addresses alias btcgrep2="grep -E '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'" #https://stackoverflow.com/questions/21683680/regex-to-match-bitcoin-addresses
#bech32 addresses only alias btcgrep3="grep -E '\bbc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71}\b'"
#both legacy and bech32 alias btcgrep4="grep -E '\b([13][a-km-zA-HJ-NP-Z1-9]{25,34}|bc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71})\b'" #http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html
#private keys alias btcgrep5="grep -E '\b[5KL][1-9A-HJ-NP-Za-km-z]{50,51}\b'" #word boundary: '\b' #https://bitcoin.stackexchange.com/questions/56737/how-can-i-find-a-bitcoin-private-key-that-i-saved-in-a-text-file
#transaction hashes alias btcgrep6="grep -E '\b[a-fA-F0-9]{64}\b'" #https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex #https://bitcoin.stackexchange.com/questions/70261/recognize-bitcoin-address-from-block-hash-and-transaction-hash
#block hashes alias btcgrep7="grep -E '\b[0]{8}[a-fA-F0-9]{56}\b'" #https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex
Flag -w is 'word bondary' and can also be set within the regex with '\b' at the ends. I haven't tested all of them. As I understand I could make one file with addresses that have balance, second file with private keys and then compare these files. I would get 2 lists of addresses: one with all funded addresses, and one list with the addresses you want to compare. If you find a match, you search your list for the private key for that address. I'd start with this: On Linux, use this to find matching addresses (after extrating the compressed .gz file of course): comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | sort | uniq) - Bitcoin_addresses_LATEST.txt: the extracted latest version downloaded from addresses.loyce.club.
- mylist.txt: your own list of addresses, one address per line.
This takes only seconds to check millions of addresses. If your text file has Microsoft formatting, you may need to use this instead: comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | fromdos | sort | uniq)
|
|
|
|
|