Dump all private keys if they had movement

[pbies]

Hello guys!

Is it possible to dump all private keys from a huge wallet in Bitcoin Core (or outside) that had any movement of BTC on them?

How could I do that?

[LoyceV]

Is it possible to dump all private keys from a huge wallet in Bitcoin Core (or outside) that had any movement of BTC on them?

I think you're looking for the dumpwallet command: click Window > Console and type "dumpwallet":

Code:

dumpwallet "filename"

Dumps all wallet keys in a human-readable format to a server-side file. This does not allow overwriting existing files.
Imported scripts are included in the dumpfile, but corresponding BIP173 addresses, etc. may not be added automatically by importwallet.
Note that if your wallet contains keys which are not derived from your HD seed (e.g. imported keys), these are not covered by
only backing up the seed itself, and must be backed up too (e.g. ensure you back up the whole dumpfile).

Arguments:
1. filename    (string, required) The filename with path (absolute path recommended)

Result:
{                        (json object)
  "filename" : "str"     (string) The filename with full absolute path
}

Examples:
> bitcoin-cli dumpwallet "test"
> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "dumpwallet", "params": ["test"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
 (code -1)

I don't think this is limited to keys with "movement" though, it dumps all keys. But for a huge wallet, I expect most of the keys to have been used.

[ranochigo]

Dumpwallet can help you narrow down your range of addresses. You might want to use listreceivedbyaddress RPC and specify your own parameters to dump the appropriate addresses.

[NotATether]

You will probably want to watch by address instead of by private key as there is already software you can run on your box that can do just that (electrum server, in particular). It would require you to keep a mapping between addresses and their corresponding private keys so I think it's only feasible in the scale of thousands.

[pbies]

I surely know dumpwallet command, but it is not what I am looking for.
Dumpwallet would be ok if it could only dump private keys of used addresses/keys.

I don't want to go to other app for that, so electrum and it's 1000 limit for one wallet is not enough for me. Simply it is blocking IP addresses with larger wallets.

Eventually I could use dump wallet but how to search it then for used addresses? I can make csv consisting of private and public keys/addresses.

I am talking here about 7.5M addresses/keys per one wallet.

[mynonce]

...
I am talking here about 7.5M addresses/keys per one wallet.

7.5M addresses/keys per one wallet? SATOSHI IS HERE!

I'm just joking    Happy new year!

[jackg]

Can you program this as a shell script or externally in any way?

I'd look at something in python and going:
dump wallet (to dump the addresses)
Iterate address over addresses
    If getreceivedbyaddress(address) not zero
        Open a file editor and file
        file add new line
        dumpprivkey(address)
        close file

Why you'd want to iterate over 7M addresses is another question entirely but I'm assuming there's a chance if someone owned a faucet or something. Or if someone's testing it's possibility before steganographically trying to secure something.

[pbies]

Yes, I can do it in Python. Bash no because it is too slow.
Your idea for script is good.
Script should do the work.

I have csv files: transactions, balances.
Only have to connect transactions and balances with public addresses/private keys.

Will think about it.

Thanks!

[LoyceV]

I am talking here about 7.5M addresses/keys per one wallet.

How is Bitcoin Core handling that? Can't you just let it sync and see what it shows?

Dumpwallet would be ok if it could only dump private keys of used addresses/keys.

I haven't used it (and don't want to test on my wallet): what does the output look like?

Bash no because it is too slow.

A few million entries isn't a problem in bash. Getting only addresses from a long list is very fast with grep.

I have csv files: transactions, balances.
Only have to connect transactions and balances with public addresses/private keys.

What are you looking for? Addresses with a balance?

[pbies]

I am talking here about 7.5M addresses/keys per one wallet.

How is Bitcoin Core handling that? Can't you just let it sync and see what it shows?

Dumpwallet would be ok if it could only dump private keys of used addresses/keys.

I haven't used it (and don't want to test on my wallet): what does the output look like?

Bash no because it is too slow.

A few million entries isn't a problem in bash. Getting only addresses from a long list is very fast with grep.

I have csv files: transactions, balances.
Only have to connect transactions and balances with public addresses/private keys.

What are you looking for? Addresses with a balance?

Bitcoin Core in current version is handling such a wallet very badly. Import takes over 24 hours and "Add to wallet" takes another several hours. Rescan is additional few hours.
Yes, in the end it is going to sync, but I am looking for a higher performance solution.

Dumpwallet gives private key, date and public address. I am only not sure if imported are only addresses with balances.

I would rather use Python than Bash because it is much faster to do things inside script than using external tools (like grep).
Grep at this point will be too slow, in effect it needs to search whole data for each call. I think I cannot run grep for hundreds of thousands parameters. Am I right?

Yes, I am looking for addresses with balances, which I have now, but looking further I need to link addresses with private keys I have. If some of them exist with balances.

[LoyceV]

Grep at this point will be too slow, in effect it needs to search whole data for each call. I think I cannot run grep for hundreds of thousands parameters. Am I right?

Why would you need so many parameters? One grep regex can grep Bitcoin addresses. You can Google the regex, or I'll look it up later (I mean next year, cause I'm off for now).

Yes, I am looking for addresses with balances, which I have now, but looking further I need to link addresses with private keys I have. If some of them exist with balances.

I have some of my favourite bash searches collected in this topic. You may also want to look for "bitcoin-tool" to get addresses out of (many) private keys using command line. Good luck

[pbies]

Why would you need so many parameters? One grep regex can grep Bitcoin addresses. You can Google the regex, or I'll look it up later (I mean next year, cause I'm off for now).

I have some of my favourite bash searches collected in this topic. You may also want to look for "bitcoin-tool" to get addresses out of (many) private keys using command line. Good luck

Thanks for the info about bitcoin-tool! I have it but didn't used it till now.
Looks like it may be handy.

This grep regex would be very useful later if you don't mind sharing it.

As I understand I could make one file with addresses that have balance, second file with private keys and then compare these files.

[jackg]

Yes, I can do it in Python. Bash no because it is too slow.

I may be the only fan of using python as something to run simple commands and using it as a docker for running other programs in different languages.

You might want to discover using os to run things in bash from a python script.

Code:

import os
os.system("echo 'hello'")

For example will run "echo 'hello'" in bash if you're using a bash shell on your system.

You can then use that same module to access the command line for running different programs.

Bash scripts are slow to be interpreted afaik, but the actual commands you run in bash are fast enough (using the os.system command and using python to handle the interactions is much faster).

Your idea for script is good.
Script should do the work.

I have csv files: transactions, balances.
Only have to connect transactions and balances with public addresses/private keys.

Will think about it.

You can just read columns and rows in CSVs though and import them all into a data structure (probably a dictionary for what you're after - of type {key,value} ={address, balance} or = {address, list[balance,txs,etc]).

[pbies]

You can just read columns and rows in CSVs though and import them all into a data structure (probably a dictionary for what you're after - of type {key,value} ={address, balance} or = {address, list[balance,txs,etc]).

Exactly!

I could make two csv files and just compare them with WinMerge. Then I can get duplicates on both sides and get private keys for that list, in lookup table having pub/priv keys.
I am using Linux also.

[pbies]

Ok, guys, just saying what I have done.

Found nothing.

But made it technically.

Well, after many sort, uniq, dos2unix and file operations I have come with:

1. 6 GB private WIFs/public addresses as a source data
2. 23M private real WIFs after selection (many of 1. were not private keys) 1.1 GB
3. 50MB of hex keys which I will convert to private keys later (hex to bin to sha256 and so on...)
4. 2.7 GB of private key - public address pairs

The result was:

1. public addresses to which I have private key - 588 MB
2. public addresses which have >0 balance - 1.1 GB
3. some Segwit "bc1q" beginning addresses/keys - will take care of them later

WinMerge didn't seen any duplicates in these two files (1 & 2).

As I don't trust 100% in WinMerge I used Linux:

sort $1 $2 | uniq -d > non-uniq.txt

[LoyceV]

This grep regex would be very useful later if you don't mind sharing it.

This is the most complete list I've seen so far:

I actually can I found this regexp on Stackoverflow:

Code:

egrep --regexp="^[13][a-km-zA-HJ-NP-Z1-9]{25,34}$" filename

With some slight changes it stops matching parts of Eth-addresses:

Code:

egrep -w --regexp="[13][a-km-zA-HJ-NP-Z1-9]{25,34}" *

I have compiled these from various sources and use them to automatically set my blockchain explorer options based on user input, and also keep them at my .zshrc :

Code:

#cryptocurrency greps

#btc1 and btc2 combined
alias btcgrep="grep -Ee '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b' -e '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'"

#legacy addresses only
alias btcgrep1="grep -E '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b'"
#http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html

#bech32 v1 and v0 addresses
alias btcgrep2="grep -E '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'"
#https://stackoverflow.com/questions/21683680/regex-to-match-bitcoin-addresses

#bech32 addresses only
alias btcgrep3="grep -E '\bbc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71}\b'"

#both legacy and bech32
alias btcgrep4="grep -E '\b([13][a-km-zA-HJ-NP-Z1-9]{25,34}|bc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71})\b'"
#http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html

#private keys
alias btcgrep5="grep -E '\b[5KL][1-9A-HJ-NP-Za-km-z]{50,51}\b'"
#word boundary: '\b'
#https://bitcoin.stackexchange.com/questions/56737/how-can-i-find-a-bitcoin-private-key-that-i-saved-in-a-text-file

#transaction hashes
alias btcgrep6="grep -E '\b[a-fA-F0-9]{64}\b'"
#https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex
#https://bitcoin.stackexchange.com/questions/70261/recognize-bitcoin-address-from-block-hash-and-transaction-hash

#block hashes
alias btcgrep7="grep -E '\b[0]{8}[a-fA-F0-9]{56}\b'"
#https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex

Flag -w is 'word bondary' and can also be set within the regex with '\b' at the ends.

I haven't tested all of them.

As I understand I could make one file with addresses that have balance, second file with private keys and then compare these files.

I would get 2 lists of addresses: one with all funded addresses, and one list with the addresses you want to compare. If you find a match, you search your list for the private key for that address.
I'd start with this:

On Linux, use this to find matching addresses (after extrating the compressed .gz file of course):

Code:

comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | sort | uniq)

• Bitcoin_addresses_LATEST.txt: the extracted latest version downloaded from addresses.loyce.club.
• mylist.txt: your own list of addresses, one address per line.

This takes only seconds to check millions of addresses. If your text file has Microsoft formatting, you may need to use this instead:

Code:

comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | fromdos | sort | uniq)