Title: mass generation of addresses using private key Post by: citb0in on November 12, 2022, 10:57:53 AM In terms of software development, what is the most efficient and performant way to generate the derived three bitcoin addresses ( uncompressed 3..., compressed 1..., segwit bc1q... ) from a private key? If you have only a handful of private keys, you can do it with all conceivable and available bitcoin tools out there. But if you have a million private keys, what is the fastest way to fulfill this task, what do you suggest? is there any python snippet you're aware of ? Thank you
Title: Re: mass generation of addresses using private key Post by: ABCbits on November 12, 2022, 12:26:04 PM But if you have a million private keys, what is the fastest way to fulfill this task, what do you suggest? If you have decent programming skill, i'd suggest to modify Bitcoin brute-force software which have GPU support. is there any python snippet you're aware of ? Thank you Python library bit[1] claim it's 100x faster compared with other Bitcoin python library. I've tried it before, but it doesn't support Bech32 address. Anyway, i did quick dirty benchmark and here's the result with 100K hex private keys. Code: from bit import Key It's much faster than i expected where the script generate 100K address in 6 seconds. Code: $ time python3 test.py Here's snippet of hex.txt and address.txt. Code: $ tail -n 5 hex.txt [1] https://github.com/ofek/bit (https://github.com/ofek/bit) Title: Re: mass generation of addresses using private key Post by: igor72 on November 12, 2022, 03:54:15 PM In terms of software development, what is the most efficient and performant way to generate the derived three bitcoin addresses ( uncompressed 3..., compressed 1..., segwit bc1q... ) from a private key? Uncompressed 3... from a private key? AFAIK it's impossible.Title: Re: mass generation of addresses using private key Post by: hosseinimr93 on November 12, 2022, 04:24:11 PM Uncompressed 3... from a private key? AFAIK it's impossible. You are right. That's not possible. When it comes to legacy addresses, there are two legacy addresses derived from a single private key. One is derived from the uncompressed private key and the other one is derived from the compressed private key. But there's only one nested segwit address and that's derived using the compressed private key. Title: Re: mass generation of addresses using private key Post by: LoyceV on November 12, 2022, 05:43:43 PM is there any python snippet you're aware of ? I used to use bitcoin-tool for this, and by the looks of it there's a segwit update (https://github.com/matja/bitcoin-tool) now. Note that I don't trust the software, so keep it offline in a VM to be safe.Uncompressed 3... from a private key? AFAIK it's impossible. As far as I know, it is possible to create uncompressed Segwit addresses, but you'll have a hell of a time trying to spend the funds. So don't do it.Title: Re: mass generation of addresses using private key Post by: ABCbits on November 13, 2022, 09:38:56 AM In terms of software development, what is the most efficient and performant way to generate the derived three bitcoin addresses ( uncompressed 3..., compressed 1..., segwit bc1q... ) from a private key? Uncompressed 3... from a private key? AFAIK it's impossible.It's fully possible to generate such address, after all it's just P2SH (Pay to script hash) address. But as @LoyceV said, spending it is very difficult to spend the coin since the transaction would be considered as non-standard and won't be relayed by most node. For reference, here are few past incident about using uncompressed public key on P2SH-P2WPKH or P2WPKH address, https://bitcointalk.org/index.php?topic=5192454.0 (https://bitcointalk.org/index.php?topic=5192454.0) https://bitcointalk.org/index.php?topic=5377781.0 (https://bitcointalk.org/index.php?topic=5377781.0) Title: Re: mass generation of addresses using private key Post by: citb0in on November 13, 2022, 11:42:53 AM Bit uses Bitcoin Core's heavily optimized C library libsecp256k1 for all elliptic curve operations (https://github.com/ofek/bit/blob/master/docs/source/community/faq.rst).
Code: $ python3 -m timeit -s "from bit import Key;k=Key()" "k.address" Quote 5000000 loops, best of 5: 49.6 nsec per loop Indeed, it is really fast. Thanks for pointing to this great tool. EDIT: By default Bit (https://ofek.dev/bit/guide/keys.html) will use the compressed version for all keys. With key.address I can get the compressed Bitcoin address (starting with 1...) and by key.segwit_address you can get the p2sh segwit address. Does anyone know if Bit is also able to generate the bitcoin segwit address starting with bc1q and but also the uncompressed bitcoin address (also starting with 1) ? How? Title: Re: mass generation of addresses using private key Post by: citb0in on November 14, 2022, 07:44:24 AM Really a pity, because otherwise bit would have been perfect. What alternative library could you recommend me, which is as fast as possible and also handles the mentioned address type?
If you have decent programming skill, i'd suggest to modify Bitcoin brute-force software which have GPU support. that would be the best option of course, however I don't have any special programming skills to implement that myself.The goal is to generate e.g. 100,000 addresses from corresponding private keys. The addresses to be generated should contain all 4 types I mentioned before. ku from pycoin offers many functions and spits out everything you want to have, however I can't really work performant with the command-line tool "ku", it would take forever to generate e.g. 100,000 addresses. Title: Re: mass generation of addresses using private key Post by: nc50lc on November 14, 2022, 08:44:08 AM Really a pity, because otherwise bit would have been perfect. What alternative library could you recommend me, which is as fast as possible and also handles the mentioned address type? I can't recommend any aside from what's already suggested by others, sorry.Note for my previous post: you can't generate uncompressed bitcoin addresses if you provided it compressed WIF, but it can if it's uncompressed WIF; otherwise you can follow their advanced tutorial: https://ofek.dev/bit/guide/advanced.html#bytes-to-wif (https://ofek.dev/bit/guide/advanced.html#bytes-to-wif) Sample: Code: import bit Code: '1L4U3Vk2Z8FqNp9JoSqcRhKVqX9V7BfQqC' Title: Re: mass generation of addresses using private key Post by: ABCbits on November 14, 2022, 11:44:34 AM Really a pity, because otherwise bit would have been perfect. What alternative library could you recommend me, which is as fast as possible and also handles the mentioned address type? I checked other library i've tried in past and found out bitcoinlib[1] utilize library called fastecdsa. It sounds good, so i decide to run quick benchmark. Code: from bitcoinlib.keys import Key Code: $ time python3 test2.py It's far slower than bit, but IMO it's still fast enough for you needs. And here's example of generated address. Code: $ tail -n 5 address_bech32.txt [1] https://github.com/1200wd/bitcoinlib (https://github.com/1200wd/bitcoinlib) Title: Re: mass generation of addresses using private key Post by: citb0in on November 14, 2022, 12:59:50 PM https://i.ibb.co/4fBQWzX/bitcoin-address-map.jpg (https://ibb.co/9vRMn61)
@ETFbitcoin thanks you for your effort. I ran some benchmark tests, too. Additionally I tried bitcoinaddress (https://pypi.org/project/bitcoinaddress/): All benchmarks involved a hex file which contains 100,000 private keys. Here are my results so far in ranking order from fastest to slowest: bitcoinlib Code: $ time ./benchmark_with_bitcoinlib.py Quote real 1m27,629s user 1m27,779s sys 0m1,356s bitcoinaddress Code: #!/usr/bin/python3 Code: $ time ./benchmark_with_bitcoinaddress.py Quote real 6m34,864s user 6m34,706s sys 0m0,129s cross-checking each output... Code: $ diff -q bech32_addresses_with_bitcoinlib.100k bech32_addresses_with_bitcoinaddress.100k Quote Output file of all those benchmark tests were the same, no difference at all.I would like to see how pycoin (https://pycoin.readthedocs.io/en/latest/) would perform in this benchmark test with 100k addresses. Without using its command-line tool ku (https://github.com/richardkiss/pycoin/blob/main/COMMAND-LINE-TOOLS.md) cause I think the command-line tool would be slow down things, correct me if I'm wrong. I didn't understand yet, how to utilize pycoin with network.parse (https://pycoin.readthedocs.io/en/latest/api.html#network-parse) to get the bech32 address of a key... EDIT: No matter how you slice it, these calculations are performed far too slowly on the CPU compared to a GPU. Even if you were to extend bit with bech32 functionality, I can't imagine that bit would be faster than GPU execution. Hence my question: Do you know any tool in this direction, which is multi-GPU capable and can generate addresses according to certain criteria ? VanitySearch won't work, because it uses for custom prefixes. Title: Re: mass generation of addresses using private key Post by: citb0in on December 15, 2022, 11:12:07 AM I would generally be interested to know if it is at all possible to achieve such speeds in Python as we know from VanitySearch, BitCrack, etc. The mentioned tools are written in C++ and use CUDA for the immense and fast computing power. Without CUDA, this would not be possible. Is it possible to use CUDA in Python to achieve the same performance or is it simply technically impossible and we have to stick with CPU calculation in Python?
The other question I'm wondering about: could we, for example, run VanitySearch in some kind of sub-process within a Python program, so that the full performance of VanitySearch would be applied, but the results would be processed by the Python program in the next step? Title: Re: mass generation of addresses using private key Post by: larry_vw_1955 on December 17, 2022, 12:13:56 AM I checked other library i've tried in past and found out bitcoinlib[1] utilize library called fastecdsa. It sounds good, so i decide to run quick benchmark. Code: $ time python3 test2.py Quote It's far slower than bit, but IMO it's still fast enough for you needs. And here's example of generated address. yeah it is. far slower. but i still am not sure how fast it would run on my machine because you didn't give any specs of your system. i imagine the more cores on the cpu, it's just multiplying the speed by that.It's much faster than i expected where the script generate 100K address in 6 seconds. here's my outputs: Code: Number of addresses processed: 125000 Quote Here's snippet of hex.txt and address.txt. Code: $ tail -n 5 hex.txt yeah i mean it seems to compute accurate results but i'm not sure i like their documentation so much. they don't even seem to show how to get uncompressed addresses in their little getting started intro. had to find out a way to do that by looking at this: https://github.com/ofek/bit/issues/163 that was about as obvious as an electron 10 feet away... >:( [moderator's note: consecutive posts merged] |