arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
December 30, 2022, 06:34:55 PM Last edit: December 30, 2022, 07:48:52 PM by arulbero |
|
The example I originally showed using iceland2k14/libsecp256k1 was about half the speed. With it I can generate 1 million addresses using all cores (in my case 16 cores) and write to the file in 4.8 seconds, this is a rate of about 208,300 keys/sec. I have modified my initial program so that you can now also configure the cores under which the program is executed. So one can select specifically "1" as value, so that we also compare apples with apples. #!/usr/bin/env python3 # 2022/Dec/30, citb0in import concurrent.futures import os import numpy as np import fastecdsa.keys as fkeys import fastecdsa.curve as fcurve import secp256k1 as ice
# how many cores to use num_cores = 1 #num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 1000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end): # Generate a NumPy array of random private keys using fastecdsa private_keys = np.array([fkeys.gen_private_key(fcurve.P256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_with_singleCore.txt', addresses, fmt='%s')
Running this under one single core, now I get 26.5 sec for 1 million generated addresses. This is a rate of 37,735 keys/second. I am curious to see your program using all available cores. Ok, finally I managed to use your multithread program with my script. Results: 30s to generate 16.4 M of addresses, about 540k addresses / s, and store them in a file. time python3 gen_batches.py > addresses.out
real 0m29,810s user 8m22,402s sys 0m6,273s
wc addresses.out
16474040 16474040 575934502
less addresses.out
18GW1MFwpYZgmRSy8R4wgjUtBnscJtXeUZ 1FXUP9HViWxrtgAvqbpgVrd69yarx6Njdp 1MuxV99fRFnhoNopdxV7kkQB4u14chCtqA 17AzXPYExdE8nxkiV6zJGLuJq32WkAZ134 12DUcGmFMkmXV8tYihM2jLQ36pc4zLBDP6 1AqwPisfi9hDHoF95SEoSEVN89fX8NSGjo 17qRbvdShkoCDw3XJGhEqxaFkQHaK36Yk1 187x8hPiVpRLGDRsvy4nreeMNW7aW9zuFv 1EGSDnc2j7p3qsPJsPakKgavCTz4szYU3w 1F4KjKLNqQSbmw9xsJurRVQ3RciL7Kox7g 1AKaHBfhMEedrtnjGgmFFsdG3SzyN3hpcZ 1GaWuuPPM747Na3CTwaR3DcewoBHc9VVSv 12VeWT4jZCep17HJswMtfCEqh14oDcRr3L 16MBC6WLWMvNeexYYX853Ucf2ZsPoYLRoS 1JVrhzEKTcgzGtSc9u9LpFjksFZiQveFT6 17kz2Wpry3vnLM6RMjB3KXoG6kRxjeDQcj 1TBh3q7vQoTmgX6pY75ADHDSkcaWkmCvd 13QVDFeeWkErhTAFqX3SP2YzrzWt3Bh7iu 1GPyzJC8Ey8PasHn2Qa8aFanjrXuE1aNK5 17QE8iNrtLnju5pinG1vdoCVZpn8zbrM7S 17nKRy1XdKdroWXPsnsd4RvxWKVySZxath 1KWaY8K824oEFk5q4tZioCZ86viLJuKNgS 141m3BUZNjMxbs4a9Cb6SbCmEzx4nPJKCL 12ueqsLkT1XCVheGM34uQWW2fpnhVvtwmE 14eH2SFvCq3fxmrK9NgKTxfMCeKixNwQ6c 1G9CaAqoKJSfTRyGcvPhVFewLRXKWzkwh7 15RthH7DBuDSmYcFCYanUhBsrdEYzBahNw 13BTmQ2gCaZ8x4T7Sba9k2MRBSDRrZLy6u 1NpbKVze4pPVCghjaa2CV7LfRkotCNu76Y 1HfQ42v7DEDHEuQMhH5caPzX2RgtSmoYWw ...
Only 1.3 s to generate 16.4 M public keys (without writing) 30s to generate 16.4 M addresses and store them in a file (24s if each thread writes on the file without waiting for the others)
|
|
|
|
citb0in (OP)
|
|
December 30, 2022, 08:13:26 PM |
|
Thank you arulbero for your efforts. Very interesting. I was curious to see the comparison of random_generated_addresses and sequentially_generated_address as you showed with your code. Following was my python program that used 16 threads and multithreading, but without / before the implementation of the ProcessPoolExecutor #!/usr/bin/env python3 # 2022/Dec/26, [b]citb0in_multithreaded.py[/b] import threading import numpy as np import fastecdsa.keys as fkeys import fastecdsa.curve as fcurve import secp256k1 as ice
# Set the number of addresses to generate and the number of threads to use num_addresses = 1000000 num_threads = 16
# Divide the addresses evenly among the threads addresses_per_thread = num_addresses // num_threads
# Create a list to store the generated addresses addresses = []
# Define a worker function that generates a batch of addresses and stores them in the shared list def worker(start, end): # Generate a NumPy array of random private keys using fastecdsa private_keys = np.array([fkeys.gen_private_key(fcurve.P256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
# Add the addresses to the shared list addresses.extend(thread_addresses)
# Create a list of threads threads = []
# Create and start the threads for i in range(num_threads): start = i * addresses_per_thread end = (i+1) * addresses_per_thread t = threading.Thread(target=worker, args=(start, end)) threads.append(t) t.start()
# Wait for the threads to finish for t in threads: t.join()
# Write the addresses to a file np.savetxt('addresses_1M_multithreaded.txt', addresses, fmt='%s')
$ time python3 citb0in_multithreaded.py
real 0m19,759s user 0m36,834s sys 0m6,259s
==> It took less than 20 seconds to generate 1 million addresses using 16 threads on my system. Then I modified that code to use only one single thread: #!/usr/bin/env python3 # 2022/Dec/30, [b]citb0in_singlethreaded.py[/b] [...] num_threads = 1 [...]
$ time python3 citb0in_singlethreaded.py
real 0m26,317s user 0m26,309s sys 0m1,568s
==> It took about 26 seconds to generate 1 million addresses using 16 threads on my system. Then I modified the program to use ProcessPoolExecutor which distributes the load to all available cores and it's multithreaded. This gave me the best results: #!/usr/bin/env python3 # 2022/Dec/30, [b]citb0in_multicore.py[/b] import concurrent.futures import os import numpy as np import fastecdsa.keys as fkeys import fastecdsa.curve as fcurve import secp256k1 as ice
# how many cores to use #num_cores = 1 num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 1000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end): # Generate a NumPy array of random private keys using fastecdsa private_keys = np.array([fkeys.gen_private_key(fcurve.P256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_multicore.txt', addresses, fmt='%s')
real 0m4,785s user 0m54,832s sys 0m1,992s
==> It took less than 5 seconds to generate 1 million addresses using all available cores (=16) on my system. Now I change the settings to use only one single core... #!/usr/bin/env python3 # 2022/Dec/30, [b]citb0in_singlecore.py[/b] [...] num_cores = 1 #num_cores = os.cpu_count() [...] np.savetxt('addresses_1M_singlecore.txt', addresses, fmt='%s')
real 0m26,271s user 0m26,077s sys 0m1,618s
==> It takes about 26 seconds to generate 1 million addresses using one single core on my system. Well, so far so good. I showed this just to summarize things up. Afterwards I modified my code according to your suggestion so the private keys are not generated randomly but instead they are sequentially generated from a pre-defined starting point. Here's the modified version: #!/usr/bin/env python3 # 2022/Dec/26, [b]citb0in_seq.py[/b] import concurrent.futures import os import numpy as np import secp256k1 as ice
# how many cores to use num_cores = 1 #num_cores = os.cpu_count()
# Number of addresses to generate num_addresses = 1000000
# Starting point (decimal/integer) for private key starting_point = 123456789
# Define a worker function that generates a batch of addresses and returns them def worker(start, end, starting_point): # Initialize the private key to the starting point private_key = starting_point
# Initialize the list to hold to private keys private_keys = []
# Generate a batch of private keys sequentially for i in range(start, end): # Increment the private key private_key += 1
# Add the private key to the list private_keys.append(private_key)
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses #return (thread_addresses, private_keys, start_int)
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end, starting_point))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_seq_singlecore.txt', addresses, fmt='%s')
real 0m10,049s user 0m10,012s sys 0m1,442s
==> By this change it takes only 10 seconds to generate 1 million addresses using one single core on my system. By replacing the private key generation from randomly to sequentially I got 2.6x better performance. Now let's activate all available cores... [...] #num_cores = 1 num_cores = os.cpu_count() [...] # Write the addresses to a file np.savetxt('addresses_1M_seq_singlecore.txt', addresses, fmt='%s')
real 0m2,481s user 0m18,081s sys 0m1,661s
==> It takes only 2.5 seconds to generate 1 million addresses using all 16 cores on my system. By replacing the private key generation from randomly to sequentially gradually resulted in higher performance. Now as final comparison to your program: [...] ('1FZqHE6MndwVFKDD9vPYjHQqCNzf9f4V4G', '1BvzQLEoausQEjdQZnaC9TsHEd8CfL54yy') ('19UvEYkDhi7WzP2rFufkWEhbJhzh1DBKed', '1geYTWHYsgokQ2HuELJALYSaBw8an5q7r') ('1N8vnptoRf7HAmdLt2iV7Tr1n6BBff5Ltd', '18x8zZjHqP4d3nUaNPLz7TWfYdYLgiHPsx') ('1PTofsPjc2FUtKoM3nWfCfCQi8KfnNMJy2', '1GYfAZ7R6mnMcRFUgTaELUqLEjcdV9CYjX') ('169x29jPKW3ehUWZcWXRgG8tCwqjDhPAn7', '1LiTZGU8B6gK5zNxBSiRBDyAxZLYSfaHnZ') ('13Xm9hWcfgNfamDEhfuisnnnAD6nCxS9k2', '1NXKfq5eqFJmmCaarjhKoqQXGqfVJKJZYq') ('1NJzBnxVzYRGUSSBqVpV5tpxGMb25hroQj', '1FtgciBUc2mmioTpxgSMvANw7JB3nQEhFc') ('19Qkdthp9m8uqJys58zBgfp4kbp6bS8ZA4', '1KiAJjTY7BvsogAF1suffpbsmB1VB6dLHu') You have just generated 1,007,862 keys (1 MKeys)
real 0m14,765s user 0m14,187s sys 0m0,448s
==> I have generated 1,007,862 addresses (about 1 million keys) in about 15 seconds using one single core with your program. ==> I have generated exactly 1 millions addresses in 10 seconds using one single core with my program.
Conclusion:The originally enormous speed advantage is therefore, according to these tests, not really to be found in the code difference but in the fact that randomly generated keys are simply more computationally intensive and cost more time than simple sequential key generation. So just as you originally said arulbero. It always depends on the purpose of use, of course, but I have my doubts that the sequential generation of private keys is not the best way. I therefore think, also in terms of security for other applications, that random private keys are always preferable to sequential ones despite the fact they are more time-intensive. Thank you very much for your effort and your valuable contribution dear arulbero. My conclusion so far remains: secp256k1 seems to be pretty damn fast and so far I haven't seen any example that can beat it. Of course I would still be happy if other suggestions flow in. In particular, I would still be interested if someone is able to move the already working code via pycuda, numba, jut or similar solutions into the GPU using CUDA to speed up the computation process. I am looking forward to further feedback and thank you all in advance.
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
December 31, 2022, 12:17:18 PM |
|
==> I have generated 1,007,862 addresses (about 1 million keys) in about 15 seconds using one single core with your program. ==> I have generated exactly 1 millions addresses in 10 seconds using one single core with my program.
Conclusion: The originally enormous speed advantage is therefore, according to these tests, not really to be found in the code difference but in the fact that randomly generated keys are simply more computationally intensive and cost more time than simple sequential key generation. So just as you originally said arulbero.
You said : "compare apples with apples" but now you are comparing a library written in C against pure python, it seems not fair For the record: I wrote a library in C with a single function: from public keys to address (without b58 encoding) Private key : 0000000000000000000000000000000000000000000000000000000000000001 Public key : x: 79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798 y: 483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8
PrKey WIF c.: KwDiBf89QgGbjEhKnhXJuH7LrciVrZi3qYjgd9M7rFU73sVHnoWn Address before b58 encode : 751e76e8199196d454941c45d1b3a323f1433bd6 Address with b58 encode : 1BgGZ9tcN4rm9KBzDn7KprQz87SZ26SAMH
I imported this function as file .so in my python script, result: 1 core, 1M of addresses time python3 gen_batches.py > addresses.out
real 0m3,961s user 0m3,910s sys 0m0,055s
16 cores, 1M of addresses time python3 gen_batches.py > addresses.out
real 0m0,527s user 0m7,404s sys 0m0,151s
1 core, 16.4M of addresses time python3 gen_batches.py > addresses.out
real 1m1,757s user 1m1,382s sys 0m0,348s
16 cores, 16.4M of addresses time python3 gen_batches.py > addresses.out
real 0m6,672s user 2m2,989s sys 0m0,698s
less addresses.out
c71c6741ae4862dadaf2d9c9295d47410a27e194 1d9a3ef1efeec75b05f0ece73ad9402cac767843 44abfeb8701c716380ed2a40aeb72370ef61d7aa d6a2aa6799f42078faa889dcda92b29c6005d84d f3a0e804269feeba68d8c1facf68206b73b2a987 7a795a7b4500f80101e489e9ad5d7bc1855edf13 d8e619b13be4c2d6182aa10c0e7237a35ba0ab05 be801790f2cefb432bdf5dce2946fd22364b36b0 531a9c231f3e42d4bdb02d44bf01104e6eccd83d 7c8391a816b578f0d6e0a56595b67e12a314f945 6d0b359ba6f3d382eb99a346f99f0d27c29e2963 aa4116886a8631699f5077d78f47e6013a1af05f 2574970ffa9d5b26d52d89a76eef9c3c1bfbe798 ...
then the speed is now about 2.4 MKeys/s. I'm pretty sure that more c functions you introduce in your python script, more fast it becomes.
|
|
|
|
citb0in (OP)
|
|
January 01, 2023, 06:01:47 PM |
|
but now you are comparing a library written in C against pure python, it seems not fair The statement is justified. I didn't know until now that the iceland2k14/secp256k1 library is written in C++ and only imported into Python. I understand that the huge speed advantage of this library is due to the native implementation. Despite the justified criticism regarding security (iceland2k14/secp256k1 is closed-source because the source code is not publicly known), the library is nevertheless widely used by various developers and tools with success and satisfaction. Nevertheless, I will include this note in my original post, as I consider it important. I'm pretty sure that more c functions you introduce in your python script, more fast it becomes.
This is understandable. Nevertheless, I am very curious and interested in running the so far simplistic and demonstrated code in the GPU using PyCuda or similar. I suspect that PyCuda could give very good results after all what I have read and understood so far on the PyCuda project website. So I'm still asking for helpful ideas and code suggestions to achieve the goal via GPU using PyCuda or similar. And how can we measure speed. We all have different hardware. And Linux will be faster than Windows.
That's right. It would be fatal to compare my results with arulbero's, because we simply use too different hardware. A comparison in this context is only valid with significance if everyone compares between different program versions on its own hardware. As you can clearly see from the results posted so far, arulbero uses faster hardware than I do, for example. But that's not a problem, you just have to be careful to compare your own results with each other on your own rig and not with the results of other users. The posted results are just to illustrate what differences the different programs provide. I see that you're using gen_private_key. This generates arbitrary secure random private keys, using urandom. urandom might be slow depending on your system - do you really need it? I assume whatever you're doing (brute force search?) will generate the private keys in some other fashion, so you can just set prvkey_dec to an integer representing the private key directly.
That's a good point as well, which I unfortunately overlooked and only became aware of when re-reading the thread. The library I originally used, FASTECDSA, really seems to be too slow for the purpose intended here, as it is too complicated. Accordingly, I have started another optimization attempt. Thereby I don't use fastecdsa anymore for the randomly generated private keys, instead I use Python's secure library: secrets.randbelow(2**256)
This resulted in a performance boost of over +32%using fastecda, 1 thread ==> 29.681 sec using secrets, 1 thread ==> 19.923 sec +32.9 %using fastecdsa, 16 threads ==> 20.736 sec using secrets, 16 threads ==> 13.612 sec +34.4 %using fastecdsa, ProcessPoolExecutor, 1 core ==> 28.961 sec using secrets, ProcessPoolExecutor, 1 core ==> 19.921 sec +31.3 %using fastecdsa, ProcessPoolExecutor, 16 cores ==> 5.120 sec using secrets, ProcessPoolExecutor, 16 cores ==> 3.744 sec +36.9 %To achieve this further speed advantage, the following change is required: - removing the fastecdsa library by removing those two lines from the top of the python code : #!/usr/bin/env python3
import fastecdsa.keys as fkeys import fastecdsa.curve as fcurve - instead we import the Python library "secrets" : import secrets
- we replace the randomly generated 256-bit private key command: # Generate a NumPy array of random private keys using fastecdsa private_keys = np.array([fkeys.gen_private_key(fcurve.P256) for _ in range(start, end)])
# Generate a NumPy array of random private keys using "secrets" library private_keys = np.array([secrets.randbelow(2**256) for _ in range(start, end)])
here's the updated complete code variant with using multicore functionality and the secrets library for enhanded speed: #!/usr/bin/env python3 # 2023/Jan/01, citb0in_multicore_secrets.py import concurrent.futures import os import numpy as np import secrets import secp256k1 as ice
# how many cores to use #num_cores = 1 num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 1000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end): # Generate a NumPy array of random private keys using "secrets" library private_keys = np.array([secrets.randbelow(2**256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_multicore_secrets.txt', addresses, fmt='%s')
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 02, 2023, 09:11:33 AM |
|
this seems to need secp256k1 library. i took from albertobsd keyhunt this folder secp256k1 but when trying to compile i get this error $ gcc -o test test.cpp
test.cpp: In function ‘int main()’: test.cpp:18:30: error: ‘class Secp256K1’ has no member named ‘GetAddress’ 18 | bitAddr = secp256k1->GetAddress(0, false, pub); | ^~~~~~~~~~
$ ls -l
drwx------ secp256k1 -rw-rw---- test.cpp
$ ls -l secp256k1/
-rw------- Int.cpp -rw------- IntGroup.cpp -rw------- IntGroup.h -rw------- Int.h -rw------- IntMod.cpp -rw------- Point.cpp -rw------- Point.h -rw------- Random.cpp -rw------- Random.h -rw------- SECP256K1.cpp -rw------- SECP256k1.h
|
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 02, 2023, 10:10:31 AM |
|
now i used the secp256k1 from the linux folder you suggested. i get this error: $ gcc -o test test.cpp test.cpp: In function ‘int main()’: test.cpp:11:23: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings] 11 | privKey.SetBase10("1"); | ^~~ /usr/bin/ld: /tmp/ccU4lCiy.o: in function `main': test.cpp:(.text+0x27): undefined reference to `operator new(unsigned long)' /usr/bin/ld: test.cpp:(.text+0x32): undefined reference to `Secp256K1::Secp256K1()' /usr/bin/ld: test.cpp:(.text+0x48): undefined reference to `Secp256K1::Init()' /usr/bin/ld: test.cpp:(.text+0x57): undefined reference to `Int::Int()' /usr/bin/ld: test.cpp:(.text+0x70): undefined reference to `Int::SetBase10(char*)' /usr/bin/ld: test.cpp:(.text+0x7f): undefined reference to `Point::Point()' /usr/bin/ld: test.cpp:(.text+0x8e): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string()' /usr/bin/ld: test.cpp:(.text+0x9d): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::basic_ofstream()' /usr/bin/ld: test.cpp:(.text+0xbb): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode)' /usr/bin/ld: test.cpp:(.text+0xea): undefined reference to `Secp256K1::ComputePublicKey(Int*)' /usr/bin/ld: test.cpp:(.text+0x1cb): undefined reference to `Point::~Point()' /usr/bin/ld: test.cpp:(.text+0x1f5): undefined reference to `Secp256K1::GetAddress[abi:cxx11](int, bool, Point&)' /usr/bin/ld: test.cpp:(.text+0x20e): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)' /usr/bin/ld: test.cpp:(.text+0x21d): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: test.cpp:(.text+0x236): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <char, std::char_traits<char>, std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' /usr/bin/ld: test.cpp:(.text+0x243): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char)' /usr/bin/ld: test.cpp:(.text+0x252): undefined reference to `Int::AddOne()' /usr/bin/ld: test.cpp:(.text+0x278): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::close()' /usr/bin/ld: test.cpp:(.text+0x28c): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::~basic_ofstream()' /usr/bin/ld: test.cpp:(.text+0x29b): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: test.cpp:(.text+0x2aa): undefined reference to `Point::~Point()' /usr/bin/ld: test.cpp:(.text+0x2d1): undefined reference to `operator delete(void*, unsigned long)' /usr/bin/ld: test.cpp:(.text+0x2f2): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::~basic_ofstream()' /usr/bin/ld: test.cpp:(.text+0x30a): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: test.cpp:(.text+0x319): undefined reference to `Point::~Point()' /usr/bin/ld: /tmp/ccU4lCiy.o: in function `__static_initialization_and_destruction_0(int, int)': test.cpp:(.text+0x365): undefined reference to `std::ios_base::Init::Init()' /usr/bin/ld: test.cpp:(.text+0x380): undefined reference to `std::ios_base::Init::~Init()' /usr/bin/ld: /tmp/ccU4lCiy.o:(.data.rel.local.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0' collect2: error: ld returned 1 exit status
|
|
|
|
NotATether
Legendary
Offline
Activity: 1652
Merit: 6962
In memory of o_e_l_e_o
|
|
January 02, 2023, 10:38:03 AM |
|
@AlexanderCurl, regarding sequential key generation:
Have you tried adding a large prime number, like 0xdeadbeef (just an example, I don't even know if that's prime)?
Sure, you'll have to check for overflow more frequently - fortunately, that's just a matter of doing a Greater-Than comparison followed by a subtraction - but a sufficiently large increment should make the keys look pseudorandom as far as the bits are concerned.
|
. .BLACKJACK ♠ FUN. | | | ███▄██████ ██████████████▀ ████████████ █████████████████ ████████████████▄▄ ░█████████████▀░▀▀ ██████████████████ ░██████████████ █████████████████▄ ░██████████████▀ ████████████ ███████████████░██ ██████████ | | CRYPTO CASINO & SPORTS BETTING | | │ | | │ | ▄▄███████▄▄ ▄███████████████▄ ███████████████████ █████████████████████ ███████████████████████ █████████████████████████ █████████████████████████ █████████████████████████ ███████████████████████ █████████████████████ ███████████████████ ▀███████████████▀ ███████████████████ | | .
|
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 02, 2023, 10:57:24 AM |
|
There is Makefile to compile. All build commands are there. Just cd to the directory with code and use make in terminal. $ make $ ./VanitySearch
i did so. see here file structure $ ls -lh
drwxrwx--- ./ drwxrwx--- ../ drwxrwx--- base58/ -rw-rw---- Base58.o drwxrwx--- bech32/ -rw-rw---- Bech32.o -rw-rw---- gen.cpp drwxrwx--- hash/ -rw-rw---- IntGroup.o -rw-rw---- IntMod.o -rw-rw---- Int.o -rw-rw---- main.cpp -rw-rw---- main.o -rw-rw---- Makefile -rw-rw---- Point.o -rw-rw---- Random.o drwxrwx--- secp256k1/ -rw-rw---- SECP256K1.o -rwxrwx--- VanitySearch*
$ ls secp256k1/
Int.cpp IntGroup.h IntMod.cpp Point.h Random.h SECP256k1.h Timer.h IntGroup.cpp Int.h Point.cpp Random.cpp SECP256K1.cpp Timer.cpp
cat gen.cpp #include "secp256k1/SECP256k1.h" #include "secp256k1/Int.h" #include <iostream> #include <fstream>
int main() { Secp256K1* secp256k1 = new Secp256K1(); secp256k1->Init(); Int privKey; privKey.SetBase10("1"); Point pub; std::string bitAddr; std::ofstream outFile; outFile.open("address.txt", std::ios::app); for(int i = 0; i < 1000000; i++) { pub = secp256k1->ComputePublicKey(&privKey); bitAddr = secp256k1->GetAddress(0, false, pub); outFile << bitAddr << '\n'; privKey.AddOne(); } outFile.close(); return 0; }
as soon as i try to compile with command i get this error gen.cpp: In function ‘int main()’: gen.cpp:11:23: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings] 11 | privKey.SetBase10("1"); | ^~~ /usr/bin/ld: /tmp/cc3i7Un9.o: in function `main': gen.cpp:(.text+0x27): undefined reference to `operator new(unsigned long)' /usr/bin/ld: gen.cpp:(.text+0x32): undefined reference to `Secp256K1::Secp256K1()' /usr/bin/ld: gen.cpp:(.text+0x48): undefined reference to `Secp256K1::Init()' /usr/bin/ld: gen.cpp:(.text+0x57): undefined reference to `Int::Int()' /usr/bin/ld: gen.cpp:(.text+0x70): undefined reference to `Int::SetBase10(char*)' /usr/bin/ld: gen.cpp:(.text+0x7f): undefined reference to `Point::Point()' /usr/bin/ld: gen.cpp:(.text+0x8e): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string()' /usr/bin/ld: gen.cpp:(.text+0x9d): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::basic_ofstream()' /usr/bin/ld: gen.cpp:(.text+0xbb): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode)' /usr/bin/ld: gen.cpp:(.text+0xea): undefined reference to `Secp256K1::ComputePublicKey(Int*)' /usr/bin/ld: gen.cpp:(.text+0x1cb): undefined reference to `Point::~Point()' /usr/bin/ld: gen.cpp:(.text+0x1f5): undefined reference to `Secp256K1::GetAddress[abi:cxx11](int, bool, Point&)' /usr/bin/ld: gen.cpp:(.text+0x20e): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)' /usr/bin/ld: gen.cpp:(.text+0x21d): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: gen.cpp:(.text+0x236): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <char, std::char_traits<char>, std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' /usr/bin/ld: gen.cpp:(.text+0x243): undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char)' /usr/bin/ld: gen.cpp:(.text+0x252): undefined reference to `Int::AddOne()' /usr/bin/ld: gen.cpp:(.text+0x278): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::close()' /usr/bin/ld: gen.cpp:(.text+0x28c): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::~basic_ofstream()' /usr/bin/ld: gen.cpp:(.text+0x29b): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: gen.cpp:(.text+0x2aa): undefined reference to `Point::~Point()' /usr/bin/ld: gen.cpp:(.text+0x2d1): undefined reference to `operator delete(void*, unsigned long)' /usr/bin/ld: gen.cpp:(.text+0x2f2): undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::~basic_ofstream()' /usr/bin/ld: gen.cpp:(.text+0x30a): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()' /usr/bin/ld: gen.cpp:(.text+0x319): undefined reference to `Point::~Point()' /usr/bin/ld: /tmp/cc3i7Un9.o: in function `__static_initialization_and_destruction_0(int, int)': gen.cpp:(.text+0x365): undefined reference to `std::ios_base::Init::Init()' /usr/bin/ld: gen.cpp:(.text+0x380): undefined reference to `std::ios_base::Init::~Init()' /usr/bin/ld: /tmp/cc3i7Un9.o:(.data.rel.local.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0' collect2: error: ld returned 1 exit status
|
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 02, 2023, 02:29:35 PM |
|
C++ code(.cpp) is compiled with g++. gcc is used for C(.c) you have just use "make" command to compile.
look. you showed this program which seems to be a modified vanity search that will generate random addresses - correct? And according to my tests pure VanitySearch code base for CPU is two times faster than used through python. Will try it out later but my guess is that VanitySearch performance for CPU can be optimized even further by placing most used functions local scope vars to global. #include "secp256k1/SECP256k1.h" #include "secp256k1/Int.h" #include <iostream> #include <fstream>
int main() { Secp256K1 *secp256k1 = new Secp256K1(); secp256k1->Init(); Int privKey; privKey.SetBase10("1"); Point pub; std::string bitAddr; std::ofstream outFile; outFile.open("address.txt", std::ios::app); for(int i = 0; i < 1000000; i++) { pub = secp256k1->ComputePublicKey(&privKey); bitAddr = secp256k1->GetAddress(0, false, pub); outFile << bitAddr << '\n'; privKey.AddOne(); } outFile.close(); return 0; }
i am trying to execute this program to see the speed of this. but i cannot run this code. what i tried so far: i downloaded from your repository bitcoin_tools and change directory to VanitySearch_Linux then i create gen.cpp with your code you showed then i run "make" and i get no error i see there is a executable called "VanitySearch" but i do not see an executable gen that was compiled from gen.cppi tried $ g++ gen.cpp gen.cpp: In function ‘int main()’: gen.cpp:11:23: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings] 11 | privKey.SetBase10("1"); | ^~~ /usr/bin/ld: /tmp/ccVxtbUk.o: in function `main': gen.cpp:(.text+0x32): undefined reference to `Secp256K1::Secp256K1()' /usr/bin/ld: gen.cpp:(.text+0x48): undefined reference to `Secp256K1::Init()' /usr/bin/ld: gen.cpp:(.text+0x57): undefined reference to `Int::Int()' /usr/bin/ld: gen.cpp:(.text+0x70): undefined reference to `Int::SetBase10(char*)' /usr/bin/ld: gen.cpp:(.text+0x7f): undefined reference to `Point::Point()' /usr/bin/ld: gen.cpp:(.text+0xea): undefined reference to `Secp256K1::ComputePublicKey(Int*)' /usr/bin/ld: gen.cpp:(.text+0x1cb): undefined reference to `Point::~Point()' /usr/bin/ld: gen.cpp:(.text+0x1f5): undefined reference to `Secp256K1::GetAddress[abi:cxx11](int, bool, Point&)' /usr/bin/ld: gen.cpp:(.text+0x252): undefined reference to `Int::AddOne()' /usr/bin/ld: gen.cpp:(.text+0x2aa): undefined reference to `Point::~Point()' /usr/bin/ld: gen.cpp:(.text+0x319): undefined reference to `Point::~Point()' collect2: error: ld returned 1 exit status
how should i compile gen.cc to run your suggested program ?
|
|
|
|
citb0in (OP)
|
|
January 02, 2023, 02:57:36 PM Last edit: January 02, 2023, 03:42:35 PM by citb0in |
|
@yoshimitsu777:
try this:
- rename main.cpp to main.cpp.bak - rename gen.cpp to main.cpp - delete existing ./VanitySearch - run "make"
then just execute ./VanitySearch and you got this program executed. Hope this helps.
@AlexanderCurl: did you manage to run it on CUDA just like VanitySearch is being capable of ?
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
January 02, 2023, 04:44:47 PM |
|
here's the updated complete code variant with using multicore functionality and the secrets library for enhanded speed: #!/usr/bin/env python3 # 2023/Jan/01, citb0in_multicore_secrets.py import concurrent.futures import os import numpy as np import secrets import secp256k1 as ice
# how many cores to use #num_cores = 1 num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 1000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end): # Generate a NumPy array of random private keys using "secrets" library private_keys = np.array([secrets.randbelow(2**256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_multicore_secrets.txt', addresses, fmt='%s')
To speed up slightly this code (with multi core), you can write on different files: #!/usr/bin/env python3 # 2023/Jan/01, citb0in_multicore_secrets.py import concurrent.futures import os import numpy as np import secrets import secp256k1 as ice
# how many cores to use #num_cores = 1 num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 1000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end, i): # Generate a NumPy array of random private keys using "secrets" library private_keys = np.array([secrets.randbelow(2**256) for _ in range(start, end)])
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys]) np.savetxt('addresses_1M_multicore_secrets'+ str(i) +'.txt', thread_addresses, fmt='%s') return
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores # Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end, i))
Your code doesn't store private keys, are you sure you want to have only a list of addresses without their private keys?
|
|
|
|
NotATether
Legendary
Offline
Activity: 1652
Merit: 6962
In memory of o_e_l_e_o
|
|
January 02, 2023, 05:35:23 PM |
|
@AlexanderCurl, regarding sequential key generation:
Have you tried adding a large prime number, like 0xdeadbeef (just an example, I don't even know if that's prime)?
Sure, you'll have to check for overflow more frequently - fortunately, that's just a matter of doing a Greater-Than comparison followed by a subtraction - but a sufficiently large increment should make the keys look pseudorandom as far as the bits are concerned.
Have no idea what you are talking about. On my computer i use VanitySearch code on a daily basis for all types of programs.(random, sequential, whatever) Whoops, now that I checked again, it turns out that I was referring to a post written by @citb0in. Sorry for the misunderstanding. ... Well, so far so good. I showed this just to summarize things up. Afterwards I modified my code according to your suggestion so the private keys are not generated randomly but instead they are sequentially generated from a pre-defined starting point. Here's the modified version: #!/usr/bin/env python3 # 2022/Dec/26, [b]citb0in_seq.py[/b] import concurrent.futures import os import numpy as np import secp256k1 as ice
# how many cores to use num_cores = 1 #num_cores = os.cpu_count()
# Number of addresses to generate num_addresses = 1000000
# Starting point (decimal/integer) for private key starting_point = 123456789
# Define a worker function that generates a batch of addresses and returns them def worker(start, end, starting_point): # Initialize the private key to the starting point private_key = starting_point
# Initialize the list to hold to private keys private_keys = []
# Generate a batch of private keys sequentially for i in range(start, end): # Increment the private key private_key += 1
# Add the private key to the list private_keys.append(private_key)
# Use secp256k1 to convert the private keys to addresses thread_addresses = np.array([ice.privatekey_to_address(2, True, dec) for dec in private_keys])
return thread_addresses #return (thread_addresses, private_keys, start_int)
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores
# Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end, starting_point))
# Wait for the tasks to complete and retrieve the results addresses = [] for task in concurrent.futures.as_completed(tasks): addresses.extend(task.result())
# Write the addresses to a file np.savetxt('addresses_1M_seq_singlecore.txt', addresses, fmt='%s')
... To save eyes from being burnt by screens at 11PM, I will paste the relevant part of the code here: # Starting point (decimal/integer) for private key starting_point = 123456789
# Define a worker function that generates a batch of addresses and returns them def worker(start, end, starting_point): # Initialize the private key to the starting point private_key = starting_point
# Initialize the list to hold to private keys private_keys = []
# Generate a batch of private keys sequentially for i in range(start, end): # Increment the private key private_key += 1 #TODO why not use a large prime number instead of 1? 1 is not random at all, but a few dozen bits varying at once can be made to look random to others.
|
. .BLACKJACK ♠ FUN. | | | ███▄██████ ██████████████▀ ████████████ █████████████████ ████████████████▄▄ ░█████████████▀░▀▀ ██████████████████ ░██████████████ █████████████████▄ ░██████████████▀ ████████████ ███████████████░██ ██████████ | | CRYPTO CASINO & SPORTS BETTING | | │ | | │ | ▄▄███████▄▄ ▄███████████████▄ ███████████████████ █████████████████████ ███████████████████████ █████████████████████████ █████████████████████████ █████████████████████████ ███████████████████████ █████████████████████ ███████████████████ ▀███████████████▀ ███████████████████ | | .
|
|
|
|
citb0in (OP)
|
|
January 02, 2023, 07:02:10 PM |
|
To speed up slightly this code (with multi core), you can write on different files: [...] np.savetxt('addresses_1M_multicore_secrets'+ str(i) +'.txt', thread_addresses, fmt='%s') return [...]
This will finish faster, yes. But it will create only one single output file which contains only 62,500 addresses. My example have all 1 million addresses listed. I'm kinda puzzled what exactly you mean, can you explain, please? Your code doesn't store private keys, are you sure you want to have only a list of addresses without their private keys?
this code is obviously made simple. My own python program saves the keys as well of course. But for the sake of easiness I want to keep the code simple so everyone is able to test and compare.
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
January 02, 2023, 07:12:14 PM |
|
To speed up slightly this code (with multi core), you can write on different files: [...] np.savetxt('addresses_1M_multicore_secrets'+ str(i) +'.txt', thread_addresses, fmt='%s') return [...]
This will finish faster, yes. But it will create only one single output file which contains only 62,500 addresses. My example have all 1 million addresses listed. I'm kinda puzzled what exactly you mean, can you explain, please? If num_cores = 10, this code saves 100k addresses x 10 files. It doesn't create only 1 file, but 1 file for each thread. On my computer works.
|
|
|
|
citb0in (OP)
|
|
January 02, 2023, 07:13:34 PM Last edit: January 02, 2023, 07:26:14 PM by citb0in |
|
Have you tried adding a large prime number, like 0xdeadbeef (just an example, I don't even know if that's prime)? Sure, you'll have to check for overflow more frequently - fortunately, that's just a matter of doing a Greater-Than comparison followed by a subtraction - but a sufficiently large increment should make the keys look pseudorandom as far as the bits are concerned.
Whoops, now that I checked again, it turns out that I was referring to a post written by @citb0in. Sorry for the misunderstanding. [...]
# Generate a batch of private keys sequentially for i in range(start, end): # Increment the private key private_key += 1 #TODO why not use a large prime number instead of 1? 1 is not random at all, but a few dozen bits varying at once can be made to look random to others.
Hi NotAtTether. Long time ago I wrote a python script that computes 16.4 million of consecutive (non random) public keys (not addresses) in 12.3 s. No numpy, pure python, only 1 thread. [...] Generating consecutive public keys is way faster than generate random public keys.
I was just following up on arulbero's comment. I wanted to try how it behaves when the key creation is not random but sequential. That was the point. It is up to you if you want to do it differently as you suggested with the help of prime numbers. You could try it out and see how the performance could be affected or not. In spite of everything, I personally don't find the way secure enough. In any case, security aspects should not be ignored when creating keys. Of course, this varies from use case to use case and requirements may differ. It doesn't create only 1 file, but 1 file for each thread. On my computer works.
Now I understand, sorry my fault. I have overseen one line and was confused now I tried on my computer and of course it works. This is a nice one, it gives not only a slight change but a very good performance boost on my side I have modified to create 10 million addresses to make the difference more clear. $ time python3 citb0in_multicore_secrets.py
real 0m38,349s user 5m47,453s sys 0m16,060s
$ time python3 citb0in_multicore_secrets_splitsave.py
real 0m25,835s user 5m57,795s sys 0m14,681s
10 million addresses generated in less than 26 seconds. Rate = 387.071 (Python). ==> this is a performance boost of additional + 32.7 % on my computer. Crazy! thank you for pointing out @arulbero. Great! As AlexanderCurl pointed out correctly, using C++ is much much faster and unbeatable so far. However, I still persist i would love to see how our current code would behave with cupy or similar interfaces and thus the code would run in the much faster GPU. Then we could compare with C++ implementations like those shown of AlexanderCurl or VanitySearch, BitCrack, etc.
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 02, 2023, 07:57:30 PM |
|
@yoshimitsu777:
try this:
- rename main.cpp to main.cpp.bak - rename gen.cpp to main.cpp - delete existing ./VanitySearch - run "make"
then just execute ./VanitySearch and you got this program executed. Hope this helps.
understand now working - thanks!
|
|
|
|
arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
January 03, 2023, 02:04:46 PM Last edit: January 04, 2023, 04:09:21 PM by arulbero Merited by ABCbits (1), citb0in (1) |
|
I have modified to create 10 million addresses to make the difference more clear. $ time python3 citb0in_multicore_secrets.py
real 0m38,349s user 5m47,453s sys 0m16,060s
$ time python3 citb0in_multicore_secrets_splitsave.py
real 0m25,835s user 5m57,795s sys 0m14,681s
10 million addresses generated in less than 26 seconds. Rate = 387.071 (Python). ==> this is a performance boost of additional + 32.7 % on my computer. Crazy! thank you for pointing out @arulbero. Great! You don't use any specific function from numpy, then I suggest you to eliminate numpy besides you have to generate a random key k with k < n = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141 not k < 2**256 you can comment the check at this line: https://github.com/iceland2k14/secp256k1/blob/main/secp256k1.py#L290This is your code optimized: #!/usr/bin/env python3 # 2023/Jan/01, citb0in_multicore_secrets.py import concurrent.futures import os import secrets import secp256k1 as ice
# how many cores to use num_cores = 10 #num_cores = os.cpu_count()
# Set the number of addresses to generate num_addresses = 10000000
# Define a worker function that generates a batch of addresses and returns them def worker(start, end, i): addr_type = [2] * (end - start) is_compressed = [True] * (end - start) n = [0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141] * (end - start) f = open("addresses_1M_multicore_secrets" + str(i) + ".txt", "w") # Generate a list of random private keys using "secrets" library private_keys = list(map(secrets.randbelow,n))
# Use secp256k1 to convert the private keys to addresses thread_addresses = list(map(ice.privatekey_to_address, addr_type, is_compressed, private_keys)) # Write the addresses in the thread file list(map(lambda x:f.write(x+"\n"),thread_addresses))
f.close() return
# Use a ProcessPoolExecutor to generate the addresses in parallel with concurrent.futures.ProcessPoolExecutor() as executor: # Divide the addresses evenly among the available CPU cores addresses_per_core = num_addresses // num_cores # Submit a task for each batch of addresses to the executor tasks = [] for i in range(num_cores): start = i * addresses_per_core end = (i+1) * addresses_per_core tasks.append(executor.submit(worker, start, end, i))
|
|
|
|
citb0in (OP)
|
|
January 03, 2023, 09:22:13 PM |
|
Absolutely correct arulbero. Thanks for pointing out. I used 2**256 as a quick'n'dirty solution and forgot about the edge of the finite field 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141 Good catch.
|
_ _ _ __ _ _ _ __ |_) | / \ / |/ (_ / \ | \ / |_ |_) (_ |_) |_ \_/ \_ |\ __) \_/ |_ \/ |_ | \ __) --> citb0in Solo-Mining Group <--- low stake of only 0.001 BTC. We regularly rent about 5 PH/s hash power and direct it to SoloCK pool. Wanna know more? Read through the link and JOIN NOW
|
|
|
arulbero
Legendary
Offline
Activity: 1915
Merit: 2074
|
|
January 04, 2023, 04:08:38 PM Last edit: January 05, 2023, 01:08:39 PM by arulbero |
|
With last code, you should save at least 2.5 - 3s on your hardware, I guess from 25,8s to < 23s to compute 10 million keys. Did you try? I'm curious. Absolutely correct arulbero. Thanks for pointing out. I used 2**256 as a quick'n'dirty solution and forgot about the edge of the finite field 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141 Good catch.
To be more precise, randbelow(n-1)+1 otherwise with randbelow(n) could occur '0' (extremely unlikely). ...
==> I have generated 1,007,862 addresses (about 1 million keys) in about 15 seconds using one single core with your program. ==> I have generated exactly 1 millions addresses in 10 seconds using one single core with my program.
Conclusion: The originally enormous speed advantage is therefore, according to these tests, not really to be found in the code difference but in the fact that randomly generated keys are simply more computationally intensive and cost more time than simple sequential key generation. So just as you originally said arulbero.
It always depends on the purpose of use, of course, but I have my doubts that the sequential generation of private keys is not the best way. I therefore think, also in terms of security for other applications, that random private keys are always preferable to sequential ones despite the fact they are more time-intensive.
If you try to substitute randbelow(n) with randbelow(10000000) (or another small number), you'll see how much time it requires to sum up many keys to get a high key. I think you may get almost 1 Million keys per sec.
|
|
|
|
yoshimitsu777
Newbie
Offline
Activity: 72
Merit: 0
|
|
January 07, 2023, 11:56:11 AM |
|
# Write the addresses in the thread file list(map(lambda x:f.write(x+"\n"),thread_addresses))
f.close() return
what is difference using with open("addresses_1M_multicore_secrets" + str(i) + ".txt", "a") as f: f.write(f'{thred_addresses}n')
randbelow(n-1)+1
how to replace this line private_keys = list(map(secrets.randbelow,n))
|
|
|
|
|