mamuu (OP)
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 73
Merit: 19
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 26, 2021, 07:59:41 PM |
|
Hi! Can I export all signatures from blockchain and parse R values of signatures? Can you suggest a method? https://gz.blockchair.com/bitcoin/inputs/from the files at this address I did it with Python but it's very slow Do you know a code with C? Thank you.
|
1DWA3Sa8i6eHVWV4AG4UP2SBhYB2XrfiHW
|
|
|
MixMAx123
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 26, 2021, 08:20:42 PM |
|
If you're looking for double r-values, forget about it! You are definitely not the first to search the blockchain for it.
I wrote a blockchain explorer that you can use to search for anything in the blockchain. But it's not finished yet and needs a lot of memory and RAM.
|
|
|
|
pooya87
Legendary
Offline
Activity: 3556
Merit: 10803
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 27, 2021, 05:17:45 AM |
|
Can I export all signatures from blockchain and parse R values of signatures?
There are billions of signatures on the bitcoin blockchain, why do you need to export "all" of them? I did it with Python but it's very slow Do you know a code with C?
Things aren't "very slow" because of the programming language, they are usually slow because of bad code or simply slow process. As for other code, there are general blockchain parsers but there is no point in extracting signatures alone so there isn't any "good" code for it.
|
|
|
|
mamuu (OP)
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 73
Merit: 19
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 27, 2021, 08:42:02 AM |
|
If you're looking for double r-values, forget about it! You are definitely not the first to search the blockchain for it.
I wrote a blockchain explorer that you can use to search for anything in the blockchain. But it's not finished yet and needs a lot of memory and RAM.
Thank you I will not be looking at repeating R values. I will use it in some tests and at the same time I am trying to learn and develop the parse operation in the blockchain. Can I export all signatures from blockchain and parse R values of signatures?
There are billions of signatures on the bitcoin blockchain, why do you need to export "all" of them? I did it with Python but it's very slow Do you know a code with C?
Things aren't "very slow" because of the programming language, they are usually slow because of bad code or simply slow process. As for other code, there are general blockchain parsers but there is no point in extracting signatures alone so there isn't any "good" code for it. Thank you for the explanation pooya87. I had such an idea after reviewing the video here. https://www.youtube.com/watch?v=C6zrPMnyWSw
|
1DWA3Sa8i6eHVWV4AG4UP2SBhYB2XrfiHW
|
|
|
|
pooya87
Legendary
Offline
Activity: 3556
Merit: 10803
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 27, 2021, 10:05:54 AM |
|
That's a pretty bad example. First of all that is not the correct way of benchmarking a code, as you can see the repeated execution of the same command reports different times (totals) that have a spread with a difference as big as 68% to my calculation. The correct way would be a warm-up, then execution of the same method (aka a function) in a loop with an accurate timer attached that measures the time it takes to run that chunk of code then reports the mean with deviance discarding any value that is way off the mark. This also eliminates the time it takes for the first execution of the program. I don't know much about python but I can write a C# program that takes a longer time to open for the first time (eg. self-contained without ReadyToRun) but then runs just as fast C. Secondly the speed of an application as a whole depends on the quality of the code and the amount of optimization that has gone into it. A bad C application runs just as slowly as A bad python application. It comes down to how familiar that developer is with the language and how correctly they can benchmark it and how effectively they can improve their code. For example there are certain cases where the following runs faster: for (int i=0; i<n; i+=4) { sum1 += data[i+0]; sum2 += data[i+1]; sum3 += data[i+2]; sum4 += data[i+3]; } sum = sum1 + sum2 + sum3 + sum4;
Finally C is faster than most other languages in certain cases and only if you have squeezed every ounce of performance from codes written in each language. Take a look at these two posts, different people write the same code in different languages some even rewrite the same code in python and get faster results than the starter: https://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-languagehttps://codegolf.stackexchange.com/questions/26371/how-slow-is-python-really-part-ii
|
|
|
|
NotATether
Legendary
Offline
Activity: 1708
Merit: 7184
In memory of o_e_l_e_o
|
Bear in mind that any program that extracts R values will need to use different procedures for segwit and non-segwit transactions, because segwit transactions have the signatures in the witness data while normal transactions (including "wrapped" segwit transactions) have them in the scriptSig.
Secondly the speed of an application as a whole depends on the quality of the code and the amount of optimization that has gone into it. A bad C application runs just as slowly as A bad python application. It comes down to how familiar that developer is with the language and how correctly they can benchmark it and how effectively they can improve their code. For example there are certain cases where the following runs faster:i
C# is actually much faster than Python because the files are compiled down to byte code that .NET runtime understands. Same with Java and other languages with a runtime. Python has an not well documented feature where it can compile the python files down into its own bytecode separately, and you can get similar performance as C# from that since the burden of parsing statements is removed. C is just compiling all its statements down to assembly code for x86 processors which is just a tiny bit faster than C# for all code that isn't doing some scientific for math work because nearly all programs for the most part spend most of their time making system calls or altering data structures for which native assembly code cannot optimize (there are SSE registers, AVX registers, the L1 and L2 cache but they are all too small to store complex data structures and are useless for shortening the overhead programs transition between user land and OS land). And of course all performance advantages of C (and C++ which for speed purposes is just C with classes) are lost the moment you use a big general-purpose library inside your program. There are bitcoin-related programs written in C and the reason they are so fast is because they use their own hand-written functions, and this works because they know their application requirements so they only write the functionality that will actually be used, unlike a general-purpose library (such as Boost) which cannot make this assumption.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3416
Merit: 17280
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
January 27, 2021, 05:57:18 PM |
|
If it helps, I made [~500 GB work in progress] Bitcoin block data: inputs, outputs and transactions: It's not complete yet, and unfortunately I only now noticed my download got stuck 7 days ago. Currently, I have 220-ish GB of data from "inputs" available at much better download speed than Blockchair.com itself offers.
|
|
|
|
mamuu (OP)
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 73
Merit: 19
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 11, 2021, 07:26:20 AM |
|
Hi My question went like this I understand asserts in python. but I didn't understand the first '48' part. my aim is to separate the "signature" and separate it as in the link below, of course together with the PubKey Can you give me information about conditions (assert) that will help me? Thanks. information that helps me https://bitcoin.stackexchange.com/questions/58853/how-do-you-figure-out-the-r-and-s-out-of-a-signature-using-pythontype :pubkeyhash spending_signature_hex : 483045022100cbee7b355c737bccdaaaf566b52e07c6e560fa33861b3035d37feffad94f66e6022 053114a529ab13eba906f9e08288bcf02dd5142ae0ecb117f10d54f56868443af014104d720973d f5c090aa1dd17adefd9f575baa2f0547dd2913fb4982bf423f1cf82623b700e0d1b7f2b7b5dcfa9 301f8c197dd0b601204d00e02b544251c5d9ac45a information I want to parse r = 00cbee7b355c737bccdaaaf566b52e07c6e560fa33861b3035d37feffad94f66e6 s = 53114a529ab13eba906f9e08288bcf02dd5142ae0ecb117f10d54f56868443af PubKey= 04d720973df5c090aa1dd17adefd9f575baa2f0547dd2913fb4982bf423f1cf82623b700e0d1b7f 2b7b5dcfa9301f8c197dd0b601204d00e02b544251c5d9ac45a def parse_element(hex_str, offset, element_size): """ :param hex_str: string to parse the element from. :type hex_str: hex str :param offset: initial position of the object inside the hex_str. :type offset: int :param element_size: size of the element to extract. :type element_size: int :return: The extracted element from the provided string, and the updated offset after extracting it. :rtype tuple(str, int) """
return hex_str[offset:offset+element_size], offset+element_size
def dissect_signature(hex_sig): """ Extracts the r, s and ht components from a Bitcoin ECDSA signature. :param hex_sig: Signature in hex format. :type hex_sig: hex str :return: r, s, t as a tuple. :rtype: tuple(str, str, str) """
offset = 0 # Check the sig contains at least the size and sequence marker assert len(hex_sig) > 4, "Wrong signature format." sequence, offset = parse_element(hex_sig, offset, 2) # Check sequence marker is correct assert sequence == '30', "Wrong sequence marker." signature_length, offset = parse_element(hex_sig, offset, 2) # Check the length of the remaining part matches the length of the signature + the length of the hashflag (1 byte) assert len(hex_sig[offset:])/2 == int(signature_length, 16) + 1, "Wrong length." # Get r marker, offset = parse_element(hex_sig, offset, 2) assert marker == '02', "Wrong r marker." len_r, offset = parse_element(hex_sig, offset, 2) len_r_int = int(len_r, 16) * 2 # Each byte represents 2 characters r, offset = parse_element(hex_sig, offset, len_r_int) # Get s marker, offset = parse_element(hex_sig, offset, 2) assert marker == '02', "Wrong s marker." len_s, offset = parse_element(hex_sig, offset, 2) len_s_int = int(len_s, 16) * 2 # Each byte represents 2 characters s, offset = parse_element(hex_sig, offset, len_s_int) # Get ht ht, offset = parse_element(hex_sig, offset, 2) assert offset == len(hex_sig), "Wrong parsing."
return r, s, ht
example_sig = None # insert_sig_here r, s, ht = dissect_signature(example_sig) print "r: %s\ns: %s\nht: %s\n" % (r, s, ht)
|
1DWA3Sa8i6eHVWV4AG4UP2SBhYB2XrfiHW
|
|
|
alexeyneu
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 351
Merit: 37
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 11, 2021, 07:45:50 AM Last edit: December 11, 2021, 06:59:29 PM by alexeyneu |
|
There are bitcoin-related programs written in C and the reason they are so fast is because they use their own hand-written functions, and this works because they know their application requirements so they only write the functionality that will actually be used, unlike a general-purpose library (such as Boost) which cannot make this assumption.
C has sse support. almost no one use it. say std::cout <<"\nCoinbase: "<< txScriptSig <<"\nPubkeyScript: "<< pubScriptSig <<"\nMerkle Hash: "<< merkleHash <<"\nByteswapped: "<< merkleHashSw <<"\nGenerating block...\n\n"; unsigned char block_hash1[32]; #ifdef _MSC_VER _declspec(align(16)) unsigned char block_hashfp[32]; #else __attribute__((aligned(16))) unsigned char block_hashfp[32]; #endif unsigned int counter=0, start = time(NULL); int drift = 4; __m128i r, m, b, camp; __m128i zero = _mm_setzero_si128();
unsigned short turt; while(1) {
SHA256((unsigned char*)&block_header, 80, block_hash1); SHA256(block_hash1, 32, block_hashfp);
r = _mm_load_si128((__m128i *)block_hashfp); m = _mm_load_si128((__m128i *)(block_hashfp+16)); switch (drift) { case 0: break; case 1: b = _mm_alignr_epi8(m, r, 31);
break; case 2: b = _mm_alignr_epi8(m, r, 30); break; case 3: b = _mm_alignr_epi8(m, r, 29); break; case 4: b = _mm_alignr_epi8(m, r, 28); break; case 5: b = _mm_alignr_epi8(m, r, 27); break; case 6: b = _mm_alignr_epi8(m, r, 26); break; case 7: b = _mm_alignr_epi8(m, r, 25); break; case 8: b = _mm_alignr_epi8(m, r, 24); break; case 9: b = _mm_alignr_epi8(m, r, 23);
break; case 10:
b = _mm_alignr_epi8(m, r, 22); break; case 11: b = _mm_alignr_epi8(m, r, 21);
break; case 12: b = _mm_alignr_epi8(m, r, 20); break; case 13: break; default:
break; } /*now it turns out like: * Hashes 6018c18fcd4bfabb523f2ee1f6c18880f38135f3eb0ff1aa76acbc2fa94eb8ce * And * b 0000000000000000 000000006018c18f */
camp = _mm_cmpeq_epi8(b ,zero); turt = _mm_movemask_epi8(camp); if (turt == 0xffff) { std::reverse(block_hashfp,block_hashfp +32); std::cout << "\nBlock found!\nHash: " << bin2hex(block_hashfp, 32) <<"\nNonce: " << block_header.startNonce << "\nUnix time: "<< block_header.unixtime << std::endl; break; } if(time(NULL) - start > 0) { std::cout << counter <<" Hashes/s, Nonce "<< block_header.startNonce<<'\r'; start = time((time_t *)(counter = 0)); } if( ++block_header.startNonce == 0 ) block_header.unixtime++ ; //trick is that to change pre-start time to find a block(really it's smth else) faster then nonce wraps counter++; }
48 stuff looks like some hex'ed ascii phase(readable)
|
|
|
|
COBRAS
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 895
Merit: 22
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 11, 2021, 06:22:30 PM |
|
Hi! Can I export all signatures from blockchain and parse R values of signatures? Can you suggest a method? https://gz.blockchair.com/bitcoin/inputs/from the files at this address I did it with Python but it's very slow Do you know a code with C? Thank you. I use this - rscan + rsz scrypt https://cryptoxploit.com/rsz-key/But data need recheck, because public key not for rsz ome time
|
[
|
|
|
mamuu (OP)
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 73
Merit: 19
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 12, 2021, 08:08:06 AM |
|
There are bitcoin-related programs written in C and the reason they are so fast is because they use their own hand-written functions, and this works because they know their application requirements so they only write the functionality that will actually be used, unlike a general-purpose library (such as Boost) which cannot make this assumption.
C has sse support. almost no one use it. say std::cout <<"\nCoinbase: "<< txScriptSig <<"\nPubkeyScript: "<< pubScriptSig <<"\nMerkle Hash: "<< merkleHash <<"\nByteswapped: "<< merkleHashSw <<"\nGenerating block...\n\n"; unsigned char block_hash1[32]; #ifdef _MSC_VER _declspec(align(16)) unsigned char block_hashfp[32]; #else __attribute__((aligned(16))) unsigned char block_hashfp[32]; #endif unsigned int counter=0, start = time(NULL); int drift = 4; __m128i r, m, b, camp; __m128i zero = _mm_setzero_si128();
unsigned short turt; while(1) {
SHA256((unsigned char*)&block_header, 80, block_hash1); SHA256(block_hash1, 32, block_hashfp);
r = _mm_load_si128((__m128i *)block_hashfp); m = _mm_load_si128((__m128i *)(block_hashfp+16)); switch (drift) { case 0: break; case 1: b = _mm_alignr_epi8(m, r, 31);
break; case 2: b = _mm_alignr_epi8(m, r, 30); break; case 3: b = _mm_alignr_epi8(m, r, 29); break; case 4: b = _mm_alignr_epi8(m, r, 28); break; case 5: b = _mm_alignr_epi8(m, r, 27); break; case 6: b = _mm_alignr_epi8(m, r, 26); break; case 7: b = _mm_alignr_epi8(m, r, 25); break; case 8: b = _mm_alignr_epi8(m, r, 24); break; case 9: b = _mm_alignr_epi8(m, r, 23);
break; case 10:
b = _mm_alignr_epi8(m, r, 22); break; case 11: b = _mm_alignr_epi8(m, r, 21);
break; case 12: b = _mm_alignr_epi8(m, r, 20); break; case 13: break; default:
break; } /*now it turns out like: * Hashes 6018c18fcd4bfabb523f2ee1f6c18880f38135f3eb0ff1aa76acbc2fa94eb8ce * And * b 0000000000000000 000000006018c18f */
camp = _mm_cmpeq_epi8(b ,zero); turt = _mm_movemask_epi8(camp); if (turt == 0xffff) { std::reverse(block_hashfp,block_hashfp +32); std::cout << "\nBlock found!\nHash: " << bin2hex(block_hashfp, 32) <<"\nNonce: " << block_header.startNonce << "\nUnix time: "<< block_header.unixtime << std::endl; break; } if(time(NULL) - start > 0) { std::cout << counter <<" Hashes/s, Nonce "<< block_header.startNonce<<'\r'; start = time((time_t *)(counter = 0)); } if( ++block_header.startNonce == 0 ) block_header.unixtime++ ; //trick is that to change pre-start time to find a block(really it's smth else) faster then nonce wraps counter++; }
48 stuff looks like some hex'ed ascii phase(readable) i will review Thanks alexeyneu
|
1DWA3Sa8i6eHVWV4AG4UP2SBhYB2XrfiHW
|
|
|
gmaxwell
Moderator
Legendary
Offline
Activity: 4242
Merit: 8647
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 27, 2021, 06:40:09 AM |
|
anyone want to take bets on the above 'precompiled binary' from the newbie account being malware?
|
|
|
|
mamuu (OP)
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 73
Merit: 19
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
December 27, 2021, 08:30:03 AM |
|
anyone want to take bets on the above 'precompiled binary' from the newbie account being malware?
We are not being optimistic anymore, I agree and increase the bet.
|
1DWA3Sa8i6eHVWV4AG4UP2SBhYB2XrfiHW
|
|
|
COBRAS
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 895
Merit: 22
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 30, 2024, 06:19:51 PM |
|
Good day
What files from this:
blockdata.loyce.club/inputs/ (159 GB - currently incomplete, growing 8 GB per day) blockdata.loyce.club/outputs/ (117 GB) blockdata.loyce.club/transactions/ (43 GB)
contain r,s,z transaction data ?
Thank you
Thank you
|
[
|
|
|
|