Bitcoin Forum
December 09, 2016, 05:46:36 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: What exactly is the midstate? (trying to decode the json data in perl)  (Read 4898 times)
j16sdiz
Jr. Member
*
Offline Offline

Activity: 37


View Profile
December 23, 2010, 01:52:25 AM
 #1

The document say midstate is the hash of first half of data. But I can not repeat this in perl...  Huh

For example, I have got a work like this (from the PMS pool)
Code:
  "data": "000000014e010ed08a87644d4a7d9b9852e8d0c5f5d3fc253917fc2b0000201600000000cec99c8365ef81d58321dbfea0473d01fd513ffa84132a0138d82ead5fd705264d12a5dd1b04864c00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000",
...
  "midstate": "4e3b9b00f856803067ebe0aa4026eac253a77a70374d6e490b50c947d549f74e"


My code:
Code:
use Digest::SHA qw/ sha256_hex /;
$data = '000000014e010ed08a87644d4a7d9b9852e8d0c5f5d3fc253917fc2b0000201600000000cec99c8365ef81d58321dbfea0473d01fd513ffa84132a0138d82ead';   # first 64 bytes
$data_raw =  pack("H*", $data );   # unpack it as a byte string
print "sha_raw = " . sha256_hex( $data_raw  ) . "\n";

$data_swapped =   pack "N*", unpack "V*", $data ;   # endian swap
print "sha_swapped = " . sha256_hex( $data_swapped ) . "\n";

Output:
Code:
sha_raw = bb4c09641e2d71c977b78edfd86c9793251e8e03f7ea211dcf7c71b7e231c7ac
sha_swapped = 8cbe144c95e66e244959719dabc1cfd4812275548ba099d78361fad1bfa7123b

Anybody can help me on this?
1481305596
Hero Member
*
Offline Offline

Posts: 1481305596

View Profile Personal Message (Offline)

Ignore
1481305596
Reply with quote  #2

1481305596
Report to moderator
1481305596
Hero Member
*
Offline Offline

Posts: 1481305596

View Profile Personal Message (Offline)

Ignore
1481305596
Reply with quote  #2

1481305596
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481305596
Hero Member
*
Offline Offline

Posts: 1481305596

View Profile Personal Message (Offline)

Ignore
1481305596
Reply with quote  #2

1481305596
Report to moderator
jib
Member
**
Offline Offline

Activity: 84


View Profile
December 23, 2010, 03:00:45 AM
 #2

SHA256 prepares the input data by appending padding and a length field to the end of it, so the total input ends up being a multiple of 64 bytes.

The "data" string you're given has this preparation step done already, which means you can't just feed it into a normal hashing function like sha256_hex and expect the right result (because sha256_hex will do this preparation step again).

Read about how SHA256 works. The midstate is the internal state after processing the first 64-byte block.

(Also, in the line of your code that swaps the data, you meant to use $data_raw rather than $data, right?)

Looking forward to quantum computing so we can have qubitcoins.
j16sdiz
Jr. Member
*
Offline Offline

Activity: 37


View Profile
December 23, 2010, 03:58:18 AM
 #3

hmm... 
I was doing this because there are no perl module allow me to access the sha256 internals (at least not without saving the state to disk).

Does it means there are no way using the OS-provided crypto function that do complete sha256 hash work?
I think the  "via" code do the extra padding as well, or does it?
jib
Member
**
Offline Offline

Activity: 84


View Profile
December 23, 2010, 04:03:11 AM
 #4

You can use the crypto functions: Ignore the midstate, take the first 80 bytes (I think) of the data (i.e the data itself but not the padding) and hash that.

Why are you doing this? If you're writing a miner in perl using built-in crypto functions, it will be much slower than existing miners.

Looking forward to quantum computing so we can have qubitcoins.
taiki
Newbie
*
Offline Offline

Activity: 4


View Profile
December 23, 2010, 05:25:42 AM
 #5

Actually I'm pretty sure you can use the built in crypto without any modification, since it is already a multiple of 64 it won't try to pad it.

Python code makes it really simple:
Code:
    def checkwork(block, difficulty_rep):
        s = hashlib.sha256(block.decode('hex'))
        val = int(s.hexdigest(), 16)
        diff = int(difficulty_rep, 16)
        return val < diff
lfm
Full Member
***
Offline Offline

Activity: 196



View Profile
December 25, 2010, 02:45:12 AM
 #6

Actually I'm pretty sure you can use the built in crypto without any modification, since it is already a multiple of 64 it won't try to pad it.

Standard hash routines would add another 64 bytes then. It is not just padding, it is a length value and a terminator bit.
lfm
Full Member
***
Offline Offline

Activity: 196



View Profile
December 25, 2010, 02:48:58 AM
 #7

I was doing this because there are no perl module allow me to access the sha256 internals (at least not without saving the state to disk).

Does it means there are no way using the OS-provided crypto function that do complete sha256 hash work?
I think the  "via" code do the extra padding as well, or does it?

ignore the midstate and only use the first 80 bytes of the data if you are using a standard sha256 library. I have used the libgcrypt this way also (it is slow tho). You can also ignore the hash/hash1 fields passed, just use your own. Then return the data with your nonce inserted if you find a candidate hash.

Using standard sha256 libraries you will probably also need to undo the byte swaps on the data field. And reswap the data to return it.

j16sdiz
Jr. Member
*
Offline Offline

Activity: 37


View Profile
December 30, 2010, 06:55:05 AM
 #8

Actually I'm pretty sure you can use the built in crypto without any modification, since it is already a multiple of 64 it won't try to pad it.

Python code makes it really simple:
Code:
   def checkwork(block, difficulty_rep):
        s = hashlib.sha256(block.decode('hex'))
        val = int(s.hexdigest(), 16)
        diff = int(difficulty_rep, 16)
        return val < diff

what is that "block" object?

I tried to print out some hash from  the "cpuminer" and repeat it in perl with no success:

perl code:
Code:
#!/usr/bin/perl
use Digest::SHA qw/ sha256_hex/;

$raw =  pack 'H*', '00000001258124a1e0837367309ed9433af69c741513067793bf1f490000c0c800000000f2a45d9b1294bf78d27fe1d77558fbedf2b1eb37bb5f1808d7b77e33d809b8fb4d1c26d21b04864c01000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000';

$head = substr($raw, 0, 0x80); # first 80 byte
$head_le = pack "N*", unpack "V*", $head;  # first 80 byte, endian-swapped

print sha256_hex($head), "\n";
print sha256_hex($head_le), "\n";


my output:
Code:
feca1e6a511bf10a01177086ae141115f0d554fc8758c640b81f9d3121d6cbc9
06f33a488390fb281b7d2fc68140a7ad69f8397caa04475c7f65f0b5f3f4922f


output generated with cpu-miner:
Code:
DBG: data:
 00000001258124a1e0837367309ed9433af69c741513067793bf1f490000c0c800000000f2a45d9b1294bf78d27fe1d77558fbedf2b1eb37bb5f1808d7b77e33d809b8fb4d1c26d21b04864c01000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000
DBG: hash1:
 697c6991ecddeb98d9af8f02a249b7e4006c8ae107a0991d4be8a88f4c9fcf1b
DBG: hash:    
 924638a8a492065a924f699ca1c13dd1fb4af43bb1c6d9ce8cab2e9bb6189ea7


patch i used again cpuminer:

Code:
diff --git a/sha256_generic.c b/sha256_generic.c
index 444e913..b9ba930 100644
--- a/sha256_generic.c
+++ b/sha256_generic.c
@@ -250,11 +250,23 @@ bool scanhash_c(const unsigned char *midstate, unsigned char *data,
                n++;
                *nonce = n;

+char *hs;
+hs = bin2hex(data - 64, 128);
+fprintf(stderr, "DBG: data:\n %s\n", hs);
+free(hs);
+
                runhash(hash1, data, midstate);
                runhash(hash, hash1, sha256_init_state);

                stat_ctr++;

+hs = bin2hex(hash1, 32);
+fprintf(stderr, "DBG: hash1:\n %s\n", hs);
+free(hs);
+hs = bin2hex(hash, 32);
+fprintf(stderr, "DBG: hash:\n %s\n", hs);
+free(hs);
+
secmff
Newbie
*
Offline Offline

Activity: 10


View Profile
April 06, 2011, 05:07:14 PM
 #9


I found two issues with that perl code. You need the first 80 bytes, not 0x80. Second, calculate back to big-endian to get the same result as the miner:

Code:
#!/usr/bin/perl
use Digest::SHA qw/ sha256 /;

$raw =  pack 'H*', '00000001258124a1e0837367309ed9433af69c741513067793bf1f490000c0c800000000f2a45d9b1294bf78d27fe1d77558fbedf2b1eb37bb5f1808d7b77e33d809b8fb4d1c26d21b04864c01000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000';
$short = substr($raw, 0, 80); # first 80 byte

$short_le = pack "N*", unpack "V*", $short; # to Little Endian

print "short_le:", unpack("H*", $short_le), "\n";

$hash1 = sha256($short_le);
$hash = sha256(sha256($short_le));

$hash1_be = pack "N*", unpack("V*", $hash1); # BE again
$hash_be  = pack "N*", unpack("V*", $hash);
print "hash1:  ", unpack("H*", $hash1_be), "\n";
print "hash:   ", unpack("H*", $hash_be), "\n";
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!