Trying to understand the getwork API function

[ahtremblay]

Hello, I am trying to understand the details about the getwork function and what it returns. When I call getwork, this is an example of what I get from my bitcoin deamon.

Code:

./bitcoind getwork
{
    "midstate" : "34c2cc4192d802888d2204188e80df8133ddbb17fd0db5ea1382096916f35369",
    "data" : "000000013cb5d00bd73e716c2d3c558a52b8c79fe8532b12307a7d91000008ba0000000007b7757adedda569a66c17a870bce5135faa2b3e5f46d22bed6e15d79296178f4ded6d461a1d932f00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000",
    "hash1" : "00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000",
    "target" : "00000000000000000000000000000000000000000000002f931d000000000000"
}

I understand how to do hashing since I have just completed a basic c++ implementation. What I do not understand is midstate, data and hash1. Why is the midstate supplied? Isn't it the job of the miner to calculate the hash. If so, why does the deamon waste cycle computing the hash for the first half of the data?

The data. Is it already stuffed and precomputed for hashing? It seems that I must do sha256(sha256(data)). I am not sure if the data has been appended a 1 at the end, then zeros, along with the message length.

What is hash1 and what do I use it for?

[Raistlan]

scanhash_c in https://github.com/jgarzik/cpuminer/blob/master/sha256_generic.c shows midstate and hash1 being used.

Basically, there are 3 chunks that the sha256_transform function in that file is called for per "nonce" value. [Here is the header definition from https://en.bitcoin.it/wiki/Block_hashing_algorithm:]

Field	Purpose	Updated when...	Size (Bytes)
Version	Block version number	You upgrade the software and it specifies a new version	4
Previous hash	Hash of the previous block	A new block comes in	32
Merkle root	256-bit hash based on all of the transactions	A transaction is accepted	32
Timestamp	Current timestamp	Every few seconds	4
"Bits"	Current target in compact format	The difficulty is adjusted	4
Nonce	32-bit number (starts at 0)	A hash is tried (increments)	4

The first chunk is for the first 64 Bytes of the header:

Field	Size (Bytes)
Version	4
Previous hash	32
Merkle root (first 28 Bytes)	28

This first chunk is constant for all of the 2^32 values of the "Nonce" that are tried, since the "Nonce" bits are in the second chunk. Therefore, the hash of the first chunk is constant, as well, and doesn't need to be re-calculated for each of the 2^32 "Nonce" values that are attempted. "midstate" and "hash1" capture the relevant state after the first chunk has been hashed, so that you can start right into hashing the second chunk for each of the "Nonce" values.

The second chunk is for the second 64 Bytes of the header:

Field	Size (Bytes)
Merkle root (last 4 Bytes)	4
Timestamp	4
"Bits"	4
Nonce	4
SHA-256 Defined bits to make the chunk 64 Bytes total	48

This chunk needs to be hashed for every value of "Nonce", since it changes [in its 13th through 16th Bytes] each time.

After that is hashed, that hash is then put in another 64 Byte chunk that gets hashed, as well. That final hash is what gets compared to the target.

Field	Size (Bytes)
Final hash of chunks 1 and 2	32
SHA-256 Defined bits to make the chunk 64 Bytes total	32

[ahtremblay]

Thank you for the answer. This cleared a lot of things up, however I am still confused about this one point.

Field	Size (Bytes)
Merkle root (last 4 Bytes)	4
Timestamp	4
"Bits"	4
Nonce	4
SHA-256 Defined bits to make the chunk 64 Bytes total	48

I do not understand how the last 48 bytes are filled. According to the sha256 standard, the first of these 48 bytes needs to be a 1. All the rest needs to be a sequence of zero, except for the last few bits. The last few bits need to be the length of the message. The length of this message is 4bytes * 4 = 16 bytes, or 256 bits. Even If we count both chunks of bytes, the length of the message is (64+16) * 8 = 640. However, the last bytes of the string returned by my bitcoind deamon are 80020000. In decimal, it is 2147614720. This is not 640 bits.

Code:

data=000000013cb5d00bd73e716c2d3c558a52b8c79fe8532b12307a7d91000008ba0000000007b7757adedda569a66c17a870bce5135faa2b3e5f46d22bed6e15d79296178f4ded6d461a1d932f00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000

[Raistlan]

The value:

Code:

data=000000013cb5d00bd73e716c2d3c558a52b8c79fe8532b12307a7d91000008ba0000000007b7757adedda569a66c17a870bce5135faa2b3e5f46d22bed6e15d79296178f4ded6d461a1d932f00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000

is Little-Endian, I believe, so the last 4 bytes are actually 0x 00 00 02 80 = 640.

The leading 1 followed by padding 0s part of the SHA-256 expansion to 64 Bytes is:

Code:

80000000000000000000000000000000000000000000000000000000000000000000000000

[ahtremblay]

The leading 1 followed by padding 0s part of the SHA-256 expansion to 64 Bytes is:

Code:

80000000000000000000000000000000000000000000000000000000000000000000000000

After counting the bits, I realized that the leading 1 followed by the padding of 0 actually comes out to:

Code:

0000008000000000000000000000000000000000000000000000000000000000000000000000000000000000

So, it appears that this padding is performed in little endian as well. Converting it, I get:

00000080 (little endian)
80000000 (big endian)

This is where the source of my confusion is at this point. I am not sure if, before performing the sha256 calculation, I must convert the whole message to big endian, only the padding section, or nothing at all.

[SebastianJu]

I have a more fundamental question for understanding this. I didnt work with hexadecimal numbers so I dont know how to split the data.

I already have a getwork-answer from testnet. The data-part is the one needed to hash. Its 256 Signs long and is hexadecimal I think. Which means numbers are from 0 to f which means 16 possibilities for each sign. But now I dont have a clue how to use this. According to https://en.bitcoin.it/w/index.php?title=Getwork the data-part is the part to be hashed. And here it seems to be a synonym for Block-Header: https://en.bitcoin.it/wiki/Block_hashing_algorithm. So when the 256 signs long data-part is the Block-Header and the block header in the table contains 4 + 32 + 32 + 4 + 4 + 4 Bytes how is this converted? I mean I should have 80Bytes as Block-Header according to https://en.bitcoin.it/wiki/Block_hashing_algorithm but have 256 signs with 16 states each. I dont get how to transform this...

Its probably a real noobquestion but till now i didnt have to bother with hexadecimal thinking-structures...