Print Page - Why does Bitcoin's SHA256 implementation work differently?

Title: Why does Bitcoin's SHA256 implementation work differently?
Post by: minerino123 on June 22, 2017, 04:09:17 PM

Hi, I am new here, so if this is the wrong forum, please let me know where this post is better placed.

I was trying to write my own miner, but simply don't understand what the Hash() function in hash.h is doing.
This is my code:

Code:

// allocate 80 bytes of 0x00s
unsigned char* memory = (unsigned char*)malloc(80);
int* result = (unsigned int*)malloc(32);
memory,0,80);
memory,0,32);

seems to be doing hashes under the SHA-256 name
hash = Hash(memory, memory+80);
hash:%s\n",hash.GetHex().c_str());
//14508459...0e57e74b

However when I try to do this with _any_ other SHA256 implementation I always get a different result (I tried three, which gave me the same result). Here is my Java code:

Code:

MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] digest = md.digest(ByteBuffer.allocate(80).array());
for(byte c : digest) {
	System.out.format("%x", c);
}
//5b6fb58e61fa475939767d68a446f97f1bff2c0e5935a3ea8bb51e6515783d8

I thought maybe I have to double hash, but this leaves me with this:

Code:

MessageDigest md2 = MessageDigest.getInstance("SHA-256");
byte[] digest2 = md2.digest(md2.digest(ByteBuffer.allocate(80).array()));
for(byte c : digest2) {
	System.out.format("%x", c);
}
//4be757e8f70eb93640c8468274ba759745a7aa2b7d25ab1e421b259845014

So the hash I get from the Bitcoin implementation differs from any other SHA256 implementation for some reason... what am I missing? Am I using the algorithm wrong? Is it initialized differently (other implementation don't give me a possibility to parametrize the function)? Does it simply work differently (not according to the standard)?

Any help would be appreciated

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: achow101 on June 22, 2017, 05:08:18 PM

The hashes that Bitcoin uses are double SHA256 hashes. The hashes are also usually in Little Endian byte order. So your second attempt with the double hash is actually correct, but with the wrong byte order. You need to byte swap your result to be in the right ordering.

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: minerino123 on June 22, 2017, 09:35:01 PM

Thanks for your reply, I still don't understand the problem though. The array consists of 80 bytes all being of value 0x00. This should be the same regardless of byte order, shouldn't it?

EDIT: I don't know really why, but I tested the byte order just to be sure... as expected it doesn't make a difference for the byte array in question.

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: achow101 on June 22, 2017, 09:56:49 PM

Quote from: minerino123 on June 22, 2017, 09:35:01 PM

It has nothing to do with what you initialized the array with nor with anything that you passed into the function. The endianness has to do with the output of the hash function. The output is different because Bitcoin uses little endian byte order for basically everything, so the output of the hash function will be in little endian byte order. However most other SHA256 implementations will output in big endian byte order so you will need to reverse the hash from other implementations in order to get the little endian byte order that Bitcoin uses.

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: physicist on June 23, 2017, 03:22:36 AM

Bitcoin uses little endian byte order. This is not what you are doing.

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: DannyHamilton on June 23, 2017, 03:24:36 AM

Quote from: minerino123 on June 22, 2017, 04:09:17 PM

- snip -
This is my code:

Code:

- snip -
//14508459...57e74b

Here, let me color code that for you. Perhaps it will be easier to see what achow101 is saying...
14508459...57e74b

Quote from: minerino123 on June 22, 2017, 04:09:17 PM

- snip -
I thought maybe I have to double hash, but this leaves me with this:

Code:

- snip -
//4be757e8f70eb93640c8468274ba759745a7aa2b7d25ab1e421b259845014

Take a look at the matching colors, see if you can spot what has happened...
4be757e8f70eb93640c8468274ba759745a7aa2b7d25ab1e421b259845014

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: Coding Enthusiast on June 23, 2017, 04:14:35 AM

Quote from: physicist on June 23, 2017, 03:22:36 AM

Bitcoin uses little endian byte order. This is not what you are doing.

Not always! There are cases which use Big-endian.

Title: Re: Why does Bitcoin's SHA256 implementation work differently?
Post by: minerino123 on June 23, 2017, 09:42:47 AM

@achow101: Thanks for clarifying... I think I looked at too many bits yesterday. This helps a lot!

@DannyHamilton: Thanks as well for putting in the effort to color it and everything. However you did fake your comment a little bit:
You wrote:

Quote

- snip -
//14508459...57e74b

I wrote:

Quote

//14508459...057e74b

See the additional zero there? That kind of bothers me still... Another implementation contains the zeros as well, so I think there is a problem with my Java implementation... I'll investigate...

EDIT: Simple problem... my Java code was swallowing leading zeros...
Bad code:

Code:

System.out.format("%x", c);

Good code:

Code:

System.out.format("%02x", c);

Thanks again to all here for helping me with this problem!

EDIT2: So just for completness sake in case someone googles this... this Java function seems to produce the same thing as Bitcoin's Hash(...) function:

Code:

static byte[] btcSha256d(byte[] memory) throws NoSuchAlgorithmException{
	MessageDigest md = MessageDigest.getInstance("SHA-256");
	byte[] result = md.digest(md.digest(memory));
	for(int i=0;i<result.length/2;i++){
		byte temp = result[i];
		result[i]=result[result.length-i-1];
		result[result.length-i-1]=temp;
	}
	return result;
}

Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: minerino123 on June 22, 2017, 04:09:17 PM