Need Help Understanding Base58Check Encoding...

[ExabyteMiner]

For fun, I am attempting to generate bitcoin WIF PrivKeys from random binary strings. I have everything figured out except the final step of converting to WIF using base58check encoding.

Here is where I am at:
Step 1) Generated random binary string

Code:

0100 0100 1101 0000 0000 1111 0110 1110 1011 0010 1110 0101 0100 1001 0001 1100
1101 0111 1010 1011 0111 1110 0111 0001 1000 0101 1101 1000 0001 1011 0110 0111
1010 0010 0011 1100 0100 1001 1000 0000 1111 0110 0010 1011 0010 1110 1101 0000
1001 0001 0100 1101 0011 0010 1011 0111 1110 1011 0001 1100 0101 0101 1000 0001

Step 2) Converted binary string to hex string

Code:

44D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

Step 3) Added 0x80 byte to beginning of hexstring

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

Step 4) Hashed new hex string to SHA256 encoded string, took result and hashed to a new SHA256 string

Code:

2897b55427f86f73136d55c1729cb4ce56d72e40e33baf8ca10e9e02cb963434

Step 5) Took first 4 bytes (checksum) and added them to end of hex string from Step 2

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C55812897b554

And this is where I am lost... No matter what information I look up online I can not seem to figure out how to turn the last string into a base58check encoded string to complete the process of turning it into a WIF PrivKey.

Thanks to "TP's Go Bitcoin Tests", I know that the end result should be

Code:

5JLbJxi9koHHvyFEAERHLYwG7VxYATnf8YdA9fiC6kXMghkYXpk

But I have no idea nor can I seem to find a clear, coherent explanation of the base58check process that changed my modified hex string to a WIF PrivKey...

To be clear, I want to actually learn how the process works, so please do not just point me at a tool online and say "Just do it here, it is much easier".

Thank you in advance to anyone who can help explain this to me!

[1714165266]

1714165266

[1714165266]

1714165266

[1714165266]

1714165266

[1714165266]

1714165266

[1714165266]

1714165266

[1714165266]

1714165266

[achow101]

To convert the last string (step 5) to base58, you just do a normal base conversion. Take the string and think of it as a massive number in base 16. Then you just need to do the standard procedure for converting between bases and convert it to base58. There is a variety of software that will do this because doing it by hand is a major pain. This should help you: https://en.bitcoin.it/wiki/Base58Check_encoding#Base58_symbol_chart

[ExabyteMiner]

To convert the last string (step 5) to base58, you just do a normal base conversion. Take the string and think of it as a massive number in base 16. Then you just need to do the standard procedure for converting between bases and convert it to base58. There is a variety of software that will do this because doing it by hand is a major pain. This should help you: https://en.bitcoin.it/wiki/Base58Check_encoding#Base58_symbol_chart

That doesn't seem right... If I take the last string from step 5 and do a normal base conversion from base16 to base58 I get:

Code:

4HJYHtf8hkGGruED9DOGJVsF6StV9Qjc7Va98cfB5hUKdgsIij8

I'm supposed to be getting

Code:

5JLbJxi9koHHvyFEAERHLYwG7VxYATnf8YdA9fiC6kXMghkYXpk

[achow101]

I found your problem. You are hashing as a string, not as bytes. You need to always hash the hex as hex bytes.

[ExabyteMiner]

Sorry, I don't understand what you mean by hex bytes...

I have had no formal coding instruction, I am entirely self-taught so I apologize if I am misunderstanding something very basic.

I tried to look up what you mean using Google but could not seem to find anything relevant.

How would I go about convert my hex string to hex bytes...?

[achow101]

Sorry, I don't understand what you mean by hex bytes...

I have had no formal coding instruction, I am entirely self-taught so I apologize if I am misunderstanding something very basic.

I tried to look up what you mean using Google but could not seem to find anything relevant.

How would I go about convert my hex string to hex bytes...?

So what you have is a hex string which is different from hex bytes. Hex bytes are the human readable form of binary. If you were to convert a set of hex bytes to binary, you would get the 1's and 0's that a computer reads. On the other hand, if you have a string, you have something completely different. At the binary level, each character in the string is a set of bytes, and when you hash the string, you are hashing those bytes, and not the bytes that you actually want which you read in the string.

To hash the bytes, you can use this site: http://www.fileformat.info/tool/hash.htm. Enter the hex string into "Binary Hash" field and it will give you the hashes down below. When you hash again, make sure you do the same thing and hash the bytes.

[ExabyteMiner]

So what you have is a hex string which is different from hex bytes. Hex bytes are the human readable form of binary. If you were to convert a set of hex bytes to binary, you would get the 1's and 0's that a computer reads. On the other hand, if you have a string, you have something completely different. At the binary level, each character in the string is a set of bytes, and when you hash the string, you are hashing those bytes, and not the bytes that you actually want which you read in the string.

To hash the bytes, you can use this site: http://www.fileformat.info/tool/hash.htm. Enter the hex string into "Binary Hash" field and it will give you the hashes down below. When you hash again, make sure you do the same thing and hash the bytes.

Thank you so much for your help, I'm getting closer and closer...

OK, so i THINK I have that step figured out...

If I'm not mistaken, when I convert the hex string from step 5 back to hex bytes it becomes:

Code:

1000 0000 0100 0100 1101 0000 0000 1111 0110 1110 1011 0010 1110 0101 0100 1001
0001 1100 1101 0111 1010 1011 0111 1110 0111 0001 1000 0101 1101 1000 0001 1011
0110 0111 1010 0010 0011 1100 0100 1001 1000 0000 1111 0110 0010 1011 0010 1110
1101 0000 1001 0001 0100 1101 0011 0010 1011 0111 1110 1011 0001 1100 0101 0101
1000 0001 0000 1001 0010 0110 1001 1001 1001 1001

Not really sure what to do from here though...



EDIT:
I am definitely doing something wrong because when I put in the hex string from step 5 into the Hexadecimal field at http://convertxy.com/index.php/numberbases/ I get:

Code:

1000 0000 0100 0100 1101 0000 0000 1111 0110 1110 1011 0010 1110 0101 0100 1001 
0001 1100 1101 0111 1010 1011 0111 1110 0111 0001 1000 0101 1101 1000 0001 1011
0110 0111 1010 0010 0011 1100 0100 1001 1000 0000 1111 0110 0010 1011 0010 1110
1101 0000 1001 0001 0100 1101 0011 0010 1011 0111 1110 1011 0001 1100 0101 0101
1000 0001 1011 0000 1001 0010 0110 1110 1001 1010

Which is exactly the same all the way up until the last 8 sets of numbers...

[achow101]

Thank you so much for your help, I'm getting closer and closer...

OK, so i THINK I have that step figured out...

If I'm not mistaken, when I convert the hex string from step 5 back to hex bytes it becomes:

Code:

1000 0000 0100 0100 1101 0000 0000 1111 0110 1110 1011 0010 1110 0101 0100 1001
0001 1100 1101 0111 1010 1011 0111 1110 0111 0001 1000 0101 1101 1000 0001 1011
0110 0111 1010 0010 0011 1100 0100 1001 1000 0000 1111 0110 0010 1011 0010 1110
1101 0000 1001 0001 0100 1101 0011 0010 1011 0111 1110 1011 0001 1100 0101 0101

Not really sure what to do from here though...

No. You are converting it back to binary. What you just need to do is keep the hex string as it is but instead of hashing it as a string, hash it as bytes. Use the website I posted earlier.

[DannyHamilton]

Are you using a programming language for this process?

If so, which language, and which packages or libraries are you using?

I'm not sure if I should be presenting suggestions and examples in javascript, java, C, C++, perl, ruby, php, or something else entirely.

[ExabyteMiner]

No. You are converting it back to binary. What you just need to do is keep the hex string as it is but instead of hashing it as a string, hash it as bytes. Use the website I posted earlier.

I tried using that website, but it outputs 16 different fields only one of which contains the word bytes and that just shows the same information from step 5 except uppercase letters have been replaced with lower case letters... Is that what I am looking for...?

[ExabyteMiner]

Are you using a programming language for this process?

If so, which language, and which packages or libraries are you using?

I'm not sure if I should be presenting suggestions and examples in javascript, java, C, C++, perl, ruby, php, or something else entirely.

So far, I have been doing all of these steps using basic DOS/batch scripts that I am writing as I go aside from cmdhashgen which is being used for the SHA256 hashing.

I have some experience and understanding in VBS, C, C++, perl and php.

[ExabyteMiner]

No. You are converting it back to binary. What you just need to do is keep the hex string as it is but instead of hashing it as a string, hash it as bytes. Use the website I posted earlier.

I think I understand what you mean now, I want to convert it to the hex form that looks like:

Code:

0x80 0x44 0xD0 0x0F 0x6E 0xB2 0xE5 0x49 0x1C 0xD7 0xAB 0x7E 0x71 0x85 0xD8 0x1B 0x67 0xA2 0x3C 0x49 0x80 0xF6 0x2B 0x2E 0xD0 0x91 0x4D 0x32 0xB7 0xEB 0x1C 0x55 0x81 0xb0 0x92 0x6e 0x9a

I'm sure that's not the correct string, but that's what it should basically look like?

[achow101]

No. You are converting it back to binary. What you just need to do is keep the hex string as it is but instead of hashing it as a string, hash it as bytes. Use the website I posted earlier.

I tried using that website, but it outputs 16 different fields only one of which contains the word bytes and that just shows the same information from step 5 except uppercase letters have been replaced with lower case letters... Is that what I am looking for...?

You have to put the hex string in the text box on the field labeled "binary hash". Then scroll down and find the one labeled "sha256" and copy the hex string and put that in text box in "binary hash". Then scroll down again and the hash in" sha256" is the hash that you want for step 4

[ExabyteMiner]

You have to put the hex string in the text box on the field labeled "binary hash". Then scroll down and find the one labeled "sha256" and copy the hex string and put that in text box in "binary hash". Then scroll down again and the hash in" sha256" is the hash that you want for step 4

OK, I took the hex string from step 3 (8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581) and ran it through the site which gave me:

Code:

b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Then i took that result and ran it through the site again, I got:

Code:

5b0e90510294762f4553a6f717a13d5dce4c4aa6137311b950f2ca90ffc9a035

Why is that result different than what I got when I used cmdhashgen to hash the string in SHA256...? Because I took the same string, ran it through cmdhashgen's SHA256 encoder, took that result and repeated but I got:

Code:

2897b55427f86f73136d55c1729cb4ce56d72e40e33baf8ca10e9e02cb963434

[DannyHamilton]

Why is that result different than what I got when I used cmdhashgen to hash the string in SHA256...? Because I took the same string, ran it through cmdhashgen's SHA256 encoder, took that result and repeated but I got:

Code:

2897b55427f86f73136d55c1729cb4ce56d72e40e33baf8ca10e9e02cb963434

Because, when you used the site, you calculated the hash on a single 32 byte value represented in hexadecimal radix.

When you used cmdhashgen, you calculated the hash on a string of 64 alphanumeric characters represented in ASCII.

The site calculated the hash using a 32 byte input of:
0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

cmdhashgen calculated the hash of a 64 byte input of:
0x62353439303965363033353265663365626464343438623635626564613933386531303336323 462323037633931323239626465366539626136303335316531

[ExabyteMiner]

Because, when you used the site, you calculated the hash on a single 32 byte value represented in hexadecimal radix.

When you used cmdhashgen, you calculated the hash on a string of 64 alphanumeric characters represented in ASCII.

The site calculated the hash using a 32 byte input of:
0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

cmdhashgen calculated the hash of a 64 byte input of:
0x62353439303965363033353265663365626464343438623635626564613933386531303336323 462323037633931323239626465366539626136303335316531

I see... Very interesting, thank you for the clarification.

For future reference how does one convert a 64 character alphanumeric hexadecimal representation to the proper 32 byte hexadecimal radix...?

In other words, how do I turn:

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

into the proper:

Code:

0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

so it can be passed to cmdhashgen for hashing...?

[ExabyteMiner]

Bumping in hopes of an answer...

I appreciate the help given so far, I just need to understand this one process.

I understand, and even if you don't feel like explaining it yourself, could you please point me in the right direction by linking me to a resource or giving me proper google search terms that will explain how:

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

becomes:

Code:

0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Thank you in advance for any help finding the answer here...

[achow101]

Code:

0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Is the hash of 

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

when that is hashed as binary data and not as a string.

[DannyHamilton]

Bumping in hopes of an answer...

I appreciate the help given so far, I just need to understand this one process.

I understand, and even if you don't feel like explaining it yourself, could you please point me in the right direction by linking me to a resource or giving me proper google search terms that will explain how:

Code:

8044D00F6EB2E5491CD7AB7E7185D81B67A23C4980F62B2ED0914D32B7EB1C5581

becomes:

Code:

0xb54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Thank you in advance for any help finding the answer here...

The problem isn't your understanding of the process (other than perhaps your understanding of the difference between a hex value and an ASCII string representation of that hex value).

The main problem is in your understanding of the tool you've chosen to use:

https://searchcode.com/codesearch/view/11088282/

cmdhashgen is a Command Line Utility that can be used to generate
various hashes for a given String or File

Since you don't want to hash the "string", that means that you'll need to create a file that contains ONLY the 32 byte value that you want to hash.

This can be done with xxd if you are using a system that has it...

First use a file editor to create a file that contains the representation that you would have gotten if you had used xxd to convert a file that originally had the value:

Code:

0000000: 8044 d00f 6eb2 e549 1cd7 ab7e 7185 d81b  ................
0000010: 67a2 3c49 80f6 2b2e d091 4d32 b7eb 1c55  ................
0000020: 81                                       .

You can see here that if you strip out the first column (which is the byte offset into the file where that line of data occurs) and the last column of dots (which is just a placeholder for the ASCII character that each byte represents) the remaining data is your 8044d00f6eb2e5491cd7ab7e7185d81b67a23c4980f62b2ed0914d32b7eb1c5581.

Next, use the "reverse" functionality of xxd to convert back to a binary representation:

Code:

xxd -r fileWeJustCreated > ourNewFileContainingTheDesiredValueToHash

We now have a file named "ourNewFileContainingTheDesiredValueToHash" that contains a 32 byte value.

Hash this file, and the result should be b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1.  I don't use cmdhashgen, but here's what I get when I use openssl:

Code:

openssl dgst -sha256 ourNewFileContainingTheDesiredValueToHash
SHA256(ourNewFileContainingTheDesiredValueToHash)= b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Now if I simply captured this output, I'd have an ASCII string representation of "b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1".  Since we need to hash this value, we want the binary output from openssl.  openssl has a "-binary" command line option that will output the actual 32 byte value instead of the 64 character ASCII string representation.

Code:

openssl dgst -sha256 -binary ourNewFileContainingTheDesiredValueToHash > newerFileContainingFinalValueToHash

We now have a file named "newerFileContainingFinalValueToHash" that contains a 32 byte value. If cmdhashgen does not have the ability to output the 32 byte value, you'll need to repeat the "reverse xxd" process that we did with the previous value to create your file.

I can now hash this file to get the final value that will be used for the checksum:

Code:

openssl dgst -sha256 newerFileContainingFinalValueToHash
SHA256(newerFileContainingFinalValueToHash)= 5b0e90510294762f4553a6f717a13d5dce4c4aa6137311b950f2ca90ffc9a035

[ExabyteMiner]

The problem isn't your understanding of the process (other than perhaps your understanding of the difference between a hex value and an ASCII string representation of that hex value).

The main problem is in your understanding of the tool you've chosen to use:

https://searchcode.com/codesearch/view/11088282/

cmdhashgen is a Command Line Utility that can be used to generate
various hashes for a given String or File

Since you don't want to hash the "string", that means that you'll need to create a file that contains ONLY the 32 byte value that you want to hash.

This can be done with xxd if you are using a system that has it...

First use a file editor to create a file that contains the representation that you would have gotten if you had used xxd to convert a file that originally had the value:

Code:

0000000: 8044 d00f 6eb2 e549 1cd7 ab7e 7185 d81b  ................
0000010: 67a2 3c49 80f6 2b2e d091 4d32 b7eb 1c55  ................
0000020: 81                                       .

You can see here that if you strip out the first column (which is the byte offset into the file where that line of data occurs) and the last column of dots (which is just a placeholder for the ASCII character that each byte represents) the remaining data is your 8044d00f6eb2e5491cd7ab7e7185d81b67a23c4980f62b2ed0914d32b7eb1c5581.

Next, use the "reverse" functionality of xxd to convert back to a binary representation:

Code:

xxd -r fileWeJustCreated > ourNewFileContainingTheDesiredValueToHash

We now have a file named "ourNewFileContainingTheDesiredValueToHash" that contains a 32 byte value.

Hash this file, and the result should be b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1.  I don't use cmdhashgen, but here's what I get when I use openssl:

Code:

openssl dgst -sha256 ourNewFileContainingTheDesiredValueToHash
SHA256(ourNewFileContainingTheDesiredValueToHash)= b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

Now if I simply captured this output, I'd have an ASCII string representation of "b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1".  Since we need to hash this value, we want the binary output from openssl.  openssl has a "-binary" command line option that will output the actual 32 byte value instead of the 64 character ASCII string representation.

Code:

openssl dgst -sha256 -binary ourNewFileContainingTheDesiredValueToHash > newerFileContainingFinalValueToHash

We now have a file named "newerFileContainingFinalValueToHash" that contains a 32 byte value. If cmdhashgen does not have the ability to output the 32 byte value, you'll need to repeat the "reverse xxd" process that we did with the previous value to create your file.

I can now hash this file to get the final value that will be used for the checksum:

Code:

openssl dgst -sha256 newerFileContainingFinalValueToHash
SHA256(newerFileContainingFinalValueToHash)= 5b0e90510294762f4553a6f717a13d5dce4c4aa6137311b950f2ca90ffc9a035

Thank you for your reply!

OK, I have xxd and openssl both and I seem to be getting some strange results.

When i create NewFile.txt with the data:

Code:

8044d00f6eb2e5491cd7ab7e7185d81b67a23c4980f62b2ed0914d32b7eb1c5581

and then run:

Code:

xxd -r NewFile.txt > NewFile2.txt

NewFile2.txt comes back empty...


Also, if I run:

Code:

xxd NewFile.txt > NewFile2.txt

I get:

Code:

0000000: 3830 3434 6430 3066 3665 6232 6535 3439  8044d00f6eb2e549
0000010: 3163 6437 6162 3765 3731 3835 6438 3162  1cd7ab7e7185d81b
0000020: 3637 6132 3363 3439 3830 6636 3262 3265  67a23c4980f62b2e
0000030: 6430 3931 3464 3332 6237 6562 3163 3535  d0914d32b7eb1c55
0000040: 3831                                     81

Which looks a bit different than the result you got...


EDIT:
Eureka! I needed to add the -p flag! So:

Code:

xxd -r -p NewFile.txt > NewFile2.txt

and then

Code:

openssl dgst -sha256 NewFile2.txt > hashed.txt

Outputs the correct:

Code:

b54909e60352ef3ebdd448b65beda938e103624b207c91229bde6e9ba60351e1

You have definitely placed me on the correct track, thank you so much! Going to see if I can finish the process from here.