bdonlan


July 14, 2010, 10:37:56 PM 

Hi, The Hash() function in util.h forms the backbone of most of bitcoin's crypto: template<typename T1> inline uint256 Hash(const T1 pbegin, const T1 pend) { uint256 hash1; SHA256((unsigned char*)&pbegin[0], (pend  pbegin) * sizeof(pbegin[0]), (unsigned char*)&hash1); uint256 hash2; SHA256((unsigned char*)&hash1, sizeof(hash1), (unsigned char*)&hash2); return hash2; }
As you can see, this tries to be more secure by hashing twice. However, this actually reduces security. To break pure SHA256, an attacker needs to find a d' such that SHA256(d') == SHA256(d), for a known d. This is also sufficient to break Hash(). However the attacker can also attack the outer layer of the hash, finding a d' such that SHA256(SHA256(d')) == SHA256(SHA256(d)), even though SHA256(d') != SHA256(d). As you can see, the double hashing here makes it _easier_ to break the hash! A better solution would be something like: template<typename T1> inline vector<unsigned char> HashV(const T1 pbegin, const T1 pend) { uint256 sharesult; uint160 riperesult; SHA256((unsigned char*)&pbegin[0], (pend  pbegin) * sizeof(pbegin[0]), (unsigned char *)&sharesult); RIPEMD160((unsigned char*)&pbegin[0], (pend  pbegin) * sizeof(pbegin[0]), (unsigned char *)&riperesult);
vector<unsigned char> ret; ret.insert(ret.end(), (unsigned char *)(&sharesult), (unsigned char *)(&sharesult + 1)); ret.insert(ret.end(), (unsigned char *)(&riperesult), (unsigned char *)(&riperesult + 1)); return ret; }
The key is to concatenate the hashes, not merge them. This means the attacker has to break both hashes  even in the worst case, this cannot be less secure than a single hash. Unfortunately, changing hashes would break the chain...






Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.




bdonlan


July 14, 2010, 10:38:51 PM 

Incidentally, a similar problem exists with Hash160, only worse  an attacker can break _either_ RIPEMD160 or SHA256, and win. At least with Hash() the attacker _must_ break SHA256.




jib


July 15, 2010, 12:41:15 AM 

The original hash isn't what you're trying to break. You're trying to break the hash bitcoin uses, which is the double hash.




Gavin Andresen


July 15, 2010, 01:05:21 AM 

As you can see, this tries to be more secure by hashing twice. However, this actually reduces security. To break pure SHA256, an attacker needs to find a d' such that SHA256(d') == SHA256(d), for a known d. This is also sufficient to break Hash(). However the attacker can also attack the outer layer of the hash, finding a d' such that SHA256(SHA256(d')) == SHA256(SHA256(d)), even though SHA256(d') != SHA256(d). As you can see, the double hashing here makes it _easier_ to break the hash!
If I understand correctly, you've got two chances to find a collision instead of one. So this decreases the security of SHA256 by a factor of 2... which is just Not a Big Deal. Bitcoin is using, essentially SHA255 instead of SHA256. It'll still take longer than forever to find a collision...

How often do you get the chance to work on a potentially worldchanging project?



bdonlan


July 15, 2010, 01:22:27 AM 

As you can see, this tries to be more secure by hashing twice. However, this actually reduces security. To break pure SHA256, an attacker needs to find a d' such that SHA256(d') == SHA256(d), for a known d. This is also sufficient to break Hash(). However the attacker can also attack the outer layer of the hash, finding a d' such that SHA256(SHA256(d')) == SHA256(SHA256(d)), even though SHA256(d') != SHA256(d). As you can see, the double hashing here makes it _easier_ to break the hash!
If I understand correctly, you've got two chances to find a collision instead of one. So this decreases the security of SHA256 by a factor of 2... which is just Not a Big Deal. Bitcoin is using, essentially SHA255 instead of SHA256. It'll still take longer than forever to find a collision... It's true that it's not a big deal, but what concerns me is that someone went out of their way to implement an unproven hashing method, without thinking it through to realize that it actually decreases security. Things like this reduce one's confidence in the system as a whole  after all, if such a mistake exists, what other problems may be lurking? What we really need is a documented protocol, so it can be audited by third parties. I'm working on reverseengineering it from the source, but it's slow going  the intent of much of the scripting system is poorly documented.




theymos
Administrator
Legendary
Offline
Activity: 2912


July 15, 2010, 01:45:03 AM 

You're trying to solve: SHA256(SHA256(d'))==256bit number
This still has 256 bits of security, and you have to do two hashes per attempt to bruteforce it.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD



knightmb


July 15, 2010, 02:12:37 AM 

You're trying to solve: SHA256(SHA256(d'))==256bit number
This still has 256 bits of security, and you have to do two hashes per attempt to bruteforce it.
I agree, basically I get what he is saying at the top, but it isn't as bad as everyone thinks. To brute force your 256bit hash, you either guess the starting string or the second hash string. Which means you have to guess, convert to 256, convert to 256 again and see if that matches. If you attack the second hash, you can't start a guess at 0000000.....1 for example, you know it's already another 256bit number that have to start with. That's like trying to brute force a password and knowing ahead of time the guy's password was 100 digits long. Does it make it any easier? Well no because the first 99 digits could be 0 and the last 1 but either way you still have to try all in between. Mathematically, I see no advantage. Either you guess the first string to generate the hash or guess the first hash that generates the second hash. Either way you are going to have to brute force the entire keyspace to do this. The only advantage is that instead of having "one" correct answer, you have "two" correct answers out of the billion trillion million guesses possible.

▄▄ ▄▄ ▄███▄ ▄███▄ ███████▄ ▄██████▄ ██████████████████▄ ████████████████████ ▄█████████████████████ ▄███████████████████████ ▄█████████████████████████ ███████████████████████████ ▀▀███████████████████████▀▀ ▀▀███████████████▀▀ ▀▀███████▀▀ ▀▀▀
 Meridian  ║ █ ║   ║ █ ║  ◣ JOIN ICO ◥ 



Gavin Andresen


July 15, 2010, 02:28:36 AM 

There's a stackoverflow question about double hashing, by the way. Consensus is that it's not less secure. It's too late for my brain to process the nuances of hashing...

How often do you get the chance to work on a potentially worldchanging project?



Bitcoiner


July 15, 2010, 02:31:52 AM 

Could this be fixed in the next release? We were just discussing what would happen if a flaw were found: Here we go, our first real test of the situation.

Want to thank me for this post? Donate here! Flip your coins over to: 13Cq8AmdrqewatRxEyU2xNuMvegbaLCvEe



bdonlan


July 15, 2010, 02:41:56 AM 

Could this be fixed in the next release? We were just discussing what would happen if a flaw were found: Here we go, our first real test of the situation. Nope! At least, not easily  block and transaction identities are based on this hash, among other things, so this would break the chain. You'd need to do a phased rollout  first, add parsing support to all clients, then obsolete older clients, then roll out a new version which generates a new version of the block serialization format using the new hashing method.




theymos
Administrator
Legendary
Offline
Activity: 2912


July 15, 2010, 02:42:43 AM 

I have no doubt that hashing multiple times is secure, since it is specified in PKCS #5 and other standards: An iteration count has traditionally served the purpose of increasing the cost of producing keys from a password, thereby also increasing the difficulty of attack. For the methods in this document, a minimum of 1000 iterations is recommended. This will increase the cost of exhaustive search for passwords significantly, without a noticeable impact in the cost of deriving individual keys.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD



Bitcoiner


July 15, 2010, 03:23:53 AM 

Could this be fixed in the next release? We were just discussing what would happen if a flaw were found: Here we go, our first real test of the situation. Nope! At least, not easily  block and transaction identities are based on this hash, among other things, so this would break the chain. You'd need to do a phased rollout  first, add parsing support to all clients, then obsolete older clients, then roll out a new version which generates a new version of the block serialization format using the new hashing method. All the better to practice now, while the network is still young! Well, if it is not a real flaw, it's not necessary, but it would be good practice. This could always be done first in the test network.

Want to thank me for this post? Donate here! Flip your coins over to: 13Cq8AmdrqewatRxEyU2xNuMvegbaLCvEe



jib


July 15, 2010, 03:52:34 AM 

I have no doubt that hashing multiple times is secure, since it is specified in PKCS #5 and other standards: An iteration count has traditionally served the purpose of increasing the cost of producing keys from a password, thereby also increasing the difficulty of attack. For the methods in this document, a minimum of 1000 iterations is recommended. This will increase the cost of exhaustive search for passwords significantly, without a noticeable impact in the cost of deriving individual keys. That quote is about using hashes to transform passwords, which is a different application to what Bitcoin is using SHA256 for. I don't think the security issues in the two cases are analogous.




theymos
Administrator
Legendary
Offline
Activity: 2912


July 15, 2010, 07:37:54 AM 

PKCS #5 is used by TrueCrypt (and GPG, I think). If each iteration reduced the difficulty of finding collisions, as bdonlan suggests, then TrueCrypt would be easy to break. You'd just attack the hashing algorithm to find the master key.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD



Some Mouse
Jr. Member
Offline
Activity: 51


July 15, 2010, 12:44:59 PM 

There's a stackoverflow question about double hashing, by the way. Consensus is that it's not less secure. It's too late for my brain to process the nuances of hashing... Note that that discussion is referring to hashing passwords  and the most upvoted comment is explicitly discussing such with that in mind. Trying to do a dictionary attack has no coherent meaning, and someone directly attacking the algorithms behind bitcoin would obviously not start there. It will attack the algorithm.




mcdett


July 15, 2010, 03:56:42 PM 

What is being hashed in this function? < let's call this item x If the total number of possible x is smaller than 256bit space that sha256 provides than an attack would come in the form of a table of all sha256(x) values. If all of the possible combination's of x is greater than the 256bit space, then here is some math for you (the attack would be at the hash value itself rather than a table on known sha256(x)): What follows is from www.atmel.com/dyn/resources/prod_documents/doc8668.pdfIf there are 256 bits in the key, then after 2^255 attempts the attacker has a 50% chance of finding the right key and after 2^256 attempts he has tried all possible keys and is guaranteed to have found the key. Here are some estimates of big numbers: 2^66 Number of grains of sand on the earth 2^76 Number of stars in the universe 2^79 Avogadro's number. The number of carbon atoms in 12 grams of coal. 2^96 Number of atoms in a cubic meter of water 2^190 Number of atoms in the sun 2^255 Number of attempts to find the key value from above But what about very well funded entities such as the US National Security Agency (NSA)? Could they build a machine to crack a 256 bit key? Assume they could build a theoretical nanocomputer that executes 10^13 instructions per second (approximate rate of atomic vibrations) in a space of a cube with a side that is 5.43nm across (This is the approximate size of a silicon lattice10 atoms wide, or a crystal containing 1000 silicon atoms). Assume that it could calculate an attempt in 10 cycles. Such a computer the size of the earth would take more than 10^13 years (roughly 58 times the estimated age of the earth) to attack a 256 bit algorithm via brute force.




laszlo


July 15, 2010, 05:55:22 PM 

I want some nanocomputers!

BC: 157fRrqAKrDyGHr1Bx3yDxeMv8Rh45aUet



Bitcoiner


July 15, 2010, 06:05:23 PM 

What is being hashed in this function? < let's call this item x If the total number of possible x is smaller than 256bit space that sha256 provides than an attack would come in the form of a table of all sha256(x) values. If all of the possible combination's of x is greater than the 256bit space, then here is some math for you (the attack would be at the hash value itself rather than a table on known sha256(x)): What follows is from www.atmel.com/dyn/resources/prod_documents/doc8668.pdfIf there are 256 bits in the key, then after 2^255 attempts the attacker has a 50% chance of finding the right key and after 2^256 attempts he has tried all possible keys and is guaranteed to have found the key. Here are some estimates of big numbers: 2^66 Number of grains of sand on the earth 2^76 Number of stars in the universe 2^79 Avogadro's number. The number of carbon atoms in 12 grams of coal. 2^96 Number of atoms in a cubic meter of water 2^190 Number of atoms in the sun 2^255 Number of attempts to find the key value from above But what about very well funded entities such as the US National Security Agency (NSA)? Could they build a machine to crack a 256 bit key? Assume they could build a theoretical nanocomputer that executes 10^13 instructions per second (approximate rate of atomic vibrations) in a space of a cube with a side that is 5.43nm across (This is the approximate size of a silicon lattice10 atoms wide, or a crystal containing 1000 silicon atoms). Assume that it could calculate an attempt in 10 cycles. Such a computer the size of the earth would take more than 10^13 years (roughly 58 times the estimated age of the earth) to attack a 256 bit algorithm via brute force. Quantum computing and encryption: http://stackoverflow.com/questions/2768807/quantumcomputingandencryptionbreaking

Want to thank me for this post? Donate here! Flip your coins over to: 13Cq8AmdrqewatRxEyU2xNuMvegbaLCvEe



SmokeTooMuch
Legendary
Offline
Activity: 872


July 15, 2010, 06:20:38 PM 

If there are 256 bits in the key, then after 2^255 attempts the attacker has a 50% chance of finding the right key and after 2^256 attempts he has tried all possible keys and is guaranteed to have found the key. That's true, but only when you bruteforce every singe hash. There are other know attacking methods for SHA algorithms, thats why it is planned to replace the SHA2 family with SHA3 soon. German source: Als Reaktion auf die bekanntgewordenen Angriffe gegen SHA1 hielt das National Institute of Standards and Technology (NIST) im Oktober 2005 einen Workshop ab, in dem der aktuelle Stand kryptologischer Hashfunktionen diskutiert wurde. NIST empfiehlt den Übergang von SHA1 zu Hashfunktionen der SHA2Familie (SHA224, SHA256, SHA384, SHA512). Langfristig sollen diese durch einen neuen Standard SHA3 ersetzt werden. Dazu ruft das NIST nach Vorbild des Advanced Encryption Standard (AES) zu einem Ausschreiben auf. Die endgültige Wahl und Benennung des HashStandards ist für 2012 vorgesehen. http://de.wikipedia.org/wiki/Secure_Hash_Algorithm#Empfehlungen




Some Mouse
Jr. Member
Offline
Activity: 51


July 15, 2010, 11:06:29 PM 

...
2^255 Number of attempts to find the key value from above
1) Your equations don't take into account reversible computing advancements at all. These techniques could halve the search space by being able to take advantage of algorithms that traditional computers find impossible. 2) Your equations assume the algorithm is perfectly secure. Given that NIST is rather gung ho about developing SHA3, I'm not terribly confident in that analysis. 3) An attacker does not need to find a specific key to compromise the system, they just need to find the keys to someone's wallet. If you want this to be a viable currency for Earth's population, plus corporate accounts, you drop the search space by 10^10 to 10^12. That 2^256 search space could get whittled down to the 2^110 range by the end of the century. Assuming no major attacks on its integrity exist. I'm not particularly sold on the technical soundness of this program, honestly. Why use SHA256 rather than Whirlpool or SHA512?




