Title: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 11, 2019, 04:16:52 PM Hello everyone.
I have a strange topic I want to ask about, I have been doing some testing with seed words and have found something interesting that I cannot explain and I am looking for some advice or possibly the answer to why this is happening. So I have a small python script that is used the old electrum code to generate the MPK seed and PK and PubK. After running it with a specific set of words I started to notice many similarity in some of the hashes being returned to me. far too many to be a coincidence. the word set being used is ['ensure', 'alter', 'segment', 'spawn', 'cupboard', 'idle', 'horn', 'enrich', 'vendor', 'smooth', 'fruit', 'stone'] Here are some of the results that came back. Partial Match 1 Code: ['ensure', 'alter', 'segment', 'spawn', 'cupboard', 'idle', 'horn', 'enrich', 'vendor', 'smooth', 'fruit', 'stone'] P2 Code: ['idle', 'smooth', 'enrich', 'segment', 'fruit', 'cupboard', 'horn', 'vendor', 'ensure', 'alter', 'stone', 'spawn'] P3 Code: ['segment', 'horn', 'enrich', 'smooth', 'cupboard', 'idle', 'ensure', 'alter', 'fruit', 'vendor', 'stone', 'spawn'] P4 Code: ['smooth', 'spawn', 'cupboard', 'idle', 'horn', 'ensure', 'alter', 'fruit', 'segment', 'enrich', 'vendor', 'stone'] P5 Code: ['segment', 'ensure', 'horn', 'fruit', 'enrich', 'alter', 'idle', 'vendor', 'spawn', 'smooth', 'stone', 'cupboard'] P6 Code: ['ensure', 'stone', 'fruit', 'horn', 'cupboard', 'spawn', 'idle', 'alter', 'segment', 'smooth', 'vendor', 'enrich'] P7 Code: ['fruit', 'stone', 'enrich', 'cupboard', 'vendor', 'idle', 'alter', 'ensure', 'horn', 'segment', 'spawn', 'smooth'] Further Breakdown Code: Words Used : For reference here is parts of the code used to generate them.. Code: def mpk_from_seed(seed): My thoughts were that the old electrum way of doing things each digit represented by a word is variable, it depends on the previous word. if this is the case the returned hashes make no logical explanation for there matching pairs for example if you look at the first two being Code: 0142aa16fea967b90142aa1700f20785 and Code: 0142aa17ff4a937e00c9d5f71003c8196 if the were to follow this rule then the hashes should be much much different as the words used are in a totally different order. I also noticed many HEX values showing up during this for color values. Code: 00509c40fed1994800a164ea0079009a = 14QqCQiTkJVyHc5KgsuJj3dvVQiZM6pWWT #00509c = Royal Blue | RGBA 90, 80, 156, 1) Dose the old electrum style wallet use color hex codes as part of the number process because from what I am seeing there seems to be some kind of detectable issue with the hashing of the same words in different orders ? I hope someone can shed some light on this matter more as I am now rather stumped on where to look into this further. Magic Title: Re: Something Odd? Thinking caps on guys. Post by: TalkStar on September 11, 2019, 05:46:07 PM You have got something interesting but in my opinion its not so easy to detect those combination by using single partial match only. If you look deeply on those set of words then you can see there that in every single combination multiple words organised in different position. Its not like that same words are using on similar place or in a common combination each time. Every single time those words are making their pairs by changing previous combination. For those pairs of words easily more thousands of combination could be generated which is kinda tough to detect AFAIK.
Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 11, 2019, 07:54:43 PM I can understand why you'd say it's strange, but normally hash would be totally different even if only 1 bit of the input is changed. I wonder if you could try it again, but using cryptographic random function instead. here is the entire script that produces them. Do you not find it strange that the words are all jumbled up but still you are getting hashes that show the same leading characters are matching. even though the words are not in the same place or even next to each other. as far as I am aware this is the old electrum code for "old style" wallets. Code: def mn_encode( message ): for example lets look at number 7 ['fruit', 'stone', 'enrich', 'cupboard', 'vendor', 'idle', 'alter', 'ensure', 'horn', 'segment', 'spawn', 'smooth'] 0: 1HX5wVovCUMmJCFuZ5PQ3JRFSvr1jeud6h: fed1b2acfef9e43800286ab300c9967a: ee4875dd10630520647378f69b0974be59b076d9443ff5b999645411bb156a6cc3934d67cfb9ab3 c7d97f5b933930d3f837b21000c91bf726fdc0d580ac6ef3d ['smooth', 'vendor', 'idle', 'enrich', 'segment', 'horn', 'alter', 'spawn', 'ensure', 'stone', 'fruit', 'cupboard'] 0: 13WJoAWkqftY9DPxBEDDkk3xCTdK6PM1Yg: fed19f9fffc3c6da0028645900c9967d: d2e3b54a56868fa4d201c72aedb6e9d99a55cad5a1215d0d87e0399cc70663e8194cdba5dbedd99 250b5110941d5e326ae32bb0a14c84a1847fc5a51cc8d174f why would it return the 2 values that are so close at the start this is not a one off by the way. if i let this run for hours 95% of them have a relation like this even with the words being used at very different parts of the seed. I think there is something more here that I am missing but those hashes should not be so close considering the functions used to create them? Thanks guys. Magic Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 11, 2019, 07:58:46 PM You have got something interesting but in my opinion its not so easy to detect those combination by using single partial match only. If you look deeply on those set of words then you can see there that in every single combination multiple words organised in different position. Its not like that same words are using on similar place or in a common combination each time. Every single time those words are making their pairs by changing previous combination. For those pairs of words easily more thousands of combination could be generated which is kinda tough to detect AFAIK. Yes if you look at the words they are not in the same positions so why or how could the produce a hash with the same leading start as the previous one? thats my point. unless there something in the code I am not seeing that's causing it but from what I can see its all electrum code from a old wallet. I am wondering if there may be some kind of flaw with the old electrum wallets that people missed I cannot understand how 2 words sets with the same words but in very different positions can produce a hash with the same leading characters unless there is a problem along the road somewhere. I may be wrong but when you have 20k of these all very very very close it tells you something is wrong very wrong. Title: Re: Something Odd? Thinking caps on guys. Post by: TalkStar on September 12, 2019, 09:37:48 AM I am wondering if there may be some kind of flaw with the old electrum wallets that people missed I cannot understand how 2 words sets with the same words but in very different positions can produce a hash with the same leading characters unless there is a problem along the road somewhere. Basically we haven't observe much incidents which actually relates with this but i think that those seed would not be detected with so ease but seed generating process is obviously backdated.I may be wrong but when you have 20k of these all very very very close it tells you something is wrong very wrong. Yeah its kinda unexpected to see common words on different combination which actually increase the chances of being too much similar on some pairs.Title: Re: Something Odd? Thinking caps on guys. Post by: buwaytress on September 12, 2019, 10:47:56 AM Interesting but I can't tell myself if there's a logical explanation other than the (Wrong) logic that those set of words for sure spit out those parts of the hash.
It's not a one off with the same set of words in different positions, but do you also get the same findings with other sets of words? Or if you progressively replace 1 word in the same set with new words from dictionary? Which v electrum? Title: Re: Something Odd? Thinking caps on guys. Post by: j2002ba2 on September 12, 2019, 11:27:55 AM ['fruit', 'stone', 'enrich', 'cupboard', 'vendor', 'idle', 'alter', 'ensure', 'horn', 'segment', 'spawn', 'smooth'] 0: 1HX5wVovCUMmJCFuZ5PQ3JRFSvr1jeud6h: fed1b2acfef9e43800286ab300c9967a: ee4875dd10630520647378f69b0974be59b076d9443ff5b999645411bb156a6cc3934d67cfb9ab3 c7d97f5b933930d3f837b21000c91bf726fdc0d580ac6ef3d ['smooth', 'vendor', 'idle', 'enrich', 'segment', 'horn', 'alter', 'spawn', 'ensure', 'stone', 'fruit', 'cupboard'] 0: 13WJoAWkqftY9DPxBEDDkk3xCTdK6PM1Yg: fed19f9fffc3c6da0028645900c9967d: d2e3b54a56868fa4d201c72aedb6e9d99a55cad5a1215d0d87e0399cc70663e8194cdba5dbedd99 250b5110941d5e326ae32bb0a14c84a1847fc5a51cc8d174f Both mnemonics have invalid checksum. For 12 words the chance to get a valid checksum is 1/16. So if you don't check it, in 15 of 16 cases you'd get garbage. Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 13, 2019, 11:43:09 AM Interesting but I can't tell myself if there's a logical explanation other than the (Wrong) logic that those set of words for sure spit out those parts of the hash. It's not a one off with the same set of words in different positions, but do you also get the same findings with other sets of words? Or if you progressively replace 1 word in the same set with new words from dictionary? Which v electrum? I was running this specific set of words from the word list. I believe most of this code is from either v1 or 2 of electrum it's like creating the "old style" wallet format. From what I am seeing come back are far to close to say there is not something there. be it the way the words are being calculated into integer or otherwise. If I take the entire word list and use them the results do not come back like such only when exactly 12 words are selected you get this same start to them. It also works if you take any 12 words and do this function which to me seems very odd. Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 19, 2019, 10:59:59 AM After some more testing the results are the same 10000's of hashes all starting the same way which is strange as said above a single position change of just 1 word would alter the hash dramatically yet there are consistently coming out with the same leading result even with the words being changed to positions completely different from the last.
I think there may have been a weakness in the "old" electrum style wallets or the process used to create them has some issue with it. Can anyone think of a logical reason why hex combinations would seem to appear at the start of the hashes on more than one occasion, At first I thought it was a fluke hash but then more and more of them started to show up I ended up with a sequence of co lours that relate to colors used in the word list of electrum. Code: black Title: Re: Something Odd? Thinking caps on guys. Post by: NeuroticFish on September 19, 2019, 11:22:22 AM Imho there's nothing odd.
Keep in mind that those words are a seed. A seed like the Random needs. Then one or more operations are done. The results can be really in any range, which cannot (and imho should not!) be related to the words and their order. Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 19, 2019, 11:27:58 AM Imho there's nothing odd. Keep in mind that those words are a seed. A seed like the Random needs. Then one or more operations are done. The results can be really in any range, which cannot (and imho should not!) be related to the words and their order. So if you take a string of words then jumble them up but still are left with the same leading characters that's not odd? Words Used : ['ensure', 'alter', 'segment', 'spawn', 'cupboard', 'idle', 'horn', 'enrich', 'vendor', 'smooth', 'fruit', 'stone'] 0142aa16fea967b90142aa1700f20785 = 1BC5kyKk4Cp8BQQnCGUrGyALKVRAFv8ZbC Words Used: ['horn', 'enrich', 'vendor', 'segment', 'smooth', 'idle', 'alter', 'fruit', 'stone', 'cupboard', 'ensure', 'spawn'] 0142aa17ff4a937e00c9d5f71003c8196 = 13XYF15nGbyTZCSAF1dzaGhmES4kdcqE3U How can the 2 hashes at the start both have 0142aa16 then 0142aa17 That's not odd? the words are in completely separate order meaning there should be no relation at all to the last hash but there is.. Unless there is something in the way the words are being used to cause this effect. Title: Re: Something Odd? Thinking caps on guys. Post by: NeuroticFish on September 19, 2019, 01:27:04 PM Hmmm... I've been looking into the code a little and one thing that bothers me is that in mm_decode there are groups of 3 and in each group the distances between the indexes seem to matter more than the actual indexes, which could be a problem. But I don't know more, I understand python just as much as it resembles with C/C++ and I didn't look too deep.
Maybe you could ask directly them? https://github.com/spesmilo/electrum/issues I expect that they have more expertise in this and could answer. Title: Re: Something Odd? Thinking caps on guys. Post by: tromp on September 19, 2019, 05:55:24 PM How can the 2 hashes at the start both have 0142aa16 then 0142aa17 That's not odd? For any list of 12 words, there are 12! = 479001600 > 16^7 possible permutations, so you expect some pair to match in the first 7 hex digits. Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on September 21, 2019, 03:33:25 PM How can the 2 hashes at the start both have 0142aa16 then 0142aa17 That's not odd? For any list of 12 words, there are 12! = 479001600 > 16^7 possible permutations, so you expect some pair to match in the first 7 hex digits. They are coming out in sequential order. Its not like i have a list and they are being found by searching they are being spat out in this sequential ordering. If I sit and tail -f the output I can see some that have no relation then there is suddenly pairs that come out one after the other. I could understand if I had a list and I did every possible combination then yes I could search and probably find some matches but they seems to come out in this order which is strange considering the process being used should in theory not have all these results come out in a ordered way like this? Title: Re: Something Odd? Thinking caps on guys. Post by: tromp on September 21, 2019, 07:52:20 PM They are coming out in sequential order. Its not like i have a list and they are being found by searching they are being spat out in this sequential ordering. Then it's exceedingly odd. I would expect such close hashes to require some form of sorting or search. How are successive permutations generated? Title: Re: Something Odd? Thinking caps on guys. Post by: DaCryptoRaccoon on December 03, 2019, 02:13:26 PM Sorry for the late reply to this,
It seems around 1 in 3 that comes out has this strange matching pattern. I will post some more in the coming weeks about this. Title: Re: Something Odd? Thinking caps on guys. Post by: BrewMaster on December 03, 2019, 05:22:48 PM i honestly don't see anything strange going on here. you take an input, pass it though a pseudo random function and get a final result. then change that input, pass it through the same function and get a different result. then you repeat this final step a lot of times in a loop until you find a partial collision.
that is not strange, that is expected. if you do this with any cryptography function you will find the same results; here is an example with MD5 (since it is a fast hash function!): Code: 1st data: 9D27E7F051FE2A5054108C5277D33E95F93713FACE1287EF58C876FAC9F71DDC the result hashes are also the same as the "addresses" or entropies you find. you can see that the jumble of the "data" gives similar hashes. it took me a nanosecond to find this collision. here is a bigger one that tool half a second: Code: 1st data: 4143C76B1A8866283ED66BF5FFA8645B65732162E684E573363B2A15868E508B Title: Re: Something Odd? Thinking caps on guys. Post by: HCP on December 03, 2019, 11:27:05 PM The term "collision" is being used very loosely here... you don't have any collisions at all... you have what are effectively hashes of some very large numbers that have a few of the most significant bytes that are the same... surprising? Not really... assuming the hash function has results that are evenly spread amongst the total hash space, you'd expect some results to be "somewhat near each other".
Its like saying... I was generating hashes from random numbers and got this "collision": 12583497236598327780860956843764674908093582309 and 12583403334565765756676575672389472394242152634 They're obviously not a "collision"... ::) Title: Re: Something Odd? Thinking caps on guys. Post by: BrewMaster on December 04, 2019, 04:27:14 AM The term "collision" is being used very loosely here... as i said, it is a "partial collision" not a "collision". a partial collision is when only a smaller portion of the digests are equal. it is a topic that becomes important if you are using truncated hashes, like using SHA256 to hash but only using first 32 bits. and it seems to me that this is what OP is doing. |