Title: Need some feedback on a search space limiter option Post by: Coding Enthusiast on April 10, 2022, 10:11:46 AM I'm working on a new option for FinderOuter to limit the search space in each recovery option. I'm currently working on the concept and want to know what you think.
For example the mnemonic recovery looks like the following pictures. User clicks Start to perform some basic checks and find out how many words are missing then it creates a set of "steps" to set what words can be used in that missing place and limit the search space. For first missing word (grace) it adds words that start with "gr": https://i.imgur.com/TBxaICw.jpg User clicks the next button (>) to move on to the next missing word. For second missing word (cruise) it adds words that contain letters "is": https://i.imgur.com/wwgx8Q6.jpg User clicks Finish button to finish up and create a SearchSpace object to be sent to the brute force service. As a result the search space is now limited to 976 permutations instead of 4,194,304. User can also add custom words one by one, or add all possible words. Q 1. What do you think of how it looks and the process? Q 2. Is there any other option you think I should add? Q 3. I'm also not sure whether I should add this option as a new window like the picture below shows or just extend the main window's height and add the option right there under an advanced expander (I'm leaning toward the later myself although my implementation so far is using the former). https://i.imgur.com/U25jRio.jpg Title: Re: Need some feedback on a search space limiter option Post by: o_e_l_e_o on April 10, 2022, 10:49:24 AM Q 1. What do you think of how it looks and the process? Looks good and straightforward to use. Great addition to your software. When you hit "Add all" it might be better to simply have a single line which says "All 2048 BIP39 words" or something similar, rather than list all 2048 words, so the user can easily confirm it is indeed searching all possible words and they haven't accidentally deleted some? Q 2. Is there any other option you think I should add? Maybe an option to combine those two. For example, words which both start with "d" and end in "e", in case the middle of the word is illegible?A really great option would be a feature to add words "similar to" a given word. So if I entered the word "air", then it would add air, aim, arm, art, chair, hair, pair, liar, affair, repair, unfair, etc., but this would be a huge amount of work I guess to go through the word list 2048 times and pick out similar looking words. Title: Re: Need some feedback on a search space limiter option Post by: LoyceV on April 10, 2022, 11:11:47 AM A really great option would be a feature to add words "similar to" a given word. So if I entered the word "air", then it would add air, aim, arm, art, chair, hair, pair, liar, affair, repair, unfair, etc., but this would be a huge amount of work I guess to go through the word list 2048 times and pick out similar looking words. If you take all words that contain at least 2 out of 3 characters (from the word "air"), you get a list of 633 words:Code: ability Code: acid Title: Re: Need some feedback on a search space limiter option Post by: Coding Enthusiast on April 10, 2022, 12:04:42 PM this would be a huge amount of work I guess to go through the word list 2048 times and pick out similar looking words. I have to test it but I think it could be simplified by using the Levenshtein distance (https://en.wikipedia.org/wiki/Levenshtein_distance) to find words that are "similar". That is rather straight forward and fast.Title: Re: Need some feedback on a search space limiter option Post by: Coding Enthusiast on May 18, 2022, 03:20:40 PM I think that's enough implementing for now so I'll release the new version very soon. I just have to cross some tees and dot some eyes. I ended up embedding the search space inside each option's window instead of a separate window. It's just under an "advanced" expander.
I ended up adding similar letters for Base58 option too which is a hard-coded list of characters I think look similar. You can see the whole list here (https://github.com/Coding-Enthusiast/FinderOuter/blob/d2ed1aadaa32914974f937401395f3de60e0057e/Src/FinderOuter/Backend/ConstantsFO.cs#L57-L78) and feel free to add to them. Some may seem silly or impossible to look alike but people always have the option to remove them from the list. Here is a preview: https://i.imgur.com/j8g9LP0.jpg And of course it also exists for BIP39 words but it works best for English words due to the very low threshold I use which is hard-coded and I didn't want to complicated this feature so I didn't add any option to the UI for user to change it. However, if the threshold is increased by modifying the code, it could also work for other languages like French in the preview below. But as you can see the result starts being less accurate. https://i.imgur.com/Xh8B8im.jpg Title: Re: Need some feedback on a search space limiter option Post by: NotATether on May 19, 2022, 02:05:11 PM I have to test it but I think it could be simplified by using the Levenshtein distance (https://en.wikipedia.org/wiki/Levenshtein_distance) to find words that are "similar". That is rather straight forward and fast. My bip39validator project has a levenshtein distance checker if you are interested in copying the algo (it's MIT-licensed and on Github). But you should probably exclude words that are not similar in such a way that there is a *contiguous group of chars* (more than 1 char) that appear in both words. I'm not sure how you would adapt LD for that. |