Why you cannot enter an arbitrary seed in Electrum

[jonald_fyookball]

I don't think knowing what part of speech might come next is meaningful here.
Not only are there enough words, but there is also plenty of flexibility in
possible sentence structures. 

For example "I would like" could be followed by a noun ("I would like ice cream")
or something else

("I would like to talk to you")
("I would like two scoops of icecream")
("I would like that")
("I would like nothing more than to smash your face lol")

And the farther you go into the sentence, the less prediction you will have
from the beginning, and since you need the entire phrase, this kind of analysis
means nothing in this context.

[1714614904]

1714614904

[1714614904]

1714614904

[1714614904]

1714614904

[1714614904]

1714614904

[1714614904]

1714614904

[oda.krell]

It's neither "paranoid", nor is the exact POS example I gave, and whether there can be exceptions, important. I just picked it as an example to make the problem more relatable.

To be clear: if your sequence of (dictionary) words is the output of a grammar, it is more predictable than a random sequence of dictionary items. See for example Shannon's classic paper on the entropy of English.

Note that Shannon makes no theoretical assumptions about which grammar underlies English. I'm not an expert in password cracking methods, so I can't give you any hard numbers, but I imagine that an n-gram based method (i.e. the classical SML model) would provide a relevant speed up in the search (under the assumption that the sequence is a grammatically valid sentence of English).

[jonald_fyookball]

It's neither "paranoid", nor is the exact POS example I gave, and whether there can be exceptions, important. I just picked it as an example to make the problem more relatable.

To be clear: if your sequence of (dictionary) words is the output of a grammar, it is more predictable than a random sequence of dictionary items. See for example Shannon's classic paper on the entropy of English.

Note that Shannon makes no theoretical assumptions about which grammar underlies English. I'm not an expert in password cracking methods, so I can't give you any hard numbers, but I imagine that an n-gram based method (i.e. the classical SML model) would provide a relevant speed up in the search (under the assumption that the sequence is a grammatically valid sentence of English).

From Shannon's paper: "This method is based on a study of
the predictability of English; how well can the next letter of a text be predicted
when the preceding N letters are known"

We should note that when brute forcing seeds, none of the preceding letters are known.

Branching out in a tree-like fashion quickly yields too combinations to make use of the predictability
methods you speak of. It's like trying to brute force guess all the moves of billions of chess games
played between relatively strong engines.  Yes, only certain moves make sense,
but the combinations still branch out exponentially.

You might find marginally higher security in choosing a pure random sequence, but
I think the overwhelming factor in having a week seed isn't found in grammatical predictability,
but instead in the simple bad decision of using a previously known combination from
a book, movie, etc.

[GODLIKE]

It's neither "paranoid", nor is the exact POS example I gave, and whether there can be exceptions, important. I just picked it as an example to make the problem more relatable.

To be clear: if your sequence of (dictionary) words is the output of a grammar, it is more predictable than a random sequence of dictionary items. See for example Shannon's classic paper on the entropy of English.

Note that Shannon makes no theoretical assumptions about which grammar underlies English. I'm not an expert in password cracking methods, so I can't give you any hard numbers, but I imagine that an n-gram based method (i.e. the classical SML model) would provide a relevant speed up in the search (under the assumption that the sequence is a grammatically valid sentence of English).

From Shannon's paper: "This method is based on a study of
the predictability of English; how well can the next letter of a text be predicted
when the preceding N letters are known"

We should note that when brute forcing seeds, none of the preceding letters are known.

Branching out in a tree-like fashion quickly yields too combinations to make use of the predictability
methods you speak of. It's like trying to brute force guess all the moves of billions of chess games
played between relatively strong engines.  Yes, only certain moves make sense,
but the combinations still branch out exponentially.

You might find marginally higher security in choosing a pure random sequence, but
I think the overwhelming factor in having a week seed isn't found in grammatical predictability,
but instead in the simple bad decision of using a previously known combination from
a book, movie, etc.

Completely agree on this.
You can try guess my sentence, but you never know if the first 3 words are ok: you will have to go with all the sentences that can be realized anyway. Still huge work.
Also note that my sentence is 140 characters long and INCLUDES PUNCTUATION, thing that the random generated words do not.

But now I throw another dice on the table: HOW LARGE IS THE DICTIONARY USED TO GENERATE THE RANDOM SEED?
Have you thought about a hacker that knows there are maybe 10 or 12 available dictionaries online and uses them to generate his sequence?
Now how QUICK would that be? Or anyway, QUICKER than hacking my sentence, with EXTRA INVENTED WORDS that the commonly used dictionary hasn't, with PUNCTUATION, that the random generated seed doesn't put in for obvious reasons, with CAPS LETTERS, that the random generated seed doesn't put in.

So, is my request so bad after all? I really don't think so.
And as you force the user to learn 12 words, you can force the user to generate a sentence that is SO LONG, contains SUCH CHARACTERS like at least 2 or 3 cap letters, and at least 3 punctuation signs.

I'm still on my position with random seed:
1) very few people will learn the seed by memory.
2) I have very good memory (tested) and IQ over 140, and still I didn't want to waste energy learning that sequence of words.
3) limited dictionary probably easily exploitable by hackers
4) people will write those words down somewhere.
Result: the seed is out of the user's brain.

User defined sentence seed:
1) easier to remember because the user can generate a sentence of his/her will.
2) harder to break: no words from more or less know dictionaries, caps letters, punctuation, invented words.
3) less people will write down the sentence, and even then... it's just a sentence, maybe a sentence on a diary, nobody could spot it if it's well hidden in plain sight.

[oda.krell]

@jonald_fyookball

> We should note that when brute forcing seeds, none of the preceding letters are known.

You don't need to /know/ any actual element of the password. The speed up comes from knowing - for English - the conditional probabilities of certain lexical items or characters, so you don't have to run an exhaustive search over all combinations over the dictionary, or rather: you can try the more likely combinations first, then the slightly less likely ones, etc.


> It's like trying to brute force guess all the moves of billions of chess games played between relatively strong engines.

Not really. It's closer to having a slightly better algorithm for finding the optimal move in a chess game than the next best alternative algorithm. Which sounds exactly like something you'd want to use if you plan to win at chess.


I said it before, I'm not an expert on password cracking algorithms, and I can't say what speedup to expect in exact numbers. But I know enough about (statistical) language models to say that I'm pretty sure it could make quite a difference if implemented right, and if the assumption is correct that you try to find not a random sequence, but something generated by "English", or "close to English".

Now then, let's see what a quick Google search comes up with...

(1)

The result of which is (usually) a more efficient way of cracking passwords. So instead of guessing every possible combination of characters incrementally, it uses a statistical model where the most common characters are used first. 'C' followed by 'a' or 'e' for example, or 'q' followed by 'u'.

from: https://www.trustwave.com/Resources/SpiderLabs-Blog/Hashcat-Per-Position-Markov-Chains/

Which describes (from what I can tell) an application to password cracking of Shannon's insight mentioned above.


(2)

The result is a series of statistically generated brute-force attacks based on a mathematical system known as Markov chains. Hashcat makes it simple to implement this method. By looking at the list of passwords that already have been cracked, it performs probabilistically ordered, per-position brute-force attacks.

from: http://www.wired.co.uk/news/archive/2013-05/28/password-cracking/page/2

That one is not even based on any underlying "English grammar", but it's the same principle: there's a set of conditional probabilities they can work with given that the sequence hasn't been chosen at random.

In a sense, the "grammar" here is the "grammar of previously discovered passwords".

Super slick, by the way, must admit that.


(3)

Both Figure 4-6 and Figure 4-7 indicate that  the  Markov  Chains  method  recovers  passwords  faster than  Brute-force.  

from: https://www.ma.rhul.ac.uk/static/techrep/2013/MA-2013-07.pdf on page 38

This one's probably the closest to what I had in mind. Password cracking based on Markov Chains that encode some form of "English knowledge" to guide the search. And, who would have thought, it's faster than brute forcing.


Sorry if this comes across as rude, but that was the last message on this topic from me.

I've made the point I believe is the one that needs to be mentioned in the context of this discussion, and that this point itself is not matter of discussion, but a mathematical certainty:

Entropy of English or near English phrases is lower than that of randomly generated sequences.

Now, admittedly, whether you think the above is worth making it more difficult for users to remember their password is a different matter. But that's a 'weighing off' decisions then, between usability and (guaranteed) safety.

And all these things considered, I think that Thomas V found an excellent solution: by default, seed generation is random, because on average, humans suck at coming up with randomness.

If however you have shown that you have a modicum of technical knowledge, you can enter your own seed, and then it's your own responsibility to ensure it is good enough.

Think of it like a "You must be this tall to ride" sign at the entrance of a roller coaster, and even placing a pair of walking stilts next to it.

If you're sure you want to roll your own, you can already do so. I don't see any need to ask Thomas to invite everyone to come up with their own seed, because the likely result is that average seed quality would decrease.


(EDIT) One thing, to be clear: I agree with you guys that it's hard, if not impossible, for most people to memorize the random seed. That's why you should probably write it down or print it, and find a way to store it away.

Hell, if your funds warrant that level of security, put it into a sealed envelope and place that one into an insured bank vault. The same principles of storing anything physical of great value applies here, only that in our favor (a) you rarely if ever need to get the item (only to recover your keys), and (b) the item is small, so hiding it or renting some safety box is easier than having to do so for a larger object.

[GODLIKE]

Hell, if your funds warrant that level of security, put it into a sealed envelope and place that one into an insured bank vault. The same principles of storing anything physical of great value applies here, only that in our favor (a) you rarely if ever need to get the item (only to recover your keys), and (b) the item is small, so hiding it or renting some safety box is easier than having to do so for a larger object.

You can 3D print a ring and stamp the words inside it, just an idea

[oda.krell]

You can 3D print a ring and stamp the words inside it, just an idea

(screw my claim that I'm not writing again in here )

You're probably joking, but I'm not so sure that "on body safekeeping" is necessarily worse than traditional safekeeping by hiding, or placing things into vaults.

Now I wonder if there's any research into this, how "on body" compares to "hiding" compares to "vaulting"...

[RealBitcoin]

You can 3D print a ring and stamp the words inside it, just an idea

(screw my claim that I'm not writing again in here )

You're probably joking, but I'm not so sure that "on body safekeeping" is necessarily worse than traditional safekeeping by hiding, or placing things into vaults.

Now I wonder if there's any research into this, how "on body" compares to "hiding" compares to "vaulting"...

Hiding is better than vaulting. If a thief breaks into your house he knows exactly where to look for valuables if he finds a vault.

If you carve the private key in a tree branch in your nearest forest, or put a piece of paper in a box, and bury it in your nearby forest, is a better solution.

[jonald_fyookball]

oda krell, interesting article.

You may be able to argue that you lose a few bits of entropy with a grammatically correct
sentence.  This is why experts really don't recommend that you create your own phrase
to begin with:  Its difficult to accurately measure the entropy.

[GODLIKE]

What if my sentence was written in... italian? (it is, actually)
Should a hacker implement several languages grammar?
I still think we are going a bit too far in the paranoid field here...

What if my sentence is written in... a dialect, of any language?

Also keep in mind there's quite some people that CAN'T write correctly.
Just to stay on the english side: many mistake IT'S with ITS, HIS with HE'S, YOUR with YOU'RE, and much more...

[RealBitcoin]

What if my sentence was written in... italian? (it is, actually)
Should a hacker implement several languages grammar?
I still think we are going a bit too far in the paranoid field here...

What if my sentence is written in... a dialect, of any language?

Also keep in mind there's quite some people that CAN'T write correctly.
Just to stay on the english side: many mistake IT'S with ITS, HIS with HE'S, YOUR with YOU'RE, and much more...

Professional password breakers, use dictionary attacks on the passwords, and they got a dictionary of all words, dialects, new words, of all languages.

Probably they focus on major ones.

If your password is in like Navajo or some really obscure language then it might be harder to break, but even then dont put your address,name, or birthdate in it.

[GODLIKE]

What if my sentence was written in... italian? (it is, actually)
Should a hacker implement several languages grammar?
I still think we are going a bit too far in the paranoid field here...

What if my sentence is written in... a dialect, of any language?

Also keep in mind there's quite some people that CAN'T write correctly.
Just to stay on the english side: many mistake IT'S with ITS, HIS with HE'S, YOUR with YOU'RE, and much more...

Professional password breakers, use dictionary attacks on the passwords, and they got a dictionary of all words, dialects, new words, of all languages.

Probably they focus on major ones.

If your password is in like Navajo or some really obscure language then it might be harder to break, but even then dont put your address,name, or birthdate in it.

Seems to me like you people want to refuse the reality.

Anyway, after thinking well around all this, I'm definitely sure that a sentence:

- more than 100 characters
- unknown language
- possibly with invented words
- possibly with grammar errors
- with caps letters
- possibly with numbers
- possibly with punctuation.

is more secure than:

- always-english
- common dictionary
- no caps
- no numbers
- no punctuation

randomly generated seed of 12 words.

AND it's much easier to remember, thus giving an actual chance to many users to just hold the sentence in their brain and nowhere else.

[RealBitcoin]

What if my sentence was written in... italian? (it is, actually)
Should a hacker implement several languages grammar?
I still think we are going a bit too far in the paranoid field here...

What if my sentence is written in... a dialect, of any language?

Also keep in mind there's quite some people that CAN'T write correctly.
Just to stay on the english side: many mistake IT'S with ITS, HIS with HE'S, YOUR with YOU'RE, and much more...

Professional password breakers, use dictionary attacks on the passwords, and they got a dictionary of all words, dialects, new words, of all languages.

Probably they focus on major ones.

If your password is in like Navajo or some really obscure language then it might be harder to break, but even then dont put your address,name, or birthdate in it.

Seems to me like you people want to refuse the reality.

Anyway, after thinking well around all this, I'm definitely sure that a sentence:

- more than 100 characters
- unknown language
- possibly with invented words
- possibly with grammar errors
- with caps letters
- possibly with numbers
- possibly with punctuation.

is more secure than:

- always-english
- common dictionary
- no caps
- no numbers
- no punctuation

randomly generated seed of 12 words.

AND it's much easier to remember, thus giving an actual chance to many users to just hold the sentence in their brain and nowhere else.

Yes but i dont understand why we are forced to only 12 words.

The devs force use to have 12 words, what if i want 30 words?

I just dont understand why are they deciding it for us with the pretense "that they know better".

I want to decide my own password and length of it.

[GODLIKE]

Anyway, after thinking well around all this, I'm definitely sure that a sentence:

- more than 100 characters
- unknown language
- possibly with invented words
- possibly with grammar errors
- with caps letters
- possibly with numbers
- possibly with punctuation.

is more secure than:

- always-english
- common dictionary
- no caps
- no numbers
- no punctuation

randomly generated seed of 12 words.

AND it's much easier to remember, thus giving an actual chance to many users to just hold the sentence in their brain and nowhere else.

Yes but i dont understand why we are forced to only 12 words.

The devs force use to have 12 words, what if i want 30 words?

I just dont understand why are they deciding it for us with the pretense "that they know better".

I want to decide my own password and length of it.

They don't allow it because, hoping that Bitcoin will become common use money, there will be a huge effort by hackers to break into users accounts.
This situation will be associated with the fact that the common user usually sets too weak passwords. This would happen with the seed as well.
But then again: as it's commonly use to force users to put caps and numbers in passwords, other mandatory rules can be added when generating a seed.
And I'm pretty sure I demonstrated that a user generated sentence with the rules I set up is more difficult to break than a random generated seed of 12 words without caps, numbers, punctuation, generated from some much used dictionary library... and there's the additional feature that you can easily remember your sentence.

[jonald_fyookball]

Anyway, after thinking well around all this, I'm definitely sure that a sentence:

- more than 100 characters
- unknown language
- possibly with invented words
- possibly with grammar errors
- with caps letters
- possibly with numbers
- possibly with punctuation.

is more secure than:

- always-english
- common dictionary
- no caps
- no numbers
- no punctuation

randomly generated seed of 12 words.

AND it's much easier to remember, thus giving an actual chance to many users to just hold the sentence in their brain and nowhere else.

Yes but i dont understand why we are forced to only 12 words.

The devs force use to have 12 words, what if i want 30 words?

I just dont understand why are they deciding it for us with the pretense "that they know better".

I want to decide my own password and length of it.

They don't allow it because, hoping that Bitcoin will become common use money, there will be a huge effort by hackers to break into users accounts.
This situation will be associated with the fact that the common user usually sets too weak passwords. This would happen with the seed as well.
But then again: as it's commonly use to force users to put caps and numbers in passwords, other mandatory rules can be added when generating a seed.
And I'm pretty sure I demonstrated that a user generated sentence with the rules I set up is more difficult to break than a random generated seed of 12 words without caps, numbers, punctuation, generated from some much used dictionary library... and there's the additional feature that you can easily remember your sentence.

yep pretty much this.

Its designed to mathematically give you 128+ bits of entropy, and each word in the dictionary is mapped
to numbers, so its random.  There's only 128 bits anyway of security in a spent address.

If you want something different, use brain wallet.

[GODLIKE]

I've been experimenting with electrum restore seed function, and its either really buggy or something is wrong.

entered frequent into the seed box, and can continue to generate a wallet.

god god god god god god god god god god god god works and I can create a wallet, however

fun fun fun fun fun fun fun fun fun fun fun fun doesnt work

abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon abandon ab works

acid acid acid acid acid acid acid acid acid acid acid acid acid acid acid acid aci also works

Well, it seems Electrum doesn't like fun.

[jonald_fyookball]

maybe that word isnt in the electrum dictionary.

[torusJKL]

maybe that word isnt in the electrum dictionary.

Fun is part of the word list:
https://github.com/spesmilo/electrum/blob/master/lib/wordlist/english.txt

[GODLIKE]

maybe that word isnt in the electrum dictionary.

Fun is part of the word list:
https://github.com/spesmilo/electrum/blob/master/lib/wordlist/english.txt

Are you telling me that... the dictionary used in Electrum is well known and available?

So I'm sorry, but this fixed automatic method to generate the seed is TOTAL AND UTTER BULLSHIT and my proposed method is infinitely better.

Why?
Because if I'm an hacker I can just test all the WORDS, I DON'T NEED TO TEST CHARACTER AFTER CHARACTER!
I know the words! So I only need to swap ENTIRE WORDS instead than CHARACTERS, it's so fuckin stupid!
If the dictionary is 1000 words, then the number of possible combinations in the seed is 1000^12!
That's 1 followed by 36 0. I don't say it's little, but it's surely waaaaaay less than a sentence human generated with some punctuation, numbers and caps.

Basically, this automatic seed generation is 12 "bytes" long: the number of words used in the seed! Only a Byte is 256 combinations, while the dictionary has 1000 combinations (I don't know how many words are in the dictionary).
But SURELY this method has NOTHING with testing characters, any hacker could just test WORDS.

Really, I wonder how nobody can see this.
Shit, I thought the dictionary was internal and encripted in the software, this is incredible, an incredible fallacy.

[jonald_fyookball]

maybe that word isnt in the electrum dictionary.

Fun is part of the word list:
https://github.com/spesmilo/electrum/blob/master/lib/wordlist/english.txt

Are you telling me that... the dictionary used in Electrum is well known and available?

So I'm sorry, but this fixed automatic method to generate the seed is TOTAL AND UTTER BULLSHIT and my proposed method is infinitely better.

Why?
Because if I'm an hacker I can just test all the WORDS, I DON'T NEED TO TEST CHARACTER AFTER CHARACTER!
I know the words! So I only need to swap ENTIRE WORDS instead than CHARACTERS, it's so fuckin stupid!
If the dictionary is 1000 words, then the number of possible combinations in the seed is 1000^12!
That's 1 followed by 36 0. I don't say it's little, but it's surely waaaaaay less than a sentence human generated with some punctuation, numbers and caps.

Basically, this automatic seed generation is 12 "bytes" long: the number of words used in the seed! Only a Byte is 256 combinations, while the dictionary has 1000 combinations (I don't know how many words are in the dictionary).
But SURELY this method has NOTHING with testing characters, any hacker could just test WORDS.

Really, I wonder how nobody can see this.
Shit, I thought the dictionary was internal and encripted in the software, this is incredible, an incredible fallacy.

ehhh...how to say this politely...You're wrong.

First of all, its 1626 words.  1626^12 = 3.4 * 10^38.
It's the same as 2^128, or 128 bits of security, which is the same for any bitcoin address
that has already spent funds.  ECDSA for Bitcoin is 256 bits which provides 128 bits of
security.  Unspent addresses benefit from additional security because of RIPEMD-160 hash,
increasing it to 160 bits.  However, electrum uses key stretching of an 100,000 round hash
once you know the seed, increasing the security to 144 bits.

You ARE correct that human generated phrases POTENTIALLY could have much higher
levels of security, but it doesn't matter because A) 128 bits is beyond brute forcing
by any form of classical computing (do the math and see how many super computers
and millions of years you need) and B) Bitcoin is limited to 160 bits of security no
matter what.

However, its impossible to measure with certainty the entropy level of a human generated
phrase and potential for error exists that does not exist with computer generated pass phrases.