HD seeds and breaking compatibility

[atweiden]

bip32 is a big upgrade for electrum. AFAIK electrum's much-anticipated multisig wallet mode is only compatible with bip32. Furthermore, in the Kivy Android app, we need bip32 to provide multi-account functionality for users who like to manage multiple wallets from the same screen.

The way Electrum is looking now, there will be two split versions of the seed serviceable from the same wallet installation. You can toggle bip32 off/on from the cmdline.

Under the hood, '0x01' is prepended to HD seeds in order to distinguish them from regular 12-word deterministic seeds. The 0x01 bits require adding one additional word to the seed to create adequate entropy. This brings Electrum HD seeds up from 12 words to 13 words.

By comparison, Multibit HD will be going with a 12 word seed: https://secure.flickr.com/photos/94861732@N04/14228288156/?rb=1

There are a few downsides to electrum's current planned split seed arrangement. One is that GUI users who wish to generate a new wallet with one type of seed need to specify that up front. For example, in Kivy on Android we're planning to default to bip32 for various reasons, but doing so means users can't enter original 12 word seeds, unless we add some type of option menu to the create account screens.

A less obvious downside is that 13 word HD seeds can only be generated by software. It's impossible to roll dice to come up with a HD seed completely offline due to the insertion of 0x01.

Again, with bip32 we can do multiple accounts and multisig wallets, which are hugely important features at least for Kivy. I see the move to bip32 as being a breaking upgrade in terms of UX for these reasons.

I can see the logic behind moving forward with support for both original deterministic seeds and HD seeds from the same codebase, but moving to only supporting bip32 from the 2.x codebase, and only supporting normal determinstic seeds from a 1.x codebase could have several advantages.

First, I assume if 2.x moved to only support HD seeds, the 0x01 bits wouldn't be needed. Which means we could default to 12 word HD seeds instead of 13. It also means the seeds could be generated completely offline.

On the plus side, there are numerous wallets supporting Electrum 12 word seeds now, and it would still be possible to host a 1.x release of Electrum, kind of like jQuery with their dual 1.x and 2.x releases.

To be clear, I see bip32 as a major upgrade, and I think new users will prefer using bip32 as the default mode. bip32 is also a bit of a breaking upgrade to Electrum. It's breaking not just in terms of seed compatibility, but breaking in the sense that bip32 enables multisig wallets, and multiple-accounts at least in our Kivy GUI. I view it as more of a foundational change, that is perhaps worthy of splitting off into a dedicated branch.

What say you to moving to a jQuery style split codebase (1.x and 2.x)?

[dabura667]

If you're technical enough to roll dice to create seeds, wouldn't you just do:

1. roll dice etc. to make 32 byte seed.
2. tack on an extra 0x01 to the beginning of the big endian string.
3. run that through the mnemonic encoder.



I would assume anyone who knows enough to roll dice and run the numbers through an offline saved version of bitaddress or brainwallet's converter functions and then convert into electrum mnemonic, would know enough to say, "add a 0x01 at the beginning of the seed before encoding." etc.

Interchangeability with other BIP32 wallets would be nice, but tbh, electrum kinda did its own thing for a while, so having electrum's BIP32 also be like "its own thing" wouldn't be too much of a problem. This way we can support all old electrum seeds AND new ones. The only drawback is 1. one extra word. 2. no compatibility with Electrum BIP32 seeds and other BIP32 services.

I don't view this as being too much of an issue... but if we don't support backwards compatibility and telling someone who just barely understood the concept of "seeds" to begin with. "oh yeah you need to download an older version from our old versions folder which is here. Most people will most likely 1. ignore such warning. 2. download latest version 3. get error on restore seed. 4. either complain in forums or just give up and throw seed away thinking "I was scammed by bitcoin"

I am all for backwards compatibility. Even if it means an extra word and not being able to use mnemonics with other programs.

[Abdussamad]

There is a third option. Kill 1.x branch. No more updates for that. Anyone who upgrades to 2.0 gets told that their coins are going to be migrated to a new wallet and asked whether they will agree to this. If they say no electrum closes. If they say yes a new wallet is generated and their coins get swept to a key in the new wallet. The old wallet file is backed up in case the user wants to refer to labels and stuff.

The above is a pretty hardcore stance but I just thought I'd mention that this is another way to do it. It would save the developers a lot of time because they could ditch the legacy code (after a few versions where you are allowed wallet upgrades).

[Abdussamad]

The only drawback is .. 2. no compatibility with Electrum BIP32 seeds and other BIP32 services.

This is a pretty big disadvantage IMO. The entire point of BIP32 was that you'd get standardized seeds across wallets.

Are you sure this is the case, though? I can understand that the mnemonic is not transferable but what about the output of mnemonic_to_seed() ?

https://github.com/spesmilo/electrum/blob/c76da17262be082828224e10399f258324886368/lib/bitcoin.py#L94

[atweiden]

After speaking with Thomas, it's possible to do a workaround for offline seed generation even assuming 0x01 bits.

I understand 0x01 is there for seed versioning, which is a development boon.

My concern with it is libraries like Bitcore and others may choose to do BIP32 without the 0x01, and *if* that is the only difference between electrum's bip32 implementation and, say, JS wallets based on Bitcore, electrum could potentially find itself incompatible with a growing number of wallets in the future for only that reason. However, Thomas informed me there would be more than 0x01 distinguishing electrum seeds from other bip32 implementations.

Interchangeability with other BIP32 wallets would be nice, but tbh, electrum kinda did its own thing for a while, so having electrum's BIP32 also be like "its own thing" wouldn't be too much of a problem

I agree with this. I'm also for backwards compatibility, my thoughts at the time were that if 0x01 was the only difference between electrum and say bitcore bip32, it may be worth looking at. Only way I could thnk of to do that is to do a 1.x / 2.x split.

I think jQuery makes this approach work for their userbase, but that's a very technical audience and even then I remember there was grinding of teeth when jQuery announced the 2.x / 1.x split. ChaiScript is the only other project I can think of right now that does this, and it has a highly technical audience as well.

FWIW I think people will want to upgrade to bip32, and they're going to need to generate new seeds to do that regardless.

The entire point of BIP32 was that you'd get standardized seeds across wallets.

I've heard standardized seeds across wallets isn't happening. Multibit is adding a timestamp to the 12 words, not sure what BitcoinJ and JS libs are doing, but I don't think there's concensus.

[fbueller]

My understanding is that people were looking at BIP0039 as a standard for seed generation. Has this been shelved? BIP39 uses a unique word list, so that in itself would distinguish the seed from the old standard?

BIP39 also uses checksums IIRC. So if a seed contains a word from the BIP39 list, and has a validating checksum, it's correct. If it only has a word from the BIP39 list, they typed it incorrectly. And if it doesn't contain a BIP39 word, then do the usual procedure for restoring older style electrum seeds.

I'm guessing just concatenation 0x01 | decoded_seed, and feeding that into the CKD function in BIP32 wouldn't be much stress, I just wonder what benefit it offers?

I'm aware of the discussions on bitcoin-development a while ago, where clients wanted to add things like timestamps to avoid reindexing the full chain.. Even going as far as to propose data about the cryptocoin network. I wonder if the point of BIP32 is diminished if everyone does what they want?

[Abdussamad]

BIP39 also uses checksums IIRC. So if a seed contains a word from the BIP39 list, and has a validating checksum, it's correct. If it only has a word from the BIP39 list, they typed it incorrectly. And if it doesn't contain a BIP39 word, then do the usual procedure for restoring older style electrum seeds.

Aren't there words that are present in both dictionaries?

Instead of trying to infer the seed version from the seed we should let the user select which version seed it is - pre 2.0 or 2.0+. Worst comes to worst he has to try both options. Not a big deal. For example Armory expects you to select the version when you restore from paper backups.

[jonald_fyookball]

I'm for backwards compatibility as well.

I don't have knowledge of a lot of things
being said here... but as a slightly technical
person, I compiled that python script cause
I wasn't sure what would happen to my coins.

I thought "JUST IN CASE" the electrum developers
won't support backwards compatibility, thinking
the chances of that were remote......so I'm
surprised that its not a foregone conclusion to do so.

 

[atweiden]

The addition of 0x01 bits to the binary seed representation is used to distinguish HD seeds from normal seeds in the current git master branch.