[Standards] Ever thought of universal wallet formats?

[keeshux]

As of 2014, I understand that wallet software interoperability has been a severely overlooked issue.

Today many end-users risk losing coins because they created a wallet with software A and weren't told that restoring it into software B is not a trivial task. Different encryption, derivation algorithms, file format/layout altogether make room for frustration and skepticism against the whole Bitcoin ecosystem. It's also worth noting that what makes the restore process difficult is geek stuff most people never heard of.

Look at this answer: http://bitcoin.stackexchange.com/questions/13687/which-bitcoin-wallets-let-you-export-to-a-universal-wallet-format

Pretty self-explanatory, all vendors/developers assume their software "is an island". I don't mean to disrespect, but I think that professional software must also support an universal (export) format other software will agree on. It's freaking absurd that B might be unable to read a wallet from A just because A saved the keypairs in a CSV fashion while B normally saves them one per line. Someone may be cra**ing his pants for this lack of consensus.

In the end it's quite clear that the most basic wallet (I'm not considering multisig in this example) only consists of:

• a number of keypairs
• a list of transactions

More metadata can also be included for convenience:

• creation time
• wallet name
• address labels
• transaction labels

The above information is enough to reconstruct the same wallet elsewhere. Of course the format would have a version header to ease migration, enable future improvements and detect unrecoverable compatibility issues. Encrypted wallets should also be described in an universal, transparent manner.

I'd honestly urge to set a starting point, not to mention deterministic wallets where obscure proprietary settings make things much, much worse.

[1714949041]

1714949041

[1714949041]

1714949041

[wumpus]

Note that the universal wallet format does not need to equal the internal wallet format used by the software. Wallets can store their internal database in any way that is efficient (and secure) for them as long as there is a way to import/export the universal format.

Bitcoin Core supports the `dumpwallet` and `importwallet` RPC commands for going from/to an interchange wallet format. This is basically just a text file with a list of private keys with metadata.

BTW: in general, unless you have a very good reason to do otherwise, I'd recommend to generate new keys and transfer your coins when you switch to a new wallet. This is safer and better supported.

[keeshux]

Note that the universal wallet format does not need to equal the internal wallet format used by the software. Wallets can store their internal database in any way that is efficient (and secure) for them as long as there is a way to import/export the universal format.

Sure, that's why I called it an "export" format in my post.

Bitcoin Core supports the `dumpwallet` and `importwallet` RPC commands for going from/to an interchange wallet format. This is basically just a text file with a list of private keys with metadata.

Not that useful until there's no common "Import Bitcoin Core wallet" feature in Electrum, Armory, Android Wallet etc. Blame me if the feature exists and I couldn't notice.

BTW: in general, unless you have a very good reason to do otherwise, I'd recommend to generate new keys and transfer your coins when you switch to a new wallet. This is safer and better supported.

I know what you mean and I agree on safety, but the need for universal formats certainly stands in backup scenarios where relying on an individual developer choice to understand my own data is highly undesirable. Many formats are badly or not documented and I'd rather refer to an accepted document to parse a wallet file than asking the software author or a forum.

Think of a "philosophical" problem: a wallet should mean something per se despite the existence of a software that uses it.

[wumpus]

Not that useful until there's no common "Import Bitcoin Core wallet" feature in Electrum, Armory, Android Wallet etc. Blame me if the feature exists and I couldn't notice.

Well, Bitcoin Core has a very simple format. You don't need any developer support to understand it:

Code:

<secretkey> <creation time> label=%s # addr=%s  (in-use keys)
<secretkey> <creation time> reserve=1 # addr=%s  (for reserve keys)
<secretkey> <creation time> change=1 # addr=%s  (for change addresses)

Time format is ISO8601 (%Y-%m-%dT%H:%M:%SZ), key format is WIF (importprivkey format).

At a certain point the Android Bitcoin Wallet export format was the same except that it was encrypted with AES.

I actually agree with you though, it would be great if different wallets had a standard export format. The first step would be writing a BIP. If the proposal is sensible I don't think much is needed to get wallet authors to implement it, as the need for this is kind of a no-brainer.

For the different deterministic wallets this is more tricky as they (currently) use different ways to generate the keys. If they all used BIP32 it'd be a lot easier.

[keeshux]

Well, Bitcoin Core has a very simple format. You don't need any developer support to understand it:

Code:

<secretkey> <creation time> label=%s # addr=%s  (in-use keys)
<secretkey> <creation time> reserve=1 # addr=%s  (for reserve keys)
<secretkey> <creation time> change=1 # addr=%s  (for change addresses)

Time format is ISO8601 (%Y-%m-%dT%H:%M:%SZ), key format is WIF (importprivkey format).

That's certainly an example of user-friendly format to start from. The point is: despite being that simple, most software seems reluctant to import it in an automated way.

I actually agree with you though, it would be great if different wallets had a standard export format. The first step would be writing a BIP. If the proposal is sensible I don't think much is needed to get wallet authors to implement it, as the need for this is kind of a no-brainer.

For the different deterministic wallets this is more tricky as they (currently) use different ways to generate the keys. If they all used BIP32 it'd be a lot easier.

Exactly. I had originally posted this question on StackExchange where I complained about HD wallets in particular. The only significant barrier is defining some default derivation behaviour (e.g. 100k rounds of SHA256) and gap limit to reduce paper backups to just mnemonics. Creation time is a mere optimization and can be optional, but any custom parameter must be exposed no matter what. As to the branching structure BIP44 is a good candidate too and it defines a default gap limit of 20 that sounds reasonable for most users.