[NEW] Common Guidelines for Wallet Developers

[BitPay Business Solutions]

Guys,

I would like to see us collaborate to define some standard features in the alternative clients, particularly the mobile clients, that can ensure compatibility with the URI standard, as well as some best practices.

Here's a start:

http://lovebitcoins.org/developers.html

If your wallet meets these standards then we can recommend it on our site to new users, and it will also be compatible with Bit-Pay's payment system and the Bit-Pay Deposit Card (which all use the bitcoin: URI).  I would encourage any other merchant systems to also use the bitcoin: URI.

My first action item here is to see if we can agree on some type of standardization for Backup and Restore of wallets, with the idea that a backup from Wallet A on Device 1 can be restored to Wallet B on Device 2.  This would allow more portability of wallets, and ultimately let developers innovate with new features instead of reinventing the wheel.  It would also help present the bitcoin community as a more collaborative and compatible eco-system, when we need to compete against the big boys.

Let me know if you have any input on the guidelines so far, if anything should be added, and if you would like to be part of the process moving forward.

Thanks
Tony

[1713889783]

1713889783

[1713889783]

1713889783

[1713889783]

1713889783

[DeathAndTaxes]

subbbed.  should be interesting but I am afraid it will end up like this ...

[jim618]

Whilst I doubt we will ever get to a "One True Standard" there are a couple of moves for interoperability that I know of.

Bitcoinj protobuf wallet format
There is a move with the bitcoinj clients to move to wallets based on a format called protobuf.

This is open but is the data format used internally for a lot of things at Google and has the advantages of:

+ compact
+ you can generate code automatically to read and write it for most languages (including C++, Java, Python)
+ you can have required and optional fields. (Meaning, say, MultiBit can add fields into the wallets for its own use and other software can read it and just ignore the unwanted data).

It is a pretty good format and I think is the best overall chance for interoperability.

Private key export file common format
A simpler task is the standardisation of private keys files.
Both Andreas's Android Bitcoin Wallet and MultiBit use a common format described here:

https://github.com/jim618/multibit/wiki/Export%20and%20limited%20import%20of%20private%20keys

edit: There is an easier to understand description here:
https://bitcointalk.org/index.php?topic=43616.msg783013#msg783013

The link also describes how you can encrypt and decrypt these files in a standard manner using openssl (a standard crypto package available on pretty much everything).

Blockchain.info can also import these private key files (the unencrypted ones).

[Mike Hearn]

I believe most Android Bitcoin wallets already can read QRcodes and bitcoin: URIs. Anything based on bitcoinj at least can parse the URIs.

bitcoinj supports private key import. It is not necessarily fast, because you need to redownload the chain from the earliest key in the wallet, but it suffices for rare operations like restoring a wallet from a backup.

bitcoinj does not presently support deterministic wallets. That means clients based on it fail the "backup to paper" option. It would be nice to support this, but there are higher priority items. I think most clients can and do support non-paper (large) backups.

I am not sure that portability of wallet backups between clients is an especially important feature. If you want to send money from one client to another, just do it with a regular transaction. It's a rare event and there's no need to confuse matters by having multiple ways to move money between apps.

[davout]

If you can pay with it, and put money in it, then it's a wallet.

Whether you can pay by scanning a URI, spelling out the firstbits to Siri, or using an overlay protocol gives the same functionality.
Whether you fund it by importing keys in X, Y or Z format, or even by photographing dollar bills is too detailed too.
Whether the keys are hosted or local seems beyond scope to me as well.

[Steve]

I am not sure that portability of wallet backups between clients is an especially important feature. If you want to send money from one client to another, just do it with a regular transaction. It's a rare event and there's no need to confuse matters by having multiple ways to move money between apps.

I believe it's important for the sole reason that with many of these wallets being web based, users need a way to download and store an encrypted backup that could be later re-imported into other wallets should the service ever disappear.  I think a simple, yet extensible format would be desirable.  I would prefer JSON over protobuf…here's just about the simplest bit of JSON you could use.  Wallets could add other things to this JSON as desired, but the important stuff (the private keys) would be stored under a standard key in the JSON structure.  In this example, the standard key name would be "privateKeys", within that is an object whose keys are the cipher algorithms (using openssl names for the algorithms) and the values are arrays of the encrypted private keys.

Code:

{
  "privateKeys": {
    "aes-256-cbc" : [
      "……",
      "……",
      "……"
    ],
    "aes-192-cbc" : [
      "……",
      "……"
    ]
  }
}

Regarding deterministic wallets, I guess you'd need a standard way of expressing the deterministic algorithm and the master key data.

[Mike Hearn]

Sure, I'm all for some client somewhere have the ability to import backed up wallets from online services. I'd say it's a relatively niche feature. Should it work? Sure. Should it have to work really slickly? No, if it's hidden away in a menu and takes a while, that's probably OK. Is it necessary for every client do support that? I don't think so. You can always import into one client and then send the money on to the one you really want to use.

At any rate, it's moot. Most client developers seem to want this and implemented it already. In the case of Andreas' Bitcoin Wallet it's just a sensible precaution to guard against wallet corruption bugs.

[Steve]

If you can pay with it, and put money in it, then it's a wallet.

Whether you fund it by importing keys in X, Y or Z format, or even by photographing dollar bills is pretty useless in such a spec.
Whether the keys are hosted or local seems beyond scope to me as well.

This is about setting some standards that wallets should adhere to in order to be recommended by lovebitcoins.org.  Hosted or local may be beyond the scope, but any hosted wallet service shouldn't have unencrypted access to the private keys (ala mybitcoin.com).  Funding methods is relevant because it has an impact on the usability and interoperability of the wallet.

[Steve]

You can always import into one client and then send the money on to the one you really want to use.

Except when the wallet website returns a 404.  

I like blockchain.info's setting that automatically emails you a backup any time the wallet is updated.

[davout]

Hosted or local may be beyond the scope, but any hosted wallet service shouldn't have unencrypted access to the private keys (ala mybitcoin.com).

That's just marketing talk unless you also require it has to be fully open-sourced too, otherwise you'd have no way to tell whether keys are shared or not.

Funding methods is relevant because it has an impact on the usability and interoperability of the wallet.

The bitcoin network itself is the best interoperability guarantee. I don't really see an advantage in devising extra import/export schemes.

My point is that you're listing technical requirements, most of them, if not all having pretty much zero impact on the actual user friendlyness and usability. The xkcd strip sounds more and more accurate the more I read in this thread

[Mike Hearn]

What I meant is, as long as you have some program that can take a wallet exported by a website and send the free coins to some other wallet, it doesn't need to be supported in every client. You can just use that one program.

[Steve]

What I meant is, as long as you have some program that can take a wallet exported by a website and send the free coins to some other wallet, it doesn't need to be supported in every client. You can just use that one program.

Yes, that's probably sufficient.  So, the guidelines my specify something like: the wallet service must provide a means of downloading an encrypted backup of the private keys in a wallet and there must be a readily available means of spending the value in the downloaded backup without being dependent on the online wallet service.  It should be as simple and user friendly as possible.

Hosted or local may be beyond the scope, but any hosted wallet service shouldn't have unencrypted access to the private keys (ala mybitcoin.com).

That's just marketing talk unless you also require it has to be fully open-sourced too, otherwise you'd have no way to tell whether keys are shared or not.

Well, the key handling bit would have to be written in Javascript and delivered to the browser…it may or may not be open source, but it would be possible to inspect that source code and convince one's self that it's not sending unencrypted private keys to the server.  If someone devised a way of signing and verifying the signature of such javascript code, it might be a good idea to recommend web based wallets use that technique.

[Steve]

Here's another item to add to the list: spend transactions should always be delivered to the bitcoin network immediately (unless the wallet has a queuing feature and the user is clearly warned that the transaction cannot be delivered immediately).  The transactions should also be well formed (not considered spammy) such that they will be propagated by nodes in the network.

With Bit-Pay, we've seen a lot of cases where users create a transaction, but they either don't have a good network connection or their client isn't caught up and it doesn't immediately deliver the transaction (the client lets you create the transaction and sometime later when the client can obtain a good connection, it will deliver it).  It would be much better if the client would warn the user in that situation and let them choose whether they want to go ahead and create the transaction (and queue it for later delivery) or cancel.

[Mike Hearn]

On network robustness, we've done a lot of work on that lately with bitcoinj. The case where your network connection is flaky should no longer be an issue. Basically all pending transactions in the wallets are announced to peers as soon as a connection is set up, rather than Satoshis approach of re-announcing them at basically random times.

One of the improvements we'll make soon is to only send new transactions to some nodes, and then watch to see if they come back to us on others. In this way transactions that don't propagate for some unanticipated reason will be flagged correctly.

BitcoinJ will sometimes create non-well-formed transactions (insufficient fee). Most wallet devs "fix" this by always attaching the minimum fee, which isn't great, but means the transactions will always propagate. That will improve soon also.

[davout]

Yes, that's probably sufficient.  So, the guidelines my specify something like: the wallet service must provide a means of downloading an encrypted backup of the private keys in a wallet and there must be a readily available means of spending the value in the downloaded backup without being dependent on the online wallet service.  It should be as simple and user friendly as possible.

How is that different from requiring that a wallet should let the user transfer his balance to any arbitrary address, at any time ?

Well, the key handling bit would have to be written in Javascript and delivered to the browser…it may or may not be open source, but it would be possible to inspect that source code and convince one's self that it's not sending unencrypted private keys to the server.  If someone devised a way of signing and verifying the signature of such javascript code, it might be a good idea to recommend web based wallets use that technique.

So that leaves out all native code applications of your guideline-compliant list ?

You're not actually specifying what a good, usable and secure Bitcoin wallet is, but more what a wallet should be to appeal to a paranoid-techno-geek audience. And don't get me wrong, I'm not implying that it is a bad thing. I'm just saying your guidelines target a specific kind of wallets, that appeal to a smaller target than what you're probably intending.

Oh, and to comment on your last post, maybe you shouldn't assume that all transaction clearing happens through the Bitcoin network

[DeathAndTaxes]

How is that different from requiring that a wallet should let the user transfer his balance to any arbitrary address, at any time ?

What happens if the wallet is unavailable.

For example lets let imagine both strongcoin and instawallet go offline tomorrow.  Operators gone nobody knows why.  Strongcoin users could use their  backup of private keys to ensure they don't lose funds.  Instawallet users are just SOL.

Now strongcoin's key backup is propreitary but imagine if all/most/some wallets understood it.  You simply select "import" give it the encryption key and are up and running.

Now I used online wallet because the case is easier to understand but the same thing could apply to standalone wallet.  A bug could prevent creating new tx.  Users risk losing funds unless they have a way to export private keys.


Most of the OP is too complicated and likely won't accompish anything but having wallets import/export keys in a uniform manner is a good feature. 

[Luke-Jr]

While I think some objective requirements might be handy, I think it would be better to run with a simple "any wallet that's usable for the desired audience" and get more specific on an as-needed basis. Maybe apply the objectivity to certifications instead?