New, standardized wallet protocol

[Luke-Jr]

I am proposing the bitcoin development community work together on creating a specification for a new wallet-access protocol. The present bitcoind JSON-RPC has numerous problems (primarily polling requirements and precision issues) which make it unsuitable for efficient usage in a client.

Please feel free to edit https://en.bitcoin.it/wiki/Wallet_protocol

The first task, IMO, should be to define requirements for the protocol. Once those are relatively stable, we can begin on actual specification of the protocol constructs.

[genjix]

So what's your proposal? I'm willing to get behind this. Show us the code.

[genjix]

Does DBUS work on Windows or Mac?

It's the default used in all Linux desktops. It's a mature, battle-hardened protocol.

http://www.freedesktop.org/wiki/Software/dbus

[BitterTea]

I'm confused. At first I thought this was a discussion for a new wallet. Do you want to replace JSON-RPC, or essentially create a new client, broken into pieces?

[Luke-Jr]

So what's your proposal? I'm willing to get behind this. Show us the code.

The goal is to create a proposal. What requirements does the development community have for such a protocol?

Does DBUS work on Windows or Mac?

It's the default used in all Linux desktops. It's a mature, battle-hardened protocol.

http://www.freedesktop.org/wiki/Software/dbus

AFAIK, DBUS only works on *nix, and is desktop-only and local-only, so would be a problem running Wallet on a server, and UI on a phone... correct me if I'm wrong.

[Luke-Jr]

I'm confused. At first I thought this was a discussion for a new wallet. Do you want to replace JSON-RPC, or essentially create a new client, broken into pieces?

The current JSON-RPC needs to be replaced (this thread), before splitting the client (or creating a split one) is even possible.

[genjix]

The current JSON-RPC needs to be replaced (this thread), before splitting the client (or creating a split one) is even possible.

Yeah well I haven't been too happy at the response. I was told: "let us know what needs to be done to make a client wrapper possible. everything should be possible through the RPC".

- Need initialisation flag. Submit patch. Patch ignored. Workaround hack (that doesn't even work for all cases) proposed instead.
- Not able to import bitcoin RPC values into Python. Propose I write my own JSON RPC lib because Python's implementation is wrong.
- Propose consistent naming scheme for JSON-RPC Api. WONTFIX. Better to leave the inconsistent usage of getblaa or doaction because it's not a problem now.

I mean wtf. Come on.

[ribuck]

- Propose consistent naming scheme for JSON-RPC Api. WONTFIX. Better to leave the inconsistent usage of getblaa or doaction because it's not a problem now.

I gotta agree with them on that one. Any time you change an API, you mess up people who are using the old API, so you end up having to support both APIs for a long time, which clutters up your software.

Of course we all like the APIs to be tidy, consistent, and easy-to-use. But the time to fix that is when a new API is being written anyway because of some other pressing need.

Refactoring an API for purely aesthetic reasons is rarely worth it.

[Luke-Jr]

But the time to fix that is when a new API is being written anyway because of some other pressing need.

Like right now, in this thread... :p

[Luke-Jr]

I would appreciate any comments/critiques on https://en.bitcoin.it/wiki/Wallet_protocol#DRAFT_0

[Jim Hyslop]

I am proposing the bitcoin development community work together on creating a specification for a new wallet-access protocol.

I'm not sure I agree. I think, at the moment at least, Bitcoin should focus on the creating and exchanging of bitcoins. The wallet is a client-specific item. How you manage your Bitcoins is very much an individual choice. How you pass the bitcoins from one person to another is (and must be) commonly agreed upon by the Bitcoin protocol.

In general, though, I've found it better to first define what you want to do, then define the protocol to accomplish what you want to do. So, what do you want to do with the wallet? Obviously, first and foremost is to accept and spend coins.

[sirius]

In general, though, I've found it better to first define what you want to do, then define the protocol to accomplish what you want to do. So, what do you want to do with the wallet? Obviously, first and foremost is to accept and spend coins.

The idea is that you can store the wallet anywhere and use it through a lightweight terminal, without connecting to the network. For example you could choose to store your coins at mybitcoin.com or your own server. Then you could use them with a Firefox extension or an Android UI.

I think it would be easiest to go with the current JSON-RPC stuff and automatically enable it, or at least add it to the options menu. Well planned is not done.

[Luke-Jr]

In general, though, I've found it better to first define what you want to do, then define the protocol to accomplish what you want to do. So, what do you want to do with the wallet? Obviously, first and foremost is to accept and spend coins.

The idea is that you can store the wallet anywhere and use it through a lightweight terminal, without connecting to the network. For example you could choose to store your coins at mybitcoin.com or your own server. Then you could use them with a Firefox extension or an Android UI.

I think it would be easiest to go with the current JSON-RPC stuff and automatically enable it, or at least add it to the options menu. Well planned is not done.

Except that JSON-RPC is utterly unusable to implement any kind of real client. Check out the ugly hacks in Spesmilo and js-remote that only workaround part of the problem.

[xf2_org]

Except that JSON-RPC is utterly unusable to implement any kind of real client.

Can you be more specific?  Sounds like vague handwaving.

[BitterTea]

Except that JSON-RPC is utterly unusable to implement any kind of real client.

Can you be more specific?  Sounds like vague handwaving.

Genjix mentioned some things, but they don't seem to be showstoppers...

Yeah well I haven't been too happy at the response. I was told: "let us know what needs to be done to make a client wrapper possible. everything should be possible through the RPC".

- Need initialisation flag. Submit patch. Patch ignored. Workaround hack (that doesn't even work for all cases) proposed instead. Bitcoin issue
- Not able to import bitcoin RPC values into Python. Propose I write my own JSON RPC lib because Python's implementation is wrong. Python issue
- Propose consistent naming scheme for JSON-RPC Api. WONTFIX. Better to leave the inconsistent usage of getblaa or doaction because it's not a problem now. not an issue

I mean wtf. Come on.

[Steve]

So what's your proposal? I'm willing to get behind this. Show us the code.

I think it's good to let actual needs drive evolution, but that doesn't mean it's not worthwhile in soliciting ideas/requirements if you're about to undertake work like this (but, I do think it would be better, for someone that is so inclined, to collect some input/feedback...and then go write some code and let the code be the specification...I wouldn't agitate too much about getting the specification agreed upon by a large number of people in advance).

So, here's my feedback...first, here's a "user story" I would like it to support:
"As a user, I would like to have very tight control over the handling of my private keys"
    - to support this story, I think you need to boil down the private key handling to the very basic methods that require the use of unencrypted private keys...I think this is basically the creation of new keys (for receiving payments) and the creation of spend transactions (is there anything else?) ...I envision a very simple module (with a very small amount of easily comprehensible code...perhaps in a very readable language other than C++) that supports an API to a) instantiate a set of key pairs from a serialized form (where the private keys are encrypted), b) request the creation of a new key pair, c) request the creation of a transaction, d) request to serialize the public portion of all key pairs e) request to serialize all key pairs (with private keys encrypted)  ...additionally, the key pairs might include a small bit of meta info (like a description of the key as the current client supports) ...an implementation of this very small API would allow for someone to run a very small and trusted bit of code that is used to maintain the privacy and integrity of their keys (about the only UI that module would require is a password prompt used for the encryption of the private keys), while at the same time allowing the rest of the system to manage physical storage and backup of the encrypted keys and create the full wallet GUI (showing a balance, a send bitcoin UI, a transaction listing, an address book, etc).  ... few additional APIs to let the components work with keys individually might also be useful

I would be very interested in a solution that utilized 0mq+protocolbuffers for this.

[Luke-Jr]

Except that JSON-RPC is utterly unusable to implement any kind of real client.

Can you be more specific?  Sounds like vague handwaving.

Read the wiki page that talks about requirements. The biggest problem is that JSON-RPC requires polling.