Protobuf wallets in C++, python, C - anyone fancy it ?

[jim618]

If you are a C++, python or C dev and fancy playing about with Protocol buffers here is a useful project for you . . .

Some time ago Andreas's Android bitcoin wallet moved their wallet format from a Java specific one to one based on protocol buffers (protobuf). I have just upgraded MultiBit to use the same format for new wallets.

It is the data format used for the majority of Google's machine-to-machine chatter and is compact and easy to extend.
You can auto-generate accessor code in a variety of programming languages using a message definition file, which for bitcoinj is here:
http://code.google.com/p/bitcoinj/source/browse/core/src/bitcoin.proto?name=release-0.5

There are protobuf generators for (at least) C++, python and C that I know of.

Having a bitcoin wallet format that is easy to read and write to in a variety of programming languages sounds like it is a Good Thing To Have so if you have the skills, the time and the inclination please, well, give it a go!

[etotheipi]

If you are a C++, python or C dev and fancy playing about with Protocol buffers here is a useful project for you . . .

Some time ago Andreas's Android bitcoin wallet moved their wallet format from a Java specific one to one based on protocol buffers (protobuf). I have just upgraded MultiBit to use the same format for new wallets.

It is the data format used for the majority of Google's machine-to-machine chatter and is compact and easy to extend.
You can auto-generate accessor code in a variety of programming languages using a message definition file, which for bitcoinj is here:
http://code.google.com/p/bitcoinj/source/browse/core/src/bitcoin.proto?name=release-0.5

There are protobuf generators for (at least) C++, python and C that I know of.

Having a bitcoin wallet format that is easy to read and write to in a variety of programming languages sounds like it is a Good Thing To Have so if you have the skills, the time and the inclination please, well, give it a go!

Jim, I really like google's protobuf library, and was looking forward to using them for something.  They seem to have great support across multiple languages.

However, for Armory wallets, I'm sticking to pure, manual, binary files.  I may just be stupid stubborn, but I like the 100.0% control over every bit in the file, and not letting any outside library touch it.  I recognize this is much smaller scope than the BSD-DB library that let to the wallet encryption bug, but I'm still paranoid about it.  The wallet is one the thing I won't outsource to any other code.

However, for just about anything else (or most other apps, where the developers aren't as paranoid as me), I think it's great

[MatthewLM]

etotheipi: You could write your own code to serialise/deserialise wallets in the Protobuf format. It seems pretty simple: https://developers.google.com/protocol-buffers/docs/encoding

[jim618]

Hi Alan,

Yes you are right anywhere where there is a translation layer there is the possibility of introduction of 'infidelity'. If it was a homebrew format I would not be comfortable but I think protobuf is established enough and our proto file is pretty vanilla. There are also a variety of roundtrip wallet tests in both bitcoinj and MultiBit (and I am sure in Andreas's codebase too).

It is orders of magnitude better than the Java serialisation we have been using previously which is so brittle it has been a real pain to work with.

One of the things I want to raise on the bitcoinj mailing list in the next couple of weeks is the best protobuf format to use for encrypted private key wallets so I would be interested in your input on that (even though I appreciate you probably would not want to use protobuf for wallets in Armory for the reasons you have stated).

I wonder how similar the message file structure is to what you use in Armory ? If they were similar you might be able to import/ export to and from protobuf whilst never using it internally.  Even as a thought exercise it might be useful as you are months ahead of what bitcoinj has in terms of encrypted wallets and deterministic wallets. I expect support for both of those to go into the protobuf wallet in time. At the moment the message definitions to support those wallet types are not done yet.

Cheers,

Jim

[apetersson]

i still remember the bitcoin-qt bug where it stored critical parts of the wallet unencrypted. this went unnoticed because hardly anyone could read the format.

so to solve this, a common, open public format that is easy to parse would really help here. maybe one day also with wallet interoperability.