Ode to the protocol ...

[Martin P. Hellwig]

While I am quietly in my limited spare time building a python implementation of the bitcoin protocol I am increasingly charmed about the ingenuity of the protocol itself.
Sure in the beginning I had of a lot of things* that I found strange but what is even stranger is that I am continuously discovering that the choices made might not be pretty, or particular easy but they share one common trait so far; the alternatives have far worse implications**.

So I would like to take my hat off and say three cheers to everybody that is working, designing and building the protocol, the software and community!

Thank you, it is truly appreciated!

<sub>* )
1 The constant byteswapping. Well once (:-)) you have it subclassed  you hardly notice it anymore.
2 Only ~21 million BTC available? Yeah but it is easy enough to increase the divisibility
3 What about lost coins, can we not have a system in place where forgotten coins are available for reminig? Neat, but this ultimately leads for regular users to a dependency on an external service provider, which opens a whole other can of worms.
4 The use of secp256k1, Well ones you look into it, it is kinda neat and there is not a clear and obvious advantage to other curves or public key algorithm, so this falls in the category why not.
5 The lack of a python implementation. Well, I am working on it, so that one is my fault for not moving my behind faster :-)

** )
My opinion, thus the validity of the made statements are not universal.</sub>

[1713579940]

1713579940

[davux]

While I am quietly in my limited spare time building a python implementation of the bitcoin protocol I am increasingly charmed about the ingenuity of the protocol itself.
Sure in the beginning I had of a lot of things* that I found strange but what is even stranger is that I am continuously discovering that the choices made might not be pretty, or particular easy but they share one common trait so far; the alternatives have far worse implications**.

Great!

It would be nice if you could write notes about what you discover. AFAIK, the protocol is only partially documented and mostly resides in "oral tradition" (if I may say so – it's obviously not oral). You could e.g. contribute to the Protocol specification page on the wiki, that still has strong tying between the network protocol and the internal structure of the original implementation (see "variable length strings", data types all over the document, etc.).

This is not a criticism. It's normal at this stage of history (the same happened with jabber/jabberd at the beginning), but an implementation-agnostic, RFC-like document is necessary if we want Bitcoin to succeed as an internet protocol, not just a program.

(Also, it's "once", not "ones".)

[ampkZjWDQcqT]

It would be nice if you could write notes about what you discover. AFAIK, the protocol is only partially documented and mostly resides in "oral tradition" (if I may say so – it's obviously not oral). You could e.g. contribute to the Protocol specification page on the wiki, that still has strong tying between the network protocol and the internal structure of the original implementation (see "variable length strings", data types all over the document, etc.).

Especially, I think the format of wallet.dat needs to be documented. A good practice it to ship the documentation of the internals together with the source code, something missing in the current reference implementation.

[davux]

Especially, I think the format of wallet.dat needs to be documented.

I don't know the subject a lot, but shouldn't the format of wallet.dat only be part of the original implementation internals?

I mean, what matters is interoperability between nodes of the network. How a given node stores its private keys (an sqlite file, MySQL database, an XML file, separate foo.key files...) is an implementation choice. If an alternative implementation wants to be able to read the original wallet.dat, very well, but this is not necessary at all.

But I do agree that a way to export/import the private keys of your published addresses would be useful though, in case they keep receiving coins.

[ampkZjWDQcqT]

Especially, I think the format of wallet.dat needs to be documented.

I don't know the subject a lot, but shouldn't the format of wallet.dat only be part of the original implementation internals?

Regardless, internals should be documented.

The reference implementation uses Berkeley DB by the way. (Paragraph added after original post)

[Gavin Andresen]

Especially, I think the format of wallet.dat needs to be documented. A good practice it to ship the documentation of the internals together with the source code, something missing in the current reference implementation.

So go ahead and document it, volunteer to keep it up to date (that's the hard part), and submit a patch. After the latest source-file re-org, there is a top-level doc/ directory where a "wallet_format.txt" file would go.

[Martin P. Hellwig]

It would be nice if you could write notes about what you discover. AFAIK, the protocol is only partially documented and mostly resides in "oral tradition" (if I may say so – it's obviously not oral). You could e.g. contribute to the Protocol specification page on the wiki, that still has strong tying between the network protocol and the internal structure of the original implementation (see "variable length strings", data types all over the document, etc.).

I have a couple of goals set:
- Split out the protocol in a pure daemon part (which does the communication and storing of the blocks/transactions) possibly with an optional SQL back-end.
- Have everything PEP-8 and Python 3 compliant.
- Have unit tests  with a 100% coverage.
- Be able to have a local "blockexplorer" that does not depend on the c++ client but instead uses the code of the above daemon.
- Create a 'thin' CLI client that interacts with the daemon (in essence this will just hold the private keys and have some UI code).
- The same as above but a GUI client.

For me when coding, the most important part is readability and maintainability, this is also the reason why I am at try number 3  because the other tries, although successfully parsing the tcpdump of an interaction, was not readable enough. So ultimately I want my implementation to be working 'pseudo code'. As I have particular strong feelings on how the Protocol Specification should be written, I think that I am not the appropriate person to contribute to that page.

As the approach I am taking is painstakingly slow, I wouldn't hold my breath, though I do intend to release the source under the new revised BSD license.
And I am sure that when I at least have reached my 4th goal, I can be persuaded to (help) create a RFC with the intention to submit an Internet Draft to the IETF.

[kjj]

1 The constant byteswapping. Well once (:-)) you have it subclassed  you hardly notice it anymore.

This is rampant in networking code.  At least in portable networking code.