"Differentiate Protocol version from client version" - denied?

[bluecmd]

Hello.

I couldn't find any other thread for this issue so here goes.

The following pull request was denied a while ago. https://github.com/bitcoin/bitcoin/pull/63
I would like to take it up for discussion since I know I am not alone to feel that it is a central thing to have the protocol version != client version.

Surely the other client makers out there can agree that this is a necessary thing?

[1713533482]

1713533482

[1713533482]

1713533482

[1713533482]

1713533482

[1713533482]

1713533482

[1713533482]

1713533482

[1713533482]

1713533482

[alkor]

The issue as I understand it is that Satoshi's implementation in C++ is considered the reference implementation, and the protocol is defined by it, hence the coupling between the protocol version and client version. Besides, despite bitcoin now being a multi-million economy, the protocol is still not fully documented, and no production ready alternative implementations exist for that reason. Bitcoin is still considered beta software, and the protocol is supposedly still evolving. Additionally, Satoshi's code is so complex that nobody really knows if the current protocol covers all the code paths in his implementation.

I also think that a lot of the core developers are opposed to alternative implementations as that will dilute their power over the bitcoin network.  

I personally believe that we should have a simple and clearly defined protocol as soon as possible, and encourage a proliferation of alternative clients.

[Gavin Andresen]

also think that a lot of the core developers are opposed to alternative implementations as that will dilute their power over the bitcoin network.  

Really?  I've been encouraging alternative implementations, who is the power-hungry core developer?

I personally believe that we should have a simple and clearly defined protocol as soon as possible, and encourage a proliferation of alternative clients.

Ok.  Start here:  https://github.com/bitcoin/netspec
Or here: https://en.bitcoin.it/wiki/Category:Technical

[TiagoTiago]

I guess this is kinda on-topic: http://dwellonit.taterunino.net/2011/05/19/reference-implementations/

[bluecmd]

Really?  I've been encouraging alternative implementations, who is the power-hungry core developer?

I personally believe that we should have a simple and clearly defined protocol as soon as possible, and encourage a proliferation of alternative clients.

Ok.  Start here:  https://github.com/bitcoin/netspec
Or here: https://en.bitcoin.it/wiki/Category:Technical

Well, that document and the Wiki differs somewhat. https://en.bitcoin.it/wiki/Protocol_specification.
Also, that text document says that the version "When a node receives an incoming connection, it will immediately advertise its version." which I think is no longer valid. What is the point of having the documentation if it is not up to date?

Also, the bootstrapping documentation is nowhere as far as I know. There are some pieces here and there but nothing that solid.

What I would like to see is something like a feed that describes all protocol changes and what motivated them. If I am to build an alternative client I want the core developers to know that it is a PITA to follow protocol changes and keep them to a minimum. Do not misunderstand me, I am all for evolvement and such, but I would like a real discussion (mailinglist anyone? bitcoin-protocol-dev?) before making anything permanent forever.

I think that using a separate protocol version will help facilitate the "keep the protocol stable" thought and also dot introduce protocol versions where the actual protocol are the same. For example, did you change anything between 32100 and 32200? As far as I know nothing P2P was changed (a bit bootstrapping though) - a protocol version increase here is accordingly to be totally uncalled for and only confuses developers.

 

[alkor]

Really?  I've been encouraging alternative implementations, who is the power-hungry core developer?

I am grateful for all the work that you are doing on bitcoin, and I didn't mean to say that you are power-hungry developer.

However, it is frustrating to see how the issue of separating the protocol version from the client version was rejected in this Github issue without any clear explanation.

[gene]

Absolutely correct that the protocol must be hammered out ASAP.

Satoshi did the crucial work of getting the reference implementation out. This fooling around with minor client changes is a waste of valuable time. More important are protocol adjustments which will strengthen the network and consistent documentation. Many different clients will follow once the documentation exists.

Obvious adjustments: perhaps consider SSL-encrypted connections (we had best anticipate the blocking of bitcoin connections by states/ISPs). IRC bootstrapping looks way too much like botnet activity and may easily be stopped by ISPs. Those kinds of steps would make things much easier and prevent the obvious "bumps in the road" that Gavin keeps mentioning.

Also, there is no manual for bitcoind. Documentation is an obvious requirement.

[davout]

Many different clients will follow once the documentation exists

https://en.bitcoin.it/wiki/Protocol_specification

Obvious adjustments: perhaps consider SSL-encrypted connections (we had best anticipate the blocking of bitcoin connections by states/ISPs).

Also, it's important that the client has no problem running on any arbitrary port. I tried to change the default port to have a mainnet and testnet client on the same box but the non-standard port client didn't get any connections. Maybe it's me that did something wrong, jsut saying.

IRC bootstrapping looks way too much like botnet activity and may easily be stopped by ISPs.

Aren't a bunch of node IPs already harcoded in the client ? (haven't got a clue about this one, but could be a good temporary fallback)

However, it is frustrating to see how the issue of separating the protocol version from the client version was rejected in this Github issue without any clear explanation.

IIRC jgarzik was arguing that the protocol version could be differentiated using a set of flags instead and that the protocol version field should be frozen, can't find that pull request anymore :/

EDIT : That was in OPs post... Protocol extension is different than protocol versioning, I think gavin should stand up

[Cdecker]

I opened (and closed) the pull request, even if I found it quite stupid to create a pull request for a semantic change...

Anyway I wanted to freeze the version number in the version message, and only change it when breaking changes are introduced. This way we could create clients that see a version number and say "ok, I speak that too", and then go ahead and communicate with each other. As it is now each new client version potentially brings breaking changes, and we have no way of being sure that the protocols are compatible.

I closed the pull request, because it was just plain stupid to have one for 2 added comments, and I thought it best to wait for the topic to come up again.

[bluecmd]

I would love to have more comments from the guys in charge - there seems to be a good number of persons who would love to see this happen and frankly I cannot see that this would be something troublesome on your part.

[Stevie1024]

also think that a lot of the core developers are opposed to alternative implementations as that will dilute their power over the bitcoin network.  

Really?  I've been encouraging alternative implementations, who is the power-hungry core developer?

I personally believe that we should have a simple and clearly defined protocol as soon as possible, and encourage a proliferation of alternative clients.

Ok.  Start here:  https://github.com/bitcoin/netspec
Or here: https://en.bitcoin.it/wiki/Category:Technical

Would be nice if there was one downloadable document containing the complete Bitcoin specification. Would be nice if the specification were set up in a way that it wouldn't have to change anymore. Would be nice that if it had to change, all the community would have a vote in it. Whole lotta woulds, I know...

[davout]

Would be nice if there was one downloadable document containing the complete Bitcoin specification. Would be nice if the specification were set up in a way that it wouldn't have to change anymore. Would be nice that if it had to change, all the community would have a vote in it. Whole lotta woulds, I know...

http://tinyurl.com/6xfhypu

[bluecmd]

Would be nice if there was one downloadable document containing the complete Bitcoin specification. Would be nice if the specification were set up in a way that it wouldn't have to change anymore. Would be nice that if it had to change, all the community would have a vote in it. Whole lotta woulds, I know...

http://tinyurl.com/6xfhypu

It has already been linked, that document is wrong in many areas and appears to be updated by people who detect errors instead of the developers when something is actually changed. There is also a git:ed document that differs from the wiki on other parts. Both are outdated.

[Cdecker]

Would you be interested in collaboratively creating a PDF in LaTex that would capture both the message format as well as the protocol behavior (something like the Pseudo-code presented in http://amzn.to/mgpDhH )? I had the idea a while back, but I'm under exams for another week.

[Stevie1024]

Would you be interested in collaboratively creating a PDF in LaTex that would capture both the message format as well as the protocol behavior (something like the Pseudo-code presented in http://amzn.to/mgpDhH )? I had the idea a while back, but I'm under exams for another week.

Actually I would. I promise you that either I will work on such a document of a competing currency or help you with one that nails down the current Bitcoin specs. However, either it's going to be a spec that is unlikely to ever change, or I'd want some guarantees that it couldn't be randomly adapted by a small group of people, other then including me |-).

[realnowhereman]

also think that a lot of the core developers are opposed to alternative implementations as that will dilute their power over the bitcoin network.  

Really?  I've been encouraging alternative implementations, who is the power-hungry core developer?

I wouldn't call this encouraging:

Good idea or not, SOMEBODY will try to mess up the network (or co-opt it for their own use) sooner or later.  They'll either hack the existing code or write their own version, and will be a menace to the
network.

Perhaps you've changed your mind in the mean time?

[davout]

It has already been linked, that document is wrong in many areas and appears to be updated by people who detect errors instead of the developers when something is actually changed. There is also a git:ed document that differs from the wiki on other parts. Both are outdated.

My bad

[Mike Hearn]

Reality check: Gavin wrote to me only days after the BitCoinJ release to tell me how happy he was to see an alternative implementation. Satoshi expressed very similar sentiments. Nobody is against alternative implementations.

What some people, especially Satoshi, have said is that there's an unusual amount of risk involved with reimplementing the full system and using that reimplementation to mine. Bitcoin is very complex and if you aren't skilled and very thorough you are likely to diverge from its behavior in small, hard to detect ways. This can fork the chain and split the economy. It's one of the few things that could instantly kill Bitcoin beyond legal harassment of its users.

Anyone who isn't completely fluent in C++ and the cryptography Satoshi used should think twice before deciding to write a new implementation. It's a huge amount of difficult work. BitCoinJ targets the simpler, less risky SPV/client-mode profile for this reason: bugs in it only will impact the user of that app and not the entire network because it can't mine. Even so, it's a large effort.

My general opinion is that if the thought of reimplementing Bitcoin from scratch doesn't make you take a deep breath, then you don't understand the system well enough to do it.

As to the version number, this is a pretty trivial thing. Does anyone serious about re-implementations actually care about it? The protocol is backwards compatible to the first version, so the number only identifies which set of features the other side supports. There are service flags for the case where you want to express optional features independent of protocol version.

Better protocol documentation would be great and I encourage people to write some (I've done my part with the articles on alt chains and contracts). All the core devs are busy right now with other things so, you'll have to dive in yourself.

[bluecmd]

Reality check: Gavin wrote to me only days after the BitCoinJ release to tell me how happy he was to see an alternative implementation. Satoshi expressed very similar sentiments. Nobody is against alternative implementations.

Good, it may just be that the core developers are busy (as you pointed out) now when Bitcoin has taken public interest.

What some people, especially Satoshi, have said is that there's an unusual amount of risk involved with reimplementing the full system and using that reimplementation to mine. Bitcoin is very complex and if you aren't skilled and very thorough you are likely to diverge from its behavior in small, hard to detect ways. This can fork the chain and split the economy. It's one of the few things that could instantly kill Bitcoin beyond legal harassment of its users.

Isn't this why we need client diversity? Bad blocks will not be accepted by other clients, having 1/3 of client A, 1/3 of client B and 1/3 of client C will guarantee that no one client will fork the chain. As it stands now the whole chain may have been hashed faulty.

Anyone who isn't completely fluent in C++ and the cryptography Satoshi used should think twice before deciding to write a new implementation. It's a huge amount of difficult work. BitCoinJ targets the simpler, less risky SPV/client-mode profile for this reason: bugs in it only will impact the user of that app and not the entire network because it can't mine. Even so, it's a large effort.

I assume you mean that the C++ knowledge is needed in order to understand the current client, right? If so, I hardly think that you need complete fluency in C++ as the client (the parts I've looked on) doesn't really use many of the advantages of C++. Sure, it has boost but that is about it. For the record, I have only reviewed the network and IRC code - might be some C++ voodoo elsewhere I assume. Anyhow, I think you are overestimating the difficulty of the task.

My general opinion is that if the thought of reimplementing Bitcoin from scratch doesn't make you take a deep breath, then you don't understand the system well enough to do it.

Of course it is a deep breath! It's huge - I think this can be concluded from all the stalled alternative clients (A bunch of libbitcoins, bitcoin-alt, pybitcoin). Implementing basic client support would be the first step - hashing and other stuff would be step two obviously.

As to the version number, this is a pretty trivial thing. Does anyone serious about re-implementations actually care about it? The protocol is backwards compatible to the first version, so the number only identifies which set of features the other side supports. There are service flags for the case where you want to express optional features independent of protocol version.

For me it may be a more ideological standpoint that if the developers do not want to merge such a trivial change that will greatly enhance the community by the looks of the discussions - then they do not appear to want 3:rd party developers. The protocol is the heart of a P2P - can you blame people for wanting it somewhat fixed?

If I were to create a client the last week, I would probably have announced it as "protocol version 32100". This week it is "32200" but as far as I can tell no changes. In my eyes, it just seems stupid.

Better protocol documentation would be great and I encourage people to write some (I've done my part with the articles on alt chains and contracts). All the core devs are busy right now with other things so, you'll have to dive in yourself.

The wiki has some nice discussions on the Talk page. There is a number of concerns how the fields are used and such, but it is hard to get a consensus on how it "is supposed to be". I would love to have a "bitcoin-protocol" mailing list that such issues could be discussed - but AFAIK the development team does not seem too excited about mailing lists.

[davout]

What some people, especially Satoshi, have said is that there's an unusual amount of risk involved with reimplementing the full system and using that reimplementation to mine. Bitcoin is very complex and if you aren't skilled and very thorough you are likely to diverge from its behavior in small, hard to detect ways. This can fork the chain and split the economy. It's one of the few things that could instantly kill Bitcoin beyond legal harassment of its users.

That's FUD and bullshit. Seriously, a wrongly implemented client? "fork the blockchain"? "split the economy"?

Breaking news! An intern deletes the blockchain due to a syntax error, hopefully he doesn't delete the whole internet with a buffer overflow.

Alternative clients are a good thing, so is protocol documentation

If the mainline client is really complex to understand and hard to read then it would be a very good thing to have a high-level language client which source is easy to read.

Mining is not really the point here, if you can check transactions and hash a block header then obviously you have everything you need to mine. So the distinction between clients than can mine and those who cannot is irrelevant.