Is there any full node implementation 100% compatible with the core client?

[hhanh00]

100%... not 99.999%. I know it's very difficult.
Do the miners actually use bitcoin core too?

[PRab]

Compatible in what way? RPC, P2P, consensus code?

[hhanh00]

Compatible in what way? RPC, P2P, consensus code?

consensus code

[PRab]

I think the best answer is "we don't know". Even bitcoin core has been found to not be 100% compatible with itself (https://bitcoin.org/en/alert/2013-03-11-chain-fork).

I believe that btcd(https://github.com/conformal/btcd) passes all known consensus tests and I have been running a node for months without forking. I don't know exactly what you are trying to accomplish, but your best approach is likely to pick a client that meets your needs and automate what you want to happen if/when a fork occurs. Even if you are on the "correct" side of the fork, if there are 2 competing forks the most prudent behavior is likely to shutdown until one fork significantly outpaces the other.

[amaclin]

Compatible in what way? RPC, P2P, consensus code?

consensus code

OK. Lets try "exception method".
What are the implementations which are not 100% compatible?

[hhanh00]

I think the best answer is "we don't know". Even bitcoin core has been found to not be 100% compatible with itself (https://bitcoin.org/en/alert/2013-03-11-chain-fork).

I believe that btcd(https://github.com/conformal/btcd) passes all known consensus tests and I have been running a node for months without forking. I don't know exactly what you are trying to accomplish, but your best approach is likely to pick a client that meets your needs and automate what you want to happen if/when a fork occurs. Even if you are on the "correct" side of the fork, if there are 2 competing forks the most prudent behavior is likely to shutdown until one fork significantly outpaces the other.

Like several people I am writing a full node for educational purposes and I want to do a good job. Its structure is quite different than other implementations (I reviewed a few) because it's in a functional language and therefore I'm sure it's not the same as bitcoin core. Unfortunately, testing is difficult because verifying a block is biased towards accepting. If your implementation rejects a block that has a valid hash, you are wrong simply because no miner would produce a bad block today. So you might as well have 'return true'.
Bitcore has a series of tests but i'm not using c++ and I'd need to port them.
Your link refers to some trip test that I will try out. Any suggestions are welcome.

[gmaxwell]

I would bet any amount of money that none of the complete reimplementation that I am aware of are consensus consistent.

If you're doing something for educational purposes, then obviously the stakes are less severe. If the prospect of inventing novel ways to assure consistency excites you, then you should do so enlighten the world... but if it doesn't, why are you implementing a Bitcoin full node?  Effectively, achieving consistency _is_ the tasks, if you don't enjoy that tasks you'd probably be better of finding a project to work on that where you do enjoy the work of it.

[tl121]

Bitcoin 0.10.0rc1 is not 100% compatible with bitcoin 0.9.3 via the RPC interface.  As a result it does not work with the current version of Electrum Server github.

[tl121]

I would bet any amount of money that none of the complete reimplementation that I am aware of are consensus consistent.

If you're doing something for educational purposes, then obviously the stakes are less severe. If the prospect of inventing novel ways to assure consistency excites you, then you should do so enlighten the world... but if it doesn't, why are you implementing a Bitcoin full node?  Effectively, achieving consistency _is_ the tasks, if you don't enjoy that tasks you'd probably be better of finding a project to work on that where you do enjoy the work of it.

You could be right, but if that's the case then either these alternate implementations do not pass all of your consensus tests (in which case they are definitely at fault) or the bitcoin core product must be considered to have one or more serious bugs in either the code or the consensus test suite. 

[hhanh00]

You could be right, but if that's the case then either these alternate implementations do not pass all of your consensus tests (in which case they are definitely at fault) or the bitcoin core product must be considered to have one or more serious bugs in either the code or the consensus test suite.  

Bugs are acceptable in consensus code. English has plenty of exceptions in grammar and spelling but as long as we follow them we understand each other.

I would bet any amount of money that none of the complete reimplementation that I am aware of are consensus consistent.

If you're doing something for educational purposes, then obviously the stakes are less severe. If the prospect of inventing novel ways to assure consistency excites you, then you should do so enlighten the world... but if it doesn't, why are you implementing a Bitcoin full node?  Effectively, achieving consistency _is_ the tasks, if you don't enjoy that tasks you'd probably be better of finding a project to work on that where you do enjoy the work of it.

I'm implementing a full node for a couple of things. 1. to make sure i have a complete understanding of the implementation that goes beyond reviewing code. For me, it's like reading a book versus writing one. 2. all the implementations i have seen follow a similar logic specially wrt blockchain management and scripting. i wanted to do it my way to see if it's better or worse. 3. i'm aiming for short. no wallet, no mining, no dos protection.
These goals in mind, it is clearly not fit for wide distribution. It's working fine though - being able to sync up from the beginning while validating, holding a mempool content identical to the core client and etc. But it's reaching a point of diminishing returns. The consensus rules I know I'm not implementing add weight but aren't particular challenging. So I was thinking about looking at another guy who faced the same problem to see where the road ends and whether it's worth going the extra few miles. If no one is consensus consistent then I need to revisit my plan.

[justusranvier]

Compatible in what way? RPC, P2P, consensus code?

consensus code

OK. Lets try "exception method".
What are the implementations which are not 100% compatible?

All of them, including Bitcoin Core.

[gmaxwell]

including Bitcoin Core.

There has at least not been a new version so far which introduce an incompatibility, that we know of (frequently repeated _incorrect_ claims otherwise aside.). It wouldn't be the most shocking thing; but enormous effort goes into preventing it.

So I was thinking about looking at another guy who faced the same problem to see where the road ends and whether it's worth going the extra few miles.

BlueMatt wrote a pretty extensive agreement testing framework and discovered several corner cases that were not previously known. He's generally of the opinion that achieving a correct reimplementation is more or less intractable. His approach made progress, especially in unearthing weird behaviours, but doesn't result in great confidence of soundness; and doesn't show an implementation free of its own buts not related to non-obvious behavior in Bitcoin Core.  Our strategy in Bitcoin Core as of late has been toward compartmentalizing and simplifying in order to make code reuse more reasonable; and also get things structured to be more agreeable to approaches that would possibly make formal analysis more realistic, but thats a long term effort.

[justusranvier]

It wouldn't be the most shocking thing; but enormous effort goes into preventing it.

Which is equally true for alternate implementations.

Since it's not possible for any implementation of Bitcoin to prove perfect compatibility with own present and past versions, treating any single implementation as special is incorrect.

They're all alternate implementations.

[grau]

It wouldn't be the most shocking thing; but enormous effort goes into preventing it.

Which is equally true for alternate implementations.

Since it's not possible for any implementation of Bitcoin to prove perfect compatibility with own present and past versions, treating any single implementation as special is incorrect.

They're all alternate implementations.

To be fair, it is easier to keep an implementation consensus compatible with its prior versions, than to achive consensus compatibility with an unrelated code base.

I think that the best option for alternate implementations is to define a side chain that is attractive for reasons other than their token, so value migrates to them.

[justusranvier]

I think that the best option for alternate implementations is to define a side chain that is attractive for reasons other than their token, so value migrates to them.

Trying to make value flow out of the Bitcoin blockchain is the same as trying to kill Bitcoin.

I wouldn't call software whose purpose in life was to extract value from the blockchain an "alternate implementation" - I'd call it an attack.

[grau]

I think that the best option for alternate implementations is to define a side chain that is attractive for reasons other than their token, so value migrates to them.

Trying to make value flow out of the Bitcoin blockchain is the same as trying to kill Bitcoin.

I wouldn't call software whose purpose in life was to extract value from the blockchain an "alternate implementation" - I'd call it an attack.

It is called competition. It is what makes things strong.

[justusranvier]

I think that the best option for alternate implementations is to define a side chain that is attractive for reasons other than their token, so value migrates to them.

Trying to make value flow out of the Bitcoin blockchain is the same as trying to kill Bitcoin.

I wouldn't call software whose purpose in life was to extract value from the blockchain an "alternate implementation" - I'd call it an attack.

It is called competition. It is what makes things strong.

Of course.

You don't call your competitors "teammates", however.

Software that tries to compete with Bitcoin is not an alternate implementation of Bitcoin - it is a competitor to Bitcoin.

[grau]

I think that the best option for alternate implementations is to define a side chain that is attractive for reasons other than their token, so value migrates to them.

Trying to make value flow out of the Bitcoin blockchain is the same as trying to kill Bitcoin.

I wouldn't call software whose purpose in life was to extract value from the blockchain an "alternate implementation" - I'd call it an attack.

It is called competition. It is what makes things strong.

Of course.

You don't call your competitors "teammates", however.

Software that tries to compete with Bitcoin is not an alternate implementation of Bitcoin - it is a competitor to Bitcoin.

The consenus of Bitcoin user created a new type of asset that is not limited to the features supported by the original code.

If people decide to move their asset to an other chain, they do it because they think it offers them something higher value.

They might be right or wrong.
- Assuming they are right, then side chains adds value to the ecosystem.
- Assuming they are wrong, then supply of Bitcoin shrinks and those sticked with it enjoy appretiation.

[hhanh00]

I ran Matt's test harness on bitcoin core 0.9.3 and an assertion was raised. Then I tried on my code and realized that the test harness relies on a specific message order or it deadlocks. After that I started debugging the test harness and got a headache. Maybe it is time to surrender.

[hhanh00]

@grau & @justus, no offense but your debate is quite off-topic. If you must discuss the pros/cons of alternate implementations on altcoins, could you create your own thread?

[work2heat]

BlueMatt wrote a pretty extensive agreement testing framework and discovered several corner cases that were not previously known. He's generally of the opinion that achieving a correct reimplementation is more or less intractable. His approach made progress, especially in unearthing weird behaviours, but doesn't result in great confidence of soundness; and doesn't show an implementation free of its own buts not related to non-obvious behavior in Bitcoin Core.  Our strategy in Bitcoin Core as of late has been toward compartmentalizing and simplifying in order to make code reuse more reasonable; and also get things structured to be more agreeable to approaches that would possibly make formal analysis more realistic, but thats a long term effort.

Is there somewhere I can read about the kinds of corner cases that were discovered and how they were dealt with?

Also, what would it take to produce a formally verifiable implementation of bitcoin? Is such a thing even possible? Can we expect a PhD thesis to deliver the matter in the next decade?

[gmaxwell]

Is there somewhere I can read about the kinds of corner cases that were discovered and how they were dealt with?

They're sort of scattered in various places, for example: http://sourceforge.net/p/bitcoin/mailman/message/29699385/

Also, what would it take to produce a formally verifiable implementation of bitcoin? Is such a thing even possible? Can we expect a PhD thesis to deliver the matter in the next decade?

One question with formal verification is what are you formally verifying?  If the question is the most important question in consensus "is this function equivalent to that function for all inputs" there is one very easy to have a strong guarantee that the answer is true: make them be exactly the same function.  Formally proving two functions written in different languages compute the same thing is much harder, and likely not practically possible if the languages are very different or the algorithms used internally substantially different.

To formally verify other properties we first need to extract the part that needs to be verified into it's smallest self contained realization. You can look at whats been done with SEL4 to see whats currently possible (and how much work it takes for a rather small program) at the state of the art.

In Bitcoin core we're working on the isolation which is an important first step on both with the "use the exact same code for the consensus critical parts" approach as well as the "formally show some code achieves certain properties". I suspect that on the latter we won't make much progress in the near term outside of some parts in the underling cryptographic code unless some researchers pick it up as an area of study, but we should hopefully be well positioned to take advantage of formal analysis tools as that space matures.

[davec]

I personally think the question is wrong here.  Not even Bitcoin Core is technically 100% compatible with the same version of itself, so asking that question is starting with a false premise.  As previously mentioned, there is a lot of work to be done in the formal verification space, but even then, due to hardware differences, compiler optimization differences, OS differences, and a myriad of other factors, I personally think it's unlikely there will ever be absolute 100% certainty.  The question then becomes, does there really have to be?

To me, what it really comes down to is probabilities.  The popular assumption is that Bitcoin Core has an extremely low probability that it will fork against itself.  So, let's go with that premise and say that the probability of Bitcoin Core forking against itself is something like 1 in a 100 trillion (1e14).

When an alternative implementation passes all consensus tests and covers all known corner cases, then it too has a relatively low probability of forking.  If that is not true, then the consensus tests need to be improved.  For the sake of argument, let's assume that such an alternative is 3 orders of magnitude, or 1000x, more likely to fork.  Well, then the forking probability relative to the Bitcoin Core forking probability previously stated would then be 1 in 100 billion (1e11).

Moreover, if/when a fork does happen, it identifies a case that the consensus tests missed and can be used to improve the consensus tests which in turn further reduces the probability for all nodes involved (including Bitcoin Core) that validate against the consensus tests.

It is estimated that in any given year the odds that you will die from an impact of an asteroid or comet are between 1 in 3,000 and 1 in 250,000.  However, that probability is still low enough that I bet you don't live in a steel box and refuse to go outside just in case.

[tl121]

I was going to make a post similar in content to davec's, but he already made my points.

One of the reasons why I didn't get involved with bitcoin at the beginning was because it was a complex distributed system that had a code base but no real specification.  It was genius and luck that bitcoin took off and still works.  There should be a formal specification, but the problem will be keeping it in sync with the code base.  This will be politically impossible so long as there is only a single code base.  So for the long term good of bitcoin it is imperative that there be different code bases.

There will be forks. There are airline crashes.  People still fly.  In the aerospace safety world, the phrase is, "All progress comes as the result of crashes."

[gmaxwell]

To me, what it really comes down to is probabilities.  The popular assumption is that Bitcoin Core has an extremely low probability that it will fork against itself.  So, let's go with that premise and say that the probability of Bitcoin Core forking against itself is something like 1 in a 100 trillion (1e14).

Probabilities are not an effective model to reason about bugs in an adversarial environment. (Even in conventional reliability engineering-- without invoking an adversary-- you must first separate systematic failure from random failure in order to reason about failure distributions.)

"If you have a website with a billion pixels and due to a bug clicking one gives the reader one of your bitcoins, and you have one visitor an hour and a normal visitor clicks on 100 pixels, how long until you've lost 100 bitcoins?"  .... This doesn't make sense, and thinking about consensus bugs in this manner doesn't make sense. Existing consensus bugs we've seen triggered against implementation were 10^-100 scale events if your model is random monkies pounding on keyboards to produce transactions.

Virtually all the errors we care about are not random failures, and to the extent that there is an element of randomness an attacker can behave adaptively to bring out the otherwise 'rare' behavior that is harmful to your interests.

the consensus tests need to be improved

Thats always true, the tests are woefully inadequate and would continue to be even if they were terrabytes in size. That said, improving them has high value. I'm looking forward to contributions on that front. Probably one of the really important things an alternative implementation can do for the ecosystem is produce new tests as a side effect of their own independent efforts to obtain agreement. (Though, sadly, there seems to have been at least some level of people keeping at least implementation errors private in order to use them for marketing purposes... with all the different ways to monetize a security flaw other than reporting it, we cannot really count on reports.)

But thinking that passing a prefab test suite that someone else wrote alone is sufficient-- which is certainly the impression given there-- sounds like magical thinking. It's not sufficient, merely necessary, and one shouldn't rely on it alone. (If this wasn't the message you were intending to convey, it wasn't clear to me.)  Anyone can grind at some tests until they pass but doing, sadly, reduces the predictive power of the tests for errors not present in the tests. (This is partially why I prefer the block tests in Bitcoin runs in CI rather than being used locally by developers; we get a concrete measurement of detectable consensus bugs which are introduced by a change.)  Yes, by some definition the tests are inadequate if there is a failure in spite of passing; but they're always going to be inadequate because they cannot be sound, at least not by themselves. (A test coupled with a formal analysis that showed it met some definition of completeness would be interesting.)

There should be a formal specification,

Philosophically there should be, but in terms of benefit to the world the value is quite possibly negative. A non-executable or not widely used specification is a dead letter. The behavior of the participants defines the system.  You can point blame all day this way that when some reading of the specification disagrees with some other reading; but its little comfort when all the funds are gone.  As a pedagogical artifact a good specification may be highly useful; but it can also cause harm by feeding deep misunderstandings like having software which you believe-does (instead of actually-does) what you believe the specification says (rather than what everyone else actually-does, regardless of what they believe the specification says) is an acceptable situation to be in. "But I followed the spec!" "Too bad, all your coins have been double spent away because you ended up on a different chain from the other participants."  Compared to other efforts to improve system integrity I think specification work beyond what exists now has fairly low returns.  (Doubly so because existing alternative implementers have frequently made mistakes in parts of the system that are clearly specified and have tests in the normal tests suites.).   In Bitcoin the behavior of the system is all there is. You have no other recourse, no higher authority.   Being surprised or unhappy with at the Bitcoin network for disagreeing with a ream of paper is, in some ways, like being mad at the universe for not being completely defined by Einsteins' description of general relativity.  Like the universe, the Bitcoin network doesn't give a darn about political gamesmanship. It does what it does, not what you think it should do. To the extent that people want to work on specification work, especially if it was specification work that clearly enough defined to permit formal reasoning, it's work I would support-- in spite of my reservations about potential net-negative value from it.

[davec]

Probabilities are entirely the wrong model to reason about bugs in an adversarial environment.

We can't just cherry pick what to apply our arguments to.  The entire argument here is entirely based around the probability that Bitcoin Core is less likely to fork than alternatives, because the exact same logic you describe applies to multiple versions of Bitcoin Core on different compilers, OSes, hardware, etc, as much as it does to alternatives.  Of course, in an adversarial environment, you have bad faith actors looking to exploit those probabilities.  You're asserting that there is a much lower probability that they will be able to do so between two versions of Bitcoin Core.  That's fair, but it doesn't make any sense to use probabilities to assert that and then turn around and say they don't matter when applied to a different implementation.

Either probabilities don't matter and the entire argument about whether Bitcoin Core is more or less likely (probability) to fork or not is moot, or they really do matter.  I claim they absolutely do matter because Bitcoin Core isn't immune to forks either and there is clearly some chance involved that forks will occur between any two pieces of Bitcoin software.

But thinking that passing a prefab test suite that someone else wrote alone is sufficient-- which is certainly the impression given there-- sounds like magical thinking. It's not sufficient, merely necessary, and one shouldn't rely on it alone. (If this wasn't the message you were intending to convey, it wasn't clear to me.)

The claim isn't that a prefab test suite is entirely sufficient, it's that it lowers the probability of a fork occurring.  When is the probability low enough that it's generally safe to transact?  If you claim there isn't one, then Bitcoin, as a whole, is unsafe to transact on because the entirety of consensus is probability based to begin with.

[gmaxwell]

We can't just cherry pick what to apply our arguments to.  The entire argument here is entirely based around the probability

I'd like to invite you to point out any place where I argued anything about probability, except to rebut your reliance on that argument. Bitcoin is an experimental system, and in some regards it is pushing the envelope about the kinds of systems we know how to engineer and is doing so under considerable resource restrictions.

[Realpra]

https://github.com/conformal/btcd

I don't know if it works and is done yet, but here you go... hahaha.


In truth though Bitcoin is an agreement between humans and as such it can never die. Hash algorithms and signing algorithms can be switched and bugs can be fixed. Consensus splits can be resolved - ALL of those have already happened before.
(Original payment method was supposed to be IP based, 2010 had a script bug letting everyone spend anything and as noted 2013 had a major fork event)

Gavin's scalability roadmap breaks the Bitcoin protocol, but because people support it and because they believe it is aligned with the concept of "Bitcoin" it will happen eventually.

However the system remains stable because you need 90+% majority for such things to work and only simple changes aligned with the human vision can attain that. It's extremely interesting and beautiful in a way.

You guys should read Asimovs Foundation book series.

[grau]

However the system remains stable because you need 90+% majority for such things to work and only simple changes aligned with the human vision can attain that. It's extremely interesting and beautiful in a way.

You are right. The real foundation of Bitcoin is the shared desire to have it, that will overcome any technical obstacles.

[work2heat]

One question with formal verification is what are you formally verifying?

Right. So maybe we can try to think about how to break bitcoin up into modular parts (eg. secp256k1, vm/script, encoding/decoding, merkle tree, create/verify block header, add block to tree, etc.). Then we can start with verifying the parts, and with verified parts you are much more likely to have a valid whole. Maybe at that point we can all band together and build an asic for it, then all run the asic instead of building software on the myriad OS's (which practically precludes the possibility of formal verification, as you've noted). Then maybe we'll have a safe bitcoin?! It's extreme but I'll entertain it.

[adam3us]

I think its fair to say its fundamentally hard (in a computer science formal sense) to make two implementations that are consensus identical in an adversarial setting.

Consensus is a new and tougher version of the host security challenges.  You dont even have to make an implementation buffer overflow in a steerable way, nor perform something it shouldnt have; just most divergences become sufficient to adversarially steer to the advantage of whoever finds the divergence first and is inclined to exploit it.

About the former simpler problem, Meredith Patterson gave an interesting research summary presentation on the limits of (computer) language security: https://archive.org/details/The_Science_of_Insecurity_

I encourage people, even those who think they know a decent amount about host security, to have a listen.  Its quite interesting.  You can see that any network protocol with messages interpreted by software forms a computer language, as well as bitcoin incorporating explicit languages like script as part of its operation.

Adam

[grau]

About the former simpler problem, Meredith Patterson gave an interesting research summary presentation on the limits of (computer) language security: https://archive.org/details/The_Science_of_Insecurity_

Great talk. Too bad, that Bitcoin messages use length encoding, even in scripts.

[DeathAndTaxes]

If you need to make use of some sort of custom node (server backend for example) I would recommend running a copy of bitcoind in wallet only mode. Then have your custom node connect only to the bitcoind under your direct control.   With bitcoind acting like a bitcoin firewall it will prevent entire categories of consensus split isolation attacks.  Just keep in mind it isn't a magic bullet.   If your consensus rules are bad enough your custom node may drop blocks passed by the bitcoind and thus "stall out" but that is preferable to being fed a false history by a malicious node.

[grau]

If you need to make use of some sort of custom node (server backend for example) I would recommend running a copy of bitcoind in wallet only mode. Then have your custom node connect only to the bitcoind under your direct control.

This is a very good advice.

I'd add that your code should do SPV validation, then a split with respect to the border router is limited to common bugs on your side.

[hhanh00]

Good advice but that defeats the reason I'm running my node. I'd like something small, light on memory usage and disk space that still does full validation. I ended up writing it myself.

[grau]

Good advice but that defeats the reason I'm running my node. I'd like something small, light on memory usage and disk space that still does full validation. I ended up writing it myself.

A wallet disabled, UI stripped satoshi client is the closest you can get at the moment to a small consensus compatible full validation.
I wish one could exclude also RPC code at compile time.

Your own code that goes beyond SPV could help understanding, aid debugging or define a side chain.

[hhanh00]

Mmmm - well, it bootstraps, syncs and validates the blockchain. It passes the acceptance tests from Matt too. As a relay node, it keeps a tx pool, validates, relays new tx and you can trim old blocks just by deleting files. I've been running for a while.
At about 2k lines of code, it fits my requirement of small code that I can fit in my brain but I understand it's not 100% compatible and will never be.

[grau]

Mmmm - well, it bootstraps, syncs and validates the blockchain. It passes the acceptance tests from Matt too. As a relay node, it keeps a tx pool, validates, relays new tx and you can trim old blocks just by deleting files. I've been running for a while.
At about 2k lines of code, it fits my requirement of small code that I can fit in my brain but I understand it's not 100% compatible and will never be.

I know it is possible to get that far in a few months, as I did myself.

The emphasis is on 100% in this discussion, that is unlikely to be reached with an unrelated code base. This is not just a theoretical problem, since the smallest disagreement is suitable to fork your node off the majority. How much that costs depends on your use.

The combo satoshi border router + SPV extension is simply a much safer bet if you really deal with money.

[Nicolas Dorier]

My original intention was to build a full node with NBitcoin.
I quickly abandoned the idea, you can't imagine the number of corner case you have to take care of... and it is a moving target.
Frankly speaking, I learned to appreciate the work that core developers were doing when I tried to do it by myself. It is boring, difficult, and high in responsability.

As DeathAndTaxes said, I end up having my own bitcoind, and my software connect directly to the trusted node.
100% is not possible, and you can expect that if the reward is worth it, an attacker will end up exploiting the 0,00000001%

[hhanh00]

Well - in my opinion, it's not a sustainable long term situation. I understand there are a lot of factors at play but putting the burden of all the core client work on a single implementation is an unfair responsibility to carry. Besides, it seems that some companies are willing to take the risk of running alternate clients. In my case, I simply aim to be at least as good as them while having a code base that I can be proud of.

[Nicolas Dorier]

Well - in my opinion, it's not a sustainable long term situation. I understand there are a lot of factors at play but putting the burden of all the core client work on a single implementation is an unfair responsibility to carry. Besides, it seems that some companies are willing to take the risk of running alternate clients. In my case, I simply aim to be at least as good as them while having a code base that I can be proud of.

The risk of having a centralized core team for bitcoin against the risk of a fork for 2 incompatible implementations.
This is a trade-off that the market will choose. I think that right now the risk of fork out weight the benefit. But maybe the market will think otherwise with a high quality alternative.

If you have demand for it, then maybe it is a good idea.

[gmaxwell]

The risk of having a centralized core team for bitcoin against

This is a false dichotomy.  Bitcoin Core is free / open source software. It isn't owned by anyone. Anyone is free to make their own modifications, run arbitrary versions of their own choosing from arbitrary sources, and people do. Even within one project the participants don't report to any authority except themselves.  Your argument actually cuts the other way when more diversity spreads thinner the review resources required to ensure that any implementation does what its users (or even its authors) thinks it does.

I think Hhanh00 comment about burden is weird; I am stuck reviewing redundant code in several language which I am not qualified to review in, and dealing with unfamiliar codebases because I have to cope with the mitigating interoperability risk created by other implementations. Node diversity is an significant cost to anyone working on the Bitcoin system who is not punting on dealing with the risk of the whole thing faulting. I am probably losing about 20% of my review time due to alternative implementations already, we have protocol enhancements hung up on concern that they'll just simply be rejected by other implementations (some of which are not very actively maintained relative to Bitcoin Core), etc.: It's a big enough task that people struggle to get things working, keeping up and working with the community is seemingly too much to ask for some. This isn't a cost we complain about-- it's a necessary consequence of an open system that people can (and inevitably will) do some of this--, but it very much is one. (This also holds for wallets and other tools, not just full nodes: I am not a developer of Bitcoin Core. I am a developer for the Bitcoin system as a whole, Bitcoin core just-- in my view-- is one of the most leveraged places to do work to benefit the Bitcoin system today.  That said, I have found and reported critical security vulnerabilities and design flaws in many wallets, services, and underlying infrastructure, sometimes fixing them by changes to the Bitcoin system when doing so appeared to be the best approach. These things are true for many of the other people who work on Bitcoin Core: We generally tend to take our share of the responsibility for the system as a whole, not exclusively the parts of it we work on most directly; as I've pointed out before: your own software being correct but not agreeing with other people is no great success in this space. Increasing ecosystem software diversity does not reduce the burden, beyond the point that it's best if no one program is asked to accommodate all possible needs.)

[Nicolas Dorier]

This is a false dichotomy.  Bitcoin Core is free / open source software. It isn't owned by anyone. Anyone is free to make their own modifications, run arbitrary versions of their own choosing from arbitrary sources, and people do. Even within one project the participants don't report to any authority except themselves.  Your argument actually cuts the other way when more diversity spreads thinner the review resources required to ensure that any implementation does what its users (or even its authors) thinks it does.

You are preaching a convinced one.
However having multiple fork of the project, with different implementation, makes the system more diverse and anti fragile. Sure there will be more fires, but one fire will not burn the whole forest.

I don't consider it a priority at this stage... maybe in 10 years. But for some people, this is a priority.

Having alternative implementation in several language might add more review power : Take myself, I am unable to contribute to core. I can't compile the C++ project, and develop on windows. (but I would if I wanted to take the time)
If there was an implementation of a full node in C#, I would be able to contribute.

I am probably losing about 20% of my review time due to alternative implementations already

Why are you doing this ? if the alternative implementation is not in consensus with bitcoin core, they will be forced to evolve or just die.
If you prefer not having diversity, I don't understand why you would waste your time saving other implementations, you have more important to do. Why not just letting them die ?

[amaclin]

I am probably losing about 20% of my review time due to alternative implementations already

Why are you doing this ?

May be someone pay for it?  

[Nicolas Dorier]

I hoped he was paid for bitcoin core so he does not need to get his brain on something as futile as a bug in alternative implementations.
If I were rich, I would pay so he does not need to do it. (Well, that's a work in progress... the being rich part... )

[davec]

Am I the only one that sees how conflicted the messaging is in this thread?

In one camp we have people saying that BC is not really centralized because it's free and open source and can therefore be forked and modified in any arbitrary fashion, but then the other camp talks all about consensus issues and why implementations not running the official BC source are a bad idea.

What's interesting about this is the moment you fork the code and start making changes to it, it is also an alternative implementation.  You could argue about the probabilities of forking risk, etc, but the fact remains, the moment you fork it and modify it, you are running something that is not the same as the majority of other nodes on the network.  It would not take much at all for somebody to fork BC and refactor some code while adding some features that they want which accidentally (or even intentionally if it's a malicious actor intending to distribute their fork) makes a mistake that very subtly breaks consensus.  This is really no different than any other alternative implementation when you get right down to it.

It's either acceptable to have Bitcoin nodes running different code bases or it's not.  I just don't see how any reasonable person can claim it goes both ways.  Either the message must be that it's too dangerous to run anything except the official Bitcoin Core sources, in which case the development truly is centralized and controlled by the small group of people who hold the ssh keys, or that it is in fact acceptable to run modified sources (which very much implies alternative implementations).  It's just not reasonable to claim that it's not centralized because theoretically anybody can fork it and do whatever they want while at the same time saying don't do that because it's too dangerous!

[grau]

It's just not reasonable to claim that it's not centralized because theoretically anybody can fork it and do whatever they want while at the same time saying don't do that because it's too dangerous!

Those who control the github keys of the most popular fork (bitcoin/bitcoin) determine the code run by the majority, this however is just common sense. Those who achive and maintain consensus and trust of people have power.

Those who know the beast called "Nakamoto consensus" are quick to point out the dangers, that are very real.

You are still free to fork or rewrite for your own benefit or loss. There are safer ways to play that too, keep asking and listening.

[amaclin]

Those who achive and maintain consensus and trust of people have power.

Barack Obama and Vladimir Putin?

[gmaxwell]

This is really no different than any other alternative implementation when you get right down to it.

I'm kind of alarmed at the false equivalence you're drawing between all possible changes. That bogus, and I know you know it.

When you're talking about a compact intentional delta you can reason about the exposure; you might fail, you might not. Normally it wouldn't make sense to do this on the balance of the costs.

The _ability_ to do something is important; it's a functional check and balance even if you don't actually pull the trigger on it or if you find those changes are just helpfully welcomed in.  When the consensus is potentially bifurcated without someone doing something awful it reduces the pressure to maintain homogenity as a justification to not deploy features which are harmful to a minority ("let them run their own nodes, and see if anyone cares").
 

the official

There is no "the official" anything in Bitcoin.

I hoped he was paid for bitcoin core so he does not need to get his brain on something as futile as a bug in alternative implementations.
If I were rich, I would pay so he does not need to do it. (Well, that's a work in progress... the being rich part... )

There is no escaping that we must care about implementations even run by a small number of people. If something happens in the network that harms them its a harm for all of us.  Outsiders rightfully do not judge Bitcoin for this implementation vs that implementation. We're all in this together.

[amaclin]

There is no escaping that we must care about implementations even run by a small number of people.

What do you mean saying "we"?
Do you include me and 7bln other people on Earth?
Should I care about any implementation of bitcoin app? Why?

[grau]

There is no escaping that we must care about implementations even run by a small number of people. If something happens in the network that harms them its a harm for all of us.  Outsiders rightfully do not judge Bitcoin for this implementation vs that implementation. We're all in this together.

Your work there is appretiated, but lets admit further benefits:

1. It is cheaper to learn from mistakes others make
2. Dangers uncovered elsewhere keep the herd together

[hhanh00]

A vast majority of the nodes is running a version or another of the core client. I don't know the exact numbers but when I look at https://blockchain.info/en/connected-nodes, it looks like it's close to 100%.
In my opinion, if most of the nodes run similar code it increases the risk of a common exploit similar to heartbleed or shellshock. Furthermore, Bitcoin core upgrades are conservative because it must avoid introducing bugs as much as possible. In my opinion, if the network was more heterogenous with let's say 10 different implementations sharing equal weight then the chances of a major fork will remain limited. An attack would need to find a flaw that affects 5 different implementations to gain majority.

Yet, it is the responsibility of whoever chooses to run a node to understand the risk taken. They could run several implementations for instance.

[grau]

In my opinion, if the network was more heterogenous with let's say 10 different implementations sharing equal weight then the chances of a major fork will remain limited.

An implementation forking from the majority view is not an issue for the majority or for the network at a whole.

BUT you personally would not want to be in the minority for any exploitable time period, because that could mean that you accept coins for your goods that are already spent in the majority view.

Your financial loss would be realized and final once you fix your software and return to majority consensus.