The duplicate input vulnerability shouldn't be forgotten

[aliashraf]

Obviously, it would be much safer for a community to take care of one implementation with fewer lines of codes.

I don't think it is necessarily best to rely on the "community" to ensure that each implementation of a bitcoin node is secure/safe to use.

Members of the community might have, at most a few million dollars worth of bitcoin of their own money at stake, but even if they make a mistake, they are unlikely to personally lose any money. On the other hand, there are several bitcoin related businesses that have billions of dollars worth of customer money, and hundreds of millions (and in some cases billions) of dollars of equity who have serious incentives to ensure these types of bugs don't pop up with software in production, and they have incentives to have fail-safes in place to prevent any actual losses if/when these types of bugs make it through the cracks.

I would point out that I am not aware of any major exchange "pausing" deposits and/or withdrawals immidiately after this bug was discovered, however anyone running the relevant software would have taken some time to stop deposits/withdrawals to upgrade their nodes (which would include reviewing the code). This leads me to believe that the majority of exchanges/businesses are running their own custom node software, maybe not exclusively, but this is at least part of what they are running.  

I thought we are done with multiple implementations proposal and it was why I brought up my software bloat diagnosis for the issue. Now I see we are recycling the same idea again and again and it is really disappointing and makes me paranoid.

Why? Why should one continue pushing for a false idea like this? How is it possible to have 10 million lines of code safer than 100K?

I suspect most of the people who are insisting on this idea, multiple implementations, are just taking advantage of an incident to retaliate against Core team. I'm sure about you not being such a person but I afraid you don't completely get how irrelevant and dangerous is this proposal.

Satoshi's original software was just like 3K lines of code, now we have bitcoin core with 100K lines and every software engineer knows what does it mean and how inevitable is having bug issues. Actually it is very impressive that an unexploited bug is the worst incident of its kind after like 10 years and Core guys deserve a lot of kudos for the job they have accomplished till now.

Now instead of a decent technical discussion about how and why this bloat happened and what measures should be taken to manage the risks involved, we are watching biased actors who are seeking more power in the community or suppose a weakened developer community is better for them are suggesting the oddest solution ever: using even more lines of code! I made a prophecy regarding this situation:

I understand there are many people with various incentives mostly not in the best interests of bitcoin who may find such an event a good opportunity to take advantage in an irresponsible and unproductive way. Although I have a reputation on this forum to be somewhat discontented with Core devs, I hope my statements here other than being helpful in the context of this topic, would show how committed I'm to bitcoin as a liberating movement instead of blindly opposing and fighting with specific parties.

The most sophisticated reasoning behind multiple implementation idea could be something like this:

One thing I've always wanted to do -- but have never had the energy for -- was to run multiple implementations of bitcoin (e.g. btcd and bitcoin core) and only transact while they are in agreeance.

For most bitcoin businesses, a few hours of not processing deposits/withdrawals is actually not a big deal, and happens pretty regularly anyway (for no fault of bitcoin itself). While on the other hand transacting after an accidental chain-split could really be devastating.

(which you, @quickseller, have merited by the way)
But it is not a matured idea as I have argued against it  before:

Firstly, it implies a new kind of consensus system, with no theoretical support.

Bitcoin consensus system is based on game theory (rational behavior of players with divergent incentives) it is NOT about fault tolerance regarding software bugs. Putting such a layer on top of bitcoin is not a trivial job because we are not talking about low level control systems and alike.

From a software engineering point of view, once we are concerned about bugs in a system, software bloat is the most important problem that we should take care of. As I have discussed in my previous post in this topic , for bitcoin, this bloat is not a mistake of an architect or a programmer it is a direct consequence of the very fact that bitcoin is a new class of system and is not (and should not be) governed traditionally, it escalates a case-by-case development process and institutional conservatism in the community according to which devs define their role as guardians.

Bitcoin has grown from 3,000 lines of code to 100,000 all the way with soft forks, i.e. downward compatibility (somehow). It is the source of the bloat
and the critical vulnerability to bugs, this situation should be reconsidered radically if there is a real motivation to do anything other than taking foolish temporary political advantages of a serious threat that should be addressed asap and before everything is lost.

[danda]

As I see it:

• The consensus layer protocol needs to be formally specified and versioned.  Bugs and all.  The spec should be updated before consensus code
• consensus layer code should be changed as rarely as possible.  if ever.

I would be very happy to see a software fork that makes a promise to its users that consensus layer code will not be changed except for critical bug fixes.   I think this will have a couple of important effects:

1. Foster growth of the bitcoin sub-community whose priority is to resist changes to the protocol.  Those who value stability, predictability, immutability in an asset/currency over shiny new things.
2. Create a technical means for this community to have its voice heard.  Possibly blocking future soft forks from occurring, or at least keeping the "old ways" alive.

my still-running pre segwit node is not affected by this bug.   just sayin.

[DooMAD]

I suspect most of the people who are insisting on this idea, multiple implementations, are just taking advantage of an incident to retaliate against Core team. I'm sure about you not being such a person but I afraid you don't completely get how irrelevant and dangerous is this proposal.

Satoshi's original software was just like 3K lines of code, now we have bitcoin core with 100K lines and every software engineer knows what does it mean and how inevitable is having bug issues. Actually it is very impressive that an unexploited bug is the worst incident of its kind after like 10 years and Core guys deserve a lot of kudos for the job they have accomplished till now.

Now instead of a decent technical discussion about how and why this bloat happened and what measures should be taken to manage the risks involved, we are watching biased actors <snip>

Putting aside the absurdity of someone who argues against any and all off chain solutions and believes every future development can be nowhere other than on layer 0 having the gall to use the word "bloat" in a sentence without a hint of irony...

How about, instead of gross generalisations and dismissing thousands of lines of code as "bloat", you name the specific parts of the code that you believe aren't needed?  This will greatly expedite the moment someone can tell you why you're wrong and we can all move on.

Fair enough if we're ruling out alternative implementations due to security concerns.  It was only a suggestion based on the conventional wisdom of "not putting all your eggs in one basket".  But, considering that you are someone who has launched multiple lengthy tirades against the current direction this project is moving in as of late, if you're using this as another opportunity to take cheap shots at LN, you can add yourself to the list of biased actors who are taking advantage of this incident.

[aliashraf]

I suspect most of the people who are insisting on this idea, multiple implementations, are just taking advantage of an incident to retaliate against Core team. I'm sure about you not being such a person but I afraid you don't completely get how irrelevant and dangerous is this proposal.

Satoshi's original software was just like 3K lines of code, now we have bitcoin core with 100K lines and every software engineer knows what does it mean and how inevitable is having bug issues. Actually it is very impressive that an unexploited bug is the worst incident of its kind after like 10 years and Core guys deserve a lot of kudos for the job they have accomplished till now.

Now instead of a decent technical discussion about how and why this bloat happened and what measures should be taken to manage the risks involved, we are watching biased actors <snip>

Putting aside the absurdity of someone who argues against any and all off chain solutions and believes every future development can be nowhere other than on layer 0 having the gall to use the word "bloat" in a sentence without a hint of irony...

More ironic would be the attitude of an "anti-FUD" troll who suddenly suggests using alternative implementations because of his common sense about "not putting all the eggs in one basket", I suppose.  

How about, instead of gross generalisations and dismissing thousands of lines of code as "bloat", you name the specific parts of the code that you believe aren't needed?  This will greatly expedite the moment someone can tell you why you're wrong and we can all move on.

Have you ever tried coding? It is not how it works. you can not pick Windows source code and put finger on this or that module to blame. It is a technical term and one should have a basic software engineering knowledge and experience to catch up.

Based on your above argument, I suppose you are not 100% qualified to judge my assessment. It would be Greg Maxwell's job to denounced bitcoin code as being in a bloated state and then my job to convince him about it. For now, 7500% growth in the code volume is enough evidence to "move on" with my analysis.

FYI: My opposition to LN and off-chain scaling solutions has nothing to do with software bloat. I think it is absolutely possible to have a smart, clean and elegant software that implements a robust and solid protocol capable of satisfying all the necessary conditions for a decentralized, permissionless and secure "p2p electronic cash system" and is absolutely capable of performing and scaling good enough to be gradually adopted by people as an alternative monetary system without compromising any of these features or projecting any part of its job to non-standard semi-centralized second layer solutions.
It is how I show my loyalty to "the cause". Unlike people like you, I haven't give up with bitcoin and crypto.

Fair enough if we're ruling out alternative implementations due to security concerns.  It was only a suggestion based on the conventional wisdom of "not putting all your eggs in one basket".  

Ok, I get it. You said something silly and now you are sorry.
Apology accepted but you should stop terrorizing my personality too.
Just keep reading and try learning instead of talking nonsense about LN in a decent highly technical topic about a critical turning point in bitcoin history.

[Lauda]

It was only a suggestion based on the conventional wisdom of "not putting all your eggs in one basket".  

Maybe you should avoid trying to apply "investment wisdom" on software engineering, especially security-related engineering. Just a thought.

Have you ever tried coding? It is not how it works. you can not pick Windows source code and put finger on this or that module to blame. It is a technical term and one should have a basic software engineering knowledge and experience to catch up.
-snip-
For now, 7500% growth in the code volume is enough evidence to "move on" with my analysis.

This says everything. I sense a strong IT background.

[Peter R]

As I see it:

• The consensus layer protocol needs to be formally specified and versioned.  Bugs and all.  The spec should be updated before consensus code
• consensus layer code should be changed as rarely as possible.  if ever.

A formal spec would be nice, I agree.  But I think it is interesting that Core's inflation bug wasn't due to nuance in the consensus rules.  Having a formal spec wouldn't have helped in this case.  The bug literally allowed coins to be created out of thin air.  Something that _obviously_ was not supposed to happen.  You might call it the most important consensus rule in Bitcoin!

As to your second point about the consensus layer not changing much, I think this is tricky in practice too. For example, a hot area of research in blockchain scaling is how to parallelize block validation.  This type of work will almost certain involve refactoring consensus critical code.  If we want to eventually processes thousands, or hundreds of thousands, of transactions per second, we will need massive parallelization.

[DooMAD]

Fair enough if we're ruling out alternative implementations due to security concerns.  It was only a suggestion based on the conventional wisdom of "not putting all your eggs in one basket".  

Ok, I get it. You said something silly and now you are sorry.
Apology accepted but you should stop terrorizing my personality too.
Just keep reading and try learning instead of talking nonsense about LN in a decent highly technical topic about a critical turning point in bitcoin history.

Poor analysis as usual.  There was an exchange of ideas and my idea has now been established to be objectionable.  So instead of continuing to plow on regardless of how many people tell me it's a bad idea, I'm accepting that decision and not continuing to pursue it.  That's something you could learn, but I suspect you won't.  

Also, I sincerely doubt this is a "turning point" on the scale you have in mind.  There will be some proposals to introduce a little more vigilance, but it sounds like you're expecting some sort of total rebuild from the ground up.  Your track record of contributing to technical topics usually consists of telling people that everything they're working on needs to go in the trash because you supposedly know best.

1. Foster growth of the bitcoin sub-community whose priority is to resist changes to the protocol.  Those who value stability, predictability, immutability in an asset/currency over shiny new things.
2. Create a technical means for this community to have its voice heard.  Possibly blocking future soft forks from occurring, or at least keeping the "old ways" alive.

Hang on, what?  You want to have a veto in what many see as a permissionless system?  No thanks.  Also, you already have a technical means to do that.  If you want to block future forks to preserve the "old ways", you're going to have to start with a fork of your own.  Keep it frozen in time forever if you like.  It stands to reason that you won't have many developers on hand when you do need to change something, though.

[harding]

I'm disappointed that almost everyone posting in this thread is making suggestions about what other people should do rather than thinking of ways to contribute themselves.

I plan to open at least one PR a month adding a useful new test to Bitcoin Core, or significantly improving an existing test, with the first PR to be opened by 31 October 2018.

What do the rest of you plan to do?

[Peter R]

I'm disappointed that almost everyone posting in this thread is making suggestions about what other people should do rather than thinking of ways to contribute themselves.

I plan to open at least one PR a month adding a useful new test to Bitcoin Core, or significantly improving an existing test, with the first PR to be opened by 31 October 2018.

What do the rest of you plan to do?

I plan to continue working on a competing implementation to Bitcoin Core. It was because of Bitcoin Unlimited that this bug was caught, when Awemany noticed it while working on the consensus changes for the November fork in BCH.  This tells me that multiple implementations and competing development teams is a good thing.

[JayJuanGee]

I'm disappointed that almost everyone posting in this thread is making suggestions about what other people should do rather than thinking of ways to contribute themselves.

I plan to open at least one PR a month adding a useful new test to Bitcoin Core, or significantly improving an existing test, with the first PR to be opened by 31 October 2018.

What do the rest of you plan to do?

newbie needs to have a track record before dictating what others could or should do, no?  Let's see if you do what you said you were going to do.  How many months are you going to report back to show that you have taken actual action and are more than just promises?

[Peter R]

I'm disappointed that almost everyone posting in this thread is making suggestions about what other people should do rather than thinking of ways to contribute themselves.

I plan to open at least one PR a month adding a useful new test to Bitcoin Core, or significantly improving an existing test, with the first PR to be opened by 31 October 2018.

What do the rest of you plan to do?

newbie needs to have a track record before dictating what others could or should do, no?  Let's see if you do what you said you were going to do.  How many months are you going to report back to show that you have taken actual action and are more than just promises?

I'm pretty sure @harding is David Harding.  Definitely not a Bitcoin newbie.

https://github.com/harding

[bones261]

I plan to continue working on a competing implementation to Bitcoin Core. It was because of Bitcoin Unlimited that this bug was caught, when Awemany noticed it while working on the consensus changes for the November fork in BCH.  This tells me that multiple implementations and competing development teams is a good thing.

I'm just disappointed that awemany hasn't received more tips. I personally tipped him .01 BCH. He hasn't even gathered 39 BCH, yet, last time that I checked. I would think the BTC and BCH community would be more grateful and giving. (As well as LTC, BTG etc. etc. communities.)

[Lauda]

I plan to continue working on a competing implementation to Bitcoin Core. It was because of Bitcoin Unlimited that this bug was caught, when Awemany noticed it while working on the consensus changes for the November fork in BCH.  This tells me that multiple implementations and competing development teams is a good thing.

I'm just disappointed that awemany hasn't received more tips. I personally tipped him .01 BCH. He hasn't even gathered 39 BCH, yet, last time that I checked. I would think the BTC and BCH community would be more grateful and giving. (As well as LTC, BTG etc. etc. communities.)

Given the absolutely shameful disaster of a post that he wrote on medium, he deserves nothing IMO.

[Ayms]

I plan to continue working on a competing implementation to Bitcoin Core. It was because of Bitcoin Unlimited that this bug was caught, when Awemany noticed it while working on the consensus changes for the November fork in BCH.  This tells me that multiple implementations and competing development teams is a good thing.

I'm just disappointed that awemany hasn't received more tips. I personally tipped him .01 BCH. He hasn't even gathered 39 BCH, yet, last time that I checked. I would think the BTC and BCH community would be more grateful and giving. (As well as LTC, BTG etc. etc. communities.)

Given the absolutely shameful disaster of a post that he wrote on medium, he deserves nothing IMO.

My opinion about all this story is that unfortunately it gives a strong feeling of not encouraging people to report bugs, @awemany is maybe thinking that he should have better sold the exploit to some dubious parties that could have used it

I don't think it's very important to know the total truth, he reported the critical bug and should have desserved a much more important reward (but he could have admitted that beardnboobies, while funny, is a kind of arrogant also), maybe he did not know to what extent it was critical but then his action prevented others from discovering it and using it

What solution among those discussed here is the best for the future? I don't know

But for sure that's always the same story, all decentralized systems failed (except bittorrent) because of the lack of incentive for people to participate (run nodes, participate to code, review, etc), fortunately bitcoin is not a decentralized system today so such situation can be controlled but it should/will be in the future, then what will be this incentive for people (lightning?)?

Simple example: I realized after the vulnerability disclosure that I was running a deviant node version 0.17.99 since I installed it from master (with a lot of difficulties), then I had to patch manually the vulnerability, I should revert to 0.16.3 and have a clean node, now why am I going to spend more time on this?

[dragonvslinux]

Well written, it's good to see a honest and constructive approach to the problem. I fully agree an LTS branch should now be implemented, starting with 16.03, this is long overdue since the overflow bug of 2010. While it'd be nice for bitcoin "companies" (companies profiting from Bitcoin transactions) to contribute to core testing, it seems like they may only do so if they have to. Maybe now they will consider it? But in my opinion it'd be more likely they would want to throw money at the problem, rather than get their hands dirty.
All in all it was a good catch, the patch was rolled out very effectively and damage was limited to $0. It's good to keep sight of this, in one sense, this is a definite victory.
To me this is merely a learning curve, we never should trust any technology 100%, there will always >1% chance of an exploit, the question is whether it can be identified through rigorous testing, or whether a malicious actor will discover it first and therefore exploit it. This is the logic we need to work on, remembering that no code is perfect.
What concerns me now is knowing that there may now be more malicious actors studying the code for any future exploits, as opposed to Core testing.
 

[Lauda]

While it'd be nice for bitcoin "companies" (companies profiting from Bitcoin transactions) to contribute to core testing, it seems like they may only do so if they have to. Maybe now they will consider it? But in my opinion it'd be more likely they would want to throw money at the problem, rather than get their hands dirty.

It's utterly disgusting how greedy some of the leading companies in this space are, and just how fraudulently their leaders tend to represent themselves with statements of support/belief in Bitcoin or similar. AFAIK, blockchain.info has hired 1 employee (although I can't recall which whop) and there was a Xapo listing for a Bitcoin Core developer some time ago (I'm not sure what happened with that). Other than those two, I haven't seen other examples of companies doing this.

[Samarkand]

...
Other than those two, I haven't seen other examples of companies doing this.

Blockstream also has hired quite a few developers that contribute to
Bitcoin Core and don´t work on their sidechain products.
Allegedly they also receive time-locked Bitcoins to ensure that it is in
their interest to maintain Bitcoin (a pretty good idea if you ask me, more
companies should do this).

...
6- Because of the last two factors, bitcoin now suffers from a software engineering problem: software bloat. Through years, incremental developments
plus efforts for keeping system downward compatible and insisting on soft forks against hard forks has ended to a situation in which bitcoin code is becoming
more and more complicated and hard to understand/contribute amd maintain.
...

Isn´t this true for any larger software engineering project?

[chek2fire]

I plan to continue working on a competing implementation to Bitcoin Core. It was because of Bitcoin Unlimited that this bug was caught, when Awemany noticed it while working on the consensus changes for the November fork in BCH.  This tells me that multiple implementations and competing development teams is a good thing.

I'm just disappointed that awemany hasn't received more tips. I personally tipped him .01 BCH. He hasn't even gathered 39 BCH, yet, last time that I checked. I would think the BTC and BCH community would be more grateful and giving. (As well as LTC, BTG etc. etc. communities.)

he deserve this guy nothing because he use it as a political propaganda tool against bitcoin. And is not very sure that is the same guy that found this bug as his claim have some timeprof mistakes.

[harding]

AFAIK, blockchain.info has hired 1 employee (although I can't recall which whop) and there was a Xapo listing for a Bitcoin Core developer some time ago (I'm not sure what happened with that). Other than those two, I haven't seen other examples of companies doing this.

Last I heard, Sjors Provoost works for Blockchain.info, Anthony (AJ) Towns works for Xapo, and Jim Posen works for Coinbase. Blockstream employees Pieter Wuille, Jorge Timon, Gregory Sanders, and several other contributors (plus two C-Lightning devs)  Several companies also help support the Media Lab's Digital Currency Initiative (DCI) that employees Wladimir van der Laan and Cory Fields (as well as several other open source Bitcoin contributors who don't normally focus on Bitcoin Core).

The other major source of employment for Bitcoin Core work is ChainCode Labs.

(I could be forgetting some companies; if so, sorry.)

[chek2fire]

Well written, it's good to see a honest and constructive approach to the problem. I fully agree an LTS branch should now be implemented, starting with 16.03, this is long overdue since the overflow bug of 2010. While it'd be nice for bitcoin "companies" (companies profiting from Bitcoin transactions) to contribute to core testing, it seems like they may only do so if they have to. Maybe now they will consider it? But in my opinion it'd be more likely they would want to throw money at the problem, rather than get their hands dirty.
All in all it was a good catch, the patch was rolled out very effectively and damage was limited to $0. It's good to keep sight of this, in one sense, this is a definite victory.
To me this is merely a learning curve, we never should trust any technology 100%, there will always >1% chance of an exploit, the question is whether it can be identified through rigorous testing, or whether a malicious actor will discover it first and therefore exploit it. This is the logic we need to work on, remembering that no code is perfect.
What concerns me now is knowing that there may now be more malicious actors studying the code for any future exploits, as opposed to Core testing.
 

In open source world we have always seen that startups or companies that use the open source programme always to have a full time payment developer to contribute to code. Great example for this is Linux.
Bitcoin is an open source paradox programme imo because the most of the companies that get great profits from it have never feel the needing to contribute to code, an example for this is Coinbase.
Many of them also want to replace it with something that full control it and act as someone that like to destroy it.
Great example to this is Bitmain, Bitpay, Blockchain info.