In re Bitcoin Devs are idiots

[MPOE-PR]

MPOE-PR: You seem to be under the mistaken impression that the hard fork was caused by something in 0.8. Not so. The block that 0.7 choked on was a valid block. The bug was lurking in 0.7 all along, and was merely uncovered by a 0.8 miner's block. You might as well blame the physician who taps your kneecap to check reflexes, for the bone cancer that he unexpectedly discovers there.

I am not under any suck mistaken impression. Please read what's actually being said, it will help.

The presence of a 250k arbitrary block limit prevented the underlying problem of the db from being discovered until that moment when a perfect storm composed out of a db upgrade and the limit removal created two almost equal miner bases. This could have easily been avoided by simply not relying on magic numbers. It will ever be the case that whenever code contains arbitrary numbers based on arbitrary assumptions that code will break spectacularly.

[1713415562]

1713415562

[gweedo]

MPOE-PR I am calling for Gavin to not get paid this month by the foundation, I would think your with me on that. Someone needs to take blame for this bug.

[justusranvier]

The presence of a 250k arbitrary block limit prevented the underlying problem of the db from being discovered until that moment when a perfect storm composed out of a db upgrade and the limit removal created two almost equal miner bases.

Maybe it wasn't the size of the block that was the problem.

https://bitcointalk.org/index.php?topic=152131.msg1614374#msg1614374

[Monster Tent]

Bitcoin is still beta software and untill recently no one got paid to do work on it.

MPOE-PR should stfu and contribute.

I bet they havent even donated to the bitcoin foundation  

Hurr Durr look how much money we made off free software.

[MPOE-PR]

The presence of a 250k arbitrary block limit prevented the underlying problem of the db from being discovered until that moment when a perfect storm composed out of a db upgrade and the limit removal created two almost equal miner bases.

Maybe it wasn't the size of the block that was the problem.

https://bitcointalk.org/index.php?topic=152131.msg1614374#msg1614374

That one requires more explanation for sure. What exactly?

[alexkravets]

Another way to state the real problem: There is no Bitcoin Protocol Spec, most semantics buried in the hairball of the C++ reference implementation

This was actually a good example of something a spec wouldn't help (though would still be good to have). The inconsistent behavior here arose out of 0.8 not faithfully implementing some implicit behavior in BDB. The behavior in question is not anywhere in the Bitcoin code itself, nor is it visible at the interface of BDB and Bitcoin.  If there were a spec it would make no mention of it— and yet the network would be broken all the same.

The key thing is that in a distributed consensus system the most important definition of right is "consistent", this is paramount to all other concerns. A spec is helpful to the extent that it makes it easier to achieve a bit identical consistent behavior in the validation of blocks across all nodes, but because the spec itself can't be executed a spec can never guarantee consistency, at least not in the real world. (well, unless the spec is code— which is effectively what we have, for worse or better)

I respectfully disagree.  

A real protocol specification would enable a whole ecosystem of alternative and interoperable implementations to appear in Scala, Java, Go, C, etc.  

These implementations would NOT use the same flawed 3rd party libraries and there would be a real diversity in Bitcoin land, instead of the monoculture of C++ with all of its "magic" and "folklore".

Once no individual implementation had large enough fraction of the hash rate, then such forks would NOT occur, instead, bugs in a given implementation would only affect some miners, merchants & users but the chain would be just fine.

Let us hope and pray that eventually there will be some effort ( funded perhaps by the Bitcoin Foundation ) to extract and publish such a spec from the C++ hairball formal enough for viable alternative implementations to appear and to interoperate.


Cheers ...

[DeathAndTaxes]

MPOE-PR: You seem to be under the mistaken impression that the hard fork was caused by something in 0.8. Not so. The block that 0.7 choked on was a valid block. The bug was lurking in 0.7 all along, and was merely uncovered by a 0.8 miner's block. You might as well blame the physician who taps your kneecap to check reflexes, for the bone cancer that he unexpectedly discovers there.

I am not under any suck mistaken impression. Please read what's actually being said, it will help.

The presence of a 250k arbitrary block limit prevented the underlying problem of the db from being discovered until that moment when a perfect storm composed out of a db upgrade and the limit removal created two almost equal miner bases. This could have easily been avoided by simply not relying on magic numbers. It will ever be the case that whenever code contains arbitrary numbers based on arbitrary assumptions that code will break spectacularly.

That is not correct.  v0.7 nodes accepted a 1MB block on testnet.  The issue was more complex then just the size of the block.  By the protocol the block which was rejected by some nodes SHOULD have been accepted.  The 250kb soft limit was simply a default for block construction.  Even with it in place nodes should have accepted properly constructed blocks up to the 1MB limit.  It also appears not all v0.7 nodes were affected.  Some accepted the problem block and some didn't.  The defect/limit in BDB wasn't documented and didn't occur in all versions/platforms.   It appears the number of transactions being changed as a result of the block validation crossed some "magic code" not in the Bitcoin source code but undocumented in some implementations of BDB.

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.   Nobody was saying/thinking oh yeah if you make a block with more than x transaction it will abort, cause a rollback, and result in a rejection.  It was a landmine.  

[MPOE-PR]

Bitcoin is still beta software and untill recently no one got paid to do work on it.

MPOE-PR should stfu and contribute.

I bet they havent even donated to the bitcoin foundation  

Hurr Durr look how much money we made off free software.

Never donated, never will donate. Things don't work like "here's a foundation we made, give us money".

For that matter, we didn't go to Taaki's stupidferences either.

[alexkravets]

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with either the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.  It was a landmine. 

Does this not argue for a diversity of implementations with different underlying 3rd party libraries ? Clearly it does. But this cannot happen until and unless there is actually a formal-enough spec to enable those to be written w/o ever "groking" all that C++ folklore, which is a moving target anyways ...

[MPOE-PR]

That is not correct.  v0.7 nodes accepted a 1MB block on testnet.  The issue was more complex then just the size of the block.  By the protocol the block which was rejected by some nodes SHOULD have been accepted.  The 250kb soft limit was simply a default for block construction.  Even with it in place nodes should have accepted properly constructed blocks up to the 1MB limit.  It also appears not all v0.7 nodes were affected.  Some accepted the problem block and some didn't.  The defect/limit in BDB wasn't documented and didn't occur in all versions/platforms. 

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with either the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.  It was a landmine. 

Notwithstanding that blocks of 1mb should have been accepted if valid, the limit prevented too many of them from being made.

If the limit had been removed at a reasonable time, those blocks would have occasionally been made, and the overall problem would have been much contained.

I would like to also point out that the alexkravets idea re specification is sound. The only way forward for the dev team, provided they wish to preserve a shred of dignity, is a. immediately start work on specification, which is the only priority, and release no further versions until such specification is complete and b. immediately start work on removing all magic numbers and assorted code smelliness from their UNreference implementation of crap, and not release any further version before they've released a clean one.

In that order.

[DeathAndTaxes]

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with either the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.  It was a landmine.  

Does this not argue for a diversity of implementations with different underlying 3rd party libraries ? Clearly it does. But this cannot happen until and unless there is actually a formal-enough spec to enable those to be written w/o ever "groking" all that C++ folklore, which is a moving target anyways ...

I agree on that 100%.  It is inevitable that to survive Bitcoin will need to be move beyond a monoculture.  Right now all the clients share the same "DNA".  If there is a fatal genetic flaw in it then the network will not survive.   This time the "landmine" was accidental, next time it may be planted.

[Atruk]

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with either the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.  It was a landmine. 

Does this not argue for a diversity of implementations with different underlying 3rd party libraries ? Clearly it does. But this cannot happen until and unless there is actually a formal-enough spec to enable those to be written w/o ever "groking" all that C++ folklore, which is a moving target anyways ...

I agree on that 100%.  It is inevitable that to survive Bitcoin will need to be move beyond a monoculture.  Right now all the clients share the same "DNA".  If there is a fatal genetic flaw in it then the network will not survive.   This time the "landmine" was accidental, next time it may be planted.

That is not correct.  v0.7 nodes accepted a 1MB block on testnet.  The issue was more complex then just the size of the block.  By the protocol the block which was rejected by some nodes SHOULD have been accepted.  The 250kb soft limit was simply a default for block construction.  Even with it in place nodes should have accepted properly constructed blocks up to the 1MB limit.  It also appears not all v0.7 nodes were affected.  Some accepted the problem block and some didn't.  The defect/limit in BDB wasn't documented and didn't occur in all versions/platforms. 

v0.7 reliance on BDB caused it to be fundamentally broken/flawed. Its actions weren't consistent with either the documented protocol, the higher level source code, or anyone's understanding/expectation of what should have happened.  It was a landmine. 

Notwithstanding that blocks of 1mb should have been accepted if valid, the limit prevented too many of them from being made.

If the limit had been removed at a reasonable time, those blocks would have occasionally been made, and the overall problem would have been much contained.

I would like to also point out that the alexkravets idea re specification is sound. The only way forward for the dev team, provided they wish to preserve a shred of dignity, is a. immediately start work on specification, which is the only priority, and release no further versions until such specification is complete and b. immediately start work on removing all magic numbers and assorted code smelliness from their UNreference implementation of crap, and not release any further version before they've released a clean one.

In that order.

Whoa... All of this... Agreement...

[cho]

MPOE-PR, your attitude makes me sick.
You've been yelling and calling developers names without having even understood what the technical issue is.
You pretend you deserve a perfect codebase and do not even spend resources towards this goal.
My opinion it that your childish rants hurt good-willed and hard-working people.
If your job is politics, then my opinion is that you suck at that job, because bitcoin as a whole does not benefit from someone technically incompetent throwing random blame around.

I, for one, do want to give a bucket of congrats to the devs for all the work done so far and for the quick response to the current problem.

One thing is interesting in this thread, it's the need for a formal spec of the protocol. alexkravets nails it real good in his posts.

[🏰 TradeFortress 🏰]

Today will go down in history as the day when Bitcoin nearly died, and its fate depended on BTC-Guild staying online. Stop and think for a minute. What are you doing here? Why are you here, really?

No, it won't. If no action were taken people will just need to upgrade to 0.8.

[alexkravets]

There is another 10+ hidden genetic flaws either in the C++ code itself or in one of the underlying libraries.

Imagine that next time instead of a fork split that can be healed, we end up leaking private key information through some side channel because of some bug in SSL libraries and the leakage gets embedded in the blockchain somehow ?

Such leakage cannot be repaired ... that will NOT be fixable and will simply lead to loss of confidence and extinction.

There's hundreds of deadly scenarios you can imagine here ...

Is anybody in the blessed circle of C++ magicians listening to this ?  

Bitcoin Foundation, is there any plan to extract the spec ?

[🏰 TradeFortress 🏰]

There is another 10+ hidden genetic flaws either in the C++ code itself or in one of the underlying libraries.

Imagine that next time instead of a fork split that can be healed, we end up leaking private key information through some side channel because of some bug in SSL libraries and the leakage gets embedded in the blockchain somehow ?

Such leakage cannot be repaired ... that will NOT be fixable and will simply lead to loss of confidence and extinction.

There's hundreds of deadly scenarios you can imagine here ...

Is anybody in the blessed circle of C++ magicians listening to this ?  

Bitcoin Foundation, is there any plan to extract the spec ?

SOMEHOW? Private keys? They're all on your client and are never transferred anywhere else.

[alexkravets]

SOMEHOW? Private keys? They're all on your client and are never transferred anywhere else.

Read up on http://en.wikipedia.org/wiki/Side_channel_attack ...

[markm]

seems like a pretty clear-cut case of insufficient test coverage.

when testing code, it is especially important to test the corner cases. this sounds like one of those corners, even without looking at the details of the issue.

Yeah but apparently bitcoin millionaires cannot afford to get test suites made so it kind of sits there waiting for hard working coders and developers to find some spare time to work on such things.

-MarkM-

15:49  gavinandresen ACTION resists the urge to talk about max block size??? must??? get??? work??? done...

Spare me the circlejerking. If there's time to be flaming Bitcoin users because they're not using Bitcoin "correctly" then there is time to stfu, swallow cox and work the testnet to the bone.

No problem, I'll leave Gavin to it and get on with doing other stuff that isn't already on the paid developer's to-do list.

LIke altcoins maybe, since bitcoin is already on the paid developer's to-do list.

Hmm, or Open Transactions maybe. So much to do, so little time to do it all.

-MarkM-

[niko]

Another way to state the real problem: There is no Bitcoin Protocol Spec, most semantics buried in the hairball of the C++ reference implementation

That would be correct. It's a huge issue.

It is also a huge issue that those suggesting we need the protocol spec are brushed off with the "you don't get it" nonsense by elitist devs.
Bits and pieces, some expired, are scattered around Wiki. Is that the best we can do?
@MP: would you be willing to lead the effort of formulating the current protocol spec?

[markm]

Lets drop the bug for bug compatible idea too while we're at it.

Tolerate any existing buggy blocks over 120 blocks old or that predate the last hardcoded checkpoint or something but don't make a religion of emulating the bugs going forward.

BDB shouldn't even be part of the spec, the spec should abstract the actual database implementation details and just maybe talk about whether it should have atomic transactions, whether we should be able to roll those back, under what circumstances to roll them back and so on.

Enshrining some bug in some particular implementation of "database back end" should be anathema, our spec should basically be telling us "BDB isn't up to spec, don't use versions of it that aren't able to perform as we need them to".

-MarkM-