Why the hell this shit was not tested on testnet?

[misterbigg]

Different versions and distros of operating systems likely linked against different versions of Berkeley DataBase too so maybe it would have needed each old version of bitcoin, with each compatible version of the BDB linked to each one

This is why I try to build only stand-alone repositories for my applications. I don't link to external libraries and I don't use submodules. Any external libraries that I use I bring in using git-subtree which makes a complete copy of the sources and inserts it into my repository. This way there is no possible way that anything can be screwed up in the manner you described. What gets built is the same for all machines.

The usual objection from GNU/Linux diehards is that I am duplicating the storage for a lot of libraries and I'm creating a large executable by avoiding shared libraries (or DLLs on Windows) but in practice it is more convenient and less risky.

You'll note that every single repository linked in my signature can be built flawlessly from a single "git clone", without having to obtain any other source trees, even though I frequently make use of multiple open source libraries (Lua, FreeType, JUCE, SQLite, TagLib to name a few).

Reference Bitcoin client source code repository should git-subtree the entire source tree of Berkeley DB (and all of its dependencies), and statically link the BDB implementation directly into the executable.

[1714175127]

1714175127

[1714175127]

1714175127

[1714175127]

1714175127

[1714175127]

1714175127

[1714175127]

1714175127

[markm]

Is there any truth in the suggestion that it was actually caused by not configuring the database correctly to match the usage pattern the app intended to use it in?

I know with e.g. MySQL one has to consider its config when planning what kind of load you plan to put on it what kind of data patterns you expect and so on.

-MarkM-

[solex]

incompatibility between versions should be called (at least) an issue imho...

My point was that exhaustive testing of 0.8 would never have revealed the bug in 0.7 that nobody knew about.

Unit testing to make sure the code was actually capable of operating at the protocol limits would have caught the problem had it been performed on 0.7

Indeed, the testing should have been conducted in the three years since Bitcoin was released. It's appalling how the edge conditions were never tested.

Not sure backward compatibility is an edge case.... generally its a big deal you plan a transition for.  What happened here was a rushed satoshi dice bailout with much discussion on filtering out their transactions while a real solution could be worked on.... which now leaves everyone feeling the pain instead of a single entertainment site.

The edge condition is referring to the blocks that were rejected even before 0.8 was released. A perfectly valid block could be rejected by every Bitcoin node. Isn't this something that should never happen?

A block rejected by all nodes is by definition invalid. Someone wil produce a new valid one, and the building will continue as normal.

Is there any evidence that all nodes would reject it? It looks like, according to Pieter's preliminary statement, that certain configurations will (correctly) accept the blocks.

Immediate solution is upgrading to 0.8, or manually setting the number of
lock objects higher in your database. I'll follow up with more concrete
instructions.

It only takes 51% of the hashing power to decide which is the next valid block. So the "certain configurations" above can find themselves in the minority; too bad for them. This is the way it has to work.

[DeathAndTaxes]

Too bad v0.7 was incompatible with v0.8 then right?  Developers should have just recommend pushing ahead and fuck those who hadn't upgraded.  You are aware that v0.8 had the majority of the hashing power right?

[solex]

Too bad v0.7 was incompatible with v0.8 then right?  Developers should have just recommend pushing ahead and fuck those who hadn't upgraded.  You are aware that v0.8 had the majority of the hashing power right?

No. I was not aware of the split in hashing power. Personally, I agree 100% that they should have pressed on with v0.8, with that knowledge. However, as I am not involved in the dev sphere, then I am going to defer to their collective judgement on the best course of action, which was to revert.

[justusranvier]

Personally, I agree 100% that they shoudl have pressed on with v0.8, with that knowledge. However, as I am not involved in the dev sphere, then I am going to defer to their collective judgement on the best course of action, which was to revert.

+1

[gyverlb]

Personally, I agree 100% that they shoudl have pressed on with v0.8, with that knowledge. However, as I am not involved in the dev sphere, then I am going to defer to their collective judgement on the best course of action, which was to revert.

+1

If they did that, all 0.7 clients would have been left dead in the water, unable to rejoin the main chain and forcing end users to upgrade on the spot. 0.8 clients can rejoin the 0.7 chain so this was why it was preferred.

[justusranvier]

If they did that, all 0.7 clients would have been left dead in the water, unable to rejoin the main chain and forcing end users to upgrade on the spot. 0.8 clients can rejoin the 0.7 chain so this was why it was preferred.

In hindsight I bet it will turn out that sticking with 0.8 would have resulted in a more rapid resolution. 0.8 already had a majority of hashing power, and now that some of the miners have downgraded it looks like the 0.7 chain is ahead but just barely. We spent several hours in the worst case of a near 50/50 split in hashing power and it's taking a long time for the new branch to pull ahead.

[gyverlb]

If they did that, all 0.7 clients would have been left dead in the water, unable to rejoin the main chain and forcing end users to upgrade on the spot. 0.8 clients can rejoin the 0.7 chain so this was why it was preferred.

In hindsight I bet it will turn out that sticking with 0.8 would have resulted in a more rapid resolution.

I'm not sure. The only way to resolve this was to force *all* 0.7 users to upgrade. Here only 2 pool ops (slush and Eleuthria) could single-handedly put the 0.7 hashrate ahead by downgrading their bitcoind nodes which will solve the problem for everybody in the following hours.

[solex]

If they did that, all 0.7 clients would have been left dead in the water, unable to rejoin the main chain and forcing end users to upgrade on the spot. 0.8 clients can rejoin the 0.7 chain so this was why it was preferred.

In hindsight I bet it will turn out that sticking with 0.8 would have resulted in a more rapid resolution. 0.8 already had a majority of hashing power, and now that some of the miners have downgraded it looks like the 0.7 chain is ahead but just barely. We spent several hours in the worst case of a near 50/50 split in hashing power and it's taking a long time for the new branch to pull ahead.

Maybe, but a fork has been averted, just a reorg event now. This is a victory in itself.

More importantly, this is a real lesson for what happens if Bitcoin comes racing up to the 1Mb block size limit and has to do an emergency change. It will be a train wreck indeed.
It clearly requires a seriously long parallel-run period, with a hand-holding "help-desk" using a LOT of cajoling and encouragement to get absolutely as many nodes, of all types, upgrading before the 1st 1Mb block gets mined.

[DeathAndTaxes]

If they did that, all 0.7 clients would have been left dead in the water, unable to rejoin the main chain and forcing end users to upgrade on the spot. 0.8 clients can rejoin the 0.7 chain so this was why it was preferred.

In hindsight I bet it will turn out that sticking with 0.8 would have resulted in a more rapid resolution.

I'm not sure. The only way to resolve this was to force *all* 0.7 users to upgrade. Here only 2 pool ops (slush and Eleuthria) could single-handedly put the 0.7 hashrate ahead by downgrading their bitcoind nodes which will solve the problem for everybody in the following hours.

Plus there was no risk in downgrading but given enough time eventually all 0.7 users would be vulnerable to a nasty and no hashpower double spend attack.  That would have created chaos, confusion, and damaged the credibility of the network.

[zebedee]

Too bad v0.7 was incompatible with v0.8 then right?  Developers should have just recommend pushing ahead and fuck those who hadn't upgraded.  You are aware that v0.8 had the majority of the hashing power right?

I tend to agree with this, though the devs have more reason to be conservative than me

[whitenight639]

Maybe the alert system should be used to force client upgrades, it is abit draconian but we can't risk the whole premise and validity of bitcoin because some people are slow to upgrade, Am I right in thinking that if everyone had upgraded straight away this wouldnt have happened?  People are always free to create there own alt coin with the bitcoin source are they not?

This though risks the communities decision to decide what changes in the client software they accept and implement, but it secures the collective currency. do everybodies funds would be safe untill they moved or converted them to an alt coin or differing implementation of bitcoin. I think we have risks of both a hard fork and soft fork when we should only have to worry about one if I understand the hard / soft analogies properly.


Is it true that the Satoshi dice filtering script is / could be the cause of this?

If the bug in the 0.7 clients is creating blocks that are too large or broken for other 0.7 clients then why have we been advised to downgrade, if 0.8 clients can accept these blocks (from the bugged out 0.7 clients) Then wouldn't a better idea have been to urge everyone to upgrade to 0.8 and issue an alert to clients telling them if they do not upgrade they may not be able to spend there bitcoin? or are there other factos at play?
As it stands it looks like the bug affecting 0.7 could still appear only with no tolerant 0.8 clients around??


please help me understand.


I think the devs should be using extensive use of stickies and polls before updates that change the block limit and extensive announcements should be made about upcoming upgrades and there benefits.

I'd like to see this mess fixed and a swift move to 8.01 with the 1mb hard limit removed, if this will result in faster confirm times, less satoshi dice ridiculessness and hope that the whole thing won't come crashing down after we get to some "limit". although i still don't fully understand the implications of the block size restriction I am going to digg up the thread discussing it.

[markm]

It was 0.8 that made, and found perfectly good, the block that caused the fork.

Some builds of 0.7, on some platforms, used implementations of Berkeley DataBase that crapped out trying to process the block.

Unmodified 0.7 nodes would never have produced such a block, which is why during the 0.7 era it was never noticed that a perfectly valid block could crap out some builds / implementations / configurations of Berkeley DataBase.

0.8 Does not use Berkely DataBase for that data (thought it does still use it for wallet.dat if I recall correctly but that is not the DB that crapped out) so it too would never have noticed this problem some cases of BDB exhibit on some platforms.

-MarkM-

[DeathAndTaxes]

Maybe the alert system should be used to force client upgrades, it is abit draconian but we can't risk the whole premise and validity of bitcoin because some people are slow to upgrade

Worst idea in the last 24 hours.  That is pretty amazing given the pretty much nonstop run of bad ideas.  There will not and should not EVER be a forced upgrade option for Bitcoin.  Not today, not tomorrow, not a century from now.  Your idea would be a "kill bitcoin" button available to anyone with the resources (either financial or through use of force) to push it.

[Killdozer]

Seriously, you made TWO major changes at once (block size and database type)

It is really not as serious as you make it sound here.
1. They didn't make the block size change. It is a "soft" setting which can be changed in separate miners. This cannot split the chain, this was a setting waiting to be raised when the right time came, and it came when the transactions got too many. So it was changed exactly when needed. This is perfectly fine.
2. Changing of the database actually solved a bug in Bitcoin! It's not the new code that produced faulty blocks or something like that, it was a bug in the old code that could'nt handle perfectly legitimate blocks. Even this is very good for the long run, even if combined with the rest of the events here cause the split.
3. This was all due to a very obscure bug in the 3d party code. Not all bugs can be found by testing. Even if the developers were the world's best coders, things like this could still happen. They handled the situation very professionally, finding the best solution fast.

[solex]

Maybe the alert system should be used to force client upgrades, it is abit draconian but we can't risk the whole premise and validity of bitcoin because some people are slow to upgrade

Worst idea in the last 24 hours.  That is pretty amazing given the pretty much nonstop run of bad ideas.  There will not and should not EVER be a forced upgrade option for Bitcoin.  Not today, not tomorrow, not a century from now.  Your idea would be a "kill bitcoin" button available to anyone with the resources (either financial or through use of force) to push it.

That's not quite right D&T because we already have a forced upgrade situation imminent!

http://blockorigin.pfoe.be/top.php

When the percentage at the bottom of that screen goes from 78% to 95% then 5% of the hashing power is forced to upgrade or cast adrift.
However, IMHO, this is the only way Bitcoin can progress from a crypto-experiment to a world-beating currency and payments system.

[nimda]

Maybe the alert system should be used to force client upgrades, it is abit draconian but we can't risk the whole premise and validity of bitcoin because some people are slow to upgrade

Worst idea in the last 24 hours.  That is pretty amazing given the pretty much nonstop run of bad ideas.  There will not and should not EVER be a forced upgrade option for Bitcoin.  Not today, not tomorrow, not a century from now.  Your idea would be a "kill bitcoin" button available to anyone with the resources (either financial or through use of force) to push it.

That's not quite right D&T because we already have a forced upgrade situation imminent!

http://blockorigin.pfoe.be/top.php

When the percentage at the bottom of that screen goes from 78% to 95% then 5% of the hashing power is forced to upgrade or cast adrift.
However, IMHO, this is the only way Bitcoin can progress from a crypto-experiment to a world-beating currency and payments system.

There is a difference between that and the aforementioned force upgrade. One gives you the "or cast adrift" option; the other changes the software on your computer. One is beneficial, the other is a "kill bitcoin" button.

[solex]

Maybe the alert system should be used to force client upgrades, it is abit draconian but we can't risk the whole premise and validity of bitcoin because some people are slow to upgrade

Worst idea in the last 24 hours.  That is pretty amazing given the pretty much nonstop run of bad ideas.  There will not and should not EVER be a forced upgrade option for Bitcoin.  Not today, not tomorrow, not a century from now.  Your idea would be a "kill bitcoin" button available to anyone with the resources (either financial or through use of force) to push it.

That's not quite right D&T because we already have a forced upgrade situation imminent!

http://blockorigin.pfoe.be/top.php

When the percentage at the bottom of that screen goes from 78% to 95% then 5% of the hashing power is forced to upgrade or cast adrift.
However, IMHO, this is the only way Bitcoin can progress from a crypto-experiment to a world-beating currency and payments system.

There is a difference between that and the aforementioned force upgrade. One gives you the "or cast adrift" option; the other changes the software on your computer. One is beneficial, the other is a "kill bitcoin" button.

That's why I said "not quite right" rather than flatly refute. There are a lot of perspectives to the argument.
Of course I reject forced changes of software as you (and whitenight) describe.