Alert: chain fork caused by pre-0.8 clients dealing badly with large blocks

[nelisky]

Well talk about bad timing. Literally as the news flash hit, I sent a large amount of coin from one PC to another. The receiving PC runs 0.8 and the sending was much much older. The transaction seems to have worked, and the recipient PC is picking up confirmations, so I am hoping these transactions are all safe.

If the coins you sent were old enough (not from a block mined during the fork) you will be just fine. Even if the receiving PC is on the wrong chain in a few hours the good chain will pick up and have your transaction.

[ChanceCoats123]

PANIC SELL!!

Price has just dropped to $30...

Im buying at $29  

On which exchange?

He's bullshitting. There really isn't a big issue. In fact, I usually don't upgrade my software until there are other versions out after the one I upgrade to, so that I can avoid any issues like this. So everyone just relax.

The issue isn't with 0.8, it's with 0.7 and lower. Because people like you didn't upgrade, we had to endure this chain fork.

Yep, my 600mhash is really fucking the network.

No raindrop takes responsibility for the flood, they say. ;3

Sad but true.

[Scratch]

Sent a lodgement to a well known BTC marketplace just before this hit, from a .7.2 Beta client. Obviously gone from the client but will it ever confirm and if no can I recover em?

[notme]

PANIC SELL!!

Price has just dropped to $30...

Im buying at $29  

On which exchange?

He's bullshitting. There really isn't a big issue. In fact, I usually don't upgrade my software until there are other versions out after the one I upgrade to, so that I can avoid any issues like this. So everyone just relax.

The issue isn't with 0.8, it's with 0.7 and lower. Because people like you didn't upgrade, we had to endure this chain fork.

Yep, my 600mhash is really fucking the network.

If all the pools, miners, merchants, and users upgraded, this would have been a non-issue.

And that is true. But I am on 0.7.2, and like I said in my post, I refuse to upgrade software to 0.8 until something newer comes out, or unless my transactions will cease to work properly because I'm on 0.7.2. Trying to pin this on me is comical at best. There are hundreds of thousands of users who haven't even upgraded to 0.7.2, not to mention the ones who haven't upgraded to 0.8. So I am the least of your troubles.

Hundreds of thousands of users?  I see less than 10k nodes:
http://luke.dashjr.org/programs/bitcoin/files/charts/branches.html

[ChanceCoats123]

I stand corrected. I thought there were more than 10k nodes worldwide.

Edit: On a side note, that's pretty unsettling. This currency is supposed to play a world-wide role and only 10k computers have the bitcoin program running? 

[HappMacDonald]

Sent a lodgement to a well known BTC marketplace just before this hit, from a .7.2 Beta client. Obviously gone from the client but will it ever confirm and if no can I recover em?

Yes. This problem was with the formation of one block, and a block is naught but a container of transactions. All the same transactions are still being put into blocks by the 0.7 miners. The wallet version you run is completely unrelated to the problem, this problem only pertains to the version of mining software. AFAWCT absolutely all transactions will be handled properly, save the block rewards for the miners who mine into the losing chain of course.

[hex]

And why are we downgrading and not pushing 0.8 fork if bug is in 0.7 ?

[bitfreak!]

Will it ever be possible to upgrade to a client which can support these large blocks? Seems to me we're going to be stuck like this.

[ChanceCoats123]

And why are we downgrading and not pushing 0.8 fork if bug is in 0.7 ?

I'm also wondering this. I'll upgrade right now if that will help fix the issue.

[alir]

I smell a mass sell off along the horizon...

I sure hope so ;)

I was away from the computer for a few hours and all this happened. The bitcoin world is so exciting!

[Scratch]

Sent a lodgement to a well known BTC marketplace just before this hit, from a .7.2 Beta client. Obviously gone from the client but will it ever confirm and if no can I recover em?

Yes. This problem was with the formation of one block, and a block is naught but a container of transactions. All the same transactions are still being put into blocks by the 0.7 miners. The wallet version you run is completely unrelated to the problem, this problem only pertains to the version of mining software. AFAWCT absolutely all transactions will be handled properly, save the block rewards for the miners who mine into the losing chain of course.

*now gets it*

Slow on the uptake there, its late here ^^ Thanks for your explanation

[Grover]

And now a more elaborate explanation:

0.7 and older nodes use BDB for storing the blockchain databases. It seems this database has a limit on the size of the modification it can make atomically to the database. With the larger blocks of the past days, it seems to have triggered the limit. The result is that 0.7 (by default, it can be tweaked manually) will not accept "too large" blocks (we don't yet know what exactly causes it, but it is very likely caused by many transactions in the block). Specifically, block
000000000000015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) with >1700 transactions.

However. 0.8 (which uses a different database system) has no such limit, and happily accepts the block. As the majority of the hash power was on 0.8, the longest chain ended up using this block, which is not accepted by older nodes.

The solution is to (for now) go back to the old chain, which has block 00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430.

Just to ask a couple of questions.

Could this have been deliberate in any way in that someone, or group, figured out the weakness of the 0.7 client?

Is this just a random occurrence with some ASIC's coming online and Hash rate going up?

In my ignorant opinion, seems to me that if it was known the 0.7 had a dbase limit then that client should have been voted off the network a while ago.

For the newbie fun I just did a cash deposit on Gox.  Was like WOW look at these prices, bought then couldn't transfer to BTCe and finally got wise looking at the chat feed there.  Thankfully, I was able to back out on Gox and go back to cash at just a little loss.

[pointbiz]

Sounds like before block 225430 all coins are safe.

I'm on 0.8.0 client and block 225451... this implies at this moment the last 21 confirmations are not safe. Ugh.

So if everyone switches to 0.7 those people on 0.8.0 need to rely on more than the typical 6 confirmations.

What version is Slush's pool on?

[tkbx]

great.... just did a transaction a couple hours ago.

You'll be fine, I did one 45 minutes ago and it's cleared.

[JoelKatz]

Will it ever be possible to upgrade to a client which can support these large blocks? Seems to me we're going to be stuck like this.

Yeah, through a coordinated process. Most likely it will go like this:

1) We'll pick a block number on which to switch.

2) A new .8 version will be released the switches as of that block number.

3) It will be announced that everyone must update before that block number.

4) The switch will take place automatically.

[HappMacDonald]

And why are we downgrading and not pushing 0.8 fork if bug is in 0.7 ?

Because 0.8 can accept the blocks generated by 0.7, 0.7 cannot accept the blocks generated by 0.8.

So on the one hand, enough miners can backpedal to make sure the 0.7 chain wins until we fix the problem, meaning everyone gets to stay on the same page, or we try to lean on the 0.8 block which will leave half or more of the world's miners and nodes on the wrong side of the fork until they all upgrade.

[fcmatt]

Will it ever be possible to upgrade to a client which can support these large blocks? Seems to me we're going to be stuck like this.

Yeah, through a coordinated process. Most likely it will go like this:

1) We'll pick a block number on which to switch.

2) A new .8 version will be released the switches as of that block number.

3) It will be announced that everyone must update before that block number.

4) The switch will take place automatically.

what happens to users who do not upgrade?

and if bitcoin requires such drastic hand holding shouldn't this be programmed into the client? some type of ksplice like update?

[notme]

Will it ever be possible to upgrade to a client which can support these large blocks? Seems to me we're going to be stuck like this.

Yeah, through a coordinated process. Most likely it will go like this:

1) We'll pick a block number on which to switch.

2) A new .8 version will be released the switches as of that block number.

3) It will be announced that everyone must update before that block number.

4) The switch will take place automatically.

what happens to users who do not upgrade?

and if bitcoin requires such drastic hand holding shouldn't this be programmed into the client? some type of ksplice like update?

That would destroy any semblance of decentralization that remains.

[dree12]

Will it ever be possible to upgrade to a client which can support these large blocks? Seems to me we're going to be stuck like this.

Yeah, through a coordinated process. Most likely it will go like this:

1) We'll pick a block number on which to switch.

2) A new .8 version will be released the switches as of that block number.

3) It will be announced that everyone must update before that block number.

4) The switch will take place automatically.

what happens to users who do not upgrade?

and if bitcoin requires such drastic hand holding shouldn't this be programmed into the client? some type of ksplice like update?

They will be left in the dust, as it is their bug. They cannot claim that the fork is against Bitcoin's spirit. This is similar to a previous hard-fork where the old chain allowed negative transaction fees, except the initial remedy is reversed. The long-term remedy is the same.

[HappMacDonald]

And now a more elaborate explanation:

0.7 and older nodes use BDB for storing the blockchain databases. It seems this database has a limit on the size of the modification it can make atomically to the database. With the larger blocks of the past days, it seems to have triggered the limit. The result is that 0.7 (by default, it can be tweaked manually) will not accept "too large" blocks (we don't yet know what exactly causes it, but it is very likely caused by many transactions in the block). Specifically, block
000000000000015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) with >1700 transactions.

However. 0.8 (which uses a different database system) has no such limit, and happily accepts the block. As the majority of the hash power was on 0.8, the longest chain ended up using this block, which is not accepted by older nodes.

The solution is to (for now) go back to the old chain, which has block 00000000000001c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430.

Just to ask a couple of questions.

Could this have been deliberate in any way in that someone, or group, figured out the weakness of the 0.7 client?

Is this just a random occurrence with some ASIC's coming online and Hash rate going up?

In my ignorant opinion, seems to me that if it was known the 0.7 had a dbase limit then that client should have been voted off the network a while ago.

For the newbie fun I just did a cash deposit on Gox.  Was like WOW look at these prices, bought then couldn't transfer to BTCe and finally got wise looking at the chat feed there.  Thankfully, I was able to back out on Gox and go back to cash at just a little loss.

From what I can tell this problem was not caused by a malformed transaction, but by a malformed, mined block. If an attacker crafted a bad block, they would have to do so as a miner which would take an awful lot of patience or resources, so I don't think it was intentional.

As to why this wasn't caught in testing, that's a great question but I'll bet it's a tricky kind of bug specific to BDB that is hard to replicate if you aren't looking for it. Once the mess is cleaned up, and once we carve a clear upgrade path to 0.8 and beyond we'll be leaving BDB and it's bugfest behind, though.