Soft block size limit reached, action required by YOU

[don giovanni]

How to raise the most amount of fees is really an interesting question.

The need to raise enough fees to incentivize enough miners to protect the network is clear. The question then is, do we raise more fees by restricting the number of transactions and bidding up the cost of each transaction, or by opening up the the number of transactions to near limitless and thus having more, but smaller fees per transaction?

It's sort of like the long debated question by politicians. Do they generate more revenue by raising taxes or lowering them?

I'm sure the answer is something similar to a bell curve.

As gmaxwell alluded to earlier, being stuck at the current limit isn't the answer, and neither is allowing infinite block sizes. There needs to be an algorithm that equitably adjusts the rate of increase of block sizes. I think that would be a much more worthwhile debate than banging our heads against the wall worrying about what constitutes spam and what doesn't.

Actually it's pretty simple. Adjust the block size to allow for a steady 10% bandwidth use of the average broadband speed across all major countries: http://www.netindex.com/download/allcountries/
Which right now would be about 1Mbit/s, which approximately translates into 100MB blocks.

A 1Mbps connection is barely viable for mining 1MB blocks. You would spend >3% of your time downloading/uploading blocks. At 30MB blocks, you will spend ~100% of your time downloading and uploading your block, forget mining.

To mine, you want to keep your time waste in block propogation to a minimum. If you are aiming for <1%, you need to be able to download and upload blocks within 6s.

Queue?

[1715487497]

1715487497

[1715487497]

1715487497

[1715487497]

1715487497

[1715487497]

1715487497

[1715487497]

1715487497

[Jutarul]

Actually it's pretty simple. Adjust the block size to allow for a steady 10% bandwidth use of the average broadband speed across all major countries: http://www.netindex.com/download/allcountries/
Which right now would be about 1Mbit/s, which approximately translates into 100MB blocks.

A 1Mbps connection is barely viable for mining 1MB blocks. You would spend >3% of your time downloading/uploading blocks. At 30MB blocks, you will spend ~100% of your time downloading and uploading your block, forget mining.

To mine, you want to keep your time waste in block propogation to a minimum. If you are aiming for <1%, you need to be able to download and upload blocks within 6s.

First you need to distinguish between a validation node and a mining node. For a mining node you can expect that they invest the necessary resources to have at least a 10x above average bandwidth.

A validation node is necessary for selecting the proper fork of the bitcoin blockchain. It gives you an economic voting power in case of a hard fork. And we need to keep that power in the hands of the masses.

[Nagato]

Actually it's pretty simple. Adjust the block size to allow for a steady 10% bandwidth use of the average broadband speed across all major countries: http://www.netindex.com/download/allcountries/
Which right now would be about 1Mbit/s, which approximately translates into 100MB blocks.

A 1Mbps connection is barely viable for mining 1MB blocks. You would spend >3% of your time downloading/uploading blocks. At 30MB blocks, you will spend ~100% of your time downloading and uploading your block, forget mining.

To mine, you want to keep your time waste in block propogation to a minimum. If you are aiming for <1%, you need to be able to download and upload blocks within 6s.

First you need to distinguish between a validation node and a mining node. For a mining node you can expect that they invest the necessary resources to have at least a 10x above average bandwidth.

A validation node is necessary for selecting the proper fork of the bitcoin blockchain. It gives you an economic voting power in case of a hard fork. And we need to keep that power in the hands of the masses.

A validation node is just that, a validation node. It allows the person running it to see what is going on in the network and whether or not the current blockchain is following the rules which are set out in his version of the client.

A validation node has no voting power whatsoever, it may at best refuse to relay blocks/txns it finds invalid. Ultimately the people who decide what rules the network is based on are the miners alone. This is why decentralisation of miners should not be taken lightly. If the hard fork makes p2pool unviable for many miners, it's not a good sign.

[Peter Todd]

A validation node is just that, a validation node. It allows the person running it to see what is going on in the network and whether or not the current blockchain is following the rules which are set out in his version of the client.

A validation node has no voting power whatsoever, it may at best refuse to relay blocks/txns it finds invalid. Ultimately the people who decide what rules the network is based on are the miners alone. This is why decentralisation of miners should not be taken lightly. If the hard fork makes p2pool unviable for many miners, it's not a good sign.

It's a bit more complex than that. Miners can't directly force anyone to accept the blocks they mine - if they break the rules the blocks aren't Bitcoin.

But if block sizes grow to the point where running a validation node is expensive the reality is the group of people who have full copies of the set of all unspent transactions, the UTXO set, essentially control Bitcoin. If there are 50,000 copies of the UTXO set it's almost inconceivable that anyone could monopolize control over it; if there are 1000 copies it's conceivable, if there are 100 copies it's downright likely. (a situation pretty much like the central banking system)

It's especially nasty because once you're at that point the remaining validating nodes and mining pools have an easy time getting together and deciding to change the rules of Bitcoin, and even though they are forcing a hard-fork, what choice will you have but to install a new client and accept it? Without the UTXO set data you can't transact your Bitcoins. All this clever cryptography stuff we've come up with for proving inflation and what not is rendered useless if the central mining pools and validating nodes decide to turn the features off.


An interesting parallel I think is actually search engine data. You need a truly enormous investment in hardware to store the indexes required to run an internet search engine. Sure you could try to start your own, but without a full index, what's the point? What do you know, Google, Bing and Yahoo are pretty much the only players in that game.

[Mike Hearn]

If in the future we have double-spend attacks happening "from time to time" I think Bitcoin would have failed to find a balance good enough to stay secure. Not enough base difficulty to make double-spend attacks unfeasible??

If trust is necessary then that's the end of it.

Well, you can gain trust in many ways. Bitcoin doesn't actually eliminate "trust" entirely regardless of how we may present it to the world, it just means you trust the majority consensus to not gang up on you and double-spend against you, instead of trusting an institution. In practice this seems to be always better than trusting an institution. But it's still trust.

Other ways you can gain trust - reputation of the counterparty, use of secure hardware, ability to find the guy who double-spent you and throw them in jail.

I think how often we see double spends happen, if ever, is one of the most interesting open questions in Bitcoin right now. It seems clear that some merchants have a higher tolerance for double-spending than others. Where the equilibrium ends up being is, I think, unknowable today.

[muyuu]

If in the future we have double-spend attacks happening "from time to time" I think Bitcoin would have failed to find a balance good enough to stay secure. Not enough base difficulty to make double-spend attacks unfeasible??

If trust is necessary then that's the end of it.

Well, you can gain trust in many ways. Bitcoin doesn't actually eliminate "trust" entirely regardless of how we may present it to the world, it just means you trust the majority consensus to not gang up on you and double-spend against you, instead of trusting an institution. In practice this seems to be always better than trusting an institution. But it's still trust.

Other ways you can gain trust - reputation of the counterparty, use of secure hardware, ability to find the guy who double-spent you and throw them in jail.

I think how often we see double spends happen, if ever, is one of the most interesting open questions in Bitcoin right now. It seems clear that some merchants have a higher tolerance for double-spending than others. Where the equilibrium ends up being is, I think, unknowable today.

Having to deal with reputation on the side of the buyer absolutely kills the anonymity factor in Bitcoin.

But then again I don't see double-spending happening "from time to time", unless the network itself is clearly weakened from the start. At most I see a PoC happening under very special circumstances. The economics of carrying it out seem extremely self-defeating unless valuation/difficulty ratio was orders of magnitude off and someone was able to carry out a very expensive operation in a very sophisticated con, while keeping 50%+ of the hashing power under his control for long enough. I will believe it when I see it.

[niko]

By default Bitcoin will not created blocks larger than 250kb even though it could do so without a hard fork. We have now reached this limit. Transactions are stacking up in the memory pool and not getting cleared fast enough.

What this means is, you need to take a decision and do one of these things:

• Start your node with the -blockmaxsize flag set to something higher than 250kb, for example -blockmaxsize=1023000. This will mean you create larger blocks that confirm more transactions. You can also adjust the size of the area in your blocks that is reserved for free transactions with the -blockprioritysize flag.
• Change your nodes code to de-prioritize or ignore transactions you don't care about, for example, Luke-Jr excludes SatoshiDice transactions which makes way for other users.
• Do nothing.

If everyone does nothing, then people will start having to attach higher and higher fees to get into blocks until Bitcoin fees end up being uncompetitive with competing services like PayPal.

If you mine on a pool, ask your pool operator what their policy will be on this, and if you don't like it, switch to a different pool.

Ahem. How much thought and work has gone into offering the first option in this list?

[pyra-proxy]

By default Bitcoin will not created blocks larger than 250kb even though it could do so without a hard fork. We have now reached this limit. Transactions are stacking up in the memory pool and not getting cleared fast enough.

What this means is, you need to take a decision and do one of these things:

• Start your node with the -blockmaxsize flag set to something higher than 250kb, for example -blockmaxsize=1023000. This will mean you create larger blocks that confirm more transactions. You can also adjust the size of the area in your blocks that is reserved for free transactions with the -blockprioritysize flag.
• Change your nodes code to de-prioritize or ignore transactions you don't care about, for example, Luke-Jr excludes SatoshiDice transactions which makes way for other users.
• Do nothing.

If everyone does nothing, then people will start having to attach higher and higher fees to get into blocks until Bitcoin fees end up being uncompetitive with competing services like PayPal.

If you mine on a pool, ask your pool operator what their policy will be on this, and if you don't like it, switch to a different pool.

Ahem. How much thought and work has gone into offering the first option in this list?

apparently not much, they went with the kick the can down the road approach and look where it got them.....

[chiropteran]

this will not fork the block chain

[dree12]

By default Bitcoin will not created blocks larger than 250kb even though it could do so without a hard fork. We have now reached this limit. Transactions are stacking up in the memory pool and not getting cleared fast enough.

What this means is, you need to take a decision and do one of these things:

• Start your node with the -blockmaxsize flag set to something higher than 250kb, for example -blockmaxsize=1023000. This will mean you create larger blocks that confirm more transactions. You can also adjust the size of the area in your blocks that is reserved for free transactions with the -blockprioritysize flag.
• Change your nodes code to de-prioritize or ignore transactions you don't care about, for example, Luke-Jr excludes SatoshiDice transactions which makes way for other users.
• Do nothing.

If everyone does nothing, then people will start having to attach higher and higher fees to get into blocks until Bitcoin fees end up being uncompetitive with competing services like PayPal.

If you mine on a pool, ask your pool operator what their policy will be on this, and if you don't like it, switch to a different pool.

Ahem. How much thought and work has gone into offering the first option in this list?

Don't blame this option, blame the lack of testing all Bitcoin versions have gone through. If a chain-splitting bug was present since 2009, and only resolved "by accident", you can hardly blame the advice banking on there being no such bug.

[niko]

My paranoid concern is that lots of thought has gone into recommending an option that is incompatible with BDB. Mike, I understand this comment is most likely bullshit, but you should understand it is legitimate. Even if bullshit, it teaches us a valuable lesson, and that is to not trust people who rush is into tweaking the system without due dilligence. That is exactly what happened, regardless of Mike's intentions.

[dree12]

My paranoid concern is that lots of thought has gone into recommending an option that is incompatible with BDB. Mike, I understand this comment is most likely bullshit, but you should understand it is legitimate. Even if bullshit, it teaches us a valuable lesson, and that is to not trust people who rush is into tweaking the system without due dilligence. That is exactly what happened, regardless of Mike's intentions.

For something as important as Bitcoin, testing is crucial. 0.8 didn't even have a second release candidate before it went live.

But sometimes we have to realize that cohesion is even more important. If something is rushed in, like 0.8, we should immediately all switch to it. In fact, 0.8 didn't introduce any new bug. Instead, it revealed a bug present in all prior versions of Bitcoin. Merchants and miners should have upgraded immediately after release, rather than delay the upgrade so late.

[rudrigorc2]

My paranoid concern is that lots of thought has gone into recommending an option that is incompatible with BDB. Mike, I understand this comment is most likely bullshit, but you should understand it is legitimate. Even if bullshit, it teaches us a valuable lesson, and that is to not trust people who rush is into tweaking the system without due dilligence. That is exactly what happened, regardless of Mike's intentions.

For something as important as Bitcoin, testing is crucial. 0.8 didn't even have a second release candidate before it went live.

But sometimes we have to realize that cohesion is even more important. If something is rushed in, like 0.8, we should immediately all switch to it. In fact, 0.8 didn't introduce any new bug. Instead, it revealed a bug present in all prior versions of Bitcoin. Merchants and miners should have upgraded immediately after release, rather than delay the upgrade so late.

Today we see the WARNING RED LETTERS on all the big sites and news.

 - If its healthy to upgrade, why the hell we dont get same media space and attention from system operators to tell everyone they should do it? thats a shame.

[niko]

In fact, 0.8 didn't introduce any new bug. Instead, it revealed a bug present in all prior versions of Bitcoin.

0.8 did not reveal the bug in all prior versions. Suggestion to increase the soft limit revealed the bug.

[Palmdetroit]

this will not fork the block chain

[Mike Hearn]

I suppose you forgot already why this thread started - we ran out of block space and transactions started stacking up, unconfirmed. This almost immediately caused lots of user complaints.

Exactly because people did NOT upgrade to Bitcoin 0.8 fast enough, which fixed the bug in 0.7 and below, now we are now back to square one - stuck with blocks that are too small to meet demand for transactions. Expect the complaints to start again soon unless filtering of SD becomes common.

The options I listed in my first post are still the only options. If you don't want to filter out SD transactions then the default "do nothing" option means that transactions will become very expensive, and small quantities of Bitcoin won't be spendable at all.

The fork is certainly a big problem: unfortunately, nobody realized it would happen. It's not even clear yet what kind of testing would have revealed this issue. Simply making big blocks is apparently not enough to trigger it. That said, we knew that BDB was generally a rats nest of weird problems which is one reason why I ported Bitcoin to LevelDB. If the block sizes had been raised a few months from now, we'd probably have just abandoned the 0.7 users to their fate, but we got unlucky with the timing.

Assuming we want Bitcoin to scale up, we'll have to fork 0.7 and lower off the chain sooner rather than later and then raise the block size limits again.

[dree12]

I suppose you forgot already why this thread started - we ran out of block space and transactions started stacking up, unconfirmed. This almost immediately caused lots of user complaints.

Exactly because people did NOT upgrade to Bitcoin 0.8 fast enough, which fixed the bug in 0.7 and below, now we are now back to square one - stuck with blocks that are too small to meet demand for transactions. Expect the complaints to start again soon unless filtering of SD becomes common.

The options I listed in my first post are still the only options. If you don't want to filter out SD transactions then the default "do nothing" option means that transactions will become very expensive, and small quantities of Bitcoin won't be spendable at all.

The fork is certainly a big problem: unfortunately, nobody realized it would happen. It's not even clear yet what kind of testing would have revealed this issue. Simply making big blocks is apparently not enough to trigger it. That said, we knew that BDB was generally a rats nest of weird problems which is one reason why I ported Bitcoin to LevelDB. If the block sizes had been raised a few months from now, we'd probably have just abandoned the 0.7 users to their fate, but we got unlucky with the timing.

Assuming we want Bitcoin to scale up, we'll have to fork 0.7 and lower off the chain sooner rather than later and then raise the block size limits again.

Testing compatibility with a bug in a previous version would require knowing about a bug in the first place, so I assume that the testing should have been conducted on older versions of Bitcoin, not 0.8.

[porcupine87]

Personally, I see the miners who are refusing all transactions (so their blocks propagate faster, less chance of orphans), who are only mining for the reward, completely ignoring that the purpose of mining is to process transactions in the first place, as being a bigger problem than Satoshi Dice is.

As long as Satoshi Dice's transactions are following the rules, that's all that matters to me. I don't care what the purpose the transactions are for. Not my business.

Look at the block explorer from time to time. I see plenty of found blocks that have 0 transactions in it, just the reward generation.

That is the big problem.

-- Smoov

Hm, I think, that the fees which are currently paid, are much too low. When I look at blocks with 250kb, they often have a Transaction volume of 10 000BTC and the fees summed up are only 0.5BTC. This is 0.005%. The other reward is 25BTC, which is 50 times more than the transaction fees. When the transactionsfees succeed over the coin base bitcoins, it would not be profitable anymore to exclude fee-based transaction...

[muyuu]

The options I listed in my first post are still the only options. If you don't want to filter out SD transactions then the default "do nothing" option means that transactions will become very expensive, and small quantities of Bitcoin won't be spendable at all.

Not really.

Just expensive enough so SD is not viable. Say, 0.03 or so. For a moderate quantity it's not expensive. For micropayments, BTC is never going to scale to global scale as to accommodate micropayments. It's simple math really...

[SgtSpike]

The options I listed in my first post are still the only options. If you don't want to filter out SD transactions then the default "do nothing" option means that transactions will become very expensive, and small quantities of Bitcoin won't be spendable at all.

Not really.

Just expensive enough so SD is not viable. Say, 0.03 or so. For a moderate quantity it's not expensive. For micropayments, BTC is never going to scale to global scale as to accommodate micropayments. It's simple math really...

I don't think BTC is ready for fees so high.  That's going to shut out a lot of commerce, not just SD.