How much slower is a big block propagating than a small block?

[SebastianJu]

There is a tip that not including transactions in a block will help propagate the block faster because of the size. Even though i dont know that this was done already it looks like many miners exclude transactions with no and low fee, probably for that reason.

Now my question is... is the slower propagation of big blocks only because it takes a bit longer to transfer the block or is there another waiting time until that block is checked for some reason. So that it might take longer to check if that block is valid until it spreads.

I mean nothing holds a miner from, for example only include transactions he gets a decent fee from. If this helps him propagating faster it might be a good move for him. But it wouldnt for the network.

So is there a way to make it more fair somehow so that big blocks have the same speed of propagation (on the assumption they are similiar connected to the network) like small blocks? A found block has to be transferred in full, checked in full and sent to other nodes in full again without a chance of speeding it up? I guess sending a block forward before it was fully received and proofed might mean a security risk of ddosing the network.

[gmaxwell]

It takes longer to transmit more data, it takes longer to run more checksig operations (the latter reduced somewhat by caching, if the transactions had already been public). Thats it.

[SebastianJu]

It takes longer to transmit more data, it takes longer to run more checksig operations (the latter reduced somewhat by caching, if the transactions had already been public). Thats it.

So its inevitable that big blocks, that contain all transactions, are slower and have the risk of being orphaned? I dont think that is good for the network.

Cant this be speed up anyhow? There is no small datapart that can be checked beforehand so its known that the block is correct. Then the incoming block could be streamed already while incoming to the next nodes without the full block proof. Nothing like that possible?

[Gavin Andresen]

See:
  http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay:  sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
  http://www.verizonenterprise.com/about/network/latency/

[cunicula]

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

[DannyHamilton]

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

Wouldn't the forgone transaction fees become the orphan cost?

[Sergio_Demian_Lerner]

See:
  http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay:  sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
  http://www.verizonenterprise.com/about/network/latency/

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.


Is this correct?

[cunicula]

It is only a floor on txn fees as long as there is a subsidy. The orphan cost comes from forgone subsidy.
Take away the subsidy and the bottom falls out.

Wouldn't the forgone transaction fees become the orphan cost?

If the other txn fees are zero, then the orphan cost is also zero.
You only begin to get positive txn fees when you run up against a block size constraint.
Here, I guess gavin is saying that there is an intrinsic block size constraint even without a coded block size limit.
Running up against this constraint will force people to pay fees.
The constraint also encourages larger pools. Orphan cost will decrease in pool size.

Taken as a whole, this seems like a bad thing to me.

[Mike Hearn]

Blocks can be easily sent as full-match merkleblocks. It just wasn't a priority to do that yet.

I'd like to see real-time metrics on block propagation and such. I suspect the picture is a lot more complicated than a fixed 80msec per kb (transferring data isn't that slow and modern nodes are much faster at verifying transactions than they used to be).

[justusranvier]

1) Make the nonce long enough that the extraNonce field is no longer needed in the coinbase transaction.

2) Now it's possible for miners to broadcast their Merkle tree as soon as they start hashing (10 minutes on average before they finish)

3) When they find a valid hash, now they only need to broadcast the block header because the rest of the network has (usually) already received and validated the Merkle tree.

4) Block header propagation is very fast and not dependent of the size of the blocks.

Using this model, miners can create large Merkle trees without worrying as much about orphans, because they get to broadcast them in parallel with searching for a valid hash. Probably miners would want to limit their Merkle tree to the amount of data they can transmit in 2 minutes to make it likely that the rest of the network has received and validated it before they finish hashing.

This is a much more efficient use of bandwidth and allows final header propagation to be extremely rapid and independent of the number of transactions in a block.

Also, it may have positive implications for the Selfish Miner attack.

[cunicula]

See:
  http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf

each kilobyte in size costs an additional 80ms delay

You can calculate the "orphan cost" of including an extra 1kilobyte of transactions in your block from that, which gives a floor on reasonable transaction fees.

RE: can we decrease that delay:  sure, up to a point. But across-the-world network latency gives a physical limit.
See, for example:
  http://www.verizonenterprise.com/about/network/latency/

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.


Is this correct?

This works for an atomisitc miner, I think.
As pool size increases, however, I think that orphan costs decrease and eventully become orphan benefits. Once you are at say 60%, the failure of  your blocks to propagate is going to mean that you get more thqn 60% of block reward.

[zvs]

Right now, it's more important that you avoid requesting a block from one of the vbitcoin nodes

I imagine it's how blockchain picked up the 268083 as an orphan before receiving 268082

re: bigger blocks.  i'm sure most of these pools are on good connections, so it's probably not much of an issue.  if you're solo mining on limited bandwidth, then, yeah, you'd want to reduce the size.

[Gavin Andresen]

If this "80 msec delay" is the delay of the block being accepted by 51% of the nodes, then it means that most miners are loosing money by accepting a fee lower than 0.00333 BTC/KByte, which is the case most of the time.

This is because each 80 msec costs 25*80/1000/60/10 = 0.00333 BTC
Most of the current blocks have lower fees per Kbyte.

In turns this means that, if miners start acting rationally, a standard transaction cost should be at least USD 83 cents, with current exchange rate. Which in turns means if header-only-push is not implemented, the Bitcoin value cannot rationally be higher than 270 USD/BTC, since a 1 USD fee seems too much for most of the common payments.


Is this correct?

Yes.

I am much more worried about transaction fees right now than "selfish mining". That is why I've been working on better fee estimation in the reference implementation.

I agree with Mike-- we need better measurement, and we need to make "full-match merkle block relaying" the default to drive that 80ms number way down.

[cunicula]

I am much more worried about transaction fees right now than "selfish mining". That is why I've been working on better fee estimation in the reference implementation.

Good to see you have your priorities straight. Algorithmic fee adjusment is a very difficult problem. Consider getting input from economists. I'm a know it all, but I recognize this problem as much too difficult for me. I'm worried that it may be impossible to prevent supply from stepping in to fake demand (mining their own txn ans paying fees to themself). The only leverage you have is the miners' loss of fee revenue due to the sacrifice in block space.

I expect you need a very clever mathematical economist armed with rigorous proofs and stuff. Without rigor, the problem is going to be too complex to grasp. (Too many things are going on simultaneously to use intuition.)