Scalability: Fast header distribution and speculative mining

[Minthos]

Today, blocks are distributed fast enough. If we scale to 10k TPS, blocks will need to be about 2 GB, right? Then it will take non-trivial time to transmit blocks over slowish links. That would put miners/pools based in countries with shitty internet connections at a disadvantage.

One solution is for the pool to have a "proxy" server with good connectivity in Europe or USA. Forcing pools to have a presence in certain regions other than where their miners stinks of centralization, so I think a better solution is needed.

My idea is to broadcast a the block header when a block is found, and broadcast the full block immediately afterwards. When a pool receives the header of a recent block that builds on the current block and meets the difficulty target, it can start speculatively hashing on top of that header without needing the whole block. The speculative hashing should be done with a special block containing only the block reward payout, to avoid including transactions with spent outputs.

If it finds a valid nonce before it receives the whole block, it must withhold it until it has the whole block and can verify the transactions in it.
If it does not find a valid nonce before receiving the whole block, it can discard the speculative block and resume normal mining with transactions included.
If it does not receive the whole block before a reasonable amount of time has passed (unlikely but not impossible) it should discard the header and resume normal mining.

I think the risk of receiving a header for an invalid block will be small enough that miners can safely mine speculatively. Since the header contains the proof of work, trolling miners by publishing a header for an invalid block or by withholding the block will be very expensive.

I don't know if this is a new idea or not, and I realize it's not a perfect solution. But I think it's better than nothing and it's one more optimization that can improve scalability. Please let me know what you think.

[1715393630]

1715393630

[1715393630]

1715393630

[d'aniel]

Today, blocks are distributed fast enough. If we scale to 10k TPS, blocks will need to be about 2 GB, right? Then it will take non-trivial time to transmit blocks over slowish links. That would put miners/pools based in countries with shitty internet connections at a disadvantage.

Miners can ensure that their peers know in advance about the txs they're mining.  Then only the first few bytes of the txids would be necessary to uniquely identify them, and this is all that would need to be broadcast in bursts when blocks are found.  Broadcasting the first say 5 bytes of the txid, instead of the full ~350 byte transaction, means a 98.6% reduction in necessary bandwidth.  So 2GB -> 29MB.

[anti-scam]

Hopefully by the time Bitcoin gets to 2 GB a block we'll see some features like mini-blockchains, pruning, and trustless off-chain transactions be finalized. That much storage is prohibitively expensive for a sufficiently diversified network even if you can transmit it quickly enough.

[Minthos]

Today, blocks are distributed fast enough. If we scale to 10k TPS, blocks will need to be about 2 GB, right? Then it will take non-trivial time to transmit blocks over slowish links. That would put miners/pools based in countries with shitty internet connections at a disadvantage.

Miners can ensure that their peers know in advance about the txs they're mining.  Then only the first few bytes of the txids would be necessary to uniquely identify them, and this is all that would need to be broadcast in bursts when blocks are found.  Broadcasting the first say 5 bytes of the txid, instead of the full ~350 byte transaction, means a 98.6% reduction in necessary bandwidth.  So 2GB -> 29MB.

Ah. This is good!