Idea/Proposal: Dramatically increase the bitcoin block propagation speed

[arnoudk]

Hello all.

We’d like to share an idea we have to dramatically increase the bitcoin block propagation speed after a new block has been mined for the first time.

Efficient bitcoin block propagation
A proposed solution to provide near-instantaneous block propagation on the bitcoin network, even with slow network connections or large block sizes. Increasing mining efficiency for everyone while decreasing transaction confirmation times and strengthening the distributed nature of bitcoin.

Short summary: we propose to introduce bitcoin-backed guarantees (“Guarantee Messages”) between miners. This would allow miners to mine on blocks that are not yet fully transmitted. This reduces the effect of slow internet connections, leveling the playing field between the 1st world fiberoptic datacenter miners and the rest of the world. We also believe it strengthens the bitcoin network by using existing processing power that is currently wasted into further securing the blockchain, and it reduces the likelihood of transactions becoming confirmed, then unconfirmed and then -hopefully- confirmed again (due to different miners finding competing blocks with different transactions at approx the same time).

It is possible to implement our idea as a fork of bitcoind, or as layer between the standard bitcoind and the mining equipment. In the future it could be incorporated in the bitcoin core if and when that becomes a priority, but that step would not make sense until it becomes a priority.

There are a lot of nuances in this idea, and the first reaction is quite probably that this is a crazy idea. We have attempted to address the most important nuances in our proposal, which is currently at v.0.2.

We cannot guarantee that there are no ‘hidden devils in the details’ and we invite you to be critical in a friendly and constructive manner. We will do our best to answer all questions that arise.

The ‘official’ proposal is at:
PDF: http://pukaki.bz/efficient-bitcoin-block-propagation-v.0.2.pdf
HTML: http://pukaki.bz/efficient-bitcoin-block-propagation-v.0.2.html

[1715000643]

1715000643

[1715000643]

1715000643

[1715000643]

1715000643

[achow101]

So if I understand this correctly, the funds in the guarantee message are locked away for 100+ blocks if the block that goes with it is valid and broadcasted within a given amount of time. If the block is not broadcasted or not valid, then the funds that are part of the guarantee message will be split either among the miners who produced a valid block off of that guarantee or as a transaction fee for a later block.

I don't see how this provides near instantaneous block propagation. It is still just like what happens now with headers being broadcasted first except this comes with some monetary guarantee that the header is valid.

[arnoudk]

The number of blocks is arbitrary (it could be much lower), but yes the guarantee message must include collateral that the sender will lose if he broadcasts an invalid guarantee message. This collateral would be given to any miner that had trusted this guarantee message and had found a valid block on top of it, or - if the block defined in the guarantee message was never sent - the value of the payment in the guarantee message is sent as miner fees. This last part is needed to discourage attackers from sending fake transaction messages.

A lower amount of blocks to keep the guarantee payment in 'escrow' is possible. A short amount of time is needed to enable miners impacted by a fake message to file a claim. In the initial implementation, this would probably be a (partially) manual process - submitting a claim could be automated but then a DRO (dispute resolution organization) would have to check this carefully.

You are correct that the headers are the part that is propagated near-instantaneous, but indeed headers with a guarantee. This guarantee is a requirement for it to work, otherwise this idea would open up unwanted vectors for attack (distracting the network to mine on invalid blocks, wasting their time and money in the process, with no risk to the attacker). The goal of this idea is to have a method by which 'just a header' is as good as a full block from the perspective of the miner. Just let me know if I need to clarify further!

[edmundedgar]

When was the last time an invalid block was created on purpose?

Presumably in the BIP 66 fork the miners would have put up, and ultimately lost, their guarantee, because they were oblivious to the fact that the network had upgraded on them.

[arnoudk]

Currently, invalid blocks would not be relayed as the node would check them first. So you would never see one. However, if you start to accept a header without the block data, you would be unable to do this verification before transmission and odds are that people would take advantage of that if it is not prevented. Therefore, you would need to ensure that this does not happen, and this is the purpose of the collateral.

You could debate whether in the event of a fork caused by an upgrade the guarantee should be lost or returned. Both are possible, but the impact of a fork caused by a BIP is a fringe case.

I've also just learned that the bitcoin block transmission in the bitcoin relay network is more optimized than I was aware of, where a 1 MB block needs ~5kB of data and the rest can be pre-sent. I'll need to look into how this is achieved, but my guess is by smart grouping of transactions, pre-checking of these transactions, and broadcasting everything as quickly as possible before a block is even mined. If so, that would be an elegant solution and we would not need to spend more time on our proposal (I'd be happy if a same or superior solution was implemented!)