Block and Transaction Propagation times

[Picard78]

According to this site http://bitcoinstats.com/network/propagation/ in 2017 it took 1.818 seconds to propagate a newly mined block to 50% of the Nodes.  However, it took 3.792 seconds to propagate a single transaction to 50% of the nodes.  (Scroll down to the bottom of the page to see the above mentioned stats)

I don't understand how a transaction would take more time to propagate than an entire block filled with thousands of transactions?

Thank you

[1715462500]

1715462500

[1715462500]

1715462500

[1715462500]

1715462500

[Carlton Banks]

I don't understand how a transaction would take more time to propagate than an entire block filled with thousands of transactions?

probably compact blocks BIP152 plays an important role

because all blocks are composed of valid UTXOs which are relayed as unconfirmed transactions before inclusion in some subsequent block, most nodes do not need the megabytes of transaction information, as they already have all (or most) of the transactions in a given block in their mempool. Compact blocks propagation is designed to leverage this fact; nodes relay only the block header and hashed transactions to other nodes, instead of the complete block. If a node does not have in its local mempool some small number of the unconfirmed transactions in that block, then they request the full transactions they are missing. Then, the node uses the hashed transactions (now that they have all of them for that block) to reconstruct the complete block.

This reduces block propagation time (and the bandwidth used to propagate blocks) considerably (and has been in use since Bitcoin 0.13.0)

[ranochigo]

Firstly, I would take the stats with a pinch of salt. I can't seem to find their sampling methodology and it is difficult to evaluate the efficacy of their sampling.

Bitcoin Core has an arbitrary interval to send inv messages to its peers. Bitcoin Core maintains a list of transaction (not the mempool) for the individual nodes connected and that the list of transaction is what your client believe that the specific peer that the list is assigned to doesn't have the transaction. The inv message is broadcasted periodically and with an arbitrary delay.

In addition, each node has their own set of rules and some may refuse to relay certain transactions due to a relay fee below its threshold or a myriad of other criteria. The number of hops that a transaction has to go through due to this has a huge factor to play.

CMIIW, since the newer client (After 0.13.0 or so) implements header synchronization, there isn't any delay when announcing blocks after the validation.

[gmaxwell]

Transaction relay is batched and relayed at random intervals because doing so greatly reduces bandwidth usage for relay (due to reducing overheads and due to reducing the number that are advertised both ways on links) and also because its important for privacy as otherwise it's much easier to tell where transactions came from based on when they showed up.  ... and also because there is no particular need for transaction relay to be extremely low latency: blocks are 10 minutes apart on average, after all.

I don't understand how a transaction would take more time to propagate than an entire block filled with thousands of transactions?

Almost all the data in blocks has already been relayed and validated.  It only takes a 6 or so bytes per transaction on average to relay a block and the only processing needed is a bit of hashing.

[Picard78]

Thank you for the answers.

[Picard78]

nodes relay only the block header and hashed transactions to other nodes, instead of the complete block.

This optimization makes sense.

It only takes a 6 or so bytes per transaction on average to relay a block and the only processing needed is a bit of hashing.

Where do the 6 Bytes come in?  A transaction ID is a SHA256 hash of the transaction done twice, which is 32 Bytes. 

[ABCbits]

nodes relay only the block header and hashed transactions to other nodes, instead of the complete block.

This optimization makes sense.

It only takes a 6 or so bytes per transaction on average to relay a block and the only processing needed is a bit of hashing.

Where do the 6 Bytes come in?  A transaction ID is a SHA256 hash of the transaction done twice, which is 32 Bytes. 

IIRC, compact block uses this https://en.bitcoin.it/wiki/Protocol_documentation#Short_transaction_ID

[Carlton Banks]

Where do the 6 Bytes come in?  A transaction ID is a SHA256 hash of the transaction done twice, which is 32 Bytes. 

compact blocks does not use the txid, it uses a shorter (64 bit) hash, and then apparently drops the 2 most significant bytes (i.e. the first 16 bits, maybe because the hashes are serialized for big endian archs? not sure). Hence, 64 bits - 16 bits = 48 bits (equivalent to 6 bytes per short-tx-id).

follow the link in ETFbitcoin's post above, that page is an invaluable reference.

[gmaxwell]

(i.e. the first 16 bits, maybe because the hashes are serialized for big endian archs? not sure)

Siphash's output is 64-bits and we needed fewer based on an analysis of collisions rates.  It didn't matter which bytes of the output got dropped, some just had to be picked.