Speed is essential, so maybe some miners don't check integrity, but hope that it is valid. If it isn't both blocks will be rejected by validating nodes. There is an argument for not including any transactions at all. When you create a large block it has to propagate quickly to competing miners so they base their work on your block instead of a competing block generated at about the same time. A small block propagates faster than a large one (bandwidth, latency, verification time etc). A large block could have more fees (profit). So it is all a question of whether the probability of loosing the propagation race can be paid by the transaction fees.
Michael Grønager posted this excellent description on the dev mailing list some time ago:
"On the optimal block size and why transaction fees are 8 times too low (or transactions 8 times too big)"
Following the discussion on the recent mining sybil trick, I reread the
article on block propagation by Decker et al.* and decided to use it for
doing a proper estimate of transaction fee size and optimal block size.
The propagation of a block depends on and is roughly proportional to its
size. Further, the slower a block propagates the higher the risk of a
fork, so as a miner you are basically juggling the risk of a fork
(meaning you loose your bounty) vs the opportunity for including more
transactions and hence also get those fees.
This alone will dictate the minimal transaction fee as well as the
optimal block size!
Lets try to put it into equations. For the purpose of this initial study
lets simplify the work by Decker et al. Roughly, we can say that the
average propagation time for a block is t_propagate, and the average
time between blocks is t_blocks. Those are roughly 10sec and 600sec
respectively. The risk of someone else mining a block before your block
propagates is roughly**:
P_fork = t_propagate/t_blocks (~1/60)
Also note that propagation time is a function of block size, S:
t_propagate = t_0 + alpha*S
where Decker et al have determined alpha to 80ms/kb. We also define the
fee size pr kilobyte, f, so
E_fee = f*S
Given these equations the expected average earning is:
E = P_hashrate*(1 - P_fork)*(E_bounty + E_fees)
And inserting:
E = P_hashrate*[1 - (t_0 + alpha*S)/t_block]*(E_bounty + f*S)
We would like to choose the fee so the more transactions we include the
more we earn. I.e. dE/dS > 0:
dE/dS = P_hashrate*{[(t_block - t_0)*f - alpha*E_bounty]/t_block -
2*alpha*f/t_block*S}
Which gives:
f > alpha*E_bounty/(t_block-t_0) ~ alpha*E_bounty/t_block
or f > 80*25/600000 = 0.0033 or assuming a standard transaction size of
0.227kb:
f_tx > 0.00076.
Note that this number is 8 times higher than the current transaction
fee! So the current optimal block size is an empty block i.e. without
other transactions than the coinbase! (miners don't listen now...)
Lets see what you loose by e.g. including 1000 transactions:
E(1000) = P_hashrate*24.34XBT
Which is a loss of 2.6% compared to not including transactions at all!
So there are two ways forward from here. 1) raise the minimum fee, and
2) make transactions smaller. We cannot make transactions much smaller,
but we can utilize that most of them have already been broadcasted
verified and validated and then just include their hash in the block***.
This changes the relevant size for a transaction from 0.227kb to
0.032kb. Which makes f_tx = 0.00011. We are almost there!
Now assume that we implement this change and raise the minimum fee to
0.00015, what is then the optimal block size (dE/dS = 0) ?
S = 1/2 * (t_block/alpha - E_bounty/f)
Which gives 1083kb for a bounty of 25 and 2417kb for a bounty of 12.5.
Optimal size in case of no bounty or an infinite fee is 3750MB.
Final conclusions is that the fee currently is too small and that there
is no need to keep a maximum block size, the fork probability will
automatically provide an incentive to not let block grows into infinity.
*)
http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf**) The calculations should be done using the proper integrals and
simulations, but I will leave that for academia
***) A nice side effect from switching to broadcasting transactions in
blocks as only their hash is that it decouples fee size from transaction
size!
---
"Even simpler minimum fee calculation formula: f > bounty*fork_rate/average_blocksize"
Last week I posted a writeup: "On the optimal block size and why
transaction fees are 8 times too low (or transactions 8 times too big)".
Peter Todd made some nice additions to it including different pool sizes
into the numbers.
However, it occurred to me that things can in fact be calculated even
simpler: The measured fork rate will mean out all the different pool
sizes and network latencies and will as such provide a simple number we
can use to estimate the minimum fee. Key assumption is that the latency
will depend on block size (# txns) and the fork rate will depend on latency.
Using the formulas from last week:
P_fork = t_propagate/t_blocks
and:
t_propagate = t_0 + alpha*S ~= alpha*S
We get a measure for alpha as a function of the average fork rate and
average block size:
alpha = P_fork*t_block/S
Further, take the formula for the minimum fee:
f > alpha*E_bounty/t_block
And insert the formula for alpha:
f > P_fork*E_bounty/S_average
Luckily the fork frequency and the average block size are easily
measurable. blockchain.info keeps historical graphs of number of
orphaned blocks pr day - average over the last year is 1.5. Average
number of blocks per day over the last year is 169, which yields a
P_fork of ~1/113. Average block size in the same time is 134kBytes,
which yields a minimum fee:
f > 0.00165XBT/kb or 0.00037XBT/txn
So the 0.0001 is only 4 times too small. Further, let us look at the
trend over the last 12 months. Pieter Wuille claimed that there has been
several improvements over the last half year that would bring down the
latency, there has also been speculations regarding direct connections
between the major pools etc - lets see if this is indeed true.
If you look instead of 360 days, only at the last 90 days the average
block size has been 131kBytes, and the fork rate has been ~1/118, which
results in a minimum fee of:
f > 0.00162XBT/kb or 0.00037XBT/txn
So a small improvement but not statistically important...
Last question, recalling that optimal revenue block size is a function
of the txn-fee (from the last writeup) - lets see what fee it takes to
support a block size of 131kBytes:
S = 1/2 * (t_block/alpha - E_bounty/f)
S = 1/2 * (S/P_fork - E_bounty/f)
f = E_bounty/[(1/P_fork-2)*S] = 0.00165XBT/kB
So a 4 times increase is still sufficient for the current load.
Anyway - the all important number is alpha, the network latency which we
expect to be dependent of various things such as interconnectivity,
bandwidths, software quality etc, where mainly the latter is within our
hands to bring down the fee. And you can actually setup the standard
client to choose a better fee, as all the parameters in the formula are
easily measured!
Secondly, transactions that are never going to be accepted don't disappear from the network after some timeout period?
They disappear as they are removed from the memory pool of the various nodes out in the wild. As far as I remember bitcoinQT only removes them when it is restarted. Note that a node that receives a transaction that spends some of the same inputs as one in its memory pool will discard the new tx as a double spend, and will not relay / mine it.
And third, I understand that miners can implement their own way of making blocks. But when for example a miner only accepts blocks with 10 transactions, is there some regulating force that counter balances that?
I think you mean "creates" instead of "accepts". The only "regulating" forces are that the miner has an incentive to keep the network healthy, and an incentive to increase his profits by collecting the fees.
According to Michaels calculations the fees are right now too low compared to the probability of loosing the propagation race.
