Transaction per block

[ltgboneBIT]

Newbie question: why don't all blocks have the same number of transactions? I could speculate but I'd like to hear a knowledgable answer.

[szmarco]

if all blocks must have the same number of tx, the mining output will not be regular.

[zimmah]

Newbie question: why don't all blocks have the same number of transactions? I could speculate but I'd like to hear a knowledgable answer.

because a block is mined roughly every 10 minutes (and this is also a random process, so some blocks take longer than others)

it depends on how many transactions were made during the time a block was formed.

So, this 10 minutes, maybe more people made a transaction than the next 10 minutes. So it has more transactions.

So it depends on two things

1) How much time was needed to find a block (more time means usually more transactions)
2) How much transactions were made during that time (of course more transactions means more transactions)

[DannyHamilton]

So it depends on two things

1) How much time was needed to find a block (more time means usually more transactions)
2) How much transactions were made during that time (of course more transactions means more transactions)

Actually, you're only partially correct.

A miner (or mining pool) that creates a block obviously can't include transactions that don't exist. Therefore, the maximum number of transactions that can be included in the block is the number of unconfirmed transactions that the miner is aware of when he creates the block.

However, a miner (or mining pool) is not forced by the protocol to include all transactions if they don't want to.  The miner (or mining pool) is free to use whatever criteria they like to choose which transactions they want to include in their block.

Since miners (or mining pools) get to keep the transaction fees from all the transactions that they include in their block, there's a financial incentive to include the transactions that pay the highest fees.  In particular, since the size of a block is limited to a 1 megabyte maximum, there's a financial incentive for the miner to include the transactions that pay the highest fee per byte.

Some miners (or mining pools) are concerned that their block might get orphaned if their block is significantly larger than a competitor's block.  Therefore some miners (or mining pools) restrict their blocks to be smaller than 1 megabyte.

Some miners (or mining pools) prefer not to include any free transactions at all (since they won't get paid for including them).  Other miners feel that including some free transactions makes the entire system (and therefore their mined bitcoins) more valuable.  Therefore, they are willing to include a limited number of free transactions in the blocks that they mine.

There are even some miners (or mining pools) that don't want to be bothered including transactions in their blocks at all.  They just mine empty blocks where the only transaction is the one that pays them their block reward.  This is allowed by the protocol, because it is assumed that the financial incentive of increased profits will motivate enough miners to include transactions.  The few miners (or pools) that mine empty blocks are simply leaving money (and profits) behind that the next miner will get to take in the next block.

[-ck]

There are even some miners (or mining pools) that don't want to be bothered including transactions in their blocks at all.  They just mine empty blocks where the only transaction is the one that pays them their block reward.  This is allowed by the protocol, because it is assumed that the financial incentive of increased profits will motivate enough miners to include transactions.  The few miners (or pools) that mine empty blocks are simply leaving money (and profits) behind that the next miner will get to take in the next block.

I admire your persistence at qualifying miner with (or mining pools) at every turn

A lot of pool software is also very slow to generate work templates as soon as a block is solved due to single threaded issues so they send out transactionless work templates immediately after as an optimisation till the next lot of work they send out which has a full transaction set. I don't do that with my ckpool software.

[DannyHamilton]

I admire your persistence at qualifying miner with (or mining pools) at every turn

It's a habit I can't seem to break.

If more miners would join pools (such as p2pool) where they get to create their own blocks rather than allowing the pool to create the blocks for them, then I probably wouldn't feel so compelled to remind people that they are turning their vote and their control over to someone else.

A lot of pool software is also very slow to generate work templates as soon as a block is solved due to single threaded issues so they send out transactionless work templates immediately after as an optimisation till the next lot of work they send out which has a full transaction set.

True.  There are a lot of very bad habits that pools have formed due to the lack of interest, lack of research, and lack of effort put forth by those that are selling their hashing power to the pool operator.  I assume that in the long run, market forces will correct this.  Unfortunately, the "long run" probably means 10 to 20 years.

I don't do that with my ckpool software.

Glad to hear it.  Have you looked into allowing your pool members create the blocks themselves?

[-ck]

Have you looked into allowing your pool members create the blocks themselves?

What you see on the forum here is probably only 15% of the remaining network. Mining left the hands of the community a long time ago, leaving only big farms, only the community hasn't woken up to this yet. The farms really could not care less about this aspect so the best I can offer is to provide them with pool software they can use themselves to not contribute their hashrate to large pool entities and that always assembles blocks from bitcoind.

[DannyHamilton]

Have you looked into allowing your pool members create the blocks themselves?

What you see on the forum here is probably only 15% of the remaining network.

I'm not certain of the exact percent, but I assume that mining is moving away from the small individual and towards a more large scale corporate structure.  This is the direction it was always heading.  Even Satoshi himself seemed to be aware that mining would eventually migrate to those with the best resources.

Mining left the hands of the community a long time ago, leaving only big farms, only the community hasn't woken up to this yet. The farms really could not care less about this aspect

Certainly.  Any farm large enough would be most profitable to stick with solo mining.

so the best I can offer is to provide them with pool software they can use themselves to not contribute their hashrate to large pool entities and that always assembles blocks from bitcoind.

That's the "best" you can offer?  Exactly why can't you do better?  Providing software that allows users to create their own blocks for the pool and to set their own transaction selection criteria would be better, wouldn't it?

[-ck]

Providing software that allows users to create their own blocks for the pool and to set their own transaction selection criteria would be better, wouldn't it?

I already explained they don't care for it. Better the default selections on bitcoind make sense as chosen by the bitcoin devs than the mining entities which largely have no informed opinion on it. What I've provided makes it easy for them to transition away from large pools to smaller solo mining operations without requiring any thought on their part. They don't want to think, they just want to mine. The core bitcoin development largely dissociated itself from mining as much as possible and the hardware vendors moved in during that period and now control it. I've tried engaging operations from 3 to 15PH in size and they would all rather mine at the lowest fee pool and concentrate on rewards only and protecting their valuable hardware investments. Any effort I put towards user controlled transaction selection and block generation would be on the rapidly diminishing proportion of community miners left - who I might add, on the whole, think exactly the same way anyway.

[-ck]

Also I'm not sure if you're aware or not if you're not a miner, but the p2pool model sounds great in theory but the model has failed, and it has nothing to do with the scalability or resource issues of running the client itself (which deserve their own attention). Pooled mining allows small miners to combine their resources to behave like large miners. The more miners that pool together the easier it is for small miners to get a consistent stream of rewards. P2pool manages to do the opposite: the larger the pool is, the more the small miners get marginalised and the less likely they are to get consistent stream of rewards till they may get nothing at all. My software acts as a proxy gateway firewall filter resource reduction to p2pool as well and I have put a lot of effort into that component so do not assume I am abandoning it entirely, but miners are voting with their farm-sized feet.

[Borisz]

First of all, thank you guys, great discussion!

Also I'm not sure if you're aware or not if you're not a miner, but the p2pool model sounds great in theory but the model has failed, and it has nothing to do with the scalability or resource issues of running the client itself (which deserve their own attention). Pooled mining allows small miners to combine their resources to behave like large miners. The more miners that pool together the easier it is for small miners to get a consistent stream of rewards. P2pool manages to do the opposite: the larger the pool is, the more the small miners get marginalised and the less likely they are to get consistent stream of rewards till they may get nothing at all. My software acts as a proxy gateway firewall filter resource reduction to p2pool as well and I have put a lot of effort into that component so do not assume I am abandoning it entirely, but miners are voting with their farm-sized feet.

I have actually seen and experienced this exactly. I have tried mining BTC and multiple altcoins on p2pool hoping that my slow miners would be (more) effective here. In fact the payouts were completely random, the overall pool speed and difficulty are so high that it took an eternity to submit a valid share and receive a payout.

Could this ever change with p2pool? Or as more miners would join slower miners would inevitably get lower and lower share count?

[-ck]

I have actually seen and experienced this exactly. I have tried mining BTC and multiple altcoins on p2pool hoping that my slow miners would be (more) effective here. In fact the payouts were completely random, the overall pool speed and difficulty are so high that it took an eternity to submit a valid share and receive a payout.

Could this ever change with p2pool? Or as more miners would join slower miners would inevitably get lower and lower share count?

No one has a workable solution at the moment. The only way currently around this is to mix a traditional and distributed pool design (by putting a regular pool in front of a p2pool back end and managing payouts like a regular pool), thereby weakening the whole distributed concept.