Decentralization idea: Encourage mining network distribution among pools

[neoranga]

Looking at how the ASIC market is moving and growing I started to think about these problems:
- Centralization of hashing on big pools.
- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.
- Possibility for a big ASIC producer/miner to control the network.
- Not possible to confirm blocks in a very long period of time if the main pool(s) are disconnected from the network (consider a scenario with 2-3 pools/companies having 90% of the current difficulty hash power, if they drop by an attack it will take extremely long time for the next difficulty adjustment to correct this)

My idea is to encourage distribution of rewards and therefore encourage distribution of mining power.

General description: Create a list of known miners with history in mining(finding blocks), assign to them a reasonable target hash to achieve (instead of mining the main block hash) with a small reward and make the approval of the block depend on both the usual high difficulty hash and a random subset of the trusted miners.

Implementation details
- Store on each block an ordered list with the bitcoin addresses that generated blocks in the past with the cumulative number of blocks created plus the time the miner took to find the last hash(record the hashing power).
- Use the Nonce plus Fibonacci series (for example) to select the next XX(e.g. 20) candidates with their public addresses from the ordered list included in the block.
- In addition to the usual hash to be found, half (10) of those XX(20) candidates have to find an adjusted SHA256 hash.
- The adjusted SHA256 hash uses a difficulty relative to the difficulty of the last block found by this miner (not the network difficulty!). So the required number is adjusted and different for each of the candidates.
- Half (10) of those XX(20) candidates have to sign (with their private key) the next block to be found including each of their adjusted SHA256 hashes.
- The first block that includes the originally required main SHA256 hash plus half(10) of those XX(20) adjusted SHA256 hashes, each with their corresponding signatures, is considered a valid block.
- The selected addresses that proved the last block by including their adjusted SHA256 hash receive a small part of the reward divided equally (e.g. half of current reward is divided equally among all selected candidates that proved the work and the other half of the reward goes to the miner that found the main SHA256 hash).
- When there are more than half(10) of the possible XX(20) addresses that submit a valid answer, the less powerful ones get the reward (this is the time it took to find the current adjusted hash). This reduces cheating.
- Original reward for miner that finds the main SHA256 hash still exists but the amount is decreased to fulfill the rewards of the candidates.
- The list of all candidates gets updated on each new block and arranged by amount of blocks found (even with adjusted SHA256 hashes* found).
- The list of selected candidates is published in the new block with their adjusted target SHA256 hash goal so all the network knows who is responsible for next block and which are the individual targets.

- Additionally, if the main SHA256 hash for the current block has not been found for 20 minutes and half (10) of the XX(20) candidates have already found and sign their corresponding adjusted SHA256 hashes, then the block is considered valid (as a recover mechanism in case of big mining network drop).

Definitions
*Main SHA256 hash: hash to be found using the network difficulty in order to confirm a block in the current bitcoin implementation 0.8.x.
*Adjusted SHA256 hash: hash found by one of the candidates when randomly selected and included in the final confirmed block together with the rest of the candidates and the Main SHA256 hash.

Pros
- Non massive mining pools get a change to continue profitable and grow.
- Massive mining corporations loose control of the network by requiring consensus from a small group of unrelated but famous miners, reducing the centralization risk.
- Recover mechanism in case of big drop of the mining network (if the massive pools are suddenly disconnected the rest of the mining network in current implementation can't confirm a block in up to two weeks).

Cons
- Pool owners will have to manage pool jumpers that join/leave depending on pool address appearing in the 20 selected candidates
- How to enter the list of candidates without having the maximum hashing power to generate a block

Related projects I've researched:
P2Pool - https://en.bitcoin.it/wiki/P2Pool
Decentralized mining protocol standard: getblocktemplate - https://bitcointalk.org/index.php?topic=108854.0

[gmaxwell]

- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.

I stopped reading here. This isn't correct. Expected return is not increased by using a larger pool. Mining is not a race, it's a random lottery.

[neoranga]

- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.

I stopped reading here. This isn't correct. Expected return is not increased by using a larger pool. Mining is not a race, it's a random lottery.

But I understood that if a pool has more people it has more chances of winning the lottery, is this incorrect?

[Peter Todd]

- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.

I stopped reading here. This isn't correct. Expected return is not increased by using a larger pool. Mining is not a race, it's a random lottery.

But I understood that if a pool has more people it has more chances of winning the lottery, is this incorrect?

Yes, but the winnings are split proportionally among those extra people.

That said, with a larger pool it's less likely that your block reward will be orphaned by another block that manages to propagate to the network faster than yours. But that's not a big effect right now in terms of profitability because fees aren't very important - a rational miner will keep their blocks small to keep that orphan risk down.

[TierNolan]

What about encouraging pools to use p2pool as their backend?

Effectively, they would mine against p2pool instead of the main chain.  The reduces pool variance for all pools.  

One of the marketing benefits of large pools is their lower variance.  This would reduce that advantage and so encourage more pools to form.

This is not in the interests of the larger pools, so it would only happen if demanded by miners.

[neoranga]

- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.

I stopped reading here. This isn't correct. Expected return is not increased by using a larger pool. Mining is not a race, it's a random lottery.

But I understood that if a pool has more people it has more chances of winning the lottery, is this incorrect?

Yes, but the winnings are split proportionally among those extra people.

That said, with a larger pool it's less likely that your block reward will be orphaned by another block that manages to propagate to the network faster than yours. But that's not a big effect right now in terms of profitability because fees aren't very important - a rational miner will keep their blocks small to keep that orphan risk down.

That makes sense to me.

The centralization problem is that a new small mining pool or an existing pool that becomes small after the arrival of a new ASICs pool has no incentives to continue mining because the expected time to find a block could become bigger than a lifetime. Therefore, it's better join the bigger pool to receive a fraction of the reward now at current difficulty than expect a large reward in an infinite time frame, specially with increasing difficulty.
This is exactly what happens right now with solo mining.

[alp]

- Decreasing chances for small pools to create blocks and attract more mining power to grow. With no reward on mid-size pools all miners should prefer to mine on big pools with better expectations for return on investment, rather than supporting new pools.

I stopped reading here. This isn't correct. Expected return is not increased by using a larger pool. Mining is not a race, it's a random lottery.

But I understood that if a pool has more people it has more chances of winning the lottery, is this incorrect?

Yes, but the winnings are split proportionally among those extra people.

That said, with a larger pool it's less likely that your block reward will be orphaned by another block that manages to propagate to the network faster than yours. But that's not a big effect right now in terms of profitability because fees aren't very important - a rational miner will keep their blocks small to keep that orphan risk down.

That makes sense to me.

The centralization problem is that a new small mining pool or an existing pool that becomes small after the arrival of a new ASICs pool has no incentives to continue mining because the expected time to find a block could become bigger than a lifetime. Therefore, it's better join the bigger pool to receive a fraction of the reward now at current difficulty than expect a large reward in an infinite time frame, specially with increasing difficulty.
This is exactly what happens right now with solo mining.

But the reward is much bigger.  Each miner can choose the appropriate risk/reward they want.  Huge pools with low variance should be able to charge more fees (since most miners prefer low variance), therefore the incentive to have a smaller pool works out.

[BombaUcigasa]

Implementation details
- Store on each block an ordered list with the bitcoin addresses that generated blocks in the past with the cumulative number of blocks created plus the time the miner took to find the last hash(record the hashing power).

So, new miners do not have an incentive to start mining yet somehow this decentralizes mining?

[neoranga]

Implementation details
- Store on each block an ordered list with the bitcoin addresses that generated blocks in the past with the cumulative number of blocks created plus the time the miner took to find the last hash(record the hashing power).

So, new miners do not have an incentive to start mining yet somehow this decentralizes mining?

The original reward for the existing POW is still there, only the amount of the reward is decreased (half for example) and the other half goes to the selected candidates that proved the work divided in equal parts. I'll update the description to make this more explicit.

[BombaUcigasa]

Implementation details
- Store on each block an ordered list with the bitcoin addresses that generated blocks in the past with the cumulative number of blocks created plus the time the miner took to find the last hash(record the hashing power).

So, new miners do not have an incentive to start mining yet somehow this decentralizes mining?

The original reward for the existing POW is still there, only the amount of the reward is decreased (half for example) and the other half goes to the selected candidates that proved the work divided in equal parts. I'll update the description to make this more explicit.

Ok, you do that, but it basically says here that if you are a new miner or require a new identity you will receive a lower payout percentage compared to an older identity.

So, if older miners receive a higher percentage, they just expose themselves as a mining pool and split the reward with these new miners.

Oh wait, that's exactly what happens now...

[jgarzik]

How about this... encourage automatic decentralization.

1. Update mining software to automatically rebalance between pools, based on a variety of factors.  Some miners and proxies already do a bit of this.

2. Measure the size of pools, and select pools to help avoid one or two pools having large shares of the network power.

[gmaxwell]

1. Update mining software to automatically rebalance between pools, based on a variety of factors.  Some miners and proxies already do a bit of this.

IgorMcScammer is looking forward to this bright future where he starts up a bunch of mining pools that gobble people's money to automatically receive traffic.

I prefer to just make it moot:  The primary useful function of pools is collecting common payments to help people average out the risk of mining. This doesn't require the pool to be a source of centralization risk for the Bitcoin network.

You ask a local bitcoind for a getblocktemplate, you ask a far away pool for a coinbase transaction.  You combine the two.  You mine that. You send shares back to the pool. The pool credits you based on work that pays coinbases they authored. With that in effect, from the perspective of the security of the Bitcoin network we don't really care if there is only one pool (though the mining ecosystem would still benefit from there being multiple pools in order to keep fees for this service down).

Doing this would remove the externalized cost of additional centralization risk from pooled mining. It's compatible with all payout regimes, doesn't require a second distributed network (like p2pool does) to decide payouts, etc.

Having your choice of a handful of very large pools to deletgate all your influence to is not decenteralization— "A choice of masters is not freedom". Even if no pool is a majority attackers-of-bitcoin can hack multiple pools, and even a pool with only 20% hashpower can still successfully reorg two-deep transactions 20% of the times they try.

When pooling decides network state, pooling and decentralization are fundamentally opposed:  Mining is decenteralized only so long as no one (group of) miner(s) is reliably deciding the next block, but pooling doesn't make your income stable unless your pool is somewhat reliably producing blocks. This conflict can be resolved by separating centralizing for payment with centralizing the network consensus.

(Personally I'm quite happy on P2pool now that it has been tuned up to work well with Avalon... but some people, especially smaller miners, prefer different pay out schemes than P2pool can accommodate. Some payout schemes, like PPS fundamentally require centralization of payout... but that doesn't mean we need to centralize anything else)