Decentralized Mining & Preventing Centralization in Pools

[bytemaster]

Anyone who has been following crypto-currencies for long has realized that any popular (and thus hard to mine) crypto-currency ends up resorting to mining pools.  These pools become new points of centralization.    In the case of ProtoShares, a CPU coin, profiteers flooded the market with hash power and forced out the little guy who wants to mine on his home computer.  These profiteers didn't care about the coin, they just saw the price difference between the coin and the cost of mining and took advantage of the profit opportunity.   

Then we have pools that came out and captured more than 80% of the hash power due to a slight optimization advantage.  These pools had a single bug in their miner that prevented more than one transaction from being included in a block and thus caused delays for the entire network like a legitimate 51% attack.

In Bitcoin land, large ASIC firms and mining pools are a potential threat.   Even having a couple large pools is a problem and represents a kind of 'special node' in the network that if taken out or compromised could enforce white lists, black lists, or transaction delays.   A new solution is required that make the use of large pools, whether ASIC, GPU, or CPU less profitable than solo-mining for individual users.   

I have a new proposal that aims to eliminate botnets and profiteers from being a threat to GPU or CPU based coins and could be useful toward encouraging solo-mining with bitcoin by increasing the costs associated with large centralized ASIC miners. 

The basic idea is this:  blocks have a random vesting time between 1 day and 1 year with an average of 6 months.   Any pool operator would have to maintain 6 months of inventory and operating expenses on their books while exposed to price fluctuations in the underlying currency.   Mining pools would have to charge much higher fees to cover the cost of capital and a 51% attack on Bitcoin would not be able to finance itself from the immediate sale of the BTC mined.   Meanwhile, solo-miners with ASICs are back to a lottery system.   New crypto-currencies wouldn't have to worry about fly-by-night pools or profiteers that mine and dump.   The average rate of currency introduction would be the same, just delayed 6 months on average and this would also prevent a lot of short-run PUMP and DUMP scams.   

http://bitsharestalk.org/index.php?topic=905.0

[AnonyMint]

Interesting proposal.

Cost of carrying extra capital means you will get lower network hashrate for the same aggregate amount spent on securing the network.

Also it means I can't reliably mine virgin coins (for removing taint issue) when I need to spend them near-term. It is a forced savings program.

I am generally against any arbitrary limit that is not helping the party who is investing. I guess you can argue it helps the overall health of the system thus coin value, but individually miners might not calculate it that way.

Perhaps you are aware of the new selfish-mining attack which is a vulnerability when any entity has control over 25% of the network hashrate. Perhaps your proposal fixes it.

[bytemaster]

Interesting proposal.

Cost of carrying extra capital means you will get lower network hashrate for the same aggregate amount spent on securing the network.

Also it means I can't reliably mine virgin coins (for removing taint issue) when I need to spend them near-term. It is a forced savings program.

I am generally against any arbitrary limit that is not helping the party who is investing. I guess you can argue it helps the overall health of the system thus coin value, but individually miners might not calculate it that way.

Perhaps you are aware of the new selfish-mining attack which is a vulnerability when any entity has control over 25% of the network hashrate. Perhaps your proposal fixes it.

I am aware of the selfish miner attack and that is solved by allowing new blocks to replace the current head block provided it is 'more difficult'.  Someone attempting to be a 'selfish miner' who starts working on a 2nd block without taking credit for the first block has a significant chance that when the rest of the network finds the first block, his second block will have been built on an invalid chain.    He would then have to find 3 blocks before the primary network found 2 blocks because miners do not switch 'chains' until their chain is more than 2 blocks behind the other chain.

[bytemaster]

Interesting proposal.

Cost of carrying extra capital means you will get lower network hashrate for the same aggregate amount spent on securing the network.

Also it means I can't reliably mine virgin coins (for removing taint issue) when I need to spend them near-term. It is a forced savings program.

I am generally against any arbitrary limit that is not helping the party who is investing. I guess you can argue it helps the overall health of the system thus coin value, but individually miners might not calculate it that way.

Perhaps you are aware of the new selfish-mining attack which is a vulnerability when any entity has control over 25% of the network hashrate. Perhaps your proposal fixes it.

I am aware of the selfish miner attack and that is solved by allowing new blocks to replace the current head block provided it is 'more difficult'.  Someone attempting to be a 'selfish miner' who starts working on a 2nd block without taking credit for the first block has a significant chance that when the rest of the network finds the first block, his second block will have been built on an invalid chain.    He would then have to find 3 blocks before the primary network found 2 blocks because miners do not switch 'chains' until their chain is more than 2 blocks behind the other chain.

To clarify, miners do not switch chains to a fork that is split based upon an easier hash unless it has more than 2 block advantage.

[AnonyMint]

Interesting proposal.

Cost of carrying extra capital means you will get lower network hashrate for the same aggregate amount spent on securing the network.

Also it means I can't reliably mine virgin coins (for removing taint issue) when I need to spend them near-term. It is a forced savings program.

I am generally against any arbitrary limit that is not helping the party who is investing. I guess you can argue it helps the overall health of the system thus coin value, but individually miners might not calculate it that way.

Perhaps you are aware of the new selfish-mining attack which is a vulnerability when any entity has control over 25% of the network hashrate. Perhaps your proposal fixes it.

I am aware of the selfish miner attack and that is solved by allowing new blocks to replace the current head block provided it is 'more difficult'.  Someone attempting to be a 'selfish miner' who starts working on a 2nd block without taking credit for the first block has a significant chance that when the rest of the network finds the first block, his second block will have been built on an invalid chain.    He would then have to find 3 blocks before the primary network found 2 blocks because miners do not switch 'chains' until their chain is more than 2 blocks behind the other chain.

Interesting idea for a solution to selfish-mining. We are getting off-topic a bit from your OP. You mean miners choose the chain with the hashes with the lowest value below the threshold for "difficulty". But the problem is that the longer we wait, the lower hashes can be. So you resolve that by choosing the chain with the lowest total of all hashes I presume. I suppose you've worked out the probabilities and game theory and convinced yourself this converges.

I assume you are aware of the equation for orphan rate:

https://bitcointalk.org/index.php?topic=260180

You've got to make sure you have convergence. This will impact on the minimum block period.

P.S. You got this conceptual idea from the mini-blockchain Clever application.

[bytemaster]

Interesting proposal.

Cost of carrying extra capital means you will get lower network hashrate for the same aggregate amount spent on securing the network.

Also it means I can't reliably mine virgin coins (for removing taint issue) when I need to spend them near-term. It is a forced savings program.

I am generally against any arbitrary limit that is not helping the party who is investing. I guess you can argue it helps the overall health of the system thus coin value, but individually miners might not calculate it that way.

Perhaps you are aware of the new selfish-mining attack which is a vulnerability when any entity has control over 25% of the network hashrate. Perhaps your proposal fixes it.

I am aware of the selfish miner attack and that is solved by allowing new blocks to replace the current head block provided it is 'more difficult'.  Someone attempting to be a 'selfish miner' who starts working on a 2nd block without taking credit for the first block has a significant chance that when the rest of the network finds the first block, his second block will have been built on an invalid chain.    He would then have to find 3 blocks before the primary network found 2 blocks because miners do not switch 'chains' until their chain is more than 2 blocks behind the other chain.

Interesting idea for a solution to selfish-mining. We are getting off-topic a bit from your OP. You mean miners choose the chain with the hashes with the lowest value below the threshold for "difficulty". But the problem is that the longer we wait, the lower hashes can be. So you resolve that by choosing the chain with the lowest total of all hashes I presume. I suppose you've worked out the probabilities and game theory and convinced yourself this converges.

I assume you are aware of the equation for orphan rate:

https://bitcointalk.org/index.php?topic=260180

You've got to make sure you have convergence. This will impact on the minimum block period.

P.S. You got this conceptual idea from the mini-blockchain Clever application.

I believe my network will converge much faster because it will never diverge in the first place because network latency is practically removed from the equation.   In the instances where there is an orphan block today it occurs because two people broadcast at about the same time.  Miners accept the first block found therefore half of the miners use one block and another half use the other.   This process is not resolved until one side of the network finds the next block.   

Under my system, two miners broadcast at the same time and everyone will immediately converge on the best chain without ambiguity.  The only potential for 'orphans' resulting in divergence is if two back-to-back blocks are found in a space less than the propagation delay AND the first block had a higher hash.   

[AnonyMint]

Under my system, two miners broadcast at the same time and everyone will immediately converge on the best chain without ambiguity.  The only potential for 'orphans' resulting in divergence is if two back-to-back blocks are found in a space less than the propagation delay AND the first block had a higher hash.  

More slightly off-topic discussion... So if a miner has already started working and a new solution comes later with a much lower hash, does the miner throw away the current work? I guess not. So miners only consider re-starting work up to the maximum propagation delay you expect? Hardcoded?

Then that still defeats selfish-mining because of the increasing probability that the hash found later will be lower than the held back hash solution discovered earlier. Effectively you've increased the gamma in the selfish-mining research paper.

[AnonyMint]

hmmm... but variance is an issue. If the selfish-miner has a very low valued hash, it is still probably worth holding it back.

Nevertheless your idea reduces orphan rate.

The tradeoff is it doesn't converge if the rate of competing block generation is less than the propagation delay?

[Soros Shorts]

The basic idea is this:  blocks have a random vesting time between 1 day and 1 year with an average of 6 months.   Any pool operator would have to maintain 6 months of inventory and operating expenses on their books while exposed to price fluctuations in the underlying currency.

If implemented, this intent could be easily defeated by an external BTC derivatives market which would undoubtedly form.

[AnonyMint]

The basic idea is this:  blocks have a random vesting time between 1 day and 1 year with an average of 6 months.   Any pool operator would have to maintain 6 months of inventory and operating expenses on their books while exposed to price fluctuations in the underlying currency.

If implemented, this intent could be easily defeated by an external BTC derivatives market which would undoubtedly form.

Damn you Soros. Why didn't I see that!

[bytemaster]

The basic idea is this:  blocks have a random vesting time between 1 day and 1 year with an average of 6 months.   Any pool operator would have to maintain 6 months of inventory and operating expenses on their books while exposed to price fluctuations in the underlying currency.

If implemented, this intent could be easily defeated by an external BTC derivatives market which would undoubtedly form.

Damn you Soros. Why didn't I see that!

I am sure such a derivatives market would form.  And it does not defeat the intent.   A coin today is more valuable than a coin in 6 months, so the cost of mining in the pool with immediate payout would still be higher than mining solo.   The coins mined would still be delivered to those taking a LONG TERM approach AND the pool operator would have to be trusted or individual selling the future contract would only provide payment upon a block being found with their private key and thus shift the trust from the pool to the long-term investor.  

There is clearly money to be made by facilitating pools, but those mining on the pools would face a much higher cost of doing business and if you push the time horizon out to a two or more years initially then the risks would be much higher and finding people willing to finance over that period of time would be much harder.  

[phelix]

interesting idea...

[niniyo]

How about people start adopting this new mining protocol? https://en.bitcoin.it/wiki/Getblocktemplate

This allows the miner to build the block rather than the pool, so the miner can implement their own policy as to what goes into the block.  This means that pool operators with a large share of hashing power cannot abuse that power to do things like exclude certain transactions from blocks, or have unreasonably high fee thresholds.  The content of the block would be based on the transaction policies of the person who happened to mine it, which would have a lot of variation since each block would likely come from a different miner.

This also means that the miner who first finds the block can broadcast the block, which makes it impossible for the pool to do "selfish mining".

[bytemaster]

How about people start adopting this new mining protocol? https://en.bitcoin.it/wiki/Getblocktemplate

This allows the miner to build the block rather than the pool, so the miner can implement their own policy as to what goes into the block.  This means that pool operators with a large share of hashing power cannot abuse that power to do things like exclude certain transactions from blocks, or have unreasonably high fee thresholds.  The content of the block would be based on the transaction policies of the person who happened to mine it, which would have a lot of variation since each block would likely come from a different miner.

This also means that the miner who first finds the block can broadcast the block, which makes it impossible for the pool to do "selfish mining".

This works well for people running 'full nodes' but it seems most 'miners' are using specialized light-weight clients. 

[eleuthria]

How about people start adopting this new mining protocol? https://en.bitcoin.it/wiki/Getblocktemplate

This allows the miner to build the block rather than the pool, so the miner can implement their own policy as to what goes into the block.  This means that pool operators with a large share of hashing power cannot abuse that power to do things like exclude certain transactions from blocks, or have unreasonably high fee thresholds.  The content of the block would be based on the transaction policies of the person who happened to mine it, which would have a lot of variation since each block would likely come from a different miner.

This also means that the miner who first finds the block can broadcast the block, which makes it impossible for the pool to do "selfish mining".

No pool at this time actually supports miners picking and choosing transactions with GBT.  They use GBT the same way pools use stratum.  They give you the block contents, you mine it.  You have no say.

There is a whole new level of complication added to the Miner<->Pool dynamic when it comes to letting miners pick transactions.  Specifically, the pool needs to know which transactions you're including every time you send in shares so it can validate your share results are correct.  This is a MASSIVE amount of bandwidth that would be required when looking at even a 10% fraction of the pool opting for it, not to mention a large increase in stale rates for most miners since they would have to upload 300-500KB to the pool to tell them the block contents.  Hell, GBT is already a complete major bandwidth hog compared to Stratum because it has to send miners the entire block contents when it pushes work instead of just a merklebranch.

[AnonyMint]

I postulated that selfish-mining can be easily defeated by spying on the pools. Bytemaster, I also mentioned your proposed solution there and linked back to this thread.

How about people start adopting this new mining protocol? https://en.bitcoin.it/wiki/Getblocktemplate

This allows the miner to build the block rather than the pool, so the miner can implement their own policy as to what goes into the block.  This means that pool operators with a large share of hashing power cannot abuse that power to do things like exclude certain transactions from blocks, or have unreasonably high fee thresholds.  The content of the block would be based on the transaction policies of the person who happened to mine it, which would have a lot of variation since each block would likely come from a different miner.

This also means that the miner who first finds the block can broadcast the block, which makes it impossible for the pool to do "selfish mining".

No pool at this time actually supports miners picking and choosing transactions with GBT.  They use GBT the same way pools use stratum.  They give you the block contents, you mine it.  You have no say.

There is a whole new level of complication added to the Miner<->Pool dynamic when it comes to letting miners pick transactions.  Specifically, the pool needs to know which transactions you're including every time you send in shares so it can validate your share results are correct.  This is a MASSIVE amount of bandwidth that would be required when looking at even a 10% fraction of the pool opting for it, not to mention a large increase in stale rates for most miners since they would have to upload 300-500KB to the pool to tell them the block contents.  Hell, GBT is already a complete major bandwidth hog compared to Stratum because it has to send miners the entire block contents when it pushes work instead of just a merklebranch.

Also there is the problem when miners don't agree about which transactions should be placed in the block, i.e. a Tragedy of the Commons outcome.

It seems to me that competition will keep the pools reasonable in size if the transaction fee is cast in stone. Because as transaction volume rises, the cost of subsidizing a pool to gain market share thus increases. However, if income from transactions is significant, then the network is vulnerable to my Transactions Withholding Attack.

If we could limit the income of the pool operator and insure all the rest goes to the pool's miners, then we could surely limit the size of pools.

[patnor1011]

Talk about care of coin, little guy, profiteering, same share for everyone.... reminds me of communism.

Would you care so much or spend so much of energy for coin with zero value?

[bytemaster]

Talk about care of coin, little guy, profiteering, same share for everyone.... reminds me of communism.

Would you care so much or spend so much of energy for coin with zero value?

I don't care about same share for everyone, I care about decentralizing control of transaction selection and increasing hash power relative to price paid (inflation/etc).

[AnonyMint]

I am thinking the pool operator (server) does so little relative to work of the pool miners that it doesn't need to charge a very high fee. Thus there isn't much ability (incentive for pool miners) to undercut competitors based on fee.

So there just needs to be a slightest incentive to encourage pool miners to seek out another pool as a pool grows large. This will encourage a proliferation of pools.

How do pool miners know that a pool server isn't cheating them by paying some of the earnings to themselves pretending to be a pool miner?

Go down that line of thought and you will discover what I am thinking.

[bytemaster]

I thinking the pool operator (server) does so little relative to work of the pool miners that it doesn't need to charge a very high fee. Thus there isn't much ability (incentive for pool miners) to uncut competitors based on fee.

So there just needs to be a slightest incentive to encourage pool miners to seek out another pool as a pool grows large. This will encourage a poliferation of pools.

How do pool miners know that a pool server isn't cheating them by paying some of the earnings to themselves pretending to be a pool miner?

Go down that line of thought and you will discover what I am thinking.

The only way you can prove a pool isn't cheating is by estimating the hash rate of the pool and comparing it to the number of blocks found.  Unfortunately, you could probably still skim a couple of a percent this way.