Bitcoin Forum

Decentralization Required, Ongoing Problem, Help Needed!

Note to moderators: I posted this in the main discussion because I believe this is a problem facing everyone, not just miners, and everyone can contribute to the solution.

I want to preface this by saying that I have nothing against any of the pooled mining operators.

Overwhelmingly they are respectable companies trying to make a profit, keep their customers happy, and are run by good people.

However, I woke up this morning and took a look at the global mining hash rate distribution on BlockChain.info.

I’m sad to report that GHash.io is back up to 42% of the global mining power, only 8% away from a global majority of mining power.

Why does this matter?

Decentralized Mining is the backbone on which Bitcoin is built.

Without a decentralized system, Bitcoin fails as a trust-less network, requiring that the party controlling 50% or more of the global hash rate be trusted.

If a pool or miner reaches 51% of the global hash power, Bitcoin can no longer be trusted.

Many people believe that the consolidation of mining power is inevitable as the barrier to entry for miners increases and the competition decreases.

Let's make a stand before it is too late and say that as a community we cannot allow this to happen, and that there is a clear solution.

I see 3 main problems for miners who would like to strengthen the network, rather than weaken it:

1. Solo mining is dead.

Unless you have a large amount of capital to invest (millions of dollars), solo mining is not an option.

For the average miner, mining solo is like buying a lottery ticket.

For example, if you took 1TH/s of mining power (~$2,500 today), and solo mined with it, it would take about 440 days to solve a single block, if your lucky.

If you’re particularly unlucky, it could take a decade or more.

Because of this, miners join mining pools to combine resources and solve blocks faster, consolidating the mining power.

2. Variance.

Centralized Mining pools solve another problem miners face, Variance. Variance is a mathematical concept, the lower the variance of a given pool the more regularly a block is found, resulting in a more predictable income.

This does not mean that miners on smaller pools make less Bitcoin, it simply means how often they make that Bitcoin becomes less predictable.

The greater the overall hash rate of a given pool, the lower the variance and the more attractive that particular pool becomes for a miner trying to predict ROI (which we all do).

3. Trust.

With 1 exception discussed later, to join a pool requires you to trust the pool operator.

This trust is essentially based on the assumption that all you’re mining power will be credited to you.

While I have no proof, I believe that most, if not all major pool operators pay their miners exactly what they have contributed to solving a given block, minus any publicly stated fees.

However, for an unscrupulous pool operator it would be a trivial thing to siphon a % of that contributed mining power for their own financial gain.

In fact, if a large pool operator did not go crazy with greed, and limited theft to a small % of contributed mining power they would most likely never even be detected and would stand to make a small fortune.

Miners need to trust the pool they are mining with, if that trust is broken, or even bent a little they will be inclined to switch pools.

If that trust remains intact, they will stay and continue mining with a given pool.

So now that we have established the dangers of continued consolidation, and the problems average miners face, how do we fix this?

Cant we just leave it up to the pool operators?

Well, the last time GHash approached the 51% mark in January of this year, they issued a press release letting us all know that they recognize the dangers of achieving over 50% of the hash power stating in the release:

“reaching 51% of all hashing power is serious threat to the bitcoin community” and also stating “the increase of hash-power in the pool is considered to be a good thing”.

Seems a little contradictory, doesn't it?

They know that reaching over 50% could be catastrophic for Bitcoin, while also displaying their desire to grow their own business, can’t really fault them for that….

We, as a community, have to solve this problem.

What is the current popular solution?

When GHash approached the 50% mark in January many miners switched to other centralized pools (Eligius, BTC Guild, Slush, etc…).

Can this solve the problem?

If their were 5 major pools, each with an equal distribution of 20% of the global hash rate, that might solve the 51% problem, but we will have still consolidated all Bitcoin transaction processing to 5 central facilities, leaving the whole network open to an attack that could bring down Bitcoin for good.

So, while distributing the hash rate between major pools solves the 51% problem, it opens up other problems that occur with centralization.

Many of us believe the best solution is a decentralized mining network, and it already exists in P2Pool.

P2Pool solves all our problems stated above:

1. Miners are not solo mining, but participating with other miners in a decentralized way.

2. The Variance of any pool decreases as more miners participate, potentially P2Pool could have a lower variance then even the mighty GHash if enough miners participated.

3. Trust-less. P2Pool is open source, anyone can download the full source and run their own node if they desire, completely controlling their own decentralized piece of the P2Pool mining pool.

4. P2Pool having over 51% of the network power does not compromise Bitcoin as it is decentralized from the start.

How ANYONE can help:

Contribute to further P2Pool development:

Make a donation to forrestv, P2Pool’s creator, for future development: 1HNeqi3pJRNvXybNX4FKzZgYJsdTSqJTbk

Make a donation to any of the other developers actively contributing to make P2Pool better (Rav3nPL is an active contributor): https://github.com/forrestv/p2pool/graphs/contributors

Hire a developer yourself and put them to work on the P2Pool source :)

Contribute to miners using P2Pool:

P2Pool has a unique feature that lets an individual donate Bitcoins that are immediately and fairly distributed to all miners currently using P2Pool, it is an awesome gift for the miners working to protect the network, and a great way to express your gratitude.

Anyone running a P2Pool node can make this donation, if you are not running a node and would like to donate there are many trusted members here who will help you out by accepting your donation and then distributing it.

Donations can be easily verified via the BlockChain, so you just need a trusted person to execute it for you.

More info: https://en.bitcoin.it/wiki/P2Pool#Donating_to_P2Pool_miners

Run a public P2Pool node:

P2Pool nodes need to be fast and reliable, if you have some technical skill you could set up your own P2Pool node for others to mine on.

There are members of this forum, who for a small fee will set up a node for you.

Mine on a P2Pool node:

Mining on your own P2Pool node is the ideal solution, if you are not inclined to set up your own node there are many public nodes available for you.

Latency (the time it takes for your data to reach the node) is the most important factor with P2Pool, more so then with traditional mining pools.

Find a few public P2Pool nodes and Ping (http://en.wikipedia.org/wiki/Ping_%28networking_utility%29) them from where you will be mining, in most cases the lowest latency P2Pool node will be the most profitable for you.

WindPath's Quick n' Dirty P2Pool latency guide:

Greater then 100ms – To far, look for a closer node
Less then 100ms – Looking good
Less then 50ms – Great, your in the zone!
Less then 30ms – It's raining hashes...
Less then 10ms – Are you sleeping in my data center?

Most importantly: Spread the word

1. Share this post
2. Add it to your signature
3. Write your own post
4. Get on social media
5. Let the world know you support a decentralized mining culture for Bitcoin.


Decentralized mining is good for Bitcoin, and it is good for you. Please support the network.



Sources:

P2Pool Main Thread: https://bitcointalk.org/index.php?topic=18313.0
P2Pool Public Pool List (main post not updated, but many pools listed in the thread): https://bitcointalk.org/index.php?topic=66182.0
BlockChain.info Hashrate distribution chart: https://blockchain.info/pools
P2Pool GitHub Repository (source code): https://github.com/forrestv/p2pool
P2Pool on the Bitcoin Wiki: https://en.bitcoin.it/wiki/P2Pool
440+ day block time @ 1 TH/s: http://bit.ly/1jtuJeN
GHash.io 51% Attack Press Release: https://ghash.io/ghashio_press_release.pdf
 

Just to add to this excellent post.  Besides being by definition completely de-centralized, p2pool also allows you to merge-mine a number of coins.  At present, you can merge-mine NMC, IXC, I0C, DVC, FSC, HUC (My apologies if I'm missed any).  Other coins have either talked about, or are actively working on supporting merged mining (MZC).

Furthermore, a number of p2pool node operators have committed to donating those proceeds right back into the p2pool.  There are even wallet addresses provided by mdude77 to which you can donate coins:

DVC: 1f56VqR9ajX3K42qSaDezvKXFmruSDg1B
IXC: xaNQ54gxowcwHyxqGtFM4vwCfS84V6fmKF
NMC: N1XhWNmvGhFL145zmQGQv7Vug6mThNh1iQ
I0C: jMDtRnMAk2WxaoAw8HUceMaxPPqAuZ8AEN

To prove he's a man of his word, mdude77's node found a block and he donated the proceeds from merge mining to the p2pool miners.

Bravo!!!... we need to spread the word on P2Pool and decentralized mining!

http://www.reactiongifs.us/wp-content/uploads/2013/08/picard_clapping.gif

Would p2pool have potential scalability issues if adopted on a scale much larger than now? For example, would the coinbase transaction be a problem with potentially tens of thousands of payout addresses? Also, if the solution to that issue is increasing p2pool share difficulty, isn't that just re-inventing something like solo mining in the long term?

You got it.  You can only split a block so many ways.  Or more correctly there is an upper bound on the number of p2pool shares per block because shares are added to a sharechain.  That means there is a lower limit on share difficulty.

If there were no pools today and p2pool had 100% of the hashpower directed at it, then with 1 share per second the p2pool share difficulty 1/600th of the block difficulty and there would be on average 600 shares found per block.

It would be fascinating then if the centralized pools actually mined on top of p2pool themselves. P2pool could act as a sort of "granularity layer" dividing the block reward hundreds of ways. We would still have a mixture of centralized pools and maybe people mining on top of p2pool themselves directly, but the variance reduction incentive to join a large pool would decline dramatically.

Nice post, even with very low hashing power I've moved to a p2pool node and have made it public on both BTC with merged mining and an ltc node.
Happy to help anyone wanting to setup one or move their miners over to a p2 node.

+1

M

Maybe a naive question here, but would P2P mining work with the getwork protocol? My miners are way outdated but enjoy cheap power so I still use them. 

Nice one OP, I completely agree. I had already decided to push my business (link in sig) to the direction of deploying P2Pool nodes for Bitcoin and others. P2Pool provides some financial incentive, but the other half is the corresponding Bitcoin Core node which is equally important to maintain the global blockchain.

I think much of the problem with P2Pool is that it is still too small and too few reliable nodes. Latency also matters a lot, and really you don't know if the node is just someones extra computer at home, or a professionally hosted server with a business grade Internet pipe. Nodes need to also be geographically close.

So, Seattle area miners, here is our first! (Lots of work going on to update our site, node front ends, etc, a work in progress  8)) The plan is to deploy several across the US as a professional pool providor using P2Pool instead of centralized servers (as I don't trust them either! Getting burned by coinex.sw was the last time I mined on a central server).

http://seattle.blockburner.net:9332/static/   Looking at a fee of 1.5%, any donations will be split 50/50 with the P2Pool devs.  (only a single Cube is mining on it from Montana, so the hashrate looks erratic, but if you want to help us test out our server, please do!)


Didn't intend this to be a shameless plug however, centralization of mining and pools is bad, and we need to fix that.



You can use Stratum proxy to get around that usually http://mining.bitcoin.cz/mining-proxy-howto

Or Multiminer can also do this (running an ASICMiner Cube on my P2Pool node above, which uses Getwork by default)

FYI my p2pool node (see sig) is in LA.  I'm committed for a year, it's on a VPS with plenty of bandwidth.  0% fees, merged mining proceeds go to everyone in p2pool.

But your point of reliable nodes is valid.  That's what backups are for.

Lastly, AFAIK, proxies don't work with p2pool.

M

maybe we need a list of nodes running on high quality data centers. my node is on the Microsoft Azure cloud so pretty damned stable and fast.
Using Azure I hope to deploy several nodes around the region being Singapore and Australia. Might do one in the US also just to provide a node out there.

Where are we in most need of high quality nodes?

I dont see how any of that solves these problems, the specialized equipment, the power consumption, the advantage of industry connections, cooling capacity, you cant stop the consolidation because bitcoin works on the scarcity model like gold, and we all know the golden run.. he who has the gold makes the rules. 

Note that I'm not trying to plug my node.  IMHO one node is the same as another, aside from geographical distance.

The one thing that conventional pools have over p2pool is merged mining.  Nodes _can_ merge, but historically the pool ops keep the proceeds to themselves to help pay for operational fees.  That's reasonable, but when it starts turning into a profit making business, it doesn't help the community and doesn't encourage miners to stay on p2pool.

Forgive me if you've seen this before, but I think it's worth repeating.

I think all p2pool nodes should convert their merged mining proceeds (after pool costs if necessary) to BTC and feed them back into p2pool, so essentially all p2pool users are merge mining.  If that's too much a bother for you, I created dedicated addresses you can send the proceeds to (unconverted) and I'll convert them to BTC and redistribute.  I believe someone posted those addresses earlier.

M

That's what Dogestreet (http://www.doge.st) is doing with proxypool (https://github.com/dogestreet/proxypool). I haven't poked around too much yet, but it would be awesome to have this for bitcoin p2pools too. It would also help a lot of small miners get on p2pools.

hmm so we currently have a bunch of independent p2pools, what about some kind of co-op venture? Entry requirements would be minimum datacentre spec and server resources as well as all nodes in the co-op would need to be merge mining and paying back those coins to the pool. i.e. the co-op all pays to one set of wallets to be converted and paid back as mdude has already set up.

we can call it co-pool the p2pool co-op that guarantees uptime, security and performance across geographically distributed nodes. (just an idea lol)

Anyway, what say you?

I like the idea. 

That said, I'm not sure I can commit to uptime for my node.  Once my host has brought it down for a few hours to migrate it (didn't tell me), and once they were down completely for an hour or so (didn't tell me).  Plus I'm still learning the ropes of running a linux p2pool node.

M

Load-balanced, zero fee, P2Pool cluster: http://p2pool.io/

I'm lucky to have some free compute on Azure. They have a nice load balancer that can now point to external endpoints. I wonder if we could have something like that automatically load balancing between nodes to ensure uptime. would need to do some testing on that...

if anyone has a company or registered for a business number or is a student I highly recommend trying to get into BizSpark or dreamspark as they both offer free MSDN benefits that include Azure compute credit. It also gives you the option of deploying to many different geographic regions and they offer both windows and Linux vm's.

Was just re-reading the Satoshi White Paper (https://bitcoin.org/bitcoin.pdf) and found this addressed directly in the conclusion:

I may look into it, maybe even creating a node. Will need to do some research on how P2P pools work.

This is a good post highlighting mining centralization and helpful countermeasures like P2Pool, but it's a bit misleading.


All Bitcoin transaction processing wouldn't be consolidated to 5 central facilities. Blockchain's hashrate distribution chart (https://blockchain.info/pools) shows about 30% is unknown, which I think is healthy. It's probably made up of tinkering solo miners, enterprise miners, less than famous pools etc., which will probably grow.

There may be 5 major/popular pools now but I doubt that will remain. GHash.io wasn't the first pool. That was Slush's Pool, if I'm not mistaken, which now has 3% of the pie slice... 3%. Just as GHash.io entered the picture and ballooned in size so too can other pools. Even now there are 9 pools not including P2Pool.

I think it's worthwhile to try and bolster P2Pool's meager 1% share, but I think we will also see more pools forming. Something they need to do, however, is take inspiration from GHash.io and get their presentation right. This was mentioned on reddit too. Let me make the point.

If you haven't seen the homepage for GHash.io do me a favor and first visit the page for Slush's Pool:

https://mining.bitcoin.cz/

Now visit GHash.io:

https://ghash.io/

It's a fricking mine shaft. No self-respecting male (of which the Bitcoin community is about 80%) would not find that appealing. Note to pool developers & entrepreneurs: you don't have to be first to be biggest. Google wasn't the first search engine. Facebook wasn't the first social network. Presentation and user experience make a difference, and that will probably become more true the more mainstream Bitcoin gets.

I am very concerned by this as well.  How ironic that the movement to decentralize currency needs to decentralize the manufacturing of currency 2.0...

I think what you have revealed here is a clear example of how easily the power and wealth of a few offsets the balance and function of any system.

This isn't about being fair, this is about being sustainable.  And the quicker we move away from comparing BitCoin in value to dollars  and other fiat currency, the more and more I think people will stop trying to make cash on bitcoin and be more reasonable about mining and/or cash exchanging.

I also think more of us need to put some serious efforts into bringing BitCoin into the real world.  No one thought the car would catch on until you started seeing people with them.  People thought the same thing about the TV, the Cell Phone and the computer.  BitCoin is the next logical step in the "things people don't know they need it yet" paradigm.

We need to all go out, tell people about BitCoin, show people the apps and the tools that are already available.  CoinBase has an amazing app for turning any tablet or phone into a payment kiosk, and even allows for tipping!  The beauty of BitCoin is the ability to put a cell phone or tablet in the hands of anyone and give them all they need to start a business.  It is time to move to the next level folks.

We just upgraded our node to a new server in a new data center, this node is blazingly fast:

http://minefast.coincadence.com

With this upgrade comes a big update to our routing system that introduces latency based routing for miners.

What this means is that as we add more p2pool nodes around the globe your miners will always be directed to the closest node, insuring the best efficiency and speed.

Announcement:


Stamford, CT: Today, CoinCadence.com, a Bitcoin startup focused on mining decentralization and transparency, announced that they have acquired the p2pool.org domain name. p2pool is a decentralized mining pool that creates a peer-to-peer network of miner nodes working together as a mining pool. p2pool is open source and any miner may host their own node and join the pool.

Milk House, LLC’s managing member said, “We have been working with the p2pool mining community to develop the most data-rich and useful p2pool dashboard available at minefast.coincadence.com (http://minefast.coincadence.com). With the acquisition of p2pool.org we now have even greater capabilities to promote the growth of p2pool and strengthen the decentralized mining network.”

The company plans a staged roll-out for the new p2pool.org website:

● The existing p2pool.org site will remain as-is until the new site is complete, mining will not be interrupted.

● By September 1st: A fresh home page for p2pool.org explaining the benefits of becoming a p2pool node operator, a public Coin Cadence p2pool node with 0% fee for those that are not able to operate their own node, and links to accredited p2pool resources and guides.

● By November 1st: Open source version of the data-rich Coin Cadence p2pool dashboard released including real time data from p2pool and the Bitcoin blockchain and an API that provides historical data.