This was a recent blog written by Mike Hearn. I cannot help but think about Myriadcoin the whole time reading this and how the decentralization problem already has been answered via Myriad. However, Bitcoin has the "first-mover's" advantage, so Myriad will climb an uphill battle. But we have one thing Bitcoin never will have--5 algorithms with which to mine the blockchain. There is no way Bitcoin hardforks--whether it is a consensus problem or a technical problem.
Mining Decentralization - The Low-Hanging FruitIn recent days the Bitcoin community has engaged in vigorous debate about how to solve the problem of a small number of players, and GHash.IO specifically, dominating the network’s hash rate. In light of GHash.IO’s decision to do nothing, many suggestions have been made for possible solutions. This blog post will review some and outline what can be done to help rectify the situation.
Firstly let’s put aside solutions that don’t address the problem, or that would make things worse. Many alt coins identify “ASIC resistance” as a design goal. It’s tempting to think about rewinding the clock by changing the proof of work algorithm. But this is fraught with difficulty. Firstly, there’s really no such thing as an ASIC-resistant algorithm. Almost by definition, the only way to force someone to use a CPU instead of an ASIC is to use “run an arbitrary computer program” as your proof of work, otherwise, you are doing a specific task and an application specific integrated circuit can always have the advantage. But nobody knows how to design a Bitcoin-like system which requires execution of arbitrarily complex and ever changing programs as the proof of work. Secondly, before the ASIC era Bitcoin had to tackle a different kind of problematic miner: botnets. Several botnets stole electricity to mine coins, which had nasty side effects. For instance, at least one of them mined only empty blocks, which is worse than the situation we have now. Losing ASICs would bring them back. Thirdly, invalidating the huge investment in ASIC farms already made would have profound consequences for the future of the project and people’s willingness to mine in future.
So if changing the proof of work algorithm is not a solution, what is?
We don’t know for sure, but we can identify useful tasks that help us move towards a more decentralised future. None of them are silver bullets, but most of them are quite easy: they’re what developers call low hanging fruit. By working together and picking enough of this fruit, we might be able to tip the balance in favour of decentralised mining. To save keystrokes, from now on I’m going to call this freemining.
Freeminers, unite!
Let’s define what this word means. It does not mean no pooling. The problem we face with mining is not pooling to reduce payout variance, it’s that in the process of doing so miners give up their ability to select their own block contents. The purpose of mining is to create blocks, thus answering the question of what happened first. When miners give up control of the blocks they mine en-masse, double spending becomes possible, as does going back into the past and maliciously rewriting the block chain. It’s effectively a form of selling your vote.
There are two ways to do pooled freemining. The problem is, both routes are riddled with potholes and speed bumps that cause many miners to not bother.
1. You can use p2pool, a peer to peer decentralised mining pool
2. You could (in future) use the getblocktemplate protocol with a regular pool that supports it
Freeminers mine in such a way that they both reduce their payout variance but also create their own blocks, a process that always requires running a fully validating p2p node like Bitcoin Core. To repeat: freemining inherently requires running a full node, and today that means Core. If you aren’t running one, you aren’t decentralising the mining process. This requirement puts people off and later I’ll discuss ways we can make it less painful.
p2pool’in
P2Pool is the best solution currently available. But it faces numerous small yet resolvable problems. Smart volunteers could tackle any of them:
1. Problem: installation is too difficult
Installation on Windows is possible via a one click installer, but on Linux and MacOS X no such easy setup exists. P2Pool is a Python program and some miners get stuck in dependency hell trying to simply install it.
Solution: One click installers need to be developed for both Linux and OS X. Distro-specific packages on Linux are not always sufficient, as they can fall behind upstream or be unavailable for the users specific distribution. Ideally p2pool would have a selection of packages well maintained by the project, and also a generic Linux installer or tarball usable by everyone else.
2. Problem: the p2pool website is hard to find
Confusingly the P2Pool website is at p2pool.in as p2pool.org was already taken. The .org page dominates the real p2pool.in website on Google. Worse, the website at p2pool.org is merely a proxy to P2pool that encourages people to mine on it without actually running a full node, and charges a 2% fee in the process. Remember, if you aren’t running Core, you are not freemining. This website confuses people into mining in such a way that they think they’re helping decentralise the network, but are not.
Solution: The bitcoin.org website needs a slick, simple miners introduction that links to the right website, explains how to use p2pool properly, and shows miners how to get started. Major bitcoin related websites could link to p2pool.in to try and boost the PageRank of the real site. The owner of p2pool.org could be contacted or bought out to try and fix the flow of miners who are getting misdirected.
3. Problem: P2Pool feels clunky compared to centralised pools
One reason for the popularity of GHash.IO is it presents nice, professionally designed stats pages and monitoring. P2Pool has nice frontends available too, but they aren’t bundled so don’t give users the same slick experience.
Solution: the best of the existing p2pool frontends needs to be located and bundled out of the box.
4. Problem: P2Pool has an increasingly high share difficulty
P2Pool is a single pool and because of how it works, the more people who mine on it the harder it becomes to find a share. This can be discouraging.
Solution: P2Pool could split itself into multiple independent p2pools, amoeba style, whenever it gets too large for small miners to benefit.
P2Pool has other problems too: it can use a lot of memory and bandwidth, and it needs more miners committing to it for the long haul to reduce the variance. But the above issues are the lowest hanging fruit for improving it.
getblocktemplate
The getblocktemplate protocol allows miners to mine at a client-server pool but without losing control over the blocks that they are mining. It represents a useful middle ground between the pure decentralisation of p2pool and the centralisation of traditional pools. A GBT pool calculates the coinbase transaction that contains payouts in a centralised manner, and then lets the miner select the rest of the block. Quoting Gregory Maxwell,
“Effectively p2pool is getblocktemplate plus a distributed system to compute acceptable coinbase transactions”
But there’s a problem: although the protocol theoretically allows for miner-side block creation, the actual software implementations of it don’t do so yet. Nobody has implemented support for freemining with GBT, so pools that support GBT today still do the block creation themselves. This in turn means that with only minor real advantages over its main competitor (Stratum, which doesn’t allow freemining), some pools and mining tools don’t support it.
We need volunteers to step up and help make getblocktemplate a competitive solution:
1. The software that implements the getblocktemplate protocol needs to be upgraded and finished off, so miners can connect their own copy of Bitcoin Core to the tools and select their own blocks. Fortunately libblkmaker (the reference GBT implementation) is designed with support for this in mind, so it should not require any major refactorings or other surgery to enable. If you’re serious about this, please contact Luke Dashjr (“Luke-Jr” on IRC).
2. Mining tools and ASIC systems that don’t natively support GBT can be bridged using a local Stratum-to-GBT proxy, but this is one more system that has to be set up. Ideally this wouldn’t be necessary, or if it must be so, then the proxy should be bundled as well so there’s no setup cost.
3. Tutorials and websites need to be created to teach miners how to configure this: ideally, with everything bundled “out of the box” so setting it up can be a one click process after selecting a GBT supporting pool.
4. More pools need to be lobbied to support GBT once it’s a suitable solution for freemining. This would reduce the pool’s role to that of tracking share submissions and calculating payouts, but they would have no real power over what blocks are mined.
Making Bitcoin Core less hardcore
What all the above solutions have in common is that you must run a full node implementation, which should be Bitcoin Core today. You can’t claim to be a decentralised miner if you don’t, because otherwise you aren’t validating blocks and thus have to follow the instructions of someone else who has.
Thousands of volunteers around the world run Core for free: node operators, we love you! But we also hear your pain. Core combines many tasks into one: it processes and validates transactions, but it also serves the block chain to newcomers who don’t have it yet. This latter process can consume large amounts of upstream bandwidth, which is often a limited resource.
In the short term, miners can stop their bandwidth being used for block chain uploads by blocking inbound connections on port 8333 or using the -nolisten flag. The next version will have a checkbox in the GUI for it. But that means the node won’t contribute to the p2p network at all. A long-term wish of the Core development team has been to teach the software how to use resources more effectively, so making Bitcoin less costly to run. As an example nodes could advertise that they can serve only parts of the block chain instead of all of it. This would allow people to give Core as much bandwidth and disk space as they want to.
This is an advanced project that requires work on the guts of Bitcoin Core and the peer-to-peer protocol. The existing developers all have their hands full with other tasks, so we’re looking for someone to step up and prototype. It means extending the IP address broadcasting mechanism so nodes can advertise how much of the chain they’re willing to serve, teaching the software how to find them, and then allowing users to tune things so they can reduce their bandwidth usage to levels they can tolerate. This should make it much less painful to run a full node at home, or in other places where upstream bandwidth is constrained.
For developers who want to get their feet wet, there’s an easier piece of low hanging fruit. Some jerk thought it would be funny to put virus signatures into the block chain, and now on Windows a few anti-virus scanners are corrupting Bitcoin Core’s database files. A simple fix is to store data obfuscated with a per-machine key so it’s no longer possible to put particular byte sequences into the database on disk. This would help freeminers who use Windows.
Building a community
There’s one last way pools like GHash.io build loyalty amongst their userbase: humans are tribal and we love to join a team. GHash/CEX run wallpaper contests and other things to help build their brand. We need a community of people who feel passionately about freemining, and help other people to get started. T-Shirt designers, this means you! But also we need forums and support networks to help people navigate the issues: no matter how easy we make the path, having someone walk it with you will always be better
Thoughts?
