Scaling bitcoin: the elephant in the room

[dinofelis]

Rewarding consensus is completely and utterly key to the security and usability of a PoW cryptocurrency - without it, there is no way to bound transaction acceptability.

Of course, with a PoW consensus mechanism, you need rewards, to compensate for the economic loss in PoW.  The problem is triple:
1) that you then get convergence because of economies of scale (see bitcoin)
2) that you waste a lot of value, in other words, your system is a net value burner.  You need to waste a high amount, because that waste is your only protection.  If the waste is not gigantic, anyone can attack from the outside
3) that you split the system in a set of "wasters/PoW industry" and a set of "users", and both have different objectives.

But that's more a problem of wanting to use PoW based consensus, than an inherent problem of finding consensus.  

As I said, rewards give rise to strategies.  Strategies can then be different than the desire to come to honest consensus.  It is very difficult/impossible to find rewards such that the optimal strategy for those rewards is going to coincide with the desired other outcome, namely consensus.   The systemic difficulty in reaching consensus increases if you reward the one proposing it, instead of making it easier.  Normally, reaching consensus on "sufficiently old mem pool messages" shouldn't be that hard.  Of course, the *current mem pool state* of all active network participants is different, because of network delays.  But a list of "old enough mem pool transactions", broadcast at a certain moment by one of the participants, is easy enough to check and confirm.  The "difficulty" of having several of those lists being broadcast nearly simultaneously, and arriving at different order at different network participants, can also easily be solved by including a symmetry-breaking merit function, part of the protocol, that will assign different preferences of the broadcast lists.  After a reasonable network delay, most nodes can assume they received all of the list candidates, the merit function indicates which one is to be preferred, and all nodes will come to the same conclusion.  That's the consensus.  When the network propagates the message that consensus has been reached, nodes can start thinking of broadcasting a next consensus list, built on the previous one.
If all of this is done without reward, and just on the basis of "altruism" because one wants the system to work, there is no incentive to "game the system", to "make others accept YOUR consensus list" and so on.  There's nothing at stake, apart from contributing to the good functioning of the system.  From the moment you introduce rewards, it becomes strategic to be the one that gets HIS/HER consensus proposal and not your peer's one.  Instead of cooperating in the network, you compete.  

Such a system is totally opaque to any form of long-range roll-back attack, simply because no roll-back is possible.  Reached consensus is reached consensus, done real-time and on-line.   The price to pay is that verification is only on-line.  If you leave the network, you trust your network peers that they continue to build the consensus.  
You can think of ways to recover from the improbable "global network split", by accepting, when the network unites again, a merge consensus, that accepts all that happened on the two split histories.    If you discover nodes that have another prong, with another history from a given point in time, you accept as well their transactions, as those in your prong.  
A priori, these two prongs should be compatible: this is like in a DAG like coin: there can normally not be double spends of the same coin, or it would mean that there was a node, sharing both half-nets and doing double spends, one on both sides.  Splitting the network is already difficult, but splitting the network and being on both at once is even harder.  If ever that happens, one could accept the double spend exceptionally.  That's just some extra coin creation, no problem if it is rare enough.  The important thing is that no roll-back is possible.
Again, this merging is quite straight-forward if consensus DOESN'T come with reward.  Because merging the prongs doesn't mess with rewards that don't exist.

In addition to that, you cannot control the value of a currency by changing the PoW difficulty. Value is derived from supply and demand, changing the difficulty only affects the supply side.

Well, if supply follows demand, price will stabilize.  And supply is value-controlled: if *making* a new coin has a fixed economic cost, you will ONLY make it if its market price is higher than that cost.  So as long as there IS a demand, this will converge to the set price.  You are right of course that if demand simply plummets, the lowest supply (namely NO supply) cannot go lower than zero.  You might introduce a systematic destruction of coins if you think that's a problem: a destroyed transaction fee, which is a given percentage of the transacted value.  It would indicate a "half life of coin": grossly the number of times you can transact a coin before it is gone entirely.  If you put that to, say, 1%, a coin's half life would be about 100 transactions, grossly.  I don't know if such friction is a good idea, though.  At least, it would make speculation impossible.  You put an upper cap on value.

[monsterer2]

The difficulty was brought in to prevent one "bad" actor with massive amounts of hashing power to mine all the blocks for himself and also to adapt to the technological improvement of processing power. So as soon as there are a massive spike in the total hashing power, then the difficulty will adjust to balance things out.

The difficulty is one of the core principles of the protocol. ^smile^

It is indeed. This proposal is radical, however it maintains the core principles of bitcoin, including the reason you state.

[dinofelis]

The difficulty was brought in to prevent one "bad" actor with massive amounts of hashing power to mine all the blocks for himself and also to adapt to the technological improvement of processing power.

In fact, the actual result of difficulty adaption is rather the opposite.  If you have "one bad actor with massive amounts of hashing power that can mine all blocks for himself", then the automatic difficulty adaption is *in the advantage* of this actor, not against this actor.  

Imagine a system where you have 100 "normal" solo miners that mine a block every 10 minutes.  Each miner has, on average, 1 chance in 1/100 to mine a block.  His average time of success is 1000 minutes.  His success series is a Poisson series with a time constant of 1000 minutes.  Each miner has an individual, independent Poisson stream of success with a time constant of 1000 minutes, and the total block rate is hence a Poisson stream (the union of these 100 individual Poisson streams) with an average of 10 minutes.

Now, imagine that our 'bad actor" arrives with about 100 times more hash rate.  Without adaption of difficulty, this doesn't influence the other Poisson streams.  Our new actor will make new blocks on average every 0.1 minutes, but our 100 other actors still have their block every 1000 minutes each.  

Come adaption: difficulty goes up x 101 (let us say, 100).  Or bad actor now has a Poisson stream of success of 10 minutes on average.  Our other actors are now only getting each a success every 100 000 minutes.  That's one block ON AVERAGE every 2 months.

It is not just that our bad actor has diminished revenue of our 100 normal miners: he also increased seriously the FLUCTUATIONS of their revenue.  Mining doesn't only become less profitable, it also becomes riskier.

So the difficulty adaption made life much worse for the "good and numerous" actors.

If you go from 100 to 1000 "good" actors at the start, they go from one block every 10 000 minutes (one block every week) to 1 million minutes (one block every 2 years).   Suppose that the average revenue still allows the good miners to make up for their PoW.  On average every week a block makes them get a relatively steady stream of income ; however, on average every 2 years a block is way, way too risky.

So, difficulty adaption favours the big "bad" actors.  It pushes the smaller miners into oblivion.  You limit the amount of independent solo miners to a small number.  in other words, you centralize, simply already by the effect of income fluctuations and hence, financial risk.  Add to that economies of scale, and you're done.

[Anti-Cen]

consensus, than an inherent problem of finding consensus.

When Bitcoin was being developed on a university campus by academia it worked fine but now
we know that it won't scale and consensus might work on a dozen or so machines but now we
have 20,000 machines doing the "consensus" which leads to a lot of wasted effort

The internet runs on an array of specialized nodes providing specific services and the Bitcoin "Every node for himself"
philosophy is a disaster but then the Lightning Network goes the other way when it comes to off-block private ledgers
running on banking hubs so we end up with a single point of failure unless paying miners Tx fees to open up lots of
channel for our wallets.

Come on boys we had web-farms providing redundancy on the internet twenty years ago to spread the
load because it's starting to look to me like bitcoin is designed just to keep the miners happy and yes someone
needs paying to provide hosting services but lets start thinking about the public when it comes to design and
put the miners more to the back of the queue.

CPU-Wars and PoW to make a living, i don't know what they will think of next

[korchenkov]

Hi there,

It seems to me that no one is talking about something quite obvious that has come to light quite recently regarding scaling bitcoin. None of the segwit/blocksize increase proposals are the right answer. That's why there hasn't been a unanimous agreement on the right path the choose. Both of the major camps are lobbying for changes which are only stop gap measures.

IMO, something quite radical needs to change in the way bitcoin works in order to facilitate proper scaling and to decrease centralisation literally as far as physically possible in a system with mining incentive.

What we need is something like this:

*) Homogenise - Remove the distinction between miners and users of the system
*) Reduce blocks to one per transaction
*) Users mine their own blocks when sending a transaction, no other user can mine another users block
*) Users can choose their own difficulty level when mining their blocks
*) Block reward is proportional to chosen difficulty (up to a an moore's law based maximum and with a spam preventing minimum)
*) Preserve orphaned branches of blocks and include them in a new LCR scoring system so we maintain deterministic, global state

These measures:

*) Allow bitcoin to scale indefinitely, as the block size is now as small as it possibly can be, and there is now no fixed block interval, as these tiny blocks arrive constantly
*) Miners can still participate, but instead of enabling transactions to be sent/received their only job is now in securing the chain by providing hashing power; they still earn their mining reward
*) Chain security remains strong; miners get paid for being on the longest (largest cumulative difficulty) branch, and now this weighting includes orphaned branches which are referenced within each block, so no history based attacks are possible
*) Decentralisation is maximised because there is no need for mining pools anymore, since variance in mining reward is now under the control of the user. Moreover, since only you can mine your own blocks (the PoW is signed by you), mining pools are unattractive anyway.

Thoughts?

Cheers, Paul.

New technologies such as blockchain have the potential to reduce cyber risks by offering identity authentication through a visible ledger

[monsterer2]

For anyone interested, I've attached a draft whitepaper explaining all the details of the proposal to the first post in this topic.