Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: plorph on February 26, 2016, 06:57:31 PM



Title: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: plorph on February 26, 2016, 06:57:31 PM
So there was an article posted, maybe here or reddit, that was like an article of an article of an article of a paper as usual. Here is the raw paper that was published by a big group of people on the technical challenges bitcoin faces to upscale to levels of visa and other payment processors. Basically they say it is going to take more than just increasing blocksize or increasing mining rate.

http://fc16.ifca.ai/bitcoin/papers/CDE+16.pdf


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: jl777 on February 27, 2016, 12:19:14 AM
So there was an article posted, maybe here or reddit, that was like an article of an article of an article of a paper as usual. Here is the raw paper that was published by a big group of people on the technical challenges bitcoin faces to upscale to levels of visa and other payment processors. Basically they say it is going to take more than just increasing blocksize or increasing mining rate.

http://fc16.ifca.ai/bitcoin/papers/CDE+16.pdf
such efforts are well underway: https://bitcointalk.org/index.php?topic=1377459.0


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: Jet Cash on February 27, 2016, 07:44:22 AM
Thanks for posting that. I was particularly interested in some of the points. The fact that blocks would propagate to 90% of nodes in 2.4 minutes, would make it appear that my suggestion of a 3 minute block interval could be managed in the current structure. The theoretical minimum of a 12 second interval, would give a good upgrade path for future reductions as the network expands. The theoretical maximum of 4Mb blocks doesn't give much hope for the future, if blocksize becomes the main emphasis.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on February 27, 2016, 01:49:25 PM
The fact that blocks would propagate to 90% of nodes in 2.4 minutes, would make it appear that my suggestion of a 3 minute block interval could be managed in the current structure.

Look up the equation for orphan rate so you can learn why your assumption is incorrect.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: Jet Cash on February 27, 2016, 03:40:13 PM
The fact that blocks would propagate to 90% of nodes in 2.4 minutes, would make it appear that my suggestion of a 3 minute block interval could be managed in the current structure.

Look up the equation for orphan rate so you can learn why your assumption is incorrect.

It wasn't my assumption, it was something that appeared to have been calculated in the posted paper.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: gmaxwell on February 27, 2016, 08:55:12 PM
It wasn't my assumption, it was something that appeared to have been calculated in the posted paper.
The paper is trying to be "conservative" in how limited they believe capacity increases from re-parameterization can come... meaning they're leaping for the largest possible increases, without regard to many considerations... e.g. how far before the existing system will certainly go off the rails.  They use this approach so they can then argue that even that much isn't enough-- a conclusion I strongly agree with.

That doesn't mean that those parameters are actually workable, however.

In terms of the interblock time, decreases have dramatic effects once you consider an adversarial setting-- honest miners end up diluted working on many forks more often, while a high hashpower attacker stays focused.  Decreased interblock times also increase the pressure to consolidate mining to a few (or one pools) by making it more profitable to do so. Many of the tools to increase reliability of shorter interblock times, like GHOST, increase other problems like selfish mining that crop up once you are considering rational actors (instead of just honest/altruistic ones).

If you chart out a simulation of how long a user has to wait for (e.g. 99.9999%) confidence that their transaction won't be reversed, as a function of the expected interblock time you end up with a chart that looks like this(ignore the scaling):

https://people.xiph.org/~greg/fn_secure_shape_napkin.png
(flip vertically for 'wait time goes up').  The scaling of this depends on factors like network hashpower distribution and latencies which are hard to measure and which change. The key take away is the derivative: if the time is somewhat longer than optimal for the network-reality, it has a fairly small effect on time-until-security ... but if it's shorter than optimal it rapidly destroys security.  This is why it's important to be relatively conservative with the interblock interval.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: nihilnegativum on February 27, 2016, 10:25:48 PM
Are there any resources on consortium blockchains mentioned in the paper?


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on February 28, 2016, 09:27:37 AM
Gmaxwell is definitely correct that you have to factor in many variables, and the fact that those with higher hashrate waste less hashrate on propagation and verification delay is one of the factors.

I did come to the conclusion that profitable mining will always centralize (https://bitcointalk.org/index.php?topic=1183043.msg13823607#msg13823607). There is no decentralized solution for as long as mining is profitable.

No one seems to talk about the fact that the Chinese mining cartel controls 65% of Bitcoin's hashrate, has vetoed every block size increase (even Classic's doubling apparently)[1], and surely their hashrate share will increase on the next halving, because marginal miners are the first to go.

Then they provably lie to us by claiming the Great Firewall of China is their justification, but they could put a pool abroad and just send hashes across the firewall. So obviously they want to fatten their oligarchy profits by forcing transaction fees up. (I am even more conspiratorial and assume they are getting free electricity charged to the collective State for a wink and a handshake). I predicted (https://bitcointalk.org/index.php?topic=340686.msg3681159#msg3681159) the block size issue and Tragedy of the Commons of mining (with a focus on block size) in 2013 and was routinely labeled loony (https://bitcointalk.org/index.php?topic=336350.0).

[1] Bitcoin has already been 51% attacked then.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: btc_enigma on March 01, 2016, 05:10:50 AM
Nice Paper


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: monsterer on March 01, 2016, 08:53:22 AM
Gmaxwell is definitely correct that you have to factor in many variables, and the fact that those with higher hashrate waste less hashrate on propagation and verification delay is one of the factors.

I did come to the conclusion that profitable mining will always centralize (https://bitcointalk.org/index.php?topic=1183043.msg13823607#msg13823607). There is no decentralized solution for as long as mining is profitable.

I am almost convinced that without profitable mining, there can be no nash equilibrium.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: smooth on March 02, 2016, 01:18:37 AM
It wasn't my assumption, it was something that appeared to have been calculated in the posted paper.
The paper is trying to be "conservative" in how limited they believe capacity increases from re-parameterization can come... meaning they're leaping for the largest possible increases, without regard to many considerations... e.g. how far before the existing system will certainly go off the rails.  They use this approach so they can then argue that even that much isn't enough-- a conclusion I strongly agree with.

That doesn't mean that those parameters are actually workable, however.

In terms of the interblock time, decreases have dramatic effects once you consider an adversarial setting-- honest miners end up diluted working on many forks more often, while a high hashpower attacker stays focused.  Decreased interblock times also increase the pressure to consolidate mining to a few (or one pools) by making it more profitable to do so. Many of the tools to increase reliability of shorter interblock times, like GHOST, increase other problems like selfish mining that crop up once you are considering rational actors (instead of just honest/altruistic ones).

If you chart out a simulation of how long a user has to wait for (e.g. 99.9999%) confidence that their transaction won't be reversed, as a function of the expected interblock time you end up with a chart that looks like this(ignore the scaling):

https://people.xiph.org/~greg/fn_secure_shape_napkin.png
(flip vertically for 'wait time goes up').  The scaling of this depends on factors like network hashpower distribution and latencies which are hard to measure and which change. The key take away is the derivative: if the time is somewhat longer than optimal for the network-reality, it has a fairly small effect on time-until-security ... but if it's shorter than optimal it rapidly destroys security.  This is why it's important to be relatively conservative with the interblock interval.

Where did the data for that chart come from? I'm interested in examining it in greater detail, along with seeing the full set of assumptions and parameters.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on March 03, 2016, 01:02:44 PM
Gmaxwell is definitely correct that you have to factor in many variables, and the fact that those with higher hashrate waste less hashrate on propagation and verification delay is one of the factors.

I did come to the conclusion that profitable mining will always centralize (https://bitcointalk.org/index.php?topic=1183043.msg13823607#msg13823607). There is no decentralized solution for as long as mining is profitable.

I am almost convinced that without profitable mining, there can be no nash equilibrium.

Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: monsterer on March 03, 2016, 01:35:31 PM
Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.

How can you convince a rational miner to behave in favour of the network when to do so always costs them more than it would to behave rationally?


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on March 03, 2016, 02:21:19 PM
Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.

How can you convince a rational miner to behave in favour of the network when to do so always costs them more than it would to behave rationally?

You can't.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: monsterer on March 03, 2016, 03:10:26 PM
Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.

How can you convince a rational miner to behave in favour of the network when to do so always costs them more than it would to behave rationally?

You can't.

Agreed. Therein lies the problem. You have to compensate them, because acting in favour of the network always costs more than being purely rational.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on March 03, 2016, 03:31:11 PM
Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.

How can you convince a rational miner to behave in favour of the network when to do so always costs them more than it would to behave rationally?

You can't.

Agreed. Therein lies the problem. You have to compensate them, because acting in favour of the network always costs more than being purely rational.

It is not a problem. You asked the wrong question.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: hv_ on March 03, 2016, 03:43:22 PM
Well that is if you assume the only possible overriding motivation for the strategies that determine a Nash equilibrium are about profit.

How can you convince a rational miner to behave in favour of the network when to do so always costs them more than it would to behave rationally?

You can't.

Agreed. Therein lies the problem. You have to compensate them, because acting in favour of the network always costs more than being purely rational.

It is not a problem. You asked the wrong question.

Guess every single wallet user / coin owner would pay a litle fee for securely enter his tx into the chain. So the wallet might do some part of mining ?


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: monsterer on March 03, 2016, 03:48:55 PM
It is not a problem. You asked the wrong question.

The only way for it not to be a problem is for acting rationally to be in favour of the entire network. I'm reasonably sure this is impossible, but if you have a solution, let's hear it?


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: TPTB_need_war on March 03, 2016, 03:51:28 PM
I told you several times in the past. I eschew repeating myself. I have 10,000 posts.


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: monsterer on March 03, 2016, 04:03:13 PM
I told you several times in the past. I eschew repeating myself. I have 10,000 posts.

Why bother to post anything at all in this thread, then?


Title: Re: Study Suggests Bitcoin Needs Major Changes to Scale UP
Post by: gmaxwell on March 03, 2016, 06:43:21 PM
Where did the data for that chart come from? I'm interested in examining it in greater detail, along with seeing the full set of assumptions and parameters.
The specifics are rubbish. Unfortunately, making an accurate estimate for the system involves things that are constantly changing and can't be measured-- and the story it tells doesn't hold when mining is very centralized.

The point I was attempting to make was that the general shape (which I believe is robust across parameters; excluding ones which centralize mining); is such that if you can't be sure to get the parameters right, you want to be fairly confident that they're wrong in the too-high direction, because being too low (relative to the network geometry) is bad compared to being too high. This has also been demonstrated in practice somewhat: there was a "liquidcoin" altcoin created with fixed difficulty, and as its hashrate increased its orphan rate went through the roof-- eventually it stopped converging completely.

(Though I do agree with the general sentiment, that if we were taking additional risk of a given amount, its more attractive to take it in the form of decreased intervals than increased blocksize; but unfortunately the decreased intervals involve strictly more uncertainty.)