I think that it is entirely possible for the Botcoin network to be scaled so that it can handle all the transactions that would be occurring during a normal day, but there has to be the implementation of the 2MB blocks for right now.
I'm not sure why this has to be such an argument and why it can't be scaled. As long as Bitcoin remains as decentralized as it currently is, would that not be beneficial to the network?
Agreed; we just have to watch out for the quadratic growth sigop verification issue. Also, there is the issue of releasing multiple changes over a short period of time to be managed. Also, a fee of 5¢/transaction isn't so burdensome, is it? Finally, we *really* need *all* wallet software to automatically and by default provide for a fee which will get the transaction blocked pretty quick and set user expectations accordingly. |
|
|
|
your assumption that quadric wont be solved in april.
True enough, I did assume the same software running on both rasPi2 and 3 (which you did too to get to your 2660 number, so I guess that was fair, no?). Do we want to then make an effort to recalculate the rest using my 2309 number and ignore the question about linear scaling with GHz? |
|
|
|
laudas evidence is not science or technical analysys.. but an image he is advertising..
Gosh, I'm pretty sure that's what we are all doing (me included). I do try hard to get the math right but I am hand waving a lot along the way trying to impress everyone; sorry. I do examine each and everyone's posting with a critical eye; nothing is taken as gospel as is. |
|
|
|
as for the raspberry pi debate
RasPi 2. 900mhz
1gm ram
RasPi 3 1.2ghz (33% increase)
1gb ram
33% increase in processing speed in just 1 year..
say rasPi2 could handle 2000 signature verification's every couple minutes.(understatement used for basic maths)
rasPi3 can handle 2660 signature verification in the same time.
Wait; that doesn't take into account the quadratic growth aspect. The correct calculation is, if rasPi2 can do 1 sigop in t and 2000²*t in a couple of minutes then rasPi3 can handle 1 sigop in 0.75t (maybe) and so in those same couple of minutes we have to solve; X²*(0.75t) = 2000²*t X=2309 Certainly a nice increase but not 2660. Does sigop verification scale linearly with GHz? |
|
|
|
Thank you so much for the link. I am reading through it but it is long and will take time to digest. But honestly I will most likely tire of it and not finish; sorry. I did find the chart on page 43 showing lines of code by developer. Please help me understand the conclusion we are meant to derive. |
|
|
|
we need to get Gavin and Vladimir in the same room and ask them these questions.
I think you'll see that the reason they disagree isn't because they disagree about technical limitations but rather different visions.
One believes bitcoin full nodes need to run on a Raspberry pie the other doesn't think thats necessary.
Ah, vision; this is always worthy of lively debate. As much as I love a deep technical dive into the underpinnings and the passions let loose, they are often misunderstood by the observers. My personal hope is those that are able to do are doing and so not available to debate. They will prove themselves in the only way that matters; deliver working code. The rest of us that can't (or won't) are left to do our best at picking up the ideas of the doers and representing them as well as we can. In my humble personal vision, I see Bitcoin being adopted very widely long before the fees climb. But blindly cranking up the block size without constraining the impact of transactions with many sigops is taking an unwise risk. |
|
|
|
I assume hardware is always getting better because HELLO! and libsecp256k1 offers 5x validation speeds.
Ah, I shall research these but they both sound like software to me. Hmm, unless "HELLO!" isn't software and you were just indicating that faster hardware is just so obvious. Certainly faster hardware is coming at us; but a quadratic growth problem will *always* out scale even the greatest conceivable hardware. A linear improvement in the software is always appreciated but again it is *only* linear (even if it is a massive 5x) as compared to quadratic growth, i.e. n². For example, suppose it takes t amount of time to process one sigop. Then a transaction with n sigops will take approximately n²*t amount time. Now we unleash the mighty libsecp256k1 5x improvement. So, we have n²*(t/5). When n is small this is great news. For example, 1²*t vs. 1²*(t/5) or 1t vs. t/5 gets us the full 5x advantage but 10²*t vs. 10²*(t/5) or 100t vs. 20t is still taking 20t and not 10t let alone 2t to do those 10 sigops. Moreover, 100 sigops works out to 10,000t vs. 2,000t; and who wants to compute 2,000t for just 100 sigops? Honestly/sincerely I am utterly delighted at the 5x offered by libsecp256k1 but against quadratic growth it pales. |
|
|
|
To scale we must either;
1) have fast enough hardware
2) improve the algorithm, i.e. verify without pairing
3) verify in the background (although the verification has to happen sometime)
4) constrain the sum of sigops/transaction in a block
Can someone please provide an example of transaction that requires two or more sigops?
this is a none issue #1 #2 is has already been done it's not like these crazy computational intensive TX are legit, you only get into trouble if you try to allow this crazy spam like TX with thousands of inputs. miners are allowed to orphen a block for ANY reason. I think it's perfectly valid to not allow spam TX designed to slow down validation time of the block. 1) Oh? What speed hardware can handle what number of sigops? With quadratic growth eventually some number of sigops will exceed the capacity of any conceivable hardware. 2) Please help me understand; I am not aware of this development yet. So, each input comes with a sigop; I see, actually that makes perfect sense. Hmm, would it make any sense to create a number of smaller transactions that consolidate the many inputs? Perhaps it would be possible/better to avoid creating addresses with small amounts in the first place. Just trying my best to understand. I appreciate your patience and efforts. |
|
|
|
|
A transaction will have one or more sigops (signature operations). Let's denote these as S1, S2, ..., Sn. Apparently verifying these involves considering pairs, e.g. (S1, S2), (S1, S3), ..., (S1, Sn), (S2, S3), (S2, S4), ..., (S2, Sn), ..., (Sn-1, Sn). This leads to quadratic growth of the compute time. To avoid running behind the live stream of blocks, transactions are limited in size; they cannot span multiple blocks. Furthermore, the block size is limited; currently to 1MB.
The least compute intensive block is composed of a single transaction with only one sigop. The most computational intensive block is also composed of a single transaction but with as many sigops as will fit. A block with two or more transactions will be less computational intense. A block full of many transactions, each with just a single sigop, is minimal in terms of compute power required to verify it.
Let's denote transactions, Tk, where k is the number of sigops. A block is composed of a set of transactions, (Tk1, Tk2, ..., Tkn). The verification time is then approximated by K12+K22+...+Kn2.
To scale we must either;
1) have fast enough hardware
2) improve the algorithm, i.e. verify without pairing
3) verify in the background (although the verification has to happen sometime)
4) constrain the sum of sigops/transaction in a block
Can someone please provide an example of transaction that requires two or more sigops?
|
|
|
|
|
Bitcoin Core contributors (Ordered by number of commits);
Wladimir J. van der Laan (3398)
Gavin Andresen (1100)
Pieter Wuille (966)
Cory Fields (332)
TheBlueMatt (288)
jonasschnelli (246)
Luke-Jr (239)
Gregory Maxwell (195)
MarcoFalke (128)
fanquake (118)
jtimon (112)
Peter Todd (92)
cozz (70)
sdaftuar (69)
morcos (59)
paveljanik (52)
pstratem (37)
muggenhor (34)
Eric Lombrozo (32)
rebroad (28)
domob1812 (25)
Michagogo (24)
dooglus (22)
dexX7 (20)
dgenr8 (18)
super3 (16)
kdomanski (15)
casey (15)
xanatos (14)
ENikS (13)
wtogami (12)
codler (10)
btcdrak (10)
wizeman (10)
maaku (10)
rnicoll (9)
ajweiss (8)
roques (8)
jamesob (8)
jmcorgan (8)
jordanlewis (8)
21E14 (8)
devrandom (8)
joshtriplett (8)
Nils Schneider (7)
forrestv (7)
freewil (7)
rat4 (7)
sinetek (7)
dcousens (7)
sje397 (7)
celil-kj (7)
sandakersmann (7)
runeksvendsen (7)
mrbandrews (7)
OttoAllmendinger (6)
mgiuca (6)
Matoking (6)
vegard (6)
zw (6)
p2k (6)
JoelKatz (6)
jrmithdobbs (6)
ashleyholman (6)
dertin (6)
Andreas Schildbach (6)
mndrix (5)
r000n (5)
roybadami (5)
vinniefalco (5)
Whit Jack (5)
fcicq (5)
ptschip (5)
maraoz (5)
federicobond (5)
alexanderkjeldaas (5)
robbak (5)
rdponticelli (5)
...
sorry I ran out of gas/time. So, pretty clearly "most" don't appear on the link from Blockstream.
|
|
|
|
From the former; Adam Back, Ph.D. Matt Corallo Johnny Dilley Alex Fowler Mark Friedenbach Ben Gorlick Francesca Hall Austin Hill Gregory Maxwell *195 James Murdock Jonas Nick Rusty Russell Gregory Sanders Patrick Strateman *37 Erik Svenson Warren Togami *12 Jorge Timón *112 Jonathan Wilkins Glenn Willen *1 Pieter Wuille, Ph.D. *966 * found in the latter; (# of commits). |
|
|
|
i would like to see the results of the tests you intend to do. as real results always outweigh opinion and guesswork. so your tests can actually help out alot of people
Although I am willing to participate in a test to measure the SigOp quadratic growth, I am not able without help. If someone will build the code then I will install, run, and report back on it. |
|
|
|
I feel the core dev team is not willing to make the appropriate trade offs.
I do believe we could of had the 2MB limit inplace months ago and all this pain/anxiety avoided.
but the Core dev team doesn't seem very concerned with end users pain/anxiety, as much as they are with theoretically more elegant scaling.
I feel they are programmers not qualified to manage and direct of the project, and they are making bad decision, maybe the decision they make are technically more elegant, but they are currently create a more elegant user experience. but they are convinced that the users dont matter because its Bitcoin birth right to replace central banking.
Bitcoin is up and running well (assuming one uses reasonable fees ~5¢/transaction) despite an onslaught of ill-intention persons. Shame on all of us for not adjusting our marketing messages earlier to set expectations better. Find me another system that has withstood as much and moves millions of dollars a day. Who pays the Core Dev Team? Are they doing all their work for Bitcoin for free? My sincerest heartfelt praise and admiration go out to Core Dev Team members! They are giving us something I couldn't do myself. That said, I do think there's room for improvement on the handling of perceptions front. I sincerely believe they have the overall good of Bitcoin *and* the users of it foremost in their minds. Lauda has done a yeoman's job representing the positions; thank goodness someone has the patience. |
|
|
|
|
Listen everyone: *** Bitcoin is a fantastic thing!!! *** Our internal debates are seen; they are taken as a measure of our ability to provide governance or not. There are folks lurking in here; picking up on the mood and reporting on it. Every posting should take this into account.
Championing Bitcoin first before espousing one technical aspect or another will help.
Avoiding the practice of nitpicking or worse will help.
Compromise and appeasement are valuable tools.
|
|
|
|
why can't the other miners simply orphen these blocks?
Exactly how do you plan on "simply" detecting these blocks and why would somebody orphan them? How do you classify this as an attack; was the TX that F2Pool did to clear the spam also an attack? somthing like if TX has >10,000 inputs it's no good? why not!? I wouldn't mind if F2Pool's tiny blocks were orphaned by other miners. maybe he'll stop making these tiny blocks if they do punish him for it? oh wait i read that wrong.... again, WHY NOT!? the TX in question WAS an attack, why include in the blockchain? if you can't be sure if its spam or not ( <1$ TX with 1cent fee ) fine included in the block, but if you are sure it is spam ( >10K inputs ) why include it? That's interesting adamstgBit. *Is* it *always* the case that >10K inputs is indeed spam? *Is* there *ever* a case where it is not? Can the same movements be accomplished by splitting it up into multiple transactions to avoid triggering the spam rejection? |
|
|
|
As hard as it might be to see, I really believe the crisis in front of us is one of perception as opposed to anything technical. Perceptions are manageable while the real work of sorting through the technical issues is taken out of the limelight.
The network is working and it's still relatively cheap (5cents pre TX) But we have threads that are titled " why is my TX not confirming ?? " or somthing to that effect Newbies are using bitcoin for the first time and are having a hard time, with their BTC tie up seemly never to confrim, and they conclude that bitcoin is not all that it's cracked up to be.... is this a problem? I remember when i was a newbie, i would check over and over waiting for the first 6 confirmations, everything went smoothly, but I didnt fully trust that it would go smoothly, i was afraid my money would get lost or somthing. slowly my confidence in the system grew as i used it more and understood it more. not sure i would have be able to build any confidence had i started using bitcoin today.... adamstgBit, you've hit it right on the head. We must do everything we can to help folks adopt Bitcoin without pain/anxiety. Our marketing messages need to set expectations appropriately; gone are the days of leading with the misleading "no fees". It's ok to indicate lower fees than the competition, e.g. wire transfers, VISA, *if* it is indeed true but don't touch the topic if it is not -- if 5¢/transaction is the going fee then transactions less than say $5, i.e. 1%, make less sense. Instead lead with our indisputable strengths, e.g. ~1 hour (6 blocks) to guarantee transfers of even large amounts anywhere in the world. It is just unbelievable that there are wallets out there that don't set an appropriate fee automatically and by default such that transactions get through quickly despite the dynamic environment. Once a new user grows accustom then they could dig in and find the overrides to try transactions with low/zero fees and see the natural consequences of long delays. |
|
|
|
So, one very large transaction with numerous sigops leads to the quadratic growth. Hmm, so blindly increasing the block size is asking for trouble. How many of these troublesome transactions are launched against us recently? Or is it an unexploited vulnerability?
Perhaps we could increase the block size *and* constrain sigops/transaction until we get SegWit out the door?
Well the problem is only minor at a 1 MB block size limit. Segwit should scale it down and make it linear. It should be released in April (from the initial estimates), so I don't see how you plan to deploy the block size limit and a sigops limitation in <2 months. and now you see why a 2mb+segwit released in april with 6month grace is not a problem i love it when lauda debunks his own doomsday scenario having the 2mb block limit included in aprils release is easy. plus it incentivises more people to download the april version ensuring a real chance of no contention, instead of having upgrades every couple months EG march april july. which just mess with the community too much Well, franky1, that's really interesting (although it could have been said just as well without the provocative words); 2MB + SegWit. We do want to be careful -- releasing multiple things at the same time can lead to confusion if things don't go perfectly well. I'd hate to see something going wrong and each side blaming the other. Also, if there would be a need to retract a feature/function then that would likely cause a media stir; something Bitcoin doesn't need anymore of at this time. |
|
|
|
So, one very large transaction with numerous sigops leads to the quadratic growth. Hmm, so blindly increasing the block size is asking for trouble. How many of these troublesome transactions are launched against us recently? Or is it an unexploited vulnerability?
Perhaps we could increase the block size *and* constrain sigops/transaction until we get SegWit out the door?
Well the problem is only minor at a 1 MB block size limit. Segwit should scale it down and make it linear. It should be released in April (from the initial estimates), so I don't see how you plan to deploy the block size limit and a sigops limitation in <2 months. Another thank you Lauda: If only more people took the time to make clear their positions like you do. Yep, I see it; with SegWit so close it would be disruptive to overlap it with a block size limit increase combined with a sigops limitation. I do feel it was a missed opportunity not to provide a modest block increase (even without a sigops limitation) many months ago but there's no going back now. *Hopefully* SegWit will come out on time, functioning well, and be embraced quickly without a lot of angst. |
|
|
|
Thank you Lauda: https://en.wikipedia.org/wiki/Quadratic_growth ... ouchie; so if a 1MB block with y transactions in it takes x seconds to validate then 32 similar 1MB blocks will take about 32x seconds but a 32MB block can be expected to take about (32y)²x seconds. Or is the quadratic growth on something other than transaction count? It can be a bit tough to understand (I feel you). The number of transactions is irrelevant from what I understand. it is possible to construct a single transaction that would fill up a block. From the Core roadmap: In 2MB blocks, a 2MB transaction can be constructed that may take over 10 minutes to validate which opens up dangerous denial-of-service attack vectors.
The validation time of the block itself is what has quadratic scaling. From one of the workshops last year (might help understand the problem): So how bad is it? There was a block consisting of one transaction in the coinbase, it was broadcasted about one month ago. It was f2pool that was clearing up a bunch of spam on the blockchain. The problem is that this block takes 30 seconds to validate. It's a 990 kilobyte transaction. It contains about 500,000 signature operations, which each time serializes the entire 1 MB transaction out, moves buffers around, then serializes it, then hashes it, it creates 1.25 GB. The bitcoin serializer is not fast, it's about where 60% of the validation time was. So 30 seconds to validate and that's on a beefy computer, I don't know what that is on a raspberrypi, it's not linear it's quadratic scaling... If you are doing 8 MB, then it's 2 hours and 8 minutes. There are some ways that gavinandresen has proposed that this can be fixed. This transaction would be considered non-standard on the network now, but miners can still generate these transactions.
Sincerely, thank you Lauda: I really do try hard to understand and your patience is much appreciated. It did seem unreasonable for the scaling to hinge on transaction count. Serialization is a classic scaling killer. So, one very large transaction with numerous sigops leads to the quadratic growth. Hmm, so blindly increasing the block size is asking for trouble. How many of these troublesome transactions are launched against us recently? Or is it an unexploited vulnerability? Perhaps we could increase the block size *and* constrain sigops/transaction until we get SegWit out the door? |
|
|
|
|
As hard as it might be to see, I really believe the crisis in front of us is one of perception as opposed to anything technical. Perceptions are manageable while the real work of sorting through the technical issues is taken out of the limelight.
|
|
|
|
|
Show Posts
