Bitcoin Forum
September 21, 2020, 09:31:29 AM *
News: Latest Bitcoin Core release: 0.20.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 [All]
  Print  
Author Topic: Current SegWit code does not fix O(n^2)  (Read 1206 times)
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 29, 2016, 03:51:41 AM
 #1

Like the title says. The current iteration of The SegWit Omnibus Changeset does not fix the O(n^2) hashing problem. At least according to Peter Todd:

Quote
We haven’t actually fixed the O(n²) signature hashing problem yet, although we’re fairly confident that we can, and there’s a open pull-req implementing the cache that we need.

- https://petertodd.org/2016/segwit-consensus-critical-code-review

Not necessarily an insurmountable problem. And I suppose PT might be... err... uninformed. However, it certainly puts some specious claims (e.g., April; e.g. safe scaling) into perspective. One wonders what other major claimed features of The Omnibus SegWit Changeset remain technical pauperism.

Might be worth a discussion, donchathink?

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
AWARD-WINNING
CASINO
CRYPTO EXCLUSIVE
CLUBHOUSE
1500+
GAMES
2 MIN
CASH-OUTS
24/7
SUPPORT
100s OF
FREE SPINS
PLAY NOW
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1600680689
Hero Member
*
Offline Offline

Posts: 1600680689

View Profile Personal Message (Offline)

Ignore
1600680689
Reply with quote  #2

1600680689
Report to moderator
crazywack
Legendary
*
Offline Offline

Activity: 1148
Merit: 1000


View Profile
June 29, 2016, 04:05:46 AM
 #2

Wow all the hype and it's not going to help clean up the mem pool right away, I'm I getting that right?

RealBitcoin
Hero Member
*****
Offline Offline

Activity: 854
Merit: 1000


JAYCE DESIGNS - http://bit.ly/1tmgIwK


View Profile
June 29, 2016, 04:11:36 AM
 #3

1 THING:  DONT RUSH THEM.

Whatever happens, dont rush the devs. They are working hard and they dont need pressure on their heads.

If you rush them, they might fuck it up, and we dont want that to happen. The network can easily work well 5-6 more months, so no need to rush things.

They should take their time, think ,research, develop, and test many times the proposals. Then we can implement it.

crazywack
Legendary
*
Offline Offline

Activity: 1148
Merit: 1000


View Profile
June 29, 2016, 04:15:16 AM
 #4

1 THING:  DONT RUSH THEM.

Whatever happens, dont rush the devs. They are working hard and they dont need pressure on their heads.

If you rush them, they might fuck it up, and we dont want that to happen.

Or do we?! Wink cheap coins.

I'm kidding. I just thought that with all the press in the BTC space of its implementation going live, we would actualy see it working and cleaning up he pool. That's all, just disappointing it's not ganna work out that way right now.

Lauda
Legendary
*
Offline Offline

Activity: 2660
Merit: 2643


Exchange Bitcoin quickly-https://blockchain.com.do


View Profile WWW
June 29, 2016, 04:27:33 AM
 #5

Wow all the hype and it's not going to help clean up the mem pool right away, I'm I getting that right?
No, you're not getting this right. This has nothing to do with "help clean up the mempool".

Like the title says. The current iteration of The SegWit Omnibus Changeset does not fix the O(n^2) hashing problem. At least according to Peter Todd:

Quote
We haven’t actually fixed the O(n²) signature hashing problem yet, although we’re fairly confident that we can, and there’s a open pull-req implementing the cache that we need.
While I must admit that I was unaware of it at first, I don't see this as something problematic especially if you consider the last part. I've read some parts of that review last night and it seems very well done.

Might be worth a discussion, donchathink?
Maybe. However, don't be surprise if you see a lot of 'unusual' accounts joining in to bash Segwit.

I'm kidding. I just thought that with all the press in the BTC space of its implementation going live, we would actualy see it working and cleaning up he pool. That's all, just disappointing it's not ganna work out that way right now.
Again, you have no idea what you're talking about. There's no reason for it "not to work out that way right now".
Foxpup
Legendary
*
Offline Offline

Activity: 3038
Merit: 2015


Vile Vixen


View Profile
June 29, 2016, 06:53:08 AM
 #6

So what? SegWit only makes linear verification time possible, it doesn't necessarily implement it. And the reason there's no rush to implement it is because scaling is not the purpose of SegWit! How many times do we have to explain that before people get it? It seems like people have the expectation that when SegWit is "implemented" (a word which seemingly nobody can agree on a definition) blocks will stop "being full" and transaction fees will drop to almost nothing so they can buy their coffee on the blockchain. These people then get inexplicably angry every time someone tries to tell them that this is not actually the case.

Will pretend to do unspeakable things (while actually eating a taco) for bitcoins: 1K6d1EviQKX3SVKjPYmJGyWBb1avbmCFM4
I am not on the scammers' paradise known as Telegram! Do not believe anyone claiming to be me off-forum without a signed message from the above address! Accept no excuses and make no exceptions!
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 29, 2016, 07:10:04 AM
 #7

So what? SegWit only makes linear verification time possible, it doesn't necessarily implement it.

I realize that SegWit -- in and of itself -- does not do anything about the quadratic verification time. That is but one reason I refer to it as The SegWit Omnibus Changeset - there is considerably more *stuff* in it than SegWit. Indeed, my impression is that SegWit itself has absolutely nothing whatsoever to do with the alleviation of quadratic verification time.

Quote
And the reason there's no rush to implement it is because scaling is not the purpose of SegWit! How many times do we have to explain that before people get it?

Well, you might be saying that, but the message that is coming through loud and clear is 'we don't need a simple increase in maxblocksize because SegWit!'

Well, that and 'The SegWit Omnibus Changeset is superior to other solutions from a scaling perspective because other solutions merely limit quadratic hashing time, rather than fixing it!'

Quote
It seems like people have the expectation that when SegWit is "implemented" (a word which seemingly nobody can agree on a definition) blocks will stop "being full" and transaction fees will drop to almost nothing so they can buy their coffee on the blockchain. These people then get inexplicably angry every time someone tries to tell them that this is not actually the case.

Maybe if the company line wasn't always changing, we'd latch onto it.

But whatevs...

You _are_ aware that The SegWit Omnibus Changeset is being sold as a solution to the block congestion problem, right?

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
Foxpup
Legendary
*
Offline Offline

Activity: 3038
Merit: 2015


Vile Vixen


View Profile
June 29, 2016, 07:47:49 AM
 #8

You _are_ aware that The SegWit Omnibus Changeset is being sold as a solution to the block congestion problem, right?
How can I be aware? I've actually never heard the phrase "The SegWit Omnibus Changeset" before your post, and a Google search for that phrase doesn't turn up much either. You say it's "more stuff" than SegWit, but what exactly is it? Who's selling it? Is anyone buying it? You need to define your terms.

Will pretend to do unspeakable things (while actually eating a taco) for bitcoins: 1K6d1EviQKX3SVKjPYmJGyWBb1avbmCFM4
I am not on the scammers' paradise known as Telegram! Do not believe anyone claiming to be me off-forum without a signed message from the above address! Accept no excuses and make no exceptions!
Carlton Banks
Legendary
*
Offline Offline

Activity: 2842
Merit: 2279



View Profile
June 29, 2016, 09:01:31 AM
 #9

OP makes out like he's the one reporting the issue. He's not.

None of the coup-attempt Developers noticed this, even though they were most incentivised to do so. lol

It's pretty sad really, jbreher, all you can do is dance around like a child sticking your tongue out. Why not do something productive with your "life"?


Vires in numeris
Lauda
Legendary
*
Offline Offline

Activity: 2660
Merit: 2643


Exchange Bitcoin quickly-https://blockchain.com.do


View Profile WWW
June 29, 2016, 10:23:14 AM
 #10

None of the coup-attempt Developers noticed this, even though they were most incentivised to do so. lol
I'd like to hear their failed attempts at gather excuses as to why they didn't notice it. Either they have failed to recognize this, or they haven't looked at it at all (which tells us more than we need to know).

It's pretty sad really, jbreher, all you can do is dance around like a child sticking your tongue out. Why not do something productive with your "life"?
Doesn't working count as being productive? Roll Eyes
Carlton Banks
Legendary
*
Offline Offline

Activity: 2842
Merit: 2279



View Profile
June 29, 2016, 10:54:10 AM
 #11

Doesn't working count as being productive? Roll Eyes

Depends on whether your work involves building or destroying. jbreher is hell-bent on destroying a productive (not to mention innovative) system, so he's no better than those sociopathic bullies that bat food out of a stranger's hand and then walk off down the street laughing at their hilarious "joke".

Vires in numeris
mayax
Legendary
*
Offline Offline

Activity: 1316
Merit: 1004


View Profile
June 29, 2016, 10:56:47 AM
 #12

this discussion belongs to Development & Technical Discussion. it's useless for 99% from those who are using BTC.
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 29, 2016, 07:09:02 PM
 #13

You _are_ aware that The SegWit Omnibus Changeset is being sold as a solution to the block congestion problem, right?
How can I be aware? I've actually never heard the phrase "The SegWit Omnibus Changeset" before your post, and a Google search for that phrase doesn't turn up much either. You say it's "more stuff" than SegWit, but what exactly is it? Who's selling it? Is anyone buying it? You need to define your terms.

All the features bundled together as part of the impending SegWit release.

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 29, 2016, 07:14:10 PM
 #14

jbreher is hell-bent on destroying a productive (not to mention innovative) system

Well, no. Destroying a productive system (assuming you are referring to Bitcoin) would be directly against my financial interests. Accordingly, I am working towards its success.

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 29, 2016, 07:17:34 PM
 #15

this discussion belongs to Development & Technical Discussion. it's useless for 99% from those who are using BTC.

I disagree. The amount of traffic dedicated to the subject shows that transaction volume is a topic of near-universal interest. In light of the fact that the O(n^2) hashing issue bears directly upon the ability of certain transactions to impair transaction volume, it would be a topic perfectly suitable for general discussion.

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
gmaxwell
Staff
Legendary
*
Offline Offline

Activity: 3178
Merit: 4301



View Profile
June 29, 2016, 07:36:49 PM
Last edit: June 29, 2016, 09:29:54 PM by gmaxwell
Merited by Foxpup (3)
 #16

Weird thread.

Fixing the O(n^2) issue required a change in how signature hashes are computed so that the work of hashing the transaction can be reused between multiple signatures.  Then it requires making use of that change van an optimization to actually reuse the computation.  (This optimization has earned the cheeky name, hashcache).

The segwit consensus rules do the former.  The segwit PR to Bitcoin core didn't include the latter optimization, because it isn't wasn't for the system to work (assuming we don't care that it can be slow), and packing everything in at once complicates review and increases risk. The correctness of the optimization is easier to verify as a change by itself, and the correctness of segwit was easier to verify without the optimization in the way.

But the code to actually make use of the format change is the queue as well https://github.com/bitcoin/bitcoin/pull/8259  (which was open 5 days ago, but the code for it was written January 19th).  This optimization is also part of the btcd implementation and has been since they originally wrote it.

Considering this, the presentation of this as some kind of flaw or surprising find make it look to me that people are desperate to find something wrong.  I suppose that is good: more eyes may reduce the risk of unbound actual issues. Hopefully the next time it won't be over a missing optimization which we've had an implementation of for six months. Smiley
franky1
Legendary
*
Offline Offline

Activity: 2884
Merit: 1753



View Profile
June 29, 2016, 08:16:31 PM
 #17

Weird thread.

Fixing the O(n^2) issue required a change in how signature hashes are computed so that the work of hashing the transaction can be reused between multiple signatures.  Then it requires making use of that change van an optimization to actually reuse the computation.  (This optimization has earned the cheeky name, hashcache).

The segwit consensus rules do the former.  The segwit PR to Bitcoin core didn't include the latter optimization, because it isn't wasn't for the system to work (assuming we don't care that it can be slow), and packing everything in at once complicates review and increases risk. The correctness of the optimization is easier to verify as a change by itself, and the correctness of segwit was easier to verify without the optimization in the way.

But the code to actually make use of the format change is the queue as well https://github.com/bitcoin/bitcoin/pull/8259  (which was open 5 days ago, but the code for it was written [January 19th).  This optimization is also part of the btcd implementation and has been since they originally wrote it.

Considering this, the presentation of this as some kind of flaw or surprising find make it look to me that people are desperate to find something wrong.  I suppose that is good: more eyes may reduce the risk of unbound actual issues. Hopefully the next time it won't be over a missing optimization which we've had an implementation of for six months. Smiley

lets get to the short and curlies of it.
lets not talk about the segnet(altcoin/sandbox) implementation. or the testnet(altcoin/sandbox) implementation.. i truly hate people back-dating availability by including the time its played around with in sandboxes of non bitcoin data

but in regards to a publicly downloading implementation that handles real bitcoin data on the real live bitcoin network.
will the optimization be included in the next release. (just needs to be pulled) or are you saying it wont be because it "complicates review and increases risk"

im asking respectfully. will the next version of bitcoin-core including segwit be "optimized" or will it be another release at a later date that the "optimization" is added.

all i ask is a simple reply without waffle. even something like "next release" or "at a later date after initial bitcoin core segwit release" would suffice

I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER.
Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
Carlton Banks
Legendary
*
Offline Offline

Activity: 2842
Merit: 2279



View Profile
June 29, 2016, 08:31:37 PM
 #18

Frankys, do something. Something real, in the real world. Anything, to prove that you're not just some computer science back-seat driver. You just have no clue about this stuff, and seriously expect to waste everyone's time with your rambling moronic over-wrought tl;dr posts.

Vires in numeris
gmaxwell
Staff
Legendary
*
Offline Offline

Activity: 3178
Merit: 4301



View Profile
June 29, 2016, 09:35:34 PM
 #19

lets get to the short and curlies of it.

Can you explain to me what the change does and what significance it has?

I think that would do more to get the details, like making it clear that your "concern" is motivated by harassment rather than actual concern.

Quote
but in regards to a publicly downloading implementation that handles real bitcoin data on the real live bitcoin network.
will the optimization be included in the next release.
I expect it to be included in any release with segwit activated.
jbreher
Legendary
*
Offline Offline

Activity: 2870
Merit: 1504


lose: unfind ... loose: untight


View Profile
June 30, 2016, 12:06:44 AM
 #20

Considering this, the presentation of this as some kind of flaw or surprising find make it look to me that people are desperate to find something wrong. I

Hello Gregory. I was not trying to insinuate anything was wrong, per se. We layfolk are not party to the detailed development plans, and that is OK. However, with several Core supporters deriding alternative node implementations for limiting the effects of the O(n^2) issue, rather than solving it head on -- specifically with the claim that The SegWit Omnibus Changeset addresses this in a better manner -- it is somewhat of a surprise to learn that this feature is not yet integrated. At what point is this feature scheduled for integration? How long has it been in test, and in what environment?

Anyone with a campaign ad in their signature -- for an organization with which they are not otherwise affiliated -- is automatically deducted credibility points.

I've been convicted of heresy. Convicted by a mere known extortionist. Read my Trust for details.
Foxpup
Legendary
*
Offline Offline

Activity: 3038
Merit: 2015


Vile Vixen


View Profile
June 30, 2016, 04:33:53 AM
 #21

All the features bundled together as part of the impending SegWit release.
So then what does that mean? The SegWit code has been released ages ago. Do you mean the version of Bitcoin Core that merges the code? It's sure to have a whole bunch of new features completely unrelated to SegWit (mostly bugfixes) and anyway nobody can be sure exactly what features it will have until it is actually released.

However, with several Core supporters deriding alternative node implementations...
"Alternative node implementations"? Is that what we're calling altcoins now?

Weird thread.
That's to be expected. SegWit bashers are weird people.

Will pretend to do unspeakable things (while actually eating a taco) for bitcoins: 1K6d1EviQKX3SVKjPYmJGyWBb1avbmCFM4
I am not on the scammers' paradise known as Telegram! Do not believe anyone claiming to be me off-forum without a signed message from the above address! Accept no excuses and make no exceptions!
Lauda
Legendary
*
Offline Offline

Activity: 2660
Merit: 2643


Exchange Bitcoin quickly-https://blockchain.com.do


View Profile WWW
June 30, 2016, 06:28:55 AM
 #22

Depends on whether your work involves building or destroying. jbreher is hell-bent on destroying a productive (not to mention innovative) system, so he's no better than those sociopathic bullies that bat food out of a stranger's hand and then walk off down the street laughing at their hilarious "joke".
Fair point. At least he doesn't have a 24/7 shift on his account. Roll Eyes

So then what does that mean? The SegWit code has been released ages ago. Do you mean the version of Bitcoin Core that merges the code? It's sure to have a whole bunch of new features completely unrelated to SegWit (mostly bugfixes) and anyway nobody can be sure exactly what features it will have until it is actually released.
If you've seen his posts anywhere else before, you'd know that he calls it the "The SegWit Omnibus Changeset". I can only assume that the changes are too complex for himself and thus they must be complex for everyone.

"Alternative node implementations"? Is that what we're calling altcoins now?
That's what altcoin supporters mostly call them, yes. That part about "several Core supporters deriding.." is also false. We have Hearn (who is definitely not a Core supporter), Garzik and Gavin. All of these people have barely contributed anything to the development in the recent times.

That's to be expected. SegWit bashers are weird people.
Apparently only their (wrong) way is the right way in their eyes.

gmaxwell
Staff
Legendary
*
Offline Offline

Activity: 3178
Merit: 4301



View Profile
June 30, 2016, 07:25:32 AM
Last edit: June 30, 2016, 07:36:10 AM by gmaxwell
Merited by Foxpup (6)
 #23

We layfolk are not party to the detailed development plans, and that is OK. However, with several Core supporters deriding alternative node implementations for limiting the effects of the O(n^2) issue, rather than solving it head on

Fundamental misunderstanding, conflating the protocol with non-normative implementation particulars.  The Bitcoin protocol has a design flaw where transaction validation can take time quadratic in the size of the transaction. No implementation can avoid this wasteful computation because it is a consensus rule normative to the protocol.

With an increase in blocksize this wasteful computation could easily be turned into a system halting denial of service.

Rather that fixing it, Bitcoin Classic implemented yet another useless hard limit of transaction sizes-- to keep the bleeding at a moderate level. (Still allowing blocks to trigger 1.2 _gigabytes_ of hashing)

Segwit's design addressed the issue in two ways: One is that the extra capacity in segwit is for witness data, which is not hashed by the signature hasher. Because of this even with no fix, the worst case possible is much less significant than a plain 2MB block.  The other is that segwit changes the data structure which is hashed to not require the quadratic computation, by making the part of the hashing that all signatures would share identical that computation can be shared-- the resulting structure can be hashed with O(N) work instead of O(N^2).  These are both done, integrated, and tested since 2015.  Both are fundamental to segwit.

The point Peter Todd was making was that the segwit implementation in Bitcoin Core doesn't make use of that changed structure in the second improvement to actually save the computation possible from that second improvement.  There is an open pull request for it it just isn't integrated yet.  Btcd's does however.    This is an implementation specific difference, on my computer it does, on yours it doesn't and we're totally compatible.  Similarly, even ignoring segwit Bitcoin Core is normally many times faster than btcd, but both are compatible.

Our focus is on correctness, compatibility, and ensuring flexibility, not in getting in every last possible optimization into the system on day one. Not changing the signature-hashing algorithm, just changing the data structure, made review for correctness easier, and also allowed compatibility testing (between the naive code, the unmerged optimization, and the btcd implementation).

The important thing is that the design flaw has been eliminated for segwit txn; allowing the implementations to implement the optimization at their leisure.  It makes not a difference at all if anyone actually makes use of the new structure until segwit is activated on the network.
Foxpup
Legendary
*
Offline Offline

Activity: 3038
Merit: 2015


Vile Vixen


View Profile
June 30, 2016, 07:28:35 AM
 #24

If you've seen his posts anywhere else before, you'd know that he calls it the "The SegWit Omnibus Changeset".
I actually haven't since he's on my ignore list (and I'm only replying to satisfy my masochistic tendencies), but I did Google the phrase and fail to find anything useful. I'm just going to assume it means "SegWit plus all other Core updates I don't understand or like".

Will pretend to do unspeakable things (while actually eating a taco) for bitcoins: 1K6d1EviQKX3SVKjPYmJGyWBb1avbmCFM4
I am not on the scammers' paradise known as Telegram! Do not believe anyone claiming to be me off-forum without a signed message from the above address! Accept no excuses and make no exceptions!
gmaxwell
Staff
Legendary
*
Offline Offline

Activity: 3178
Merit: 4301



View Profile
June 30, 2016, 07:33:06 AM
 #25

If you've seen his posts anywhere else before, you'd know that he calls it the "The SegWit Omnibus Changeset".
I actually haven't since he's on my ignore list (and I'm only replying to satisfy my masochistic tendencies), but I did Google the phrase and fail to find anything useful. I'm just going to assume it means "SegWit plus all other Core updates I don't understand or like".

Don't be so hard on him on this (every other reason is good to go).  SegWit Omnibus changeset sounds like something _I_ would say-- I'd use it to refer to the pull request that implemented the segwit consensus rules, the segwit wallet support, and the huge amount of testing infrastructure.
Pages: 1 2 [All]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!