Bitcoin Forum

I don't think there's a thread about this yet (after conference), so here it is.
https://i.imgur.com/vq132ho.jpg

Here is a transcript of the presentation. (http://diyhpl.us/wiki/transcripts/scalingbitcoin/hong-kong/segregated-witness-and-its-impact-on-scalability/) A hard fork is possibly not even required.

Gavin's explanation: (http://gavinandresen.ninja/segregated-witness-is-cool)

If this (https://bitcointalk.org/index.php?topic=1210235.msg12706195#msg12706195) is correct and witness information is prunable, how is it a solution to scaling? It would still require a block size increase. Maybe I thought I knew what scaling is, but I'm not quit grasping the concept...

So, for a lay person like me, this is basically a simple, efficient way to patch the malleability exploit while simultaneously increasing block size? And to do that we only have to make a soft fork?
Is this correct?

Scaling is not all about the block size what majority understand. Scaling could be a better way of storing data, a different level approach like LN, or something else.
From what I understood is that they could discount the witness data by 75% right now, which means that 1 MB blocks could theoretically have as much transaction volume as 4 MB blocks. Or they increase the block size for the witness part to 4 MB (the non-witness part stays at 1 MB). This is how I understood it so far. This is still a fairly new concept so I'm also still learning (Bitcoin is a constant learning process though).

Simple? Not exactly. This adds complexity, which BIP101 and XT supporters are probably going to use as an argument. If society wanted simplicity and not harness the benefits that came with complexity, we should have remained at the stone age. This kills all cases of unintentional malleability and can be implemented with a soft fork. This is correct.

Still learning; but making a post titled "Segregated witness - The solution to scaling".

Lauda, explain Segregated Witness to me like I'm five.

And to me as if I'm just born

It's a bit hard to correctly explain something so complex without leaving out important information. Let me try this: "Normally the transactionID is the hash of the signature and transaction", with the segregated witness the signatures are being excluded (as they consume 60% of the data on the blockchain now). In other words, they are going to re-work how this data is being stored (simplistic explanation without merkle tree) by excluding it from the block.

The positive outcome of this is an effective block-size of 4 MB with a soft fork. With effective I mean that they don't have to change the actual block size (that most people know of today).

---

---

Read the bolded part. Addition: By changing how the data is stored, they are saving a lot of space (hence the effective block-size of 4 MB).

But doesn't the signature verify the transaction was created by the real owner of the address? What about multisig? Is that gone with this new system? Sounds very flaky to me.

You are correct. However, I'm not talking about removing the signature data; I said excluding from the blocks.
It does not eliminate multisig, it actually solves malleability as I've previously stated and as seen on the slide.

Ok, so how are the transactions signed and does it increase the possibility of address collision? Hal Finley proposed batch signature verification long ago where it was believed the shortcut to secp256k1 would bring as much as a 20% speed increase to signature verification. By the time it was modified and implemented, in order to protect security, there was almost no speed advantage. Removing the sig verification from the mined blocks will most likely have some kind of security leak issue. I'm just not knowledgable enough to tell you what it will be. I'll be eagerly watching the development.

The data comes after the block and is connected via a hash IIRC. I don't think it increases the possibility of address collision why would it? Apparently it has been in testing mode for 6 months now, and I'm pretty sure that they would not miss a significant security leak just like that. Besides, they won't be rushing this out either way. For the exact specifics I'll have to get back to this thread as I'm very busy now and will head out (possibly stay disconnected). 

Video of great work done-

https://www.youtube.com/watch?v=fst1IK_mrng#t=36m

The size of the block chain can be cut down considerably by moving all signatures(not needed to be stored anyways) into a separate data structure and only keeping the transactions without signatures in the block chain.

Side benefits as well
    Much simpler future opcode additions/upgrades
    Solves malleability problems
    Fraud proofs for every single consensus rule, making SPV much much more secure and lazy validation
    
Also can be deployed with softfork which without this upgrade would have been extremely difficult to implement.

Basicly, this is an example of a scaling solution with absolutely no tradeoffs, The consequences are all positive. This is just one peice of the puzzle that needs to be rolled out to scale but objecting to this improvement is non-sensical.

Sildes- https://prezi.com/lyghixkrguao/segregated-witness-and-deploying-it-for-bitcoin/

Best yet, code is already done and been tested for over 6 months --

https://github.com/ElementsProject/elements/commit/663e9bd32965008a43a08d1d26ea09cbb14e83aa
https://github.com/sipa/bitcoin/commits/segwit


"Size = 4*BaseSize + WitnessSize <= 4MB. For normal transaction load, it means 1.75 MB, but more for multisig."
https://twitter.com/pwuille/status/673710939678445571

to me this translates to creating a new chain. called the witness(pruned) chain. where all the old blocks are pruned of the the signatures.
this is not a soft fork.. this is remaking a new chain.

also because there still needs to be a chain containing the signatures.. that would be the real bitcoin which will still bloat..
and if anyone still wants to be a full node. then they need to now have 2 chains.. meaning more data as some tx data is duplicated by holding both

witness chain would supposedly be used for liteclients. but its much easier to just let lite clients only download the tx data of addresses they control. and then find a real solution to bitcoins data bloat, without creating a new chain or risking security bugs related to not having signature checks

Yes, Soft forks are new chains in all cases.



There is only one chain. The softfork would allow old clients and implementations to bloat the blockchain still and the newer clients would prune off the unneeded signatures. You are insinuating that the security model would change introducing potentially new bugs. This is false as the security model remains exactly the same.

The blockchain signatures are segregated-- one has the transactions minus signatures, and one has just the signatures. Full nodes will still download and verify both, so security isn't reduced or compromised .

This is basically creating a better SPV/lite client that has the ability to use fraud proofs for authentication. It is not meant to replace Full nodes!

An Airline only allows you to bring a limited amount of weight and bags onboard a plane for your luggage. They have to be fair and have a consistent policy for their passengers while at the same time recognize real life physical limitations and risks. There is a careful balance to be made where they make their clients happy with allowing them more luggage but at the same time keep the weight considerations reasonable for speed , limiting fuel costs and safety.

They previously had a policy of allowing 2 carry on bags weighing 10 Kg in total, but overtime with a growing economy their clients grew more wealthy and needed to travel with more bags because they were flying to Aruba for 2 week vacations, instead of 1 week. The clients were demanding more bags and up to 40 Kg of luggage weight. They had a choice to increase the size of the plane but the accountants and engineers were concerned because the fuel costs would increase and the safety of the planes may be compromised as well with so much weight. The sales department also voiced an objection that larger planes would cut down on their flying routes leading to less possible destinations clients could visit.

It was known since 2011 a safe and effective solution would be to separate the less critical luggage on a train or boat and the important luggage could go with the client to solve these problems but the airline kept delaying these changes because of distribution and contract problems with the railroad/shipyard companies. Redirecting every railroad or shipping route would be a logistical nightmare. Another concern is passengers didn't want their luggage being mixed or misplaced within the trains/boats as they were keen on some of the new quick luggage check-in and fraud protection processes the airlines were developing and there is no way they could get any of that with dumping their luggage on a train or boat.

One day a bright young engineer recommended a novel approach to solve these concerns. He indicated, "Look, we already have shipping agreements with Fedex/DHL and lease out a certain amount of space on their cargo planes that flyout from the same airports. We would have to increase capacity on them but they are efficient and we won't have to reroute all the trains or deal with the nasty railroad companies and our passengers could still bring their important luggage through our new fraud protection and quick check-in processes. Our passengers won't have to pay more and get everything they want. We will have to secure larger contracts with the airfreight companies but ultimately they will benefit from more business and we will benefit from more flight routes and clients." ... followed by an applause from the board of directors and some quick recommendations to roll out the long awaited changes immediately.



***edited to better explain the complexities and tradeoffs in the analogy.***

But what amount of transaction per second would this deliver? can this really compete against LN in terms of being able to be at the same level of VISA and the like? This looks good on the surface but I would wait for someone like gmaxwell to comment on it to see potential flaws. Did any of the core devs speak on this?

Whew, now that was a good explanation

Bitcoin can and will surpass VISA alot. just be patient and wait some years my friend.

"Size = 4*BaseSize + WitnessSize <= 4MB. For normal transaction load, it means 1.75 MB, but more for multisig."

More than 1.75 but less than 4 effective. Thus between 6 to 28 tps vs current 3-7tps.



It isn't meant to compete but merely be one piece of the puzzle that solves many other concerns as well. Sipa is positive towards other BIPs and LN. This softfork is not meant to replace a future hardfork to address scalability.


All of the core devs are familiar as this has been discussed since 2011. The only real objection is from the fact that previous to luke-jrs brilliant discovery they assumed a hard fork would be needed which would be very difficult to accomplish. I know of no core devs against this implementation because if you understand it, it is a no -brainer.

Sounds alright to me so far, but I will need to familiarise myself better with it even still.



The presentation directly after Pieter Wuille's is a good watch. Demonstrates the (exemplary) capacity of:

• Basic Lightning (CLTV only)
• Intermediate Lightning (CLTV + CSV)
• Advanced Lightning (CLTV + CSV + Segregrated Witness)

Looks like the kind of transaction rate scaling we're after (100's of millions of users supported under the Advanced scheme)

1. no, its not about tx per second. its about relaying speeds of solved blocks to clients imagine it this way.
a pizza is 1mb and oneside has pepperoni, the other has anchovies.. the proposal is to cut the pizza in half but you as a full and hard worker(full node miner) still eat the same 1mb whole pizza.. thats right the full 1mb limit nothing has changed.

but because its in 2 halves, its now easier for your witness to CLONE(relay) just the pepperoni half of the pizza because he doesnt like anchovies(signatures). and thus he as a separate thing holds onto just 500kb pepperoni of his copy of the pizza

so each 10 minutes you(full node/miner) are still eating a whole 1mb pizza and your witness has cloned a copy of the pepperoni side meaning he is only eating 500kb per blocktime.

for the full nodes mining the real bitcoin blockchain. nothing has changed, the data totals are the same and thus no extra transactions can fit into bitcoins blocks. the only difference is that the miners data has a split in the middle to make it easier to relay a copy of a half to the network of lite clients..

its not changing the transactions per second. and is only reducing the data per block for the witness(liteclients) who dont want to download everything.

2.
witness wont scale anything.. its just a tool for SPV wallets to not be as bloated.. nothing to do with the real blockchain that gets mined properly
LN has more potential to scale however i have seen a few limitations that dont make it perfect.

Ok, that doesn't seem so bad at all. It's just a sidechain solution.

It's not?

This is all 100% true in itself as Segregating the signature isn't intended to increase TPS by itself, but with a soft fork core can increase the witness data set limit to 4MB thus indirectly increasing TPS to 6 to 28 tps. It isn't the Segregation in itself that increases the TPS but the partial and limited blocksize growth that only applies to part of the data set. Thus after the softfork there will be a 1 and 4MB limit concurrently depending upon which dataset.

Full nodes will take on more data, but lite nodes will have new features like fraud proofing (although this isn't as thorough as using a full node) and take on as much data as the blocksize limit never increased and have 6 to 28 tps.

There will than be 3 categories of nodes-
1) Traditional and the least secure SPV nodes
2) Lite / Prune nodes with SW nodes that have some fraud proofs
3) Full nodes

Well it was tested as a sidechain in elements alpha , but this is actually intended to be an immediate soft fork on the main chain.

well its not a side chain in itself.. but yea if you have 20 blockchains linked together.. some lite clients may want a reduced history of all chains.
although the full nodes/miners still have to deal with 20 full blockchains. lite clients can have a basic summerised copy.

i think the solution to lite clients. is to not send out a command to the network with a block height attached asking for all new blocks since X.. (to resync)
but instead send out a command to the network with a public key attached asking for all tx data of that key.

that way liteclients only store the data relevant to the keys it holds

if you increased the tx data to 4mb but kept signature to 1mb... thats just the same as having a 5mb block.. and full nodes will still have to mine upto 5mb of data whether its split or together..

so witness is not itself causing more tx per sec.. it is however usingits ninja skills to just up the block limit to 5mb under the subterfuge that its something special.

so without meandering..
explain the benefits to miners and bitcoin full node users who (imagine last week) agreed to a simple 5mb block limit increase.. and today you are trying to sell them the idea of witness..
forget about lite user benefits.. explain the benefits to the miners/full node network

You know, I'm quite disappointed. Have you even looked at the picture shown in the original post? Of course you have not, else you would not be asking the ignorant question of why this is something special. All the side-effects of this are presented on that picture and they are just pure pros. Unintentional transaction malleability gets fixed, script upgrades become simpler, fraud proofs, etc.

Yeah, I can understand that. Those of us that primarily use SPV clients will still be secure. Miners will hash reduced size blocks and full nodes will be the only ones burdened with an even larger carrying capacity. Works for me. I only use lite clients now and haven't run a full node in over a year.

Close, its actually more complicated than this because of multisig and is essentially reflecting miners having an effective increase between 1.75MB limit but less than 4 MB limit . Thus if blocks are 100% full expect ~3MB of bloat per 10 min avg for full nodes.



Many user benefits but to address your point concerning miners and full nodes:

Negatives vs status quo-
Increased Disk bloat and bandwidth requirements (Just like rasing the rate to 1.75- 3.5MB limit)

Nuetral-
UTXO unaffected

Positives-
Increased TPS capacity thus more mining fees and Better job security as profits are now diversifies from tx volume instead of reward alone which will be quickly dropping

There are other positives to miners from benefiting the overall health of the bitcoin ecosystem but I think you are trying to focus on the short term selfish incentives of miners alone absent from external motivations.



I think franky is just trying to focus on the incentive structure for miners to see how well this will work. It is a fair question to make and try and remove any external goodwill of miners and consider worst possible outcomes. Sometimes some skepticism is healthy and we shouldn't rely on the goodwill of miners to secure us into the future.... that is merely another layer of security that involves humans I appreciate.

Well there has been 6 months of testing and Pieter Wuille did suggest we ASAP implement this improvement so I would expect a few weeks to allow outside criticsm to be taken into account but it is going to happen quickly as his statement from the transcript is -

"This is my proposal that we do right now. We implement segregated witness right now, soon. "

I see. Thanks for responding. I have deleted my post since I am in the process of reading more about it. I didn't want to clog up the thread too much.

I must say that this is a very interesting concept. This also gives us time, if implemented, to find another long-term solution since I doubt that even a 4 times increase will be enough in the future!

I see the commits from Pieter Wuille (sipa) for Segregated Witness at https://github.com/sipa/bitcoin/commits/segwit

Appears that this is a done deal, am I correct in that statement? If so when will the soft fork take place. Thanks.

Ok, I needed to update my ELI5 analogy to more accurately reflect Segregated witness.


"This is my proposal that we do right now. We implement segregated witness right now, soon. "  --  Pieter Wuille

I have watched this video twice but still don't really get it

So far as I understand Pieter's proposal, a spending transaction is first validated by the mining nodes by checking its signature, then after it is validated, the node will include the transaction in the next mined block, but without that signature

However, this will create a security question for the other nodes receiving such a block: Without signature, how do they know the transaction is valid? Is it possible for a rougue node to forge a block with lots of invalid transctions but appears to be all validated by that node?

Normally all the adjacent nodes have similar mempool which contains similar transactions. If a transaction is already in their mempool they will know it is valid or not. However, if a new block arrives from far apart which contains many transactions not in their mempool, they need all the data of the transaction, especially the signature to validate it

Otherwise you would be able to spend bitcoin at any address without the signature

I'm not very sure how it works today, e.g. what kind of check other nodes do to make sure it is a valid block with all valid transactions, have to check it back


A side thought: I suppose the original Satoshi client design tried his best to be both secure and efficient. If there is such a large room of efficient increase without impacting security, why it has not been implemented before?

I remember 2-3 years ago there was a talk about the separation of transaction data using two chains to get rid of the forever growing block size pain, then after some deeper discussion it turns out you can not secure the separated part from being tampered, so it seems everything in a transaction is a whole and need to be included and secured by hash power altogether, which is the most efficient way

"What are the advantages of this? It allows you to drop the signatures from relay whenever you are relaying to a node that is not actually doing full-validation at the time. "

So this solution still does not help full-validation nodes, it only improve the relay speed for SPV nodes. But SPV nodes could just connect to the latest full node to get faster access, they are not the bottleneck for the whole network

What are you talking about? Advantages: 4x increase in traffic that is possible via a soft fork, fix to malleability, simpler script upgrades (quite important as well), fraud proof, less bandwidth for light nodes and historical sync. I think that it could help full nodes (not sure yet) by not syncing all of the historical data (i.e. signatures), because it isn't really needed. More discussion about this is definitely needed.

https://www.reddit.com/r/Bitcoin/comments/3vurqp/greg_maxwell_capacity_increases_for_the_bitcoin/


Here is an exact breakdown of an example of the savings in overhead and why capacity can increase-

All in all, seems to be even less than a x2 capacity increase:

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011869.html

For how many month will it be 'the solution' in 2016, the year of the Great Halvening? April or even May?

Lol you clearly still cant read with any proficiency.


You're thinking of proposals that people dislike, not proposals that have received a warm welcome. Pieter Wuille is highly respected, mainly because of his amazing computer science work so far. This sounds to be more of the same, and all you can do is lie and hate? I feel genuinely sorry for you Zara, it can't be much fun, for you or your friends, being such a bitter person.

I don't understand any of this, maybe there can be something written for the less technical among us?

Just ignore anyone who ignores this:
This is very important (the first 3). This proposal received a warm welcome on the conference.

---

---

There already was an airplane analogy. Re-read the whole thread.

I've updated the thread title a bit.

It was never presented or intended to be "the solution" and merely a small peice of a puzzle in a comprehensive and holistic approach to addressing Scalability and capacity.

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011865.html

Your stupid projections. Are you crazy? We welcome a 1,8x increase. But I say it will not be the solution. That's even not enough to survive until the halving date.

"I think that right
now capacity is high enough and the needed capacity is low enough that
we don't immediately need these proposals, but they will be critically
important long term."

Is this a joke? What is long term? 1 month before the halving date oder 1 month after?

No developer is suggesting it is "the solution". The fact that you cannot see that after we keep clarifying and providing contrary evidence suggests you are acting irrationally.


Serious people do not pretend to know the how quickly we will need to scale the capacity needs in the future.

This is why a very responsible approach is outlined here-




If we make sloppy and dumb capacity upgrades in fear and haste we don't have the right incentives to make the right tradeoffs and develop optimal solutions which benefit all. There is no harm in having tested backup plans if the need arises and demand increases more than expected.

The paypal and Visa network have outages all the time and the ecosystem doesn't simply crumble in chaos. with bitcoin the worst fear is some transactions being delayed from confirmation while payment processors rely on 0 conf verification more heavily while an agreed upon and tested hardfork gets deployed. This is far less of an issue than the Visa payment network/paypal being down.   

If this is your impression of being welcoming, then I'd certainly avoid any event where you were greeting the guests.

And more doomsday deadline scaremongering, after last time? Come on now.

Some say that a 'monster softfork' would be a dumber 'immediate' short term last minute solution than a simple increase of the block size, when everyone can see that the capacity is at the limit right now and the halving event is just 7 month away.

 It is almost as if he believes that the only thing holding bitcoin back is capacity...and as soon as we build the 100k seat stadium it will be filled. Even if these optimistic delusions are correct it would be disastrous for the bitcoin ecosystem to have that rapid of growth in such short order.

I am so glad there are enough rational and calm developers contributing and who are aware of the nuances and trade offs in decentralization and security.


Define simple. Bitpays BiP 101 patch actually has more lines of code than the SW softfork.

Ah, the ignoring user changed the title of his thread.

More projections out of thin air. Crazy.

ETA update-

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011875.html

"- Segwit BIP is being written, but has not yet been published.

  - Gregory linked to an implementation but as he mentions it is not completely
    finished yet. ETA for a Segwit testnet* is later this month, then you can test as well.

Wladimir"


I assume Wladimir is refering to rolling segwit into the main bitcoin testnet instead of being tested merely in elements sidechain testnet .

Pieter Wuille is highly respected because he is one of the devs that made the right conservative approach during the 2013 fork. Still, his proposal can not be taken without careful review

We know that every large player here in bitcoin community never listen to anyone else but only themselves, so unless a proposal can be understand by them it will just be ignored. People ignore Gavin's solution just because they don't understand the potential risk for his radical change in block size limit. Similarly, if Pieter's solution is so complex (much more complex than Gavins) that it is not understandable for majority of the large players, it will just be ignored. You can never convince the large mining pools with those slides

Am I wrong to assume that the large mining pool owners aren't well versed in the rudimentary basics of bitcoin? If I can understand it, I am sure they can.

It's mostly a lack of communication and language barrier. (with regards to chinese mining pool)

Back to the basic: How can you validate a transaction without the signature? You can not, otherwise a node will be able to spend anyone's coin and bitcoin will immediately worth nothing overnight

It seems that after the validation and the transaction is finalized in the block, if node can verify the validity of the block, then it requires no detailed transaction data, since everything in that block is regarded as valid

However, if a rogue node send out a block appears to be valid but with transactions with wrong signature, then if other nodes do not validate each transactions in the block, they have no way to know if those transactions are valid, if they approve those blocks and start to build block above this block then those invalid transactions can even spend satoshi's coins

Therefore bitcoin is designed so that any nodes are able to independently verify every transaction, so that it is secure on every node. And that will include lots of data in each block

In order to come over this limitation, Pieter suggested to redesign bitcoin, where the hashes of all the signature goes into the coinbase. But still, without the original transaction data together with signature, you have no way to prove that the hashes are correct, you would still need all those data to prove the validity of each hash. However, those data would be in another chain, then it will reduce the data on main chain, but put the data into a side chain. And in future, that side chain will have all the capacity problem that bitcoin chain have today, even more difficult to deal with

1. Not a side chain, a parallel chain. For every header'ed block, there must be a corresponding SegWit block. Not a side chain.

2. SegWit chain is prunable. Future chain bloat not a problem. You haven't taken it all in yet (and neither have I)

To be exact it's more like a parallel merkle tree than a chain if I understand it correctly.

Do you really understand what Pieter is suggesting? And the possible consequence in future if it is adopted?

We have observed, even simply raise the blocksize limit to 8MB which is simple enough for anyone to understand will result in huge resistance, this kind of complex solution which requires decision makers to have deep understanding of the inner workings of blocks and nodes would never get serious acceptance if any at all

The objection to BIP101 was on the basis of it's efficacy, not it's simplicity.

If you're looking for a simple system, a trustless cryptographic consensus network is possibly the wrong place to look.

Not only these. I guess chinese mining farms  are mostly driven by large capitals which using the mining infrastructure to move capital out of china due to capital control in China. These investors can hire hundreds of developers in a large project, so they might not be too interested in what devs are saying. For large capitals, the only thing they care about is risk, anything increase the risk will not be considered

So far from what I've gathered there's clear understanding of what Pieter is doing within the dev community.

It is not as complex as you make it seem but certainly more complex than a simple change in constant. Yet the upsides are clear and significant.

Maybe Greg could pop up in this thread and clear things up a bit.

Raised level of complexity will bring much more trouble later in robustness and sustainability. That's why those complex network design in early ages of internet eventually died over time

Even it is a working solution, the gain in that is not even a magnitude, the risk/reward ratio is not worth it. I prefer the simplicity over complexity. I think Lightning network's settlement based design can increase the capacity for magnitudes, that is more worth the effort

A full-scale deployment of Lightning would require segregated witness to be implemented.

I insist that you are exagerrating the complexity of the proposal. It may not be simple for laymen but clearly it is generally well understood by the developer community from the logs of bitcoin-wizards I've looked at. This is also something that's already been tested to an extent under the Sidechains Elements platform.

By all accounts the security model is exactly the same and while the direct transaction gains are not in the order of magnitude improvement, they are significant. Moreover it enables a new type of semi-verifying SPV using fraud proofs, an important progress.

That cannot happen.

If the right signatures are not included along with the transactions data inside the blocks fully verifying nodes will reject them.

The biggest drawback of complexity is it increases the risk of centralization: If only a few guys knows how it works, then if these guys are compromised then the whole system is down. Currently bitcoin is understandable by thousands of developers, but if you go two chain implementation it will take decades to reach that level of understanding, at mean time simple solutions will gain more and more supporters

Ok, so the fully verifying nodes are not going to benefit from the new design, then how does this design improve the communication speed between fully verifying nodes (which is the bottleneck of the current design) ?

Look, you need to step back and realize that you clearly don't understand the proposal. Granted that may be because it wasn't communicated well enough yet seeing as all details haven't been hashed out.

I understand your precautionary approach but may I suggest you wait until the full BIP proposal is out?

jonnyj, your avatar text has always read "Beyond Imagination". May I submit that you have gone too far. Come back!

Yes, need some more material, especially some examples showing how a transaction is verified in the new design

Thanks, but that's Pieter's imagination to re-design bitocin:

"What if we could redesign Bitcoin from scratch? What if you're designing an altcoin, there's really no reason why you would want to do this in Bitcoin. This is actually something we did in sidechain alpha."

It doesn't.

It was clearly implied that data propagation and bandwidth is not directly addressed by this proposal.

Have you read Greg's scalability "roadmap"? There are other solutions on the drawing boards in that regard:

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011865.html

Yes, soft fork is a much better way to bring in a change, get it thoroughly tested and maybe becomes a hard fork when majority of users have tested it. It also takes time to test. Bitcoin has been tested in live traffic for almost 7 years to reach today's maturity, any large change would also need such kind of test by time

Do you mean that a trustless system which depends on users trust in the "brainpool"... isn't really a trustless system anymore? But isn't this already the case?

Besides all the benefits already described, one benefit with SW is full nodes could also skip transferring old signatures which is an unnecessary task.(Existing full nodes already do not validate signatures in the far past)

Let me borrow Peter Todd's famous word: If it is already so then why make it worse  ;)

My worry here is that we seem to have a large cadre of proponents of this new feature that are not able to articulate answers to reasonable questions. I see a lot of demurring of the nature of "perhaps the devs can come by and explain it better". It makes be think that perhaps these proponents likewise don't understand the details of what is being proposed deeply enough to understand the implications upon the questions being asked.

Maybe because we've encountered segwit only 2 days ago? Maybe it is because Bitcoin is a learning process and these things take more time as people have their personal lives to attend to? I've already said in the thread that I'm also learning on the go. There's nothing wrong with not knowing the answer to a bit more complex questions.

At the moment everything goes in the block.

With segwit, only the important stuff goes in the block. The other stuff goes into an 'attachment'.

This way more transactions can be put into a full block without increasing the blocksize limit.

i am not sure atm.
am i correct that miners dont need the "attachment" and can mine blocks with transactions with the "smaller" chain with proofs and a UTXO?
that would reduce bandwith for them (well a little...they still need to receive tx) as they only have to broadcast smaller blocks without the full tx data. - which would surely help

same goes for nodes: lower storage requirements (at home i would maybe only store my transactions and not all) but who will store and share the full history and why?
is the full history even needed (which IMHO would change bitcoins security model if it is dropped completely)?

Those things are way outside the scope of a "ELIJustBorn" ;)

Which is fine. I get that. But 'I really don't quite understand yet how this is going to work, exactly' is kind of hard to square with 'The solution to Scalability'. If one does not understand all the considerations, how is one able to meaningfully advocate it as any kind of solution?

Like anything else in life, if it's difficult to figure out at first, then continuous exposure to the logic of the inner workings will eventually bridge the gap. That's how I learned about Bitcoin; not over 2 days worth of reading/thinking, but over 4 years. I'm still going.

Fair enough. The difference I see is that when I entered the Bitcoin world, it was already a demonstrably working system. This SegWit thing, OTOH, which is merely said to have been tested, has in my mind the burden of proof. Is it an answer to the scalability issues? Maybe ONE answer - but seemingly a short-term minor fix even if all claims are validated - certainly not THE answer.

So by all means, let us investigate the efficacy. But in the meantime, let's not shout down those that are asking reasonable questions, and let us not argue for this on the mere appeal to authority. That is not how one sciences.

I have been here for a while and as a non coder and pretty average IQ person I have done a good job in understanding the various BIP's in a logical and understandable way but this segregated witness one is probably too abstract for me to understand. It sorts like a way to efficiently compress  the weight of blocks by removing something that's not needed when possible.

As merely one question, can we really consider the signature as something that's not needed?

I get that we're not _eliminating_ the sig, merely putting it in a separate (segregated) container, apart from the rest of the transaction. But any entity that wants to operate bitcoin in a trustless manner is going to need to be able to fully validate each transaction. Such entities will need the signature, right? Accordingly, such entities will need both components, so no data reduction for them, right?

Currently, relay nodes verify each transaction before forwarding it, do they not? If they are denied the signature, they can no longer perform this verification. This seems to me to be a drastically altered division of responsibilities. Sure, this may still work, but how do we know whether this is a good repartitioning of the problem?

Further, does this open a new attack vector? If 'nodes' are going to stop validating transactions before forwarding them, then there is nothing to stop them from forwarding invalid transactions. What if an attacker were to inject many invalid transactions into the network? Being invalid, they would be essentially free to create in virtually unbounded quantities. If nodes are no longer validating before forwarding, this would result in 'invalid transaction storms', which could consume many times the bandwidth of the relatively small number of actual valid traffic. If indeed this is a valid concern, then this would work exactly contrary to its stated goal of increasing scalability.

Note I am not making any claims here, but I am asking questions, prompted from my incomplete understanding of this feature.

Gavin Andresen has posted a pretty good explanation that is not overly technical:

http://gavinandresen.ninja/segregated-witness-is-cool (http://gavinandresen.ninja/segregated-witness-is-cool)

It is a kind of "return punch" to those BIP101 and XT propaganda posters. Obviously this seems like a very good solution for short term which is going to buy the developers enough time to work out others solutions. In addition to increasing the capacity this proposal has additional benefits which make it even better. These are the things that we need; i.e. better infrastructure and not just changing the block size to random numbers hoping for them to be the right call.

You should not be asking that here. You should ask it somewhere where it is very likely that the developers are going to see and answer. Apparently it has been tested for 6 months, so I'm pretty sure that they know those potential attack vectors (or some). Besides, they aren't going to rush this. It should be available on testnet this month IIRC.

Full nodes need all data, it means signatures as well.




You can choose to run lite node instead, this does not require the signatures, and you basically have to trust the mined block has all valid transactions (I mean you have to trust the transactions are signed right). It mean you cannot trust 1 confirmation tx with lite node.


BTW, is there any demand for such lite nodes (I mean who would trade security for little bandwich + HDD space, I mean -33% to -75% savings but trusting miners the transactions were signed right...) ?

I like that it seems we are making slow but steady progress in reaching a consensus, my question is why is gavin advocating for a hard fork to do this when soft fork should be enough?

He posted this on the dev mailing list:

A more elegant implementation but he's supportive of the soft fork as proposed. Interesting point on it being a sidechain earlier, I hadn't thought of it that way but that's kind of right and there's may be ways of exponentially increasing security with parallel chains.

Lauda, can you update the OP with the detailed explanation of the idea?
I'm sometimes just too lazy to search it to remeber exactly how would be made this idea.

I've added Gavin's explanation with a link to it. Hopefully that helps everyone. I've also updated the thread title by adding a question mark; hopefully now it fits better.

I agree with you about the scaling issues. It's not going to give us the kind of scaling needed to serve billions on it's own. But, the reasoning from Peter Wuille is that it lays important groundwork for scaling up to billions of users, at the same time as providing ~3.5x the transaction rate for the immediate term.


I agree with you here also, less caps-lock and I'll be convinced you're not shouting at anyone :)

Thanks. I wasn't meaning to shout, so much as to ensure my emphasis (on two words in a single post? c'mon) survived any quoting.

Exactly. If a solution is not understandable for users with average IT expertise, then it will never be understandable for anyone with even less IT knowledge. And typically owners of large mining farms and exchanges do not have time to do those learning, so they tends to select the solution that they can understand or listen to people they like. This will turn the decision making into politics, and who are good at lobbying and PR will push in their changes. And this is not people would like to see in bitcoin. So, the knowledge gap of different participants decided that you really can't reach a wide consensus upon a radical or complex solution, XT's failure already proved that

I still don't really understand how that can be implemented as a soft fork. Softfork means backward compatible, when the upgraded SW clients broadcast new blocks through out the network, how come the original core client can accept such kind of strange block which does not contain signature data?

Some of us are suffering from a sort of whiplash... we've been told (by some factions and their hangers-on) for months that raising max block size even to 2MB is highly dangerous for decentralization. But now, completely reorganizing some of the basic functions of the protocol, with a (somewhat unnecessary) requirement that there be no hard fork... has led us to the point where the same group with those concerns... is offering a fairly drastic solution that effectively raises the requirements for fully validating nodes to a 4MB(or 2?) max equivalent.

SegWit is widely agreed to be a net positive to incorporate into Bitcoin (especially if it can kill malleability problems), but the burden of vetting and testing should be much more involved than a one line patch like BIP102. My fear is that we will be into 2017 before anything is deployed, and we will continue to be without the base data that garzik's 102 would provide. And, the precedent that "hard forks r bad n scary" would still be firmly in place, and would be rolled out to stifle any possibility of future main chain capacity growth.

Understanding can be of different levels: conceptual, algorithmic, implementational... I bet most people don't quite grasp how Bitcoin's Script stack machine is implemented, though it doesn't prevent them from utilising it, if they know conceptually at least. What's enough for most people is that a particular component has been peer-reviewed thoroughly to prove it's safe to use it.

There are two modifications to be made for it to be soft-fork compatible:
1) SW outputs are made anyone can spend, so that for older clients it won't matter how they are spent, the scriptSig will be empty.
2) The merkle tree root of SW data hashes is stored in the coinbase.

I don't think you have to worry about that.

And also without the 1+ hour long block validations a simple "just increase the constant to 2MB" enable. :)

Yes, this is a very interesting scaling strategy. Quadruple the cap to get a double throughput is okay. Quadruple the cap to get a quadruple throughput is not okay.

But BIP102 stills a hard fork, with all the stress of needing to everybody upgrade their Bitcoin servers ASAP, no?

And sorry, but why the blocks would need more than 1 hour to validate? This segregate witness proposal is that bad?

but this will not solve the problem completely, when we need to increase again the block in the future, it will only delay it

in this case it seems that we have a margin of 3 more mega, it will be effectively like having a block of 4MB, but when we need 5MB we will be forced to increase the block anyway

therefore this is only a temporary solution...one problem at time i understand...

https://www.reddit.com/r/bitcoinxt/comments/3w2w17/segregated_witness_is_cool_gavin_andresen/#cxt01bu

No solution can be final in this. There is always going to be a need for more upgrades. I don't see how that could possibly be an argument.

And that's OK! What we need now is buy some time, get something done and observe how this solution is impacting the whole network.

Also a good message needs to be sent out to the whole community that something is being done towards a long term solution of this problem!

ok
full nodes (the real bitcoin-core) that mining POOL operators and true bitcoin fanboys will keep needing to store both tx data and signatures..
thus to them changing Block=1mb into blockA=0.25mb blockB=0.75mb makes no difference. its still 1mb bloat per blocktime..
thus to them changing Block=4mb into blockA=1mb blockB=3mb makes no difference. its still 4mb bloat per blocktime..
you can paint certain data any colour.. it doesnt make it invisible to full nodes
you can put certain data into different drawers.. it doesnt make the cabinet any lighter

secondly miners (not pool operators) dont need the full blockchain.. unscrew a mining rig and you will see no 60gb hard drive.. so yea miners do not care, they know how to grab what they need to do the job, and how the data is saved means nothing to them..

thirdly lite users. can easily code a liteclient right now(without protocol changes) that can read the blockchain and simply not save the signature part of the json data to file so they dont even need anything new to do this right now.. and in actual fact anyone wanting to not download bitcoin core. definitely aint going to want to have 20gb of lite segwit blockchain either... its an "all or nothing" game.. not something in the middle.

all i can see is that talking to a 5 year old
kid(lite):"mum theres pea's(sig) on my plate i just want the meat(tx), i dont want the pea's(sig)"
mom(full node):"ok here is a bigger plate. let me put everything on it.. and now move the pea's to the side. now shut up and grab your meat in your lite hands and ignore the pea's"
kid:"mom there is still pea's on the plate, every day you are still going to cook(store both) meat and pea's and all you are doing is putting it on a bigger plate, telling me i can just take the meat. your not helping yourself because your still making pea's. yea i know i will never eat(store) pea's, but you know you cant take the pea's off the plate because all the other moms will tell you its not a healthy(verified) meal. yes i can just grab the meat and eat it from my light hands separately but i could have done that anyway... but just putting it on a bigger plate means nothing.. if you think it means you can now cook 10x more meat you have to realise that you still end up cooking more pea's aswell.. if there is more meat theres more peas, simple fact.. you have not solved never needing moms to cook pea's nor have you solved me not needing to grab the meat off the main plate as i could always do it, even if you tell me that its on 2 plates and i only see the plate with the meat on it you have still cooked meat and pea's"


so segwit WILL NOT resolve scaling.. because upping the limit is just the standard thing to do and not special feature segwit is offering. the meat and pea ratio will still be there mining still will produce meat and pea's databloat for true nodes. you just increasing the meat and pea's which is no different than just making a larger limit..

using gavins example
wrong

bitcoin-core users will still have 170bytes per tx.. whether you want to colour 90byte green and colour 80byte red, its still 170byte saved to full nodes hard drives
trying to con people into thinking that making a plate 4 times bigger and saying oh look you can fit 8x more green bytes.. is just wrong.. full node blocks will still be the same 170byte total all that is happening is splitting the chain into two and branding the green chain as "bitcoin" and the red chain as "please dont look"
but full nodes will still be holding both chains and thus the total data a full node stores is still 170bytes on a basic tx...

so take a 2014 simple tx of 170bytes. thats 5882 tx a block
so just up the block limit to 4mb. 23529 tx a block

now seg wit
simple tx of A=90 B=80 full node storage is still 170byte = 23529 tx per 4mg block. but segwit lite clients storage is 2.117mb for 23529 tx segwit block

lite clients could have 90byte per tx. but their chain is not the real chain and wont help the network security nor will it help lite users that dont want any bloat
lite clients wont be part of the network security and so this is not a solution to help real network supporting users (bitcoin core), its not helping lite users either

lite clients can already have 90byte just by looking at a full tx and ignoring the json strings they dont need when saving files.
ive been doing it for years now. as my lite client only grabs tx data of addresses the client holds. and just saves the txid's, vins vout's and values.. lite clients wont want to store 20gb of useless history that doesnt help the network.. they either want full history to protect the network which they can verify, or just data that applies to them specifically to sing transaction, which is far far less than 20gb

having 20gb of non secure tx data is not a lite client. its a medium weight client. which to be honest ill say it again. anyone can make their own medium weight client right now. only saving part of the json data to file without doing anything special to bitcoins protocol.

so now onto the malleability..
once tx is confirmed.. its locked into history.. and then when segwit grabs just a portion of the block data.. ofcourse is malle proof.. BECAUSE ITS ALREADY CONFIRMED!
which is the same as anyone grabbing tx data on confirmed transactions has the same malle proof,..
now onto bandwidth
segwit lite clients will not just relay 90byte of unconfirmed tx's, as mining pools need the whole thing and each relay needs to check it.. so segwit will still transmit full 170bytes. full nodes will still store/transmit 170bytes too, and thus its not helping bandwidth of the network.

anyone right now can create a client that only grabs txid, vins vouts and values of relevant addresses of a user.. right now without any soft or hard forks..
i still cant see why people think segwit is so special..

summary
i still cannot rationalise why bitcoin-core needs to split the blockchain. just for useless lite clients..who are not going to help the network.. nor want any bloat
lite clients can more effectively grab the json data, put the json strings into individual variables.. and then just not save the signature variable to file..
this to me seems like a dysfunctional attempt at a solution
far easier to just keep the chain as 1 chain. and just put code in to raise limit to 4mb and solve the malleability by having code that ignores relayed tx variant if same vin has already been relayed by another tx saved in mempool, thus stopping people using the same vin until its confirmed(goodbye doublespend)

Which developer is claiming it will "resolve" scaling? segwit is indeed providing more capacity and scalability and thus is part of the puzzle in scaling bitcoin.  

You are ignoring the nuanced benefits SW provides(vs simply increasing the block limit) that allow for better scalability in the future:

- one benefit with SW is full nodes could also skip transferring old signatures which is an unnecessary task.(Existing full nodes already do not validate signatures in the far past but still have the burden of transfering them)

- resolves Tx malleability which is an important step that needs to be accomplished to roll out LN. Yes there are numerous ways to fix tx malleability but this is a simple and elegant one.

- Allows for lite nodes to have fraud proofs where we are adding an extra layer of security to potentially compensate for further centralization of full nodes due to partially increasing block limit

You appear to be insinuating that we should just take the simpler approach and increase blocklimit.... which is something the core devs are suggesting we do in addition to segwit when needed. Why don't they simply increase the blocklimit? Because of the benefits cited above that increasing the blocklimit doesn't resolve.

Indeed, during early days of bitcoin, developers have much more freedom to do whatever they want, partly due to that no one cares about it, and partly due to that there are no major interested stake holders because of its low value

But now situation is different, the network has attracted so much venture capital and investors, these guys all have their own agenda thus the political landscape has changed. A good example is kncminer, they took the crowd funding money, realized their projects and start to drive their own mining operation secretly

At this stage, posting on a forum or reddit or checkin some code in git does not make a lot of sense, because the decision making power is not in the hand of developers, but in the hand of large mining pools, exchanges and payment processors. If devs represent a complex solution which those large players do not understand thoroughly, they would just ignore it (they have to protect their million dollar investment as best as they can). They could just keep running the old client, and build their clearing and settlement channel to avoid the scaling problem altogether

Imagine that when the blocks are full and each transaction cost a lot to clear, then only large service providers would be able to use blockchain to clear with their business partners. Users will find out that using web wallet services will cost just a few cents as usual and clears instantly, but using core client will cost $100 and maybe confirmed after 1 day, so they will definitely move to use blockchain.info or similar web wallet instead

You see, this is also a solution, since the risk on individual service provider is much smaller than the risk of the whole network, it can be accepted. And this solution is much easier for every investor to understand than that Segregated witness complication. In fact, most of the people are still very used to centralized service provider, so they would accept a locally centralized solution easily

The best scenario is that all the large players out there have deep IT expertise and can easily get what those new changes' pros and cons, but in my experience it is not the case. Rich people have totally another set of criteria in decision making

It's weirder than that. The 'drastic change' (i.e. moving the signatures to a separate data structure) does absolutely nothing to address scalability for fully validating nodes. To fully validate, such nodes need all the block data and all the signature data. No reduction there. I merely reduces demands on _non-validating_ nodes, by a factor of 1.8x or so.

What the entire SegWit proposal does to address scalability at fully validating nodes is not the segregation, but rather a simple _increase_in_the_block_size_. In Wuille-speak, this is represented as "Discount witness data by 75% for block size Or: block limit to 4MB, but only for witness".

At least as far as I can tell.

Bait & switch?

standard disclaimer: I have an incomplete view of SegWit at this time.

As far as I can tell, the only component of the omnibus SegWit proposal that does anything about capacity or scalability is a simple increase of the block size to 4MB (I presume he means 4MiB). You can doublespeak this as "Discount the signature by 75% for block size" if you want, but that's really all it is.

so, core devs are now being racists? :P


(i dont get the sig being "segregated" to some soft (alt?!) fork.. aren't sigs a very basic and important 'feature' for Bitcoin?)

As far as capacity is concerned it doesn't even increase it to 4 MiB , but allows heavy multisig to extend that limit up to 4. It is better to assume that it equivalent to a 1.8-2.5MiB limit increase.

The one direct scalability benefit of segwit that isn't found with simply raising the block limit is full nodes could also skip transferring old signatures which is an unnecessary task.(Existing full nodes already do not validate signatures in the far past but still have the burden of transferring them)

All the other segwit benefits are only indirectly related to capacity increases.  

Segwit isn't being promoted as the solution to capacity problems by developers. It is a elegant change that solves many problems and only slightly increases capacity. The core developers are being very conservative and want to resolve all other optimizations like Segwit and the relay network before drastically increasing the limit.

Gavin does have a fair point to getting these capacity increases completed immediately because it will take a a long time to deploy them and complete the hard fork. I would like for the core devs, miners, and wallet developers to have the code ready and tested and a plan in place to increase the blocksize as an emergency measure if the fee market produces unfavorable results and there is a huge backlog.

My understanding is that decision making power in this case very much rests with the developers. The consensus so far seems that Segregated Witness will be proposed as a soft fork when the BIP is published. Mining pools, exchanges and processors have no direct say. All that is needed for the BIP to be merged is agreement among the lead developers. That would be Van der Laan, Gavin Andresen, Jeff Garzik, Gregory Maxwell and Pieter Wuille. Correct me if I missed a name or am in error.

That's not what I derived from Wuille's talk.

http://imgur.com/HdnFO7x
http://imgur.com/HdnFO7x

I may have missed it when he said something more clearly about the actual new block size limit. Shall we look at the code to check?

http://imgur.com/am6m5PT
http://imgur.com/am6m5PT

Oh. Dear.

http://imgur.com/BEXIhpH
http://imgur.com/BEXIhpH

What to do?

Here is the code -

https://github.com/sipa/bitcoin/commits/segwit

Transcript -
http://diyhpl.us/wiki/transcripts/scalingbitcoin/hong-kong/segregated-witness-and-its-impact-on-scalability/

Wuille does discuss 4x(sipa should have stressed up to 4x), but it is more complicated than that as the 4MiB limit increase is only on the whitness merkle tree(parallel chain) with a 1 MiB still on the main chain, thus only heavy multisig will use all 4 MiB.

Here is some quick math to show you an example-

Miners have most of the power. They can immediately protest and scare the developers into changing their proposal or adopting another. They also are the ones that have to ultimately accept the code so whatever the developers implement can be ignored or rejected.
The only power the developers have over the miners ultimately is the fact that they can walk away as volunteers and refuse to contribute if the miners don't accept their updates. This isn't of much consequence because other (likely less talented ) developers would fill the void. The miners, node operators, merchants and exchanges will likely accept the soft fork because they agree with it and trust the judgment of the developers. 

No.

We do. (assuming you hold any bitcoins)

The power dynamics is a stratification of users/nodes/developers/miners/merchants and exchanges; each exists with differing capabilities to exert influence on each other.

Many Miners have the most power as they are in at least 3 of those categories and sometimes 4. A user that doesn't host a full node is only in one of those categories therefore is more limited in their vote/abilities.

Meritocracy is really an critical aspect within bitcoin. It isn't just Proof or work with regards to hashing but proving ones work with developed and tested code, proving ones work with a hosted node, proving ones working with the most hashing power, proving ones work by client base or political power, ect...

If you only own bitcoin than your vote is counted far less than if you actively contribute to the ecosystem.

salt..



http://qntra.net/2015/12/after-xt-failure-gavin-andresen-supports-jim-crow-for-signatures-on-the-blockchain/


and pepper... http://trilema.com/2015/theres-a-one-bitcoin-reward-for-the-death-of-pieter-wuille-details-below/

That's absolutely backwards.

The best contribution to Bitcoin is every owner's respective economic share.

Yes, that absolutely implies that large holders are de facto more powerful than those with less coins.

No, your "activity" has nothing to do with this.

TLDR: The investors run Bitcoin http://nakamotoinstitute.org/mempool/who-controls-bitcoin/

This will be very true if you add Btc guild's owner and dial back time to 2013, where they successfully prevented an accident fork by commanding more than 60% of the hash power and forced the chain go back to their desired direction

And today? Btc guild does not even exist, large mining pools from China have dominated hash power distribution, and even inside the core devs there are large disagreement

The core devs are losing dominance because bitcoin is open source. Suppose that a group of large mining pools are not satisfied with the solution proposed by core devs, they could easily deploy their own R&D people and make a version that they prefer, and because they have dominant hashing power, when they roll out the change, their fork will be the strongest chain immediately. Of course they could not persuade others to join them, and they risk being abandoned by the other pools and exchanges, but having so much hash power definitely can affect lots of things

This was not the case during early days of bitcoin when mining pools were just a couple of hobbyists, they didn't have resources to implement such large code change, they only listen to core devs (I still remember the deepbit pool refused to upgrade to any latest version and stayed at 0.3 for a very long time :D That could be the earliest case of mining pool not following core devs)

Fortunately, chinese mining pools are not very interested in codes, so core devs still hold a lot of saying power, but when there is a disagreement inside core devs, we have already seen on BIP101, miners have veto right

Let me quote Bitmain’s Pan Zhibiao's word here:
“There are many technical solutions. Every solution has pros and cons. The miners are put on a pedestal to be a jury. Right now there is no lawyer on both side. We need a lawyer, we need more discussion, more evidence. ”

I happened recall the movie "Divergent", there are 5 factions, each doing its own job, maintaining a balanced society, hopefully we can reach that status of balance here

If we speak about SW in particular, I don't see it too complex to be understood by miners. The recent BIP65 rollover, which is a necessary step towards fully-functioning Lightning Network, is going very fast. Of course SW is a fair bit more complex than BIP65, I guess it shows us that miners either a) possess enough expertise to evaluate proposals; b) trust core devs' expertise. I'd prefer 'a', to be honest ::)

They are totally different, SW is a redesign of bitcoin, while BIP 65 is a planned patch toward lightning network which is easy to understand since it is very similar to a clearing based solution. Lightning network make things more simple (combining thousands of transactions into one) while SW make things more complex (split one transaction into two chains)

what if all the miners agree to leave the bitcoin mining scene, does we can save it by simply holding our coins? i don't think so

i could agree that also merchants matter, but i can't see how a simple holder waiting for dumping his stash could matter here

The large miners own only a fraction of the hardware. Changing the pool credentials is easy for the hardware owner.  ;)

If the hardware owners leave the mining scene they have to sell their machines to recover (parts of) their investment.

Also just to add, I think that we will need all of these solutions when the real mass adoption hits! So in my opinion it is very good to see that many solutions are being proposed, hard forks, LN, as well as this segregated witness proposal which is really just a smart move to buy us more time.

Remember guys, we will need them all, simple as that!

No we wont, there is no solutions to be found to a non problem.

Seriously you should take a deep breath instead of cheerleading all the BIPs ph0rking initiatives.

Sick of the highjack BS. Let it be people, or make your own "blockchain"!

Bitcoin is, and has features, not bugs.

Could you please re-tell this topic in words that ordinary people, like me, will be able to understand? So there is something about malleability, stopping fraud and increasing the block-size, that is all I got from that.

Oh ok. Now I get it. So they are going to take away the signatures, which in turn will lower the amount taken in a block, and then they are going to make the block-size bigger. That makes a lot sense. What are the downsides of doing this?

No, it is not. Even Pieter said that it was not possible anymore IIRC. If they could redesign it from scratch they would approach it differently than they're going to now. This added complexity is only going to play a downside role on other developers, but that's pretty much it. Users should not feel the difference (aside from the benefits).

Aside from added complexity, there are no downsides (yet).

removing the cryptography in cryptocurrency would seem a downside to me.

They're not doing that. Splitting something up does not mean removing either piece of the original thing.

new core nodes won't be sending witness data to old nodes.

That old data already does not get validated, so it's redundant to transfer it.

So, in layman's terms..

They have taken all the script sig data for all the P2SH signatures in a block, and put them in a merkle tree. Only the root hash of that tree is mined into the header.

Then, a week later, way after validation has occurred, you can safely through that merkle tree of data away, BUT still keep all the input/outputs in the blocks, to check that all the numbers add up.

?

If so.. I like it. Well coded!

I've been droning on about putting just the txn hash in the blocks, and using a similar disposable merkle tree of txn data, and squish everything, but I think this is a really nice solution. As you can still check that all the amounts add up.
 
And, who knows, maybe some day they will use this same trick, disposable merkle trees of data, once studied and better understood surviving in the wild, on the entire UTXO set.

 

This planned patch is a step towards LN. Judging by its adoption, it seems that miners are favoring LN.

They (SW and LN) are different, yet their complexity is of comparable scale. In fact, what SW does is 'solve' malleability, and thus paves the way for full LN.

Incidentally, I spent more time understanding LN than SW.

What you argue here (as I understand it) is that miners won't be able to 'understand' the SW patch, so they won't run it. I highly doubt both an assertion and a conclusion. I am sure it will be rolled over quite fast. But I have no real proof so we have to wait.

Dunno about laymans terms, but that's pretty good way of summing it up in technical terms! The lay Bitcoiner maybe! Also, I think the P2PKH sigs will end up in the SegWit chain too.

I dunno man, smells fishy.

Not that I understand code as well as devs, still I feel bitcoin protocol should not be 'fixed/improved' to fit external needs.

Besides, USGavin thinks segwit is 'cool'...

So to me it is just yet another attempt at highjacking bitcoin, weakening its protocol and introduce falsification risks.

Sigs are essential and part of the deal for bitcoin to stay strong (ie. valuable).

I am not playing the 'racist card' on the sigs, yet I understand noobs could not care less about it, and would rather have their friggin coffee cups on the blockchain.



Now everybody is alining behind this segwit soft fork because its is perceived as 'better than BIP101' basically..

But that is far from being enough, and they still want to twist the holy protocol.

Soft fork is even more pernicious than hard fork because it is more tricky to reject it.

Funny how not even coupla days after announcing this 'new plan' we got this satoshi hoax stealing the attention from it all.

Can MP just spank his fortune on developing a time machine and piss off back to 2009 already?  If all he's ever going to do is spend the rest of forever bitching and whining every time someone proposes a change, it would be mutually beneficial for all.

Until security experts make such statements that is useless speculation. Average users can't really (correctly) tell if there are security risks involved. Let's wait for test net before drawing to conclusions.

Bitcoin *is* change.

WTF is this insatiable urge to change it?

Only TPTB really have a motivation for such governance coups, which are to essentially prove there is a central point (of failure) where change can be imposed to all.

Their hard fork failed, now they try at soft forking the cryptography part out of the blockchain.. for moar coffee cups!

If that were true, we'd all still be running 0.1.5 alpha, which he's more than welcome to run if he doesn't like recent developments.  Or better yet, as I keep suggesting, a closed-source coin would suit his motives far better.  He'll never have to worry about change again and we won't have to hear more petulant hissy fits.

I concur, you seems like an open person, just dont jump off to cheerleading this simply for the sake of an 'improvement'.

Besides, testnets are irrelevant, and experts have already drawn conclusions: http://trilema.com/2015/theres-a-one-bitcoin-reward-for-the-death-of-pieter-wuille-details-below/#selection-105.0-291.48

You should know better than to call Mircea Popescu a expert. My 4 year old cousin is probably more of an expert than that guy. This is not a security analysis but rather a expression of his opinion with an attack on Pieter included. After so much time, you should have learned not to listen to this guy. He's also very biased regardless of expertise.
I'll wait for a 3rd party to explain why this is bad (if it actually is).

Is anyone else finding hdbuck's crisis of derived authority rather delicious?  :D

No, really, that's what happens now. The sigs get kept, but "checkpointed" periodically (which presumably constitutes a hash of some set of the tx data such that it proves bulk sig validity up to the checkpoint), only some few thousands of the most recent sigs actually get individually verified by Bitcoin Core, it's been like that for a while. I can't remember how far apart the checkpoints are spaced, but there's some margin of safety involved. It's just impractical to verify the whole thing for a brand new chain download, at least with OpenSSL doing the verifying (not so in upcoming 0.12, which uses another long term project, the in-house libsecp256k library, which might turn out to be an important crypto library for other software)


You'd probably be surprised at the changes you've not accounted for, although I know you're fairly knowledgeable about how Bitcoin works. You should know that we're not even running 2013 Bitcoin right now, let alone 2010 Bitcoin. And it's better like this, 0.11 is a significant improvement over 2 years ago.

But you're right to have conservative instincts about something that's already valuable as it is; Gavin and Mike have already demonstrated that well-meaning people can be misled to bcak a bonkers "improvement" scheme; I remember your palpable anger when people first started arguing for it publicly, I felt the same.

But I propose that Bitcoin does, can and should be changed over time to make best use of the technological resources available to it, made as available as is economically viable (chain security and decentralisation bound up in that property amongst other parts), and that the 1MB blocks design fulfills that right now (but is becoming seriously stretched also). And so at the risk of sounding slightly too much like VeritAss, I think the will of the overall community will end up alighting on the conservative approach that reflects, because we've demonstrated we're treating changes to Bitcoin very carefully, and so looking carefully at any significant proposals also.

And from what I've seen so far, people like Pieter Wuille are who you want for that conservative engineering task; if you want to abandon the verification checkpoints and scan your blockchain whole, wait till version 0.12, it'll be many, many times quicker (as well as half-way practical). Send your thanks to Pieter. (and gmaxwell)

For me raised level of complexity is the biggest downsides of any change. Typically in an enterprise environment such change will cause huge problem in the test lab where many test case failed due to some strange behavior they have never been designed for

In this case I have asked several questions and it seems no one has been able to give any concrete answer, which is already a dangerous sign, means centralization of knowledge

To migrate to a large change in infrastructure, you need to not only understand the solution, but also fully aware of the impact to the existing system and all the other systems that are dependent on them, potential security risk, evaluate its risk/reward ratio, and finally and most important, you should always be able to roll back to the old version if something went wrong

A soft fork qualify the last criteria, but still, from risk/reward point of view, I don't feel it worth the effort given the risk it involves. You change to a new untested architecture, what if after 1 years when majority of the nodes were upgraded to SW and found out that there is a deadly security hole that can not be covered, thus hackers can spend anyone's coin?

Bitcoin's value relies purely on its security model. Existing architecture worth a lot because it is robust and time tested for almost 7 years. It worth a little in the beginning, since there are too many possible security risk to break it apart, thus it must survive the test of time to gain its value. Now if you change to another untested architecture, it will basically reset its value to zero, and take equal long time to establish people's trust

No, it does not mean that. Obviously when something is so young, the inventors of it are the ones who are going to have all of the knowledge. For everyone else it will take quite some time to fully understand everything. I'm actually not bothered by this complexity and people are way too scared of it. When I program myself, I usually end up solving the problem the complex way without realizing it (it just depends on individuals). Also, just remember that everything about Bitcoin was "complex" to someone at a certain point in their life.

This is growing tiresome. Do you fully understand the complexities of the ACH/EFT system? I bet, just like me, you have a general understanding of how it works but have never seen the code behind it. I would also bet you have or have had at least one debit card and credit card in your life because almost everyone in western society has. There were 22 billion ACH transactions in 2013 with a total value of $38.7 trillion and almost no one really understands how it works. The way the ACH/EFT system continues to operate is by government and private oversight. People make a career out of working in finance and programming who oversee the system and there are laws that govern the form of that oversight.

The users of Bitcoin believe that system is flawed. They see potential in Bitcoin and are willing to use it. That requires faith in the original architect and the developers that continue to improve it because there is no governing body to regulate its continued operation or bailout its failures as there is with ACH/EFT. That leaves two possibilities for our acceptance and use of Bitcoin. Either build up a world class understanding of at least C++, Python, Java and elliptic curve cryptography or trust the people that are doing the job for you. I've learned to trust some of the scientists that work on this project based on their actions. Others I don't trust at all. Unless a mistake becomes blatantly obvious or someone I trust decides to speak out against this change I'm going to trust it because my understanding is limited. I suggest you do the same.

everyone says that segwit wants less data.. risking security just so lite clients can have less bloat.. well here goes.. i have a solution that doesnt involve messing with bitcoin protocol of splitting the chain..
firstly as described by gavin an old and common 2014 fullnode tx looks like:
[txdata+sig]=170bytes

segwit proposed 2016 softfork fullnode tx with fraudproof opcode will look like:
[[txdata][sig]opcode]=182
yes more fullnode bloat

and the same softfork in 2016 but for segwit liteclient (not full nodes) will look like:
[[txdata]opcode]=102

thats only 68bytes saving.. compared to 2014(no new opcode version)
i can think of easier ways of saving 68bytes (well80 if you include this new malle fix opcode)
just rename the namespaces of the variables:
hex=hx(1)
txid=id(2)(3 saved)
version=vr(5)(8 saved)
locktime=LT(6(14 saved)
Vin=Vi(1)(15 saved)
txid=id(1)(16 saved)
Vout=Vo(2)(18 saved)
scriptsig=ss(7)(25 saved)
hex=hx(1)(26 saved)
sequence=sq(6)(32 saved)
Vout=Vo(2)(34 saved)
Value=Va(3)(37 saved)
scriptPubKey=SPK(10)(47 saved)
OP_HASH160=oph160(4)(51 saved)
OP_EQUALVERIFY=OPEV(10)(61 saved)
OP_CHECKSIG=OPCS(6)(67 saved)
hex=hx(1)(68 saved)
reqsigs=rs(5)(73 saved)
addresses=adr(6)(79 saved)

thats 79bytes per tx saved without risking security at all and allowing more transactions per block(if full nodes also truncated namespaces(not advised)) without having to cut the block into two.
so my idea gives liteclients  79 less bytes on hard drive along with still getting signatures and still having the opcode for this magical malle fix

so now instead of
2014fullnode [txdata+sig]=170  or  2016fullnode [[txdata+sig+malleopcode]=182
its
2016segwit[[txdata+sig+malleopcode]=103 which is similar to
2016 segwit [[txdata]opcode]=102

but my idea still includes signatures. allowing full relay/seeding privileges for people syncing their chains, rather than being a limited function non secure copy of the blockchain that people cant trust when syncing

so while full nodes still use full length namespaces of the variable, lite wallets can truncate the namespaces with a simple translator bit of code
EG (laymans)
full nodes can continue doing this(no change just used as comparison to the SWlite clients difference)
  receive data to json_class Fjson{{hex:12345}{sequence:54321}{scriptsig:a1b2c3d4e5}}
    Fjson.hex
    Fjson.sequence
    Fjson.scriptsig
  save Fjson to file {{hex:12345}{sequence:54321}{scriptsig:a1b2c3d4e5}}

where as SWliteclients
  receive data to class Fjson {{hex:12345}{sequence:54321}{scriptsig:a1b2c3d4e5}}
  create class SWjson
    SWjson.hx=Fjson.hex
    SWJson.sq=Fjson.sequence
    SWJson.ss=Fjson.scriptsig
  save SWjson to file{{hx:12345}{sq:54321}{ss:a1b2c3d4e5}}

and the lite clients saves 79bytes per tx without losing the signatures. and this little change to save 79bytes per tx is not done in the main bitcoin protocol which we all admit will still store signatures no matter how you paint it..
but will be done by the lite clients themselves thus not affecting bitcoin protocol at all

the lite clients can relay and seed data simply by translating back when sending to full nodes
where SWliteclients:

  open file data to class SWjson {{hx:12345}{sq:54321}{ss:a1b2c3d4e5}}
  create class Fjson
    Fjson.hex=SWjson.hx
    Fjson.sequence=SWJson.sq
    Fjson.scriptsig=SWJson.ss
  relay Fjson to network {{hex:12345}{sequence:54321}{scriptsig:a1b2c3d4e5}}

its just that simple for lite clients to save 79bytes while still remaining part of the main network securing and relaying data..

the other good thing about letting lite clients do the pruning without it being a bitcoin protocol implementation is that liteclients can if they choose to, still prune off the signatures..
where SWliteclients
  receive data to class Fjson {{hex:12345}{sequence:54321}{scriptsig:a1b2c3d4e5}}
  create class SWjson
    SWjson.hx=Fjson.hex
    SWJson.sq=Fjson.sequence
    SWJson.ss= - ignore -
  save SWjson to file{{hx:12345}{sq:54321}}

allowing lite wallets to save even more data, without changing bitcoin-core

So, bitcoin is about believing and trusting people?

No, not if you're a scientific genius, it's not. For the rest of us, yeah, it is.

Exactly, instead of trusting a group of bankers you trust a group of developers, then this is not a trust-less system

The problem here is that an open source project does not take any financial responsibility

At enterprise level, if a company push in a software change which resulted in customer loss, the company will have to pay the fine/compensation to customer based on the service level agreement or similar contract. So there are usually strict change and release management procedure in place to avoid such kind of loss as much as possible

However here in bitcoin world, there is no such kind of financial responsibility. If devs push in a change which resulted in customer loss, the devs are not responsible for the damage, or to say, they don't have enough money to compensate the loss from millions of users. The users are all on their own

Imagine that in future if pension fund and social security fund are all invested in bitcoin, and devs pushes out a change and let their bitcoin stolen, who is going to compensate for the losses caused for millions of people that relies on these funds for their retirement?

Of course no one can insure against everything, there is always risk, but it is a general practice to not take drastic change in financial projects unless in a crisis, and we are far from that yet

Being a scientific genius is far from enough, there are so many professors financially broke because they lack of financial knowledge. In fact human all make wrong decisions, and different people have different interest, the point of bitcoin is to remove this kind of trust

if they mess up bitcoin, the dev's own income and savings is also lost..
thats why they dont want to mess up. and each new release is tested. and in the result of a bug in a new release. people downgrade to previous version and the bug is gone until its sorted out for an amended new release.

bitcoin is actually more secure then you think, you are not relying on 1 coders say so. there are usually 1000+ coders all using new releases before the general users even get the option to upgrade. so if there was any issue, the coders would be the first to get affected.

Right, so essentially we need to use at our own risk and trust that they will make the right choices and implement the correct improvements. It's a little more risky than trusting the current proven system with 22 billion transactions a year and millions of oversight workers but I so hate the current system that I'm willing to take the risk. I use the same method to determine when to jump out of Bitcoin that I use for investments or banks, I watch what's happening, ask questions, seek the council of trusted advisors and make my own decision. I've voted with my feet a few times now, only returning when I feel it's safe. Everyone can do the same. Honestly, none of us really have any say or influence over what happens with Bitcoin. All we can do is choose to use it or move on.

banking is proven system??
2015 crash http://www.telegraph.co.uk/finance/newsbysector/epic/hsba/11830336/HSBC-hit-by-payments-crash-on-payday.html
2014 crash http://www.telegraph.co.uk/finance/personalfinance/bank-accounts/10667064/NatWest-suffers-payday-mobile-banking-crash.html
2013 crash http://www.theguardian.com/money/2013/mar/07/natwest-banking-crash-what-need-to-know
2012 crash http://www.bbc.co.uk/news/business-18561426

i picked 1 bank per year to make a point.. but there were alot more incidents.. need we forget the great crash of 2007-2008. or the goings on with banks in europe holding customers funds to ransom..

i much more trust bitcoin

Of course it's a proven system that has bailouts happen all the time. I've already said I'm willing to take the risk. Now, would you like me to start listing Bitcoin's failures starting with malleability and work backwards to be fair? Before you answer realize I'm going to fill a couple of pages of not only functional code problems but, because of the links you used, I'm going to list every theft that happened from human error or because Bitcoin made those thefts possible. BTW: no one bails out Bitcoin users and the FDIC doesn't reimburse them.

and i will write every single card cloning, every single pick pocket. every single bank note loss..
yes thats right bank notes are not insured.. if its stolen by some cunning theif its gone..
i bet the number of victims of bank notes. including total value lost will exceed bitcoin losses.

lehman brothers lost $600 billion+.. bitcoin is only $5 billion cap and losses far far far below that

if bitcoin v0.12 broke. we can just import our private keys into v0.11..
if our bank broke. we have to wait for a bank bailout.. which millions of people are still waiting for even today 7 years later..
so bitcoin fix=10 minutes.. banking mega crash.. 1day-7years

There can't be a mega crash in Bitcoin compared to a world wide economy because Bitcoin is a flea compared to an elephant. There can't be a housing market crash in Bitcoin because there are no home loans denominated in Bitcoin. Bla bla bla....  I've used fiat banks for 50 years and never lost a dime. But I've lost thousands of dollars with Bitcoin.

You so desperately want to be right and defend your beloved Bitcoin that you're missing my point completely. I use Bitcoin. I'm willing to take the risk because I don't like the current system either. Bitcoin use is a personal choice. Use your own best judgement and trust it or not. But please don't pretend to have any influence on what's happening. If you want to influence what's happening start posting your own changes on Git and join the debate there. Be prepared though, they know what they're talking about. This is mostly a users forum. You can't help here.

Yes, being the stake holder in the system will prevent devs from intentionally harm the system. But still they can make mistake, and a mistake in bitcoin network will bring real financial impact, like reversed transactions, lost mining income etc...  I still remember that during 2013 fork, those miners lost hundreds of coins due to they were on the wrong chain by running Gavin's new version, eventually their loss was compensated by the mining pools, so it was mining pools took the responsibility, not core devs. And that directly resulted in that almost none of the mining pools favor Gavin's code this time

Or by ignore new updates and let others become lab rats ;)

Why can´t the Bitcoin Lightning network, sidechains or colored coins be the answer to this problem? al least it would avoid tempering with the code

Well, that is one way, I suppose. As long as you don't end up on the losing side of a fork.  :)

LN needs two things to work properly:
 - free blockspace
 - fixed transaction malleability

the first one is not really solved (in my opinion at least) but the second one is.

You mean, the second one will be after segwit gets implemented? Otherwise the statement is incorrect because right now malleability is still a problem. The other complex solutions need time to develop.

yes, sorry. after segwit is implemented the second one is fixed (malleability) but the first one is not solved (the blocksize problem has just been moved to the future)

im not saying it is fool proof.. im just saying its not as bad a risk as you think..
and how have you lost thousands of bitcoins.. if there on your keys there on your keys.. it wont matter if they are on keys from 2009 or 2015, they are stil on the keys..
lets say you are a business and you were delivering products in exchange for bitcoins.. you havnt lost bitcoins.. you just never got paid because of a scammer handing you a counterfeit.. which is the same as the real world..

lets say you put funds into dodgy exchanges.. well i can tell you millions of dodgy fiat boiler room scammers. how many "nigerian prince" scams cost people..

scams are not bitcoin protocols problem.. they are problems of human judgement. which no code can sort out.. if your going to throw funds at a nigerian prince, it doesnt matter if its bitcoin or fiat.. your going to lose..

but if you have and keep the coins on your private keys.. there is no way for a nigerian prince to run off.. there is no way a bank can go bankrupt and take your money..

im sorry but i do realise that bitcoin is not perfect. but nigerian princes setting up dodgy boiler room bitcoin exchanges has nothing o do with the bitcoin protocol

lauda, you keep emphasising the malle proof..
but have you actually read HOW

come on paste in the exert that explains it. and tell me how it cannot be done without splitting the blockchain.. and how only wuille can do this,

As you can see, unless you heavily tamper the code, none of these solutions is going to work

Still I prefer the simplest solution of direct clearing and settlement between service providers, two major benefit:
1. you don't need to change the bitcoin code, thus maximum code stability
2. clearing based solutions is a mature solution in financial industry, maybe more risky at each service provider level but no risk for bitcoin network, it can give you 0 transaction fee and instant payment, which bitcoin network can never achieve no matter what kind of scaling solution

tx malle can be fixed easily with a softfork without segwit.
just let all miners and nodes reject one kind of tx (be it the positive or the negative one - doesnt matter)
if you have the biggest miners on board it wont take long and its done ;-) to help with the transition miners may help users by "mallating" the tx themselves to the new correct format (some people [forgot the name..sorry] already do that).

i know.. im just poking at Lauda because he seems to really want to push segwit, as if its the ultimate solution.. and the only solution..
yet many other people know there are better solutions that dont mess up the main bitcoin chain..

Transaction malleability aside (because I'm not sure I believe the millions of dollars were actually lost from this), I still know that Bitcoin isn't secure enough for anyone to "go all in" as they say. Not only because the protocol isn't well refined yet but because there aren't systems in place to secure users from the inevitable teenaged hacker. I have had fraudulent transactions on one of my bank accounts. You know how much I lost? Nothing! You need to make your own choice how much you are willing to use and possibly lose. Your right, of course, if you're willing to have all your keys unusable on a paper wallet then Bitcoin is a fortress. The rest of us have our money constantly in play.

This Segwit solution is a change that can help, at least in the short run. I'm really unclear why you are so opposed to it. It seems like a solution that will give the developers a little breathing room to come up with a permanent solution to blocksize while solving the malleability problem without even the need for a hard fork. What's wrong with that?

Edit: If you have real concerns and are trying to save us all then please go here and make them known. https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

This is incorrect. Did you notice the question mark in the thread title? There is no "ultimate solution" and there never will be.

i am still undecided on segwit.
i just know that added complexity always leads to more problems. bitcoin is simple atm. i'd prefer to keep it that way (if possible)

so we have to look at the benefits to make an educated decision.

the only valid benefit i see is reduced storage - which is not bad but is it important? i dont think so. AFAIK nodes running on home-intrnet-lines are adding a burden on the network anyway.

so...still undecided ;-) but i like to learn more...

Malleability fix, simpler script upgrades, fraud proof.. those are not 'valid benefits' ? The least important benefit here is reduced storage. Most people don't run nodes so they don't even care about that.

1. i personally am all in.. my bank account has not seen any deposit in years. i get paid in bitcoin and i move a majority to cold store and a few fractions of bitcoin to a hotwallet (treating it like bank note spending)

2. yes i have lost a few 'bank note spending' fractions of bitcoin due to many reasons. but my cold store has been filling up quite nicely since 2012.

3. then think of the cold store as a bank and the hot wallet as bank notes.. and just dont walk round town with more then your ready to lose

4. segwit pretends to be a hard fork, but how data is saved is not benefiting true bitcoin core nodes.. and most of the benefits can be done by liteclients already.
i can think of 20 different solutions to bloat/malle. all of which are less offensive to bitcoin-core and less security risking for the network.

yes segwit has features.. but the benefits and issues it will cause are not as worthy, and other solutions can do a better job

i missed: simpler script updates and fraud proof
do you have a link?
i thought i had read this thread carefully together with gavins blog entry...are there other sources?

Ok, I'll assume you're actually trying to help. Please see the edit on my last post.

you original title on december 7th was
Segregated witness - The solution to Scalability
and now
Segregated witness - The solution to Scalability (short term)?

so just admit that even you have lost the faith seeing as you edited the title

If you look at the picture in the OP, you are going to notice those benefits. Here's a transcript of the original presentation (http://diyhpl.us/wiki/transcripts/scalingbitcoin/hong-kong/segregated-witness-and-its-impact-on-scalability/) and more information can be found on reddit.

No. The title was changed a few days ago. The initial title was misleading; I think it fits better now.

---

---

Off-topic: Milestone reached.
https://i.imgur.com/cmmw2li.png

so you admit its not the ultimate solution and never was... ok, but i still wonder why your trying to push people soo hard..

and yea i seen the image 20 times.. its not showing how..to me, i dont read a cerial box of ingrediants.. id rather do my own tests
and yea i read the article 5 times. and its solutions seem flimsy at best and i can see 20 other ways the same benefits in the image can be met without messing with the real blockchain.

i do believe bitcoin needs a new opcode for the scripting/malle stuff.. but the whole splitting signatures part.. no way should that happen, lite clients can ignore signatures themselves or reduce data saved like i explained on a long past on the other page (https://bitcointalk.org/index.php?topic=1279444.msg13225642#msg13225642).. but no way should bitcoin-core be altered just for liteclients lazy sake of 15 lines of code in their lite client software

Pushing hard? I don't think so. You're the guys who are mostly leading the discussion. Look at how many posts I've made (percentage wise) of the total posts here. I like this idea; test-net is going to be out this month (IIRC). You can then do your own tests. Would I mind a simple block size increase to 2 or 4 MB right now? No, I would not. I don't think those "20 ways" could work else some developer would have already proposed it/coded it. Simpler script upgrades and a malleability fix is really good, without those the added complexity/potential attack vectors would not be worth it at all.

thank you.. here are a few thoughts (if segwit is implemented):

• in case of a bigger chain-split and merge it might not be possible for a miner to include tx from the wrong chain into the correct one because of the missing proof
• segwit fixes tx malleability for P2SH transactions which we dont have a solution for today (bip62 dont work - and its relevant for LN)...nice
• easier script changes: neat... soft-forks for script updates. and to easily support other crypt-algos seems good too in case current ones get broken we have an upgrade path (ofc schnorr as a possibility is nice too).
  i dont understand (yet) why P2SH does not work for this. AFAIK it is a a hash of a script placed in the block which also resolves to ANYCANSPEND. isnt it possible to use this method for further script-changes?
• fraud-proof: IMHO unneeded as only nodes which has storage/bandwidth problems would benefit (and we did agree this is not a big problem anyway)

please correct me where i am wrong ;-)

In fact, this possible change in bitcoin architecture raised a question: Are your bitcoin safe in a cold storage?

I used to believe that it is protected by the public-private key cryptography, e.g. without the signature generated from private key, the coin at certain address can not be spent

But now I realized that this really depends on the client running on the nodes

If a group of nodes are running a new version which does not need the signature to spend coins, then that version can spend anyone's coin without their signature (The new version can use a new signature scheme to protect their new address). Of course this transaction would not be known to the old client since that is not part of the old protocol, so in the old client coins are still there but in the new client the coins have already been spent. After the old client upgraded to new, the coins are gone

And there is really motivation in doing this: Since by the time when over 99% of the client is running new software, the old client essentially becomes minority thus have to upgrade to the new version because almost no node is using the old version anymore. So, by successfully rolling out a new version you can steal other's coins, especially Satoshi's 1 million coins, doesn't that sounds like a good idea?

I'm not talking about developer's ethics here, it is just a technical possibility that will attract lots of criminals, and criminals really does not care about bitcoin's long term success, they just need to cash out the stolen coins at exchanges and they are done. In a word, if nodes could not prevent the protocol from being changed to something malicious, then you essentially can not protect your bitcoin at all. And the more complex the code is, the easier to hide malicious implementations

This thing is pretty genius to be honest, even Andreas dedicated a post to sigwit and described it as a turning point in Bitcoin. This is exactly what we needed before Lightning Network which disipates all doubts of Bitcoin being able to wait until LN is operative, with sigwit we will be able to deal with any extra traffic of new people jumping in in the next year or so.

Also I noticed you are still not Legendary, pretty bad luck considering it starts randomly at 775 activity+ randomly I think :p

IMHO thats only the case if:
 - majority of miners runs new version
 - satoshi moves his coins to a new address
 - majority of miners decide to roll back to old version

or did i miss something?

In this order:

1. Some large miners start to run a new version that can spend satoshi's coin without signature (in new version you can redefine what is a valid transaction)
2. These miners promote the new version to be widely accepted by exchanges and merchants
3. These large miners moved satoshi's coin to their own address in the new version
4. They sell those 1 million coins and gone

Since no one else except Satoshi will notice the difference, and in this case majority of the miners already get Satoshi's coins and be satisfied with the new version. Even Satoshi comes out and protest, it does not make any sense any more

i dont think 3 would work ;-)
why should exchanges and merchants accept such a bitcoin version?

First of all, signatures are separated only for transactions that are spending from new SW-compatible outputs. As Gavin explains it (http://gavinandresen.ninja/segregated-witness-is-cool), the scriptPubKey will be like this:
Old transactions will still employ the current mechanism. This 'old' mechanism will be preserved, and there's no real chance spending from old outputs will be made obsolete (there's a chance sending to 'old' addresses will be made non-standard though, but I also doubt that, given the implications).

I do not understand what an attack vector you are discribing here. Old versions will have decreased security because they will have to assume (w/r/t to those transactions they won't be able to fully check) that the longest chain is the valid one. This kind of an assumption is already here for SPV wallets, which, to my knowledge, are an overwhelming majority these days. But thanks to fraud proofs, the SW will be able to strenghten their security.

Anyway, it's always been that full nodes provide the highest security possible. The full node verifies that the coins you receive are valid. Full nodes act as a check against dishonest miners. It will stay this way.

Sorry, I have changed the order a little bit to make it more realistic, they should first push for mass adoption and then do the malicious transaction

Exchanges and merchants accept such a new version simply because they heard that it can bring more transaction capacity, can fix bugs, can reduce the block size and increase performance. etc... And because the code and the implementation is so complex they don't have time to check every detail

its all theory but here is a story

mining pools will still be full node
in 2015 a tx looks like [txdata&sig]. in 2016 SW softfork would look like [txdata][sig] to a full node. so no worries about miners (i hope)

but it would only be [txdata] relayed/saved to a SWClient wallet..

what i could then do, is hack your SWClient so that im the only relay node you connect to.
i do this for 4 people just so your not curious about lack of network connects.
so i could make a [txdata] that is satoshi -> franky 50btc.
i send the [txdata] to you all. and because you all have the same [txdata].. you accept it (remember you cant contact a fullnode to check signature as you are in my hacked circle).
i then say 'bob im satoshi's friend as you can see he gave me 50btc,  i want to give you 50btc if you send me 5000LTC or $15,000.' your happy because its a cheap deal and also you think you will get fame for receiving funds originally from satoshi.. afterall the [txdata] shows that the satoshi funds came to me
you agree so i make a new [txdata] that shows franky -> bob.
you also see for other connections with the same [txdata] crediting you with 50btc
you then send me the litecoin/dollar funds..
i release you from my hacked circle, where you realise that the [txdata] is all fake.. and ive just run off with your litecoins or dollars.. all because you did not have the signatures stored to check locally while you were not able to check the real data.

I see you concerns and I must admit that such a situation is theoretically possible, but there are two things I must note that make it less likely practically:
1) there are many experts that are evaluating proposals. It raises the bar for any controversial change.
2) SW is not as complex as you are painting it.

Anyway, the possible soft-fork is still months away, so you'll have time to evaluate it yourself.

If full nodes are still needed then they are still the slowest bottleneck of the system, the SW implementation won't improve the bottleneck then what's the benefit?

Anyway this is just a generalized talk, I am not aiming SW, just showing by changing the protocol you can do whatever thing to bitcoin. So a change to protocol should be very carefully tested and reviewed, but due to the complexity of the codes, the review will be quite difficult

What constitutes a full node might change over time, as more advanced features, like UTXO commitments, are implemented. But yes, they are still the bottleneck, and the SW doesn't claim fixing it. The benefits are those listed in the OP, on the picture.

lol.. i love it how a few people are so commited to pointing out the picture but without explanation..
so here is my picture
http://www.amitbhawani.com/blog/wp-content/uploads/2010/04/Olive-Zipbook-3G.jpg

now tell me..
will it play Call of duty on full res.
will does it have HMDI
is the screen technicolor or just black and white
is the pre-installed windows set to english or japanese.
does it come with a power cable?

yea pictures with list of features can say one thing.. but they dont explain the details people really need.. so can people stop saying 'look at the picture' as it explains nothing

So, have you posted your concerns on the Bitcoin-dev discussion board yet?

Ok, ask a question and I'll try my best to answer it. But as you are waiting for a detailed answer, I'll have to spent a significant amount of time replying. So please make it concrete.

once i get home. i will

i know splitting the blockchain into 2 is not needed.. SWliteclients can prune what they like how they like without messing with the real blockchain.
i understand the benefits of adding new opcodes..

but the real question is.. how is the fraud proof so special that only SW proposition can implement it. explain how the fraud proof works in your own words

Exactly, and this is just one of the security risk that is possible under the new implementation, and typically an honest dev will barely understand or test how malicious users exploiting the weakness of the system. So the real damage might only happen when the software is in live traffic

Good, please keep us informed. I'd like to know if I'm on the wrong side on this. I'm not really a big fan of throwing money away. It's obvious that something major will be done in 2016 but I'd prefer to not sit out the whole year waiting for the dust to settle.

I'm sorry but that's not knowing the thought process of current core devs.

Adversarial environments are all Peter Todd talks about.

That's what she said.

"Have you stopped beating your wife?" Who argues that only SW can implement fraud proofs? It's a neat additional feature that is relatively straighforward to implement with SW.

Fraud proof is simply some compact data telling you that Bitcoin rules were violated. For many of the rules we can create compact proofs that they were violated. But there are two types of fraud that we can't compactly prove right now. These are subsidy fraud (inflated fees) and spending from a non-existent input. Pieter Wuille suggests we store additional data with SW, that isn't present with blocks currently, so by having it we can compactly prove these types of fraud. These data are input values, and a 'path' of inputs (original block height containing that input and a position within that block). Once this data is made a consensus rule, full nodes can simply broadcast, say, a message telling you "This input is invalid because it doesn't exist within the block it claims to", then you check that and decide to ignore the chain with this invalid input.

Fraud proofs are still more of a theoretical approach, and will require careful engineering to make them operational. But making all consensus rules compactly provable is a necessary step towards that.

or more simply.. if someone has already checked it.. instead of writing in an opcode to say its invalid.. they just dont relay it..
relying on someone else to do checks, to trust them when they say everything is ok, and only double check when they say is wrong seems a little backwards

even right now i can check a tx from back in 2009 and see if its valid just searching the txid....
i agree that adding in the blockheight and tx position of originating confirmed funds in a new unconfirmed tx can make searching the original tx easier.. but if its invalid.. you simply dont relay it.. bt i still dont see this a fraud proofing.. just making the checking mechanism slight faster (microseconds) by having the header and tx index.
i hopefully dont want to see anyone relaying an invalid tx just to tell someone its invalid. and they blindly trust opcodes without checking source.

i think every bitcoin software whether its SPV, SW, LN or core all check every transaction before saving to file or relaying. never should they just see
OP_Check="im legit" and accept it blindly. nor should they ever receive a invalid transaction, thats already been checked as invalid

that said im all for blockheight txindex.. just not the other part

SPV doesn't check signatures.


That's actually how a good share of the ecosystem presently operates. What did you think everyone runs a full node?

It seems to me there are fundamentals holes in your understanding that prevent you from making any progress in your comprehension of SW.

as i said, i insinuated SPV SHOULD... not does

but anyways,
ive got a full node. and i also have my own lite wallet. and guess what even though my lite wallet doesnt store the blockchain. i still know basic code get it to connect to other nodes and grab that specific blockchain data as and when needed.. id never ever ever blindly trust a unconfirmed tx based purely on how it was presented to me.. i always check txdata against other nodes history.. never would i ever just accept it and pass it on because it said op_check="im legit".

as i said im all for blockheight and txindex to be included in new transactions as it would make things far far easier to check.. but never should anyone just receive a tx and accept it on face value.

and now im more shocked that, with knowing how simple it is to make a lite wallet. why these other spv wallets are being so lazy.. its not rocket science.

businesses should not be running crappy lite versions..
if a business trains their cashiers on how to check if a bank note is counterfeit then the same should be true to bitcoin businesses using any wallet
they should all atleast be running some sort of lite client that checks history.. and that can be done without even needing to store said 7 years data.

I had a feeling that this solution seems quite sudden and not very well prepared, might because they are afraid of a transaction volume surge during the next rally, but in that case I'd prefer Jeff's approach to simply raise the block size limit, it does not change the code too much and can buy people more time to work on more long term solution

In fact I think even we do nothing, bitcoin still works. People will adjust themselves to their beloved bitcoin and create solutions outside of blockchain to solve the scaling problem. It is a habit that devs always want to do something to prove that they are working, but even as large as FED, considered to be the top wisdom on the planet, what they do is just print a bit more money or print a bit less money, change interest rate by step of 0.25%, nothing more than that, and they did that for decades. When you are commanding a monetary system, every step of change should be extremely cautious and small

I remember one of the core dev said a few month ago that we should wait until the block become full and see how the situation develop and then start to apply measures accordingly, I still think this is a good approach. People won't die because the banks are closed during week end, similarly, if the block become too congested, they will just reduce the transaction frequency and plan their transaction accordingly, that gives us at least 3-4 times of block space to maneuver, at least one years time to react

Nope. Still not getting it. A block with all associated signatures already contains everything needed to determine whether or not it is fraudulent.

It does, but not everything can be proven fraudulent compactly at the moment, in a way compatible with lightweight clients. Fraud proofs are allowing for a lite client to be aware of bad blocks without fully downloading and verifying them.

By fully verifying the blockchain you are proving that it's valid (good). By using fraud proof you are proving that something is invalid (bad). Notice the difference.

no... because full nodes make blocks (via mining pools ofcourse).. and if other users with full nodes see a invalid tx/block.. they ignore it, dont relay it.. end of
the real stupid thing would be to relay a tx/block knowing its bad/wrong..

its a simple concept.

imagine bank notes.. someone hands you a hand written piece of paper with scribbles on it.. you throw it in the bin and tell the person who created to go screw themselves.. you dont accept it. mark it as counterfeit and then hand it out to the next customer..

The default policy is already not to relay invalid data. But it doesn't help under adversarial circumstances. It doesn't help if there's miner collusion, and you are getting invalid data from them and have no means to know it's invalid. It doesn't help if they are relaying invalid data to you on purpose.

SW with these modifications allows for a more scalable and safe lite node approach, where you verify (perhaps randomly) some percentage of data.

miner collisions are just 2 miners that solved at nearly the same time.. the data isnt invalid, they just solved relatively at the same time. each of these miners wont invalidate their block. its for the other full nodes to check both blocks and when both are valid. then check who solved first. and the not relay the loser..

there is no point marking the loser as invalid and still relaying it.
there is no point in a miner marking their block as invalid. as at the time of solving and relaying to that very first peer, they dont know they lost.
thats for the network to decide

Of course it's collusion, not collision. I made a typo.

well thats why bitcoin-core is there.. to validate properly.. and drop invalids.... and that should never ever change for bitcoin core.. handing off invalids to other peers once bitcoincore knows its invalid.. is as i said a few times, a rediculous notion.

i would never in my life trust any data coming in on face value. and soo all these litewallets will fail by trusting opcodes to say its valid but never actually checking..

we should not be thinking of way to add more data to a tx/block in full nodes.. just so that litewallets can be lazy and ignorant.
i originally thought segwit would originally be a better lite wallet. but when i started reading that its a medium weight wallet that does no check and takes data on blind faith of who sent it. without checking origins.. i facepalmed

Now the one facepalming is me.

miner(dodgytx)opcheck='imlegit' ->RoadchainSWlite -> otherSWlite-> otherSWlite ->otherSWlite -> otherSWlite

roadchain does no checks and on the data and see's the fraudproof opcode 'imlegit'.. and accepts it and relays it on.. now 4 people have dodgy information.. for no reason.. they cant use it but the 2000 people using the SWlite keep passing it on.. all blindly thinking its valid..

miner(dodgytx)opcheck='imlegit' ->frankyFullNode -||BIN||

franky ignores the opcode'imlegit' and checks the real data. see's its really dodgy and deleted it, its not relayed and its business as usual
then waits for a valid block to be solved, checking that and then relaying that only if satisfied the rules are met

And it's worse, as now it will be essentially free for an attacker to create unbounded numbers of invalid transactions, and the medium-weight SegWit wallets will blindly relay them throughout the network, clogging it up with crap.

shhhhhhh i wanted roadchain to realise it all by himself. :D next time write 'spoilers'

Why stop there?  Lets put TXIDs, scriptsigs, and addresses into separate data structures and calculate merkle trees for each.  Now the block chain doesn't need to have ANY of that data in it.  We can SHA256 a cat of all the merkle roots and now every block has 256 bits only!  We can now fit infinite spam in the chain. 

Congrats Lauda!   ;D

many a true word spoken in jest (well ridicule in this case)

You're still not getting it. Your sarcastic tone is completely unjustified and only shows your lack of understanding.

I dunno where you're getting this thing "opcode 'imlegit'", and it looks extremely stupid on your side.

If a verifying node sees a fraudulent block, it constructs a compact fraud proof and relays it instead. The particular mechanism and criteria are not clear yet, but the idea is something like that. DoS protection in necessary anyway.

What you're doing here is making up a stupid non-existent solution that no sane person will accept and then ridicule it. You basically ridicule your own idea :D

What's more funny is that this whole idea of fraud proofs is present in the Bitcoin Whitepaper under the name 'alerts', in section 8. "Simplified Payment Verification".

Yes, they can use Litecoin, Viacoin, Dogecoin, Monero, instead of Bitcoin, where the stream is blocked.
This seems to be a great strategy of the core devs. The Altcoiners should applaud it, and they do.

Exactly.

Mike Hearn and Matt Corallo didn't implement Satoshi's complete idea for SPV in the first place, we've been living with the implications (lower grade security SPV wallets) of that ever since. SegWit completes the design in the original white paper, so all this "re-design" rhetoric is deluded hand-waving.

Still remains to be seen whether SegWit lives up to it's promises IMO. The upcoming public testnet should be interesting in that respect. I'm optimistic, but I also find myself agreeing with those that say to do this as a hard fork. 12 months is a long time in Bitcoin, and a hell of a long time for a fork rollout.

merklecoin!

ok here is my thought process
https://i.imgur.com/gwSInGG.jpg

(1) is obvious that both fullnodes and SWLite clients will relay data.. please use as a simple reference example to compare to the other 3 theories
remember because SWLite only has 25% data they wont relay TO fullnodes but fullnodes(100% data) can relay to SWLite and other fullnodes

(2) is the idea that when its a dodgy data fullnodes dont relay to anyone. thats the basic rule of bitcoin!!.. if it dont meet protocol rules.. drop it!
but dumb SWlites dont check. leaving the SWlites thinking everything is fine and dandy.. which is something that the
'fraudproof' needs to address as these SWlites are too lazy to confirm themselves and blindly trusting data received.

(3) ignoring the key in the bottom right for a moment imagine the red lines were blocks with an opcode added to say 'its dogdy'.. all other nodes can have the data to double check without having to request more data from fullnodes

because
(4) now use the key in bottom right as you can see in (4) receiving just a alert(red line) and then requesting more data(blackline) is just alot of bandwidth bloat. and this double checking due to alerts would all fall apart if a SWlite does not have a fullnode connected to it, to verify alerts accusation. so would just say fake accusation

so thats why i envisioned opcheck='its legit/dodgy' inside a block(3). as it wouldnt need the black line after an alert.. because the alert would be inside the block.. saving the need for a black line data request.
but that brought me onto the point of bitcoins cardinal rule.. if its dodgy why even relay anything if the recipient isnt going to save it, just dont send it..

now back to (4) for other reasons
yes the alerts have stopped SWlite1,2 and now SWLite1,2 are just alerting their peers, but thats only because SWLite1,2 had fullnodeA connected to them, and SWLiteX,Y,Z had fullnode1

 if there was not a full node to check, as the case is with SWLiteA then they would continue relaying and it would again look like (2) because the SWLites would treat it as a false accusation due to lack of proof..(no fullnode to corroborate data).

so this is why i thought SegWit's solution was to break the cardinal rule of bitcoin. by not dropping the block on rule break and instead keep it, just to allow lazy clients to check(silly i know)

the funny thing is.. if SWLite does have rule checking code to be able to perform actions after alert and request(black line).. as proven in (4) then it can be done right from the start.

so
(5) because SWLites do have the rule checking code, just checking it and dropping it, if its dodgy means:
SWLiteA drops it, SWlite1,2,X,Y,Z has nothing to argue about
fullnodeA drops it, fullnode1,X,Y has nothing to argue about

afterall they fullnode and swlites are all 'listening' to the network and taking in data. where SWLite then just truncate the variables they dont need when saving to file to save space

which is the fundemental rule of bitcoin.. dont relay data that doesnt fit the rules but check data you do receive

and now for your quote of a quote
[/quote]

i emboldened the real important part
segwit wants to verify in a very haphazard way (4) when its far quicker to just be (5) knowing your protecting the network at every point.
sending out dodgy data and then alerts and then verifying dodgy data. to then alert new peers who then need to verify said dodgy data.. is just mind bending..
easier to just drop the data and end the endless cycle of bandwidth bloat.

If implemented correctly, there would be little to no bandwidth bloat. It means it would be runnable by low-end hardware, while achieving security higher than current SPV. Those demanding more security would still run full nodes.

my theory but from different angle
https://i.imgur.com/JOBRymb.jpg

(1) this is what the fullnode blockchain does.. checks the rules, if violation, drop it dont relay it.. dont alert anyone.. it didnt exist
now
(2) is segwit. it DOES NOT check data at first. but waits for an alert. as you can see SWLiteA alerts the next Swlite1, even though it has confirmed its duff.. the alerts keep happening..SWLite1 alerts SWLiteX who alerts SWLite& and so on and so on endlessly. look at all the extra queries each client is making.

(3) is segwit. that DOES check data at first. as you can see SWLiteA drops the data and doesnt relay.. now the rest of the network does not need to worry or check

One would hope.


Unsupported assertion. Could you make a proper defense of this?


With the 'low end' that this enables not even saving half over a full node, I rather doubt that many would opt for this unsecure near-full-node. (Thank goodness.)


But still insecure.


This I can agree with.

This is rather far from reality. How do you think block relay works? You don't simply send a full block to everyone. Instead, to every connected node you send a short 'inv' (inventory) message, containing a hash of a block/tx you've just received (and verified). Then, these nodes can request this data from you, if they don't already have it. The point is that you don't know if nodes already have this block, and in order for a block to reach every node, you need this small overhead, you need to tell everyone that you have it.

The same is with alerts. In order for everyone to know about fraud you have to relay it to every node that wants it. I guess it can also be done by firstly notifying nodes with a short inv message. This way the overhead would be comparable to block propagation.

Anyway this is theoretical, I don't know how this will be implemented, there's no formal protocol.

I don't know how you estimated all this, but nevermind.

i understand that, i was trying to keep things simple.. rather than waffling.
EG
as you say
1.fullnodeA connect to node fullnode1
2.fullnodeA sends "i have upto blockheight 400,000"
3.fullnode1 only has 399,999, so requests 400,000 from fullnodeA, fullnodeA sends 400,000

now using (1) in previous image where X is 400,001

4.dodgy miner broadcast it has 400,001 height
5.FullnodeA is only at 400,000 so asks for 400,001 from miner and dodgy miner sends 400,001
6.FullnodeA sees rules broken. deletes it and listens for anyone with a valid 400,001 while broadcasting it only has 400,000 still
7.Fullnode1 only has 400,000 and is patiently listening to anyone with a valid 400,001

translate to layman
1.miner sends 400,001 , fullnodeA sees rules broken. deletes it and wont relay it..

sorry, but i was not trying to waffle using 7 lines of jargon. just to make a 1 line point..
i didnt want to have to explain how they handshake and resync the chains.. its errelavant to SWLite checks i just wanted to point out that fullnodes when seeing a dodgy block wont relay it.

i always try to keep things laymans simplified.. as then the general public can get their minds around the concept without waffle.

when talking to people about computer hardware.. i dont say RAM or Harddrive i say short term memory and long term memory, web camera is eyes, mouth is the speaker microphone is the ears,

where some would say
a CMOS webcam is: 640 x 480
a HDwebcam is: 1920 x 1080

i say
a CMOS webcam are: eyes with cataracts
a HDwebcam is: perfect 20:20 vision

im sorry that my explanations are not whitepaper material.. but this is not bitcoin-dev, im glad though that the only thing you can knit pick is my lack of technical jargon and waffle

but as i show in (3) when SWLiteA checks if X block exists.. im not talking about reync standard protocol.. as SWLite will need to do more than just check blockheight..

eg
fullnodeA receives 400,001 hash: h3hdkdfksdlfksjlkfj checks and finds rule is broken, deletes it to say it only has 400,000
fullnodeA receives 400,001 hash: ldpffdp4989df988i from different miner, checks and finds rule is good, stores it and now has 400,001

if i was to confuse people by talking about standard resyncing relationship. they would not understand me saying
SWLiteA asks does fullnodeA have 400,001
as eventually the answer would be yes..

my image was not about resyncing. (checking who has height) it was about a separate check
SWLiteA asks does fullnodeA have 400,001 hash: h3hdkdfksdlfksjlkfj
nope
which if we just done resyncing method you elluded to it would be
SWLiteA asks does fullnodeA have 400,001
yes
so again im sorry i was to simplistic.. and not talking about separate checks and hash's and txindex's etc.. but "does X exist" was much simpler to say

i just didnt bother to explain all the variables SWLite would need to check.

and that is why i didnt want to confuse people by even mentioning the standard resyncing part, and treated it as a different call, purely for easy understandings sake

When your banks are closed during weekend, do you use Chinese RMB because their banks works during weekend? If you are afraid of being taken over by alt-coins, get some alt-coins, just in case  ;)

As statistics show, most of the users don't spend their bitcoin, simply because they will spend depreciating fiat money first and hold bitcoin to protect them from inflation. If you have $4000 and 10 bitcoin, which one do you spend first?

So, given people mostly purchasing bitcoin for long term saving, then they will purchase once a while and should not be very sensitive to transaction frequency and fee. Also, today there are more and more realtime mobile payment system that charges user 0 fee for instant transactions, it is a waste of time trying to scale bitcoin in order to get close to the speed and cost of any centralized solutions today

Maybe there is a real pressure for pools, exchanges and wallet service providers. But these organizations, being centralized, they should pursue clearing based solution to solve their problem, instead of giving pressure to core devs to let the blockchain serve them: Bitcoin is not designed to serve the institutions but person to person

Let's recheck Gavin' quote here:
"Segregated witness transactions won’t help with the current scaling bottleneck, which is how long it takes a one-megabyte 'block’ message to propagate across the network– they will take just as much bandwidth as before. There are several projects in progress to try to fix that problem (IBLTs, weak blocks, thin blocks, a “blocktorrent” protocol) and one that is already deployed and making one megabyte block propagation much faster than it would otherwise be (Matt Corallo’s fast relay network)."

So, a solution changes nodes' architecture, reduces their security, but does not help to reduce the major bottleneck, why rush with that? It is a smart way of thinking but the involved code change and potential security risk just make it less attractive than simply raise the block size to 2MB to deal with current block size limitation. In fact it is a very long term solution to totally change the bitcoin architecture,  thus require much more time and effort to test

https://www.youtube.com/watch?v=NOYNZB5BCHM

Peter Wuille: Segregated witness and its impact on scalability at SF Meetup with more time for questions and clarifications.

Also a good summary by StephenM347
http://bitcoin.stackexchange.com/a/41771

Wait a minute.  Lets just stop right there shall we?  Because I was under the impression that a transaction, and in fact any information at all, could already be divided up in to ones and zeros and handled any way we liked. 

Isn't a "single chunk of data" actually a bunch of ones and zeros that any client, miner, observer, bot, or individual, and do whatever they want with - put in tables, trees, hash them, store them.. whatever?  That is, already? 

What am I missing here?  How is this a thing to be discussed or debated?  Go, split your transactions up any way you like.  Nobody is stopping you. 

Personally I have some code that only keeps the first 20 base58 digits of btc addresses, dropping the rest, to save space.  I didn't make a post on a forum about it or a talk at a conference though.  You don't have to ask permission to manipulate public data. 

I think SegWit brings up an interesting point.

Namely - What data is essential to Bitcoin ?

What do we need to store and what can we through away ?

Which bits matter for the security and integrity of the system ?

..

I'm of the opinion that the inputs and outputs could also be put into an external disposable merkle tree (maybe the same one..).

The only bits that matter are the block headers, to show the POW, and the UTXO set. (this would probably aid anonymity as well)

It would of course require storing the UTXO in some fashion, ( the root hash of that tree would be mined into the block headers ) but certainly not beyond the capabilities of 'bitcoin developers'..  ;)

Those are the only bits that matter IMHO and then we could seriously talk about shrinking bitcoin 100 fold (in storage requirements).

1. imagine you have a pair of pants(fullnode) and in the pockets(chain) you carry receipts for all your spending..
your girlfriend(segwit) loves wearing skinny jeans hates filling her pockets up.. she now wants you to keep the transaction part of the receipt in your left pocket and the part with the shops logo of the receipt in your right pocket.. the pants still weight the same as the total paper equals the same amount but she wants you to only think about the left pocket. and although you pretend the right pocket doesnt exist, you know its still weighing you down..

the only benefit of splitting up the receipts. is so that your girlfriend(segwit) can quickly grab receipts from your left pocket and read data without having to see useless logo's or "thank you for shopping with us" footers. she cant tell where you shopped or if the receipt is real.. but she can see that money has been spent and naively does accounts trusting you.

2. the girlfriend can photocopy(be relayed) the receipts and she can bin the logo part of the receipt.. but the guy in the pants (fullnode) will still need the right pocket of store logo's(signatures), because he needs to be a seeder for anyone else who may ask for the receipts. so they can do proper checks that the receipts came from proper and real shops..

in short. splitting the receipts does not benefit fullnodes and only benefits lazy lite clients. that can easily split the chain themselves by deciding what details they dont want to save to file.

yes some say this means the left pocket is only 40% full, but with each new receipt, part goes into the left part goes into the right.. meaning the pants still get just as heavy as they would if they just made bigger pants.. instead of messing with the pockets.

so now the guy has to not only cut up the transactions to put into each pocket. not only give the girlfriend the 40% half copy of the receipt so they can fit into her smaller skinny jeans.. but if she questions where the receipt comes from, she questions the guy and he then has to check the other pocket. read which store it came from and tell her if its legit. causing more arguments (bandwidth bloat)

and also the shop logo receipt part(signature), the guy has to write in the receipt number and timestamp(hash) to it so that he can link the parts together (data bloat by having extra data on 2 parts instead of 1 data on one part

id much prefer a girlfriend who didnt ask me to re-arrange my pants, and that girlfriend simply didnt wear any pants at all and simply asked me about a particular receipt when she needs to know it.. rather then asking for every receipt for the last 7 years cut up and handed to her. where in 99.99% of the time she wont need to look at as the receipts in only 0.01% has relevance to her.

after all its not like her copies of the receipts can be of any use, especially if i have to be the one to tell her that one of the receipts has an error.. thus there is no point her handing the receipts onto anyone else either. as she would need to inform them of issues too and they would then need to check with me. which would get annoying

It makes no sense to stop right there. You can't just split a transaction without it losing its integrity, because we need the whole transaction to calculate its txid. With SW we can do that by introducing a new consensus rule (witness merkle hash), and now we will have two hashes representing a transaction: txid and witness hash. The benefit is that only txid is used as a reference in the blockchain, and this way we solve the malleability issue introduced by scriptSig.

You are painting it as if you would need to manually do all this stuff. It's all casually done by the software, you likely won't notice any changes.

You are arguing against a semi-full-node architecture that is better than SPV. Why would you do that? Are you against SPV as well? Do you understand flaws in its current design? Don't you see how it could get much better with segwit? Do you think we should all run full nodes? Do you think it's realistic?

Thanks for your reply.  Well certainly we don't want to lose integrity!  That is important emphasis thank you.  

One can split data apart, move it around, stack it in different piles, but if one has thrown out the instructions for how to put it back.. well, one is royally fucked at that point or at least out the original data.

So a couple things come to mind:

1) Malleability of TXID has never been an issue or a problem

2) The fundamental data we need for integrity is still the fundamental data we need for integrity.  Nothing has really changed.  If you come up with a clever way to store data that helps miners be efficient, that's great.  Not that any of them would bother looking here anyway for such ideas.  Nor would they care if pull requests are "approved".  Just grab the code and run it if you like.  
    

Here? No but they certainly look up to the Core devs, right or wrong.

"Just grab the code and run it" is pretty much the definition of a miners enforced soft fork, no?

For you? Because if we're to implement full Lightning Network and possibly other sophisticated solutions, we need malleability solved.

I'm not sure that's the case, but of course I could be wrong.  Can't you build it to actually check receipt of funds?  I mean, that should in theory be the important part of making a payment anyway, rather than thepayment ID.  Please let me know what I am missing here.     
 

Lightning Network can work just fine with tx Malleability but wont be as efficient and cannot scale as long as it remains. This is why solving it is one of the requirements to proceed with the project.

https://blog.bitgo.com/malevolent-malleability/
https://lightning.network/lightning-network-paper-DRAFT-0.5.pdf

Sorry I am slow.  What won't be as efficient?  I guess I don't understand the problem exactly.

https://blog.bitgo.com/malevolent-malleability/  <-- I see here a list of how folks who didn't know what they were doing could make mistakes, and a conclusion: "The general consensus up to this point has been that malleability is an annoying but not critical system-wide problem. "

Anyway, if for some reason you can't figure out a way to actually check that funds have been sent and arrived at the proper address, or to not reference any TXs by number, you can always pay a miner a few extra milliies to put your TX with exactly the bit order you like into the block just where you want it. 

The whitepaper details it a bit more. In laymans terms, tx malleability allows interesting and complex attack vectors on non-confirmed transactions. Since the Lightning network is a caching layer where contracts are made between lightning nodes before confirmations appear than one has to assume all implications in a hostile environment. Fixing malleability allows for decentralized and untrusted parties to cache these tx's. In order to cache tx with malleability than one has to have centralized sources of trust which basically amounts to a coinbase/circle model of off the chain txs. Centralized off the chain solutions do provide a valuable service to our ecosystem but have heavy inherent regulatory, human , and insurance overhead. If bitcoin is to compete with other payment systems and fullfil its true vision it must eliminate these inefficient and corruptible sources of security.

It seems that's the main reason here, so let's drop Lightning Network

If a clearing based solution, require so much change to bitcoin's architecture, then it must be able to provide lots of benefit to worth the risk. Currently I don't really see a big difference between lightning network and traditional clearing solutions, which require no changes for bitcoin at all

BTW, I just heard that Adam Back said that you need insurance for lightning network to work properly (https://www.reddit.com/r/bitcoinxt/comments/3wty7s/dr_adam_back_believes_that_insurance_may_be/)

Ok, if lightning network need insurance to work, and traditional clearing solution also works perfect given insurance, then why don't just use existing mature clearing based solutions? I thought the biggest benefit Lightning network has against traditional clearing solution is that it requires no trust, but it seems not the case. If you need insurance to be trustworthy, then there must be some fundamental weakness with the design of lightning network. I have not looked into details about this statement, too low signal to noise ratio there, but it is very natural that LN like any new system have many security problem which only time will tell if it is a robust design

In my not-so-deep understanding of LN, they are using a similar design as NashX exchange's mutually assured destruction model to keep it trustless, however, that model does not work well under certain circumstances. That's also the reason those so called P2P exchanges can not gain any momentum against localbitcoins: You eventually need an authority to solve a complex dispute, blockchain can not be this authority since it lacks judgement

Ok, everyone can go home and sleep, no work needs to be done, bitcoin is perfect, just raise the block size limit to 2MB for the time being  :D

With that kind of attitude you might have just continued to use fiat instead of Bitcoin. But you've managed to comprehend Bitcoin. You can do the same with segwit or LN. It just means you must spend time on that, and refrain from making judgements until you're finished.

Mark Friedenbach: https://www.reddit.com/r/btc/comments/3woin3/to_adam_back_we_are_hereby_officially_requesting/cxzpcpw

The problem is that time is the only precious resource that is constantly reduce for anyone. If you can not present a system that is simple enough for others to fully understand in a couple of days, they will just ignore it

In a centralized organization you don't need other's understanding to enforce a change, but here ...

He was discussing a hypothetical where insurance might possibly be necessary with certain conditions like many more tx and extremely cheap rates.

https://www.reddit.com/r/Bitcoin/comments/3siyff/z/cwxp1oi

The way you phrased it sounds like it would be required for the LN to even work which is very misleading.

Thanks for your reply.  I think the details are important here.  "I have a way to make a secure lightning network if TXs were not malleable" is not the same as "I have a proof that lightning networks will only work securely if TXs are not malleable".   Sadly I'm not qualified to tell you at the moment whether one, both, or neither of these are true.

/begin rant

This "if bitcoin is to compete with other payment systems" stuff is all nonsense.  Bitcoin is a unit of account.  A public, verifiable, noncounterfeitable unit of account.  Visa, Swift, and ACH don't offer that.  There's no competition. All of these work on top of bitcoin just as well (or better) than they did on top of funbux.  .    ,

/end rant

Anyway, even if someone were to come up with a proof that certain forms of payment channels are only possible with a hardfork level protocol change..  it still wouldn't happen.  I'm still interested though!  :D   

I would love to listen to this since the latest lecture was very limited in time as there was a bunch of other speakers in queue, but I just can't stand having to hear a voice from only one of my headphones...

Is there a way to make the sound mono? It's not that hard to do it before you upload it, so much for tech geniuses that fail to duplicate the left audio channel into the right one before uploading :P

First world problems. Just sayin'.

You could always just short the left hot to the right hot. An adapter could be soldered up in five minutes tops, and that would include the time spent rooting around for the soldering iron. Or go ghetto and slice into the leads, twisting the wires together.

^(partially) in jest.

....or wait till he gets home and listen with speakers instead of headphones at work. I bit tedious we have to point out the obvious

or even a simpler hack..
you know when you put an audio jack into the socket. you feel it click twice.. pull it out slightly so you feel only one click..you should now get mono in both headphones

Ahh genius ...

Garziks concerns with segwit---

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011976.html

A summary of the scalability consensus status-


There also appears to be a disagreement between developers between how quickly a soft fork vs a hard fork can be deployed. Matt Corallo suggests that it will take 1.5-2.5 years while garzik suggests it can roll out quicker than the segwit soft fork.

Either way, there may be a need to for trustless off the chain solutions using CLTV in the event of potentially negative effects from a Fee market event.

https://bitcoinmagazine.com/articles/segregated-witness-part-how-a-clever-hack-could-significantly-increase-bitcoin-s-potential-1450553618
Good explanation of segwit

In fact, the possibility of carrying out such a soft fork to change bitcoin's architecture without being rejected by old nodes revealed a fundamental security problem for bitcoin that needs to be addressed: You can even increase the total amount of coins in a similar way, because the old nodes can not see new coins, only new nodes can see

Soft Fork to Increase the 21M Limit? (https://bitcointalk.org/index.php?topic=1285211.20)

Of course the money supply limit is prohibited from being changed, but there could be other changes which users do not fully understand its dangerous potential today. Now I understand why Satoshi said it will either worth nothing or a lot, there are just so many uncertainties on the road ahead which can wipe out this ecosystem altogether overnight

But this is bad, bitcoin is trust-less since all the trust is placed on this network and mathematics. I think at least we should make a system that is trust-backward-compatible, e.g. any aspects that affects trust should continuously strengthen the trust, not weaken it

For example, the changes I think are trust-backward-compatible: Any change that increase the security of the system; Any change that increase the mining decentralization (encourage more and more private miners, p2pool for example); Any change that increase the communication and understanding of different actors in the ecosystem

Seems the forkers just can't get enough of their governance illiteracy, rolling out the heavy artillery after the XT debacle, soft or hard forking for every little feature that make their business plan irrelevant.

Such time and energy spent in lame attempts at highjacking bitcoin from within the inside. Banksters and corporatists trying to preserve their privilege. Almost makes you sorry for them. But not.

That's pretty crazy to be honest. How I solved it: Get a Firefox plugin that allows you to download videos, open it up in a video editor and make the audio track mono (there are free editors that can do this). Render it and voila, you have your headphone ready video.

I like this analogy, it is a good representation of SW: Because currently there is a limitation on size of your left pocket on this pants (1MB), and it is not easy to change the size of this pocket (Difficult to reach consensus, the pants might be too heavy etc... ), so the solution is to cut the receipts in half and put the other half in another newly made pocket (SW data)

I'm still evaluating the security consequence of the new architecture on full nodes under SW. Is there a 100% sure way to prove a specific set of SW data belongs to a specific block? I think nodes still needs to go through every transaction to verify both the block and SW

This is also a "kick the can down the road" approach, provide one time increase of block capacity, while made large change to bitcoin's architecture, so the question is: Is it really worth the effort comparing with simply raise the block size to 2MB?

I'm still evaluating what type of user will want to run a medium-weight client - which seems to have none of the security of a full node, while still requiring about half the storage and bandwidth thereof. Is there really space in the market in between a full node and an SPV client? Maybe if the demands were 10%. But at half, I kinda doubt it. Who is that user?

With SW the fraud proofs required to bring SPV security up to 'almost as good as' fullnodes will finally be available. Look for 'Alerts' in the Nakamoto white paper.

Trusting others to provide 'proof of fraud' is still trusting others.

You are correct , we should have a healthy amount of full nodes distributed across the world that SPV clients with Fraud proofs can depend upon. Hopefully, we can work together to reverse node centralization and create incentives beyond "security" that increase full node count.

Any thoughts or ideas to incentivize full nodes besides - https://bitnodes.21.co/nodes/incentive/ which only gives nodes a very small chance weekly to win 10 USD? This is nice but clearly not enough. ~1/100 chance of winning 10 usd in a year.

Still that's a lot better than current headers-only SPV design and gives more freedom designing lite wallets.

I guess I'm missing it. Seems to me that needing to trust others is needing to trust others. How is this an improvement?

SPV nodes with Fraud proofs raise the standard and allow for fraud alerts and limited auditing with SPV lite nodes that currently don't have these capabilities. As long as having these fraud proofs doesn't incentivize us to drop full nodes than security will be improved within the ecosystem. The SPV client isn't directly dependent upon one full node but can audit many full nodes with a fraud proof.

Further reading in different proposals -
https://www.reddit.com/r/bitcoin_devlist/duplicates/3vzm35/why_sharding_the_blockchain_is_difficult_peter/
https://gist.github.com/justusranvier/451616fa4697b5f25f60

Your point is still valid in our need to have more full nodes as well.

OK. I'm just not seeing what type of user would switch from an SPV client to a SegWit 'medium-weight' client. Especially when the SegWit 'medium-weight' client makes not only on the same order of magnitude, but around half as much, demands on their system as does a full node.


Nothing in SegWit seems to address this.

Within separated signatures (I'm going to start calling the upgrade SepSig, as segwit is misleading and a horrible name*) there is an incentive structure to lower tx fees and encourage adoption with a discount . This might not even be necessary as most users simply accept and download the update which will upgrade them automatically to "medium weight" SPV client.


Agreed, that is out of separated signatures scope. We need other solutions to address this problem as bitnodes lottery clearly isn't enough and centralization will get worse when SepSig allows for the blocksize capacity to increase from 1.8-4x in capacity. I have been thinking long and hard about various solutions and am testing things now. My guess is various "off the chain" solutions to encourage full nodes might be good enough.


* "segregated" has negative connotations and "witness" usually refers to probabilistic proof in which signatures aren't.

Edit- Developers rallying behind SepSig and Gregory Maxwell's scaling roadmap:
https://github.com/bitcoin-dot-org/bitcoin.org/pull/1165
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011865.html

BTCC skeptical but appears to give 3 months or room to see rollout before hard forking
https://twitter.com/Excellion/status/679066301168418816

http://blog.oleganza.com/post/135710722553/how-segregated-witness-is-not-the-same-as-bumping


How segregated witness is not the same as bumping block size limit

the irony: "trust others" to provide "proof of fraud" ::)


https://thedilettantesdilemmadotcom.files.wordpress.com/2015/04/accountability-2.jpg


bitcoin is trust.

See, that's not making any sense at all to me. Currently, Bitcoin Core performs complete validation. If most users download the 'update', they end up not with an upgrade, but rather a downgrade that requires them to trust that others will not feed them false data.

So we go from full nodes with zero trust requirement to medium-weight nodes that need to trust others. > net loss of security

Meantime, SPV users make no change, so their net security is unchanged.

Unless you want to make the argument that SPV users are going to adopt this new medium-weight client -- which in terms of resource requirements is much closer to full node than SPV. I just don't see it happening.

This entire SegWit 'lighter-weight' claim seems like a veneer meant to disguise the other changes being ushered in.

I have no issue with trusting others to provide proof of fraud. If fraud has been proven, I would like to know. But proof of fraud is a by no means a substitute for proof of lack of fraud. There's a chasm wide enough to absorb all the funds one can imagine between those two proofs.

Full nodes like Bitcoin core will continue full validation even with SepSig, its just the SPV nodes that will slowly start upgrading to fraud proof layers of security when their devs roll out the changes.

Those that currently run full nodes will download Bitcoin Core/or other implementations and continue with full validation and security. Those that run spv wallets like those found on their cell phone will automatically upgrade from a regular SPV client to a SPV client with fraud proofs when it starts being used. On the mailing lists many of the SPV wallet devs support this and have indicated a quick roll out . To the user it will be seamless and not change anything but upgrade their security.

If you are insinuating that new fraud proof layers will convince some to switch from full validation to partial validation I would disagree as there is no evidence for that. Those that currently run full nodes do so because they are miners or understand the importance of them, or simply are resistant to changing what they are used to ... because doing so is a heavy burden compared to SPV nodes. Right now there is, unfortunately, strong incentives to run lite or web clients :'( We need to upgrade those SPV clients with fraud proofs and figure out ways of increasing full node count simultaneously.

yes, my thoughts exactly, there is no "proof of not fraud", and hence this will open a fraud -ie security- breach, at core level.
something that is way more crucial than increasing transaction capacity, which is not a bug, but a feature for such decentralized system to prevail.

but cant stop the progress it seems, the socialmedia forces have been pushing hard for a bloatchain if not outright denaturing of the satoshi codebase.

I think every thing about trust eventually comes from risk, if there is no risk involved then you can trust anyone

There are many ways to manage risk. People still manage the risk very well in today's legacy financial system since they diversify the financial risk in different area or different country, not trusting any single entity, but multiple of them

Just like a bank's opinion, trust is not always a bad thing, it can dramatically increase the efficiency. A trust-less system is always of low efficiency since you have to independently verify every piece of data from anyone

For small money, I would rather just trust a service provider and do not even bother with those light nodes. A good example is localbitcoins, although they are centralized p2p exchange, but you do get lots of benefit from it over true p2p exchanges, like having human admin to resolve complex dispute or scam, and even frozen coins from a scammer, instant and fee free coin transaction between users etc... you only need to be careful to not put lots of money there. A true P2P exchange however are not consumer friendly, due to enormous amount of scammers and hackers out there

https://bitcoin.org/en/bitcoin-core/capacity-increases-faq

Is the segregated witness soft fork equivalent to a 4MB block size increase, a 2MB increase, a 1.75MB increase, or what? I keep hearing different numbers.

The current proposal for soft fork segregated witness (segwit) counts each byte in a witness as 0.25 bytes towards the maximum block size limit, meaning the maximum size of a block is just under 4MB.

However, blocks are not expected to consist entirely of witness data and each non-witness byte is counted as 1.00 bytes towards the maximum block size limit, so blocks near 4MB in size would be unlikely.

According to some calculations performed by Anthony Towns, a block filled with standard single-signature P2PKH transactions would be about 1.6MB and a block filled with 2-of-2 multisignature transactions would be about 2.0MB.

Segregated witness sounds complicated; will the ecosystem be prepared for its deployment?

Some ideas are easy to explain but hard to execute. Other ideas are easy to execute but hard to explain. Segregated witness (segwit) seems to be the latter.

Segwit can be deployed incrementally without breaking compatibility, so no significant preparation of the ecosystem is necessary. Developers who want immediate hands-on experience with segwit can begin to test their software on the segwit testnet being deployed in Dec 2015.

Initially, only miners who wish to support it need to upgrade in order to activate it and enforce it on the mainnet. Existing applications only need to change if they wish to take advantage of the new features.

Segregated witness transactions will require lower fees, will afford much greater performance optimizations, and can support multistage smart contracts and protocols such as bi-directional payment channels that can scale without writing extra data to the blockchain. Wallets are strongly encouraged to upgrade but can continue to operate without modification as the deployment does not break backwards compatibility.

Segregated witness sounds complicated; will the ecosystem be prepared for its deployment?

Some ideas are easy to explain but hard to execute. Other ideas are easy to execute but hard to explain. Segregated witness (segwit) seems to be the latter.

Segwit can be deployed incrementally without breaking compatibility, so no significant preparation of the ecosystem is necessary. Developers who want immediate hands-on experience with segwit can begin to test their software on the segwit testnet being deployed in Dec 2015.

Initially, only miners who wish to support it need to upgrade in order to activate it and enforce it on the mainnet. Existing applications only need to change if they wish to take advantage of the new features.

Segregated witness transactions will require lower fees, will afford much greater performance optimizations, and can support multistage smart contracts and protocols such as bi-directional payment channels that can scale without writing extra data to the blockchain. Wallets are strongly encouraged to upgrade but can continue to operate without modification as the deployment does not break backwards compatibility.

Segregated witness still sounds complicated. Why not simply raise the maximum block size?

There’s a single line of code in Bitcoin Core that says the maximum block size is 1,000,000 bytes (1MB). The simplest change would be a hard fork to update that line to say, for example, 2,000,000 bytes (2MB).

Hard forks are anything but simple:

    We don’t have experience: Miners, merchants, developers, and users have never deployed a hard fork, so techniques for safely deploying them have not been tested.

    This is unlike soft forks, whose deployments were initially managed by Nakamoto, where we gained experience from the complications in the BIP16 deployment, where we refined our technique in the BIP34 deployment, and where we’ve gained enough experience with BIPs 66 and 65 to begin managing multiple soft forks with BIP9 version bits in the future.

    Upgrades required: Hard forks require all full nodes to upgrade or everyone who uses that node may lose money. This includes the node operator, if they use it to protect their wallet, as well as lightweight clients who get their data from the node.

    Other changes required: Even a single-line change such as increasing the maximum block size has effects on other parts of the code, some of which are undesirable. For example, right now it’s possible to construct a transaction that takes up almost 1MB of space and which takes 30 seconds or more to validate on a modern computer (blocks containing such transactions have been mined). In 2MB blocks, a 2MB transaction can be constructed that may take over 10 minutes to validate which opens up dangerous denial-of-service attack vectors. Other lines of code would need to be changed to prevent these problems.

Despite these considerable complications, with sufficient precautions, none of them is fatal to a hard fork, and we do expect to make hard forks in the future. But with segregated witness (segwit) we have a soft fork, similar to to other soft forks we’ve performed and gained experience in deploying, that provides us with many benefits in addition to allowing more transactions to be added to the blockchain.

Segwit does require more changes in higher level software stacks than a simple block size increase, but if we truly want to see bitcoin scale, far more invasive changes will be needed anyway, and segwit will gently encourage people to upgrade to more scalable models right away without forcing them to do so.

Developers, miners, and the community have accrued significant experience deploying soft forks, and we believe segwit can be deployed at least as fast, and probably more securely, than a hard fork that increases the maximum block size.

Will there be a hard fork before or as part of the segregated witness implementation?

No. That is not part of the roadmap.


How will segregated witness transactions work for wallets?

Wallets that currently support P2SH can migrate to full segregated witness in two phases:

    Phase 1: Scripts are hashed twice, first to 256 bytes and then to 160 bytes. The 160 byte hash will be compatible with existing P2SH addresses, so upgraded wallets will be able to send and receive bitcoins to and from currently existing wallets.

    Phase 2: Scripts are hashed once to 256 bytes. This format will not be compatible with existing wallets but will allow more efficient use of block space and will offer better security due to greater collision resistance.

If no one is forced to upgrade, why will anyone bother to upgrade? I heard P2SH took almost two years to become widely deployed.

Each byte of the witness part of a segregated witness (segwit) transaction will only count as 0.25 bytes towards the size of the transaction. Since transaction fees are based on the size of a transaction, this is effectively a 75% discount on fees for that part of a transaction—but only for people who use segwit.

David Harding provided a table of estimated savings at various fee/transaction levels. That is, if the fee for a typical 250-byte transaction is $0.01 USD, using segwit will save about $0.003 when spending a P2PK-in-P2SH transaction output.

Transaction    Bytes saved    $0.01/250B    $0.05/250B    $0.25/250B    $1.00/250B
P2PK-in-P2SH    79/107    $0.003    $0.015    $0.079    $0.316
1-of-1 P2SH multisig    83/112    $0.003    $0.016    $0.083    $0.332
2-of-2 P2SH multisig    163/219    $0.006    $0.032    $0.163    $0.652
2-of-3 P2SH multisig    189/254    $0.007    $0.037    $0.189    $0.756

(We don’t expect fees to get as high as the highest seen in this table; they are just provided for reference.)

Web wallets and exchanges that send large numbers of transactions each day at fixed rates (such as for free or for 1% per spend) are expected to be early adopters—even the small savings per spend seen in the table above adds up to significant amounts of money if repeated hundreds or thousands of times a day.

“Why would miners adopt the SegWit format, given that it does not provide any savings of bandwidth, storage, or processing time to them?”

Most previous soft forks have not provided these benefits to miners either. For example,
BIP16 (P2SH)    New transaction type
BIP30 (duplicate txids)    Required checking for duplicate txids
BIP34 (height in coinbase)    Reduced miner coinbase space by 4 bytes
BIP65 (OP_CLTV)    New opcode

The BIP66 (strict DER) soft fork which activated in July 2015 will soon be providing reduced processing time by making it possible to switch to libsecp256k1 for validation as described elsewhere is this FAQ. The reduced validation time make makes it fairly unique among soft forks in providing direct benefits to miners.

What segregated witness (segwit) does is provide several major benefits to anyone who uses it to create transactions:

A permanent fix for third-party malleability, allowing multi-stage smart contracts to flourish. A modest reduction in fees. Easy future upgrades to Bitcoin Script, so wallets can more easily gain access to new features.

Through the previous soft forks, and through conversations such as the Miners’ Panel at Scaling Bitcoin Hong Kong, miners have repeatedly shown that they want Bitcoin to be the most useful system possible even if they don’t receive any direct benefits. Segwit and the other improvements in the roadmap provide significant usability enhancements.

In addition, segwit allows miners to put more transactions in their blocks, which may allow them to increase their per-block revenue.

Jeff and Gavin are not in the list. I had a feeling that block size limit has long become a political weapon for each faction to take control of the development process thus realize their own vision of bitcoin

And this is a result when lots of money is involved  ;D

In today's society, when you are talking about a project worth $6.6 billion in an enterprise, developers has no decision making right at all. It is the biggest shareholder make decisions at board level, and technical solution is only a small part of the decision making process

However in a decentralized open source project like bitcoin, there is no board, no one knows how it is going to work

Concern over validationless mining is incentivized by SepSig

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012103.html


# Summary

1) Segregated witnesses separates transaction information about what
coins were transferred from the information proving those transfers were
legitimate.

2) In its current form, segregated witnesses makes validationless mining
easier and more profitable than the status quo, particularly as
transaction fees increase in relevance.

3) This can be easily fixed by changing the protocol to make having a
copy of the previous block's (witness) data a precondition to creating a
block.


# Background

## Why should a miner publish the blocks they find?

Suppose Alice has negligible hashing power. She finds a block. Should
she publish that block to the rest of the hashing power? Yes! If she
doesn't publish, the rest of the hashing power will build a longer chain
than her chain, and she won't be rewarded. Right?

Well, can other miners build on top of Alice's block? If she publishes
nothing at all, the answer is certainely no - block headers commit to
the previous block's hash, so without knowing at least the hash of
Alice's block other miners can't build upon it.


## Validationless mining

Suppose Bob knows the hash of Alice's new block, as well as the height
of it. This is sufficient information for Bob to create a new, valid,
block building upon Alice's block. The hash is needed because of the
prevhash field in the block header; the height is needed because the
coinbase has to contain the block height. (technically he needs to know
nTime as well to be 100% sure he's satisfying the median time rule) What
Bob is doing is validationless mining: he hasn't validated Alice's
block, and is assuming it is valid.

If Alice runs a pool her stratum or getblocktemplate interfaces give
sufficient information for Bob to figure all this out. Miners today take
advantage of this to reduce their orphan rates - the sooner you can
start mining on top of the most recently found block the more money you
earn. Pools have strong incentives to only publish work that's valid to
their hashers, so as long as the target pool doesn't know who you are,
you have high assurance that the block hash you're building upon is
real.

Of course, when this goes wrong it goes very wrong, greatly amplifying
the effect of 51% attacks and technical screwups, as seen by the July
4th 2015 chain fork, where a majority of hashing power was building on
top of an invalid block.


## Transactions

However other than coinbase transactions, validationless mined blocks
are nearly always empty: if Bob doesn't know what transactions Alice
included in her block, he doesn't know what transaction outputs are
still unspent and can't safely include transactions in his block. In
short, Bob doesn't know what the current state of the UTXO set is. This
helps limit the danger of validationless mining by making it visible to
everyone, as well as making it not as profitable due to the inability to
collect transaction fees. (among other reasons)


# Segregated witnesses and validationless mining

With segregated witnesses the information required to update the UTXO
set state is now separate from the information required to prove that
the new state is valid. We can fully expect miners to take advantage of
this to reduce latency and thus improve their profitability.

We can expect block relaying with segregated witnesses to separate block
propagation into four different parts, from fastest to propagate to
slowest:

1) Stratum/getblocktemplate - status quo between semi-trusting miners

2) Block header - bare minimum information needed to build upon a block.
Not much trust required as creating an invalid header is expensive.

3) Block w/o witness data - significant bandwidth savings, (~75%) and
allows next miner to include transactions as normal. Again, not much
trust required as creating an invalid header is expensive.

4) Witness data - proves that block is actually valid.

The problem is #4 is optional: the only case where not having the
witness data matters is when an invalid block is created, which is a
very rare event. It's also difficult to test in production, as creating
invalid blocks is extremely expensive - it would be surprising if an
anyone had ever deliberately created an invalid block meeting the
current difficulty target in the past year or two.


# The nightmare scenario - never tested code ~never works

The obvious implementation of highly optimised mining with segregated
witnesses will have the main codepath that creates blocks do no
validation at all; if the current ecosystem's validationless mining is
any indication the actual code doing this will be proprietary codebases
written on a budget with little testing, and lots of bugs. At best the
codepaths that actually do validation will be rarely, if ever, tested in
production.

Secondly, as the UTXO set can be updated without the witness data, it
would not be surprising if at least some of the wallet ecosystem skips
witness validation.

With that in mind, what happens in the event of a validation failure?
Mining could continue indefinitely on an invalid chain, producing blocks
that in isolation appear totally normal and contain apparently valid
transactions. It's easy to imagine this happening from an engineering
perspective: a simple implementation would be to have the main mining
codepaths be a separate, not-validating, process that receives "invalid
block" notifications from another process containing a validating
implementation of the Bitcoin protocol. If a bug/exploit is found that
causes that validation process to crash, what's to guarantee that the
block creation codepath will even notice? Quite likely it will continue
creating blocks unabated - the invalid block notification codepath is
never tested in production.


# Easy solution: previous witness data proof

To return segregated witnesses to the status quo, we need to at least
make having the previous block's witness data be a precondition to
creating a block with transactions; ideally we would make it a
precondition to making any valid block, although going this far may
receive pushback from miners who are currently using validationless
mining techniques.

We can require blocks to include the previous witness data, hashed with
a different hash function that the commitment in the previous block.
With witness data W, and H(W) the witness commitment in the previous
block, require the current block to include H'(W)

A possible concrete implementation would be to compute the hash of the
current block's coinbase txouts (unique per miner for obvious reasons!)
as well as the previous block hash. Then recompute the previous block's
witness data merkle tree (and optionally, transaction data merkle tree)
with that hash prepended to the serialized data for each witness.

This calculation can only be done by a trusted entity with access to all
witness data from the previous block, forcing miners to both publish
their witness data promptly, as well as at least obtain witness data
from other miners. (if not actually validate it!) This returns us to at
least the status quo, if not slightly better.

This solution is a soft-fork. As the calculation is only done once per
block, it is *not* a change to the PoW algorithm and is thus compatible
with existing miner/hasher setups. (modulo validationless mining
optimizations, which are no longer possible)


# Proofs of non-inflation vs. proofs of non-theft

Currently full nodes can easily verify both that inflation of the
currency has no occured, as well as verify that theft of coins through
invalid scriptSigs has not occured. (though as an optimisation currently
scriptSig's prior to checkpoints are not validated by default in Bitcoin
Core)

It has been proposed that with segregated witnesses old witness data
will be discarded entirely. This makes it impossible to know if miner
theft has occured in the past; as a practical matter due to the
significant amount of lost coins this also makes it possible to inflate
the currency.

How to fix this problem is an open question; it may be sufficient have
the previous witness data proof solution above require proving posession
of not just the n-1 block, but a (random?) selection of other previous
blocks as well. Adding this to the protocol could be done as soft-fork
with respect to the above previous witness data proof.

Note that this fix can be designed to retain the possibility of
validationless mining, by allowing empty blocks to be created if the
previous witness data proof is omitted. This would achieve the same goal
as Gregory Maxwell's blockchain verification flag(1) but with
significantly less ability/reason to lie about the status of that flag.

1) [bitcoin-dev] Blockchain verification flag (BIP draft),
   Gregory Maxwell, Dec 4th 2015,
   http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/011853.html

This part concerns me: Although from individual miner's point of view, producing an invalid header is expensive, but if a malicious user want to attack the network by broadcasting invalid header, he doesn't need to process large amount of hash power, maybe even 1% is enough to cause large scale of orphan for the network every day

So from this point of view, I guess even in a SW architecture, all the data must be broadcasted as one single package and full nodes need all of them to validate the block. And SPV miners will have to pay for the orphan cost by themselves if they use header-only validation

Hmm. Doesn't inspire much confidence when the developers advocating the change (Peter Todd in this case) can't construct a simple logical argument.

What is the 'This' that is being 'easily fixed' in 3) above?

'This' is in the (2) -  'validationless mining', mining without validating witness data beforehand. Like 'SPV' mining, which is mining on top of headers without fully validating the previous block. This kind of an issue caused July, 2015 network split (https://bitcoin.org/en/alert/2015-07-04-spv-mining).

Is there anyway to stop or block segregated witness? From what I understand, it hits testnet in two days... :(

There is huge difference between a free software open source project (Linux) and a decentralized monetary system (Bitcoin). Imagine that if an enterprise or a group of programmers could take over the control of Federal Reserve, how many people would like to give it a try?

Bitcoin currently does not have self validation, it does not detect the change at protocol level and reject it, and miners only make sure the blocks are authenticate, not software itself, this makes its rule very flexible and constantly changing, a point of weakness

If some kind of code that make the validation of the client is also built in the blocks and pushed into thousands of nodes, then no one would be able to change the protocol without a hard fork. (I'm not sure if this is achievable since it is a chicken and egg problem)

This is the opposite direction of a soft fork, hardening the bitcoin thus make all the future improvements off-chain, can also be considered if no consensus can be reached upon core devs, or the risk that core devs become compromised is too large

The only people who want to stop it so far are ones that either hate on Core developers or any of their proposals, or who lack the understanding for Segwit.

I understand it and don't hate the core developers. Seg wit is a thoughtful solution, but not an elegant one. IMHO, it's just more spackle over the cracks and a further delay to increasing block size. 

Meh, people need to stop fucking around with The Protocol.

People are free to fork off and insert whatever 'solution' in their shitty altcoin.

There is no scaling problem for bitcoin that can be adresses by retardedly bending the rules.

Fuck the masses, fuck the industry, fuck the devs.

Bitcoin does not need them.

The delay is intentional as the core developers need more time to test and optimize the protocol to anticipate increasing the blocksize which all but Luke and Peter appear to be in favor of.

This fact stood out to me and stressed the importance to carefully prepare and study all attack vectors and changes that need to be done beforehand:



The average non-developer likely naturally assumes that 2MB blocks is a safe change to make and conservative, but a targeted DDOS attack exploiting that 10 minute validation delay(up from 30 seconds for 1MB ) would be disastrous.  



This is rather simple to answer. Gavin is in support of SepSig(although prefers a hardfork) , Hearn is no longer interested in XT or working with Bitcoin directly. So your first task is to amass a separate group developers to maintain a github fork and than convince enough nodes and miners to adopt your version whether its Bitcoin XT or Bitcoin Unlimited, or something else. I encourage you to do this as I think a diversity of choice and implementations is a good thing for bitcoin. I also encourage any supporters of alternate implementations to be both proud of their work and not play the victim if their implementation doesn't get adopted en masse.

Once I thought better of you, now you are acting equal to that Veritas and Zara something guy. Segwit is not bad; it's actually very far from that.

So? Nobody ever said it was elegant.

Yes.

Bitcoin has never been simple or elegant. Much of the work done by developers has been to clean up the messy tarball of code Satoshi created. Those wanting to have a simple increase in blocksize have good intentions but may be missing out on some finer nuances that make such increase premature. It will be great to support other implementations now however because no one has a good idea of what is going to happen if and when a  "Fee Market Event"* occurs. It will be great if we have working and tested backup solutions.

* There is good evidence that suggests we already have a fee market for tx's

No one answered your question, so I will.  The answer is yes, all you have to do is find unfixable problems with it and exploit them on testnet.  Doing the same with fixable problems will delay it at the very least while also ensuring it is more secure if it is ultimately deployed live.  Several potential attack vectors have been discussed in this thread, if any of them truly exist, you can "take advantage" of them on testnet and protect bitcoin at the same time.

Yes , that is the another solution to potentially either stop SepSig or improve it when solutions are discovered. This should naturally be the first course of action one should take as all other implementations can learn from and or use SepSig as well because it is a valuable solution to many problems. I highly recommend everyone test and attack SepSig on the testnet for the betterment of our ecosystem. Please post any results in this thread and mailing lists.

The second solution I already mentioned can be carried out simultaneosly if you wish -

This discussion sometimes has gotten heated and bitter, but without evidence to the contrary, I will assume good faith and welcome other ideas and competing development teams that we can all learn and share from.

Good observation

In change management, the first question is not about if a change is good or bad, it is why must we make this change. It is always the motivation behind the change worth looking

It seems SegWit is invented to temporary circumvent the hard coded 1MB limitation of the block size, so that the traffic can still grow without trigger a fee event

So the next question will be: Why are we so fear of a fee event?

Jeff gave some answer here:

I don't think this so called "chaos" is convincing enough, so the next question: Who are these bitcoin projects and businesses, and is bitcoin's goal to benefit average people or serve these projects/businesses?

Although institutions have large capital and influence in the industry, I don't think bitcoin's purpose is to become banks' another payment network (Banks being the highest form of business, a business large enough will start to do banking)

In fact, businesses can always pass the fee cost to customer, and those customers are not fee sensitive (Statistics showed that majority of users come to bitcoin and use it for long term value store and high value international remittance, both are not very sensitive to fee and transaction frequency) So a higher fee will not affect business either. And large businesses can establish clearing channel to dramatically reduce the fee cost, this is a common practice in financial world, they don't need to change bitcoin architecture to do that

So I think the motivation behind the architecture change of bitcoin is still not enough convincing. Since no one have seen a fee event, it might not be the "chaos" that is predicted by Jeff, so people must see it with their own eyes to be convinced that it is a problem that really need to be solved. What if it is not a problem at all? Banks are still closed during weekends and holiday, is that a problem for our financial system?

Even a fee event negatively affect majority of the user experience, the way to future scaling should still follow Satoshi's vision as much as possible. Anyway this is his invention, no one except him have the right to change it to something totally different

I must admit that I am guilty of such an assumption. Time linear with block size seems rational on the surface before looking into the matter.

Are you asserting that the worst cast for a 1MB block size today is less than 30 seconds on the same hardware that would have a worst case of 10 minutes if the only variable is a blocksize doubled to 2MB?

What are the characteristics of such an aberrant block?

Great answer. IF there are no killer problems in SegWit, I can be swayed to support it (my current position, as a conservative measure, is to try to uncover faults). If indeed we can throw all possible attacks at it on the test net, and it comes through unscathed, then what would be the downside to adopt it on the main net? (he asked rhetorically)

There is already some evidence that a fee market has existed :

https://rusty.ozlabs.org/?p=564

Even wallets like mycelium have 4 fee settings at the point of payment (low priority , economy, normal, priority) to address the fee market that has already occurred and is occurring.



Satoshi did build a lot of extensibility and op codes within the original design so bitcoin could grow, evolve, and use layers like the lightning network. While I do respect Satoshi we shouldn't worship him and treat everything he has done as sacrosanct as he has made many mistakes. What is more important is us respecting the investment contract we have all agreed to over the years about respecting the core fundamentals that makes bitcoin unique. Satoshi can always sign a PGP key and jump and and make a comment if he has some serious concerns as well.