Gold collapsing. Bitcoin UP.

[molecular]

The way I've envisioned it is the UTXO hash becomes a part every block's structure (just as the coinbase transaction) and it serves as a marker for the UTXO set as of the current block. This enables nodes to optionally ask for: a) all prior blocks, or b) the UTXO set only.

If there is a re-org, then the new chain's blocks simply have their own UTXO hash in each block and provide a valid reference for the UTXO set on that branch. Each separate chain would have their own UTXO hash that is valid at every block. The cost is adding 256 bits to each block, but this doesn't even have to be a part of the header structure and instead is part of the block itself.

Until yesterday I thought UTXO commitments were a good idea. After the discussion here I oppose it for 2 reasons:

• It gives additional capabilities to a 51% attacker
• Since the UTXO set is actually an implementation detail, making it part of the bitcoin protocol seems inelegant and uneccessarily cluttering

[1498449340]

1498449340

[1498449340]

1498449340

[1498449340]

1498449340

[1498449340]

1498449340

[1498449340]

1498449340

[cypherdoc]

Open questions;
i) UTXO hash in blockchain is vulnerable to re-orgs, so it needs to be >120 or 288 blocks deep or w/e your 'trust' level is ...

ii) Also some UTXO could discard/prune long unspent dust/spam or other 'bad' TX so not all UTXO sets will be created equal ...

The way I've envisioned it is the UTXO hash becomes a part every block's structure (just as the coinbase transaction) and it serves as a marker for the UTXO set as of the current block. This enables nodes to optionally ask for: a) all prior blocks, or b) the UTXO set only.

If there is a re-org, then the new chain's blocks simply have their own UTXO hash in each block and provide a valid reference for the UTXO set on that branch. Each separate chain would have their own UTXO hash that is valid at every block. The cost is adding 256 bits to each block, but this doesn't even have to be a part of the header structure and instead is part of the block itself.

During a re-org a node could ask for either: a) all new blocks since the branch or b) the UTXO set of the latest block on the new chain. Both would be valid options to function on the new branch.

What I like about this is it allows each node to make optimal decisions that alleviate bandwidth pressure on the network. Every time a node restarts after x number of days, it could quickly determine which is the fastest method to catch-up, a) download the newest block or b) download the UTXO set, whichever is smaller. This would allow nodes to minimize their download requests.

Wouldn't the UTXO hash have to be in the header since otherwise it could get pruned?

[cypherdoc]

looks to me like the $DJI is starting to get dragged down by the $DJT:



also, RUT has broken down below the support line of the ascending wedge, depending on how you draw it:

[cypherdoc]

The way I've envisioned it is the UTXO hash becomes a part every block's structure (just as the coinbase transaction) and it serves as a marker for the UTXO set as of the current block. This enables nodes to optionally ask for: a) all prior blocks, or b) the UTXO set only.

If there is a re-org, then the new chain's blocks simply have their own UTXO hash in each block and provide a valid reference for the UTXO set on that branch. Each separate chain would have their own UTXO hash that is valid at every block. The cost is adding 256 bits to each block, but this doesn't even have to be a part of the header structure and instead is part of the block itself.

i have a few questions:

Until yesterday I thought UTXO commitments were a good idea. After the discussion here I oppose it for 2 reasons:

when you say "commitment", you mean by requiring a UTXO hash to be embedded into a block?

• It gives additional capabilities to a 51% attacker

you mean that because the UTXO hash is embedded (committed) into blocks, it leaves room to be manipulated?  how so?

• Since the UTXO set is actually an implementation detail, making it part of the bitcoin protocol seems inelegant and uneccessarily cluttering

by implementation detail, do you mean that the UTXO set is maintained at the client level and is never actually used in the protocol?

[molecular]

i have a few questions:

Until yesterday I thought UTXO commitments were a good idea. After the discussion here I oppose it for 2 reasons:

when you say "commitment", you mean by requiring a UTXO hash to be embedded into a block?

yes. I hope I'm using the term correctly.

• It gives additional capabilities to a 51% attacker

you mean that because the UTXO hash is embedded (committed) into blocks, it leaves room to be manipulated?  how so?

thezerg explained an approach and I think the implications are valid here (bolding mine):

If the above is useful, next we create a soft-fork new op code that identifies this as a hash of the UTXO and miners don't accept block with an invalid UTXO.  Miners must put this UTXO hash in every (block number%31 ==0).  Clients don't use the most recent UTXO hash but the prior one.

So now we can have a very small client that is also trustless (well, a single miner would have to find 32 blocks in a row or 51% the network to fool a client).  Small clients could use even older UTXO sets if they are less trusting of the diversity of the miner network.  

The only problem with the above is that today 51%ing the network only lets you get all the newly minted coins and deny the ability of others to spend.  This is a pretty cool feature

But with the above a 51% miner could "mint" new coins (at least from the perspective of the small clients) and remove existing ones by creating a fake UTXO set.  You'd have to remove existing UTXOs or the # of coins would not match the coinbase issue rate.  But if a 51%er ever did this, the non-small clients would see it instantly and so there would presumably be a huge PSA.  So this idea is acceptable within the "weak" validation model...

I'm really no sure about this point after re-reading what thezerg said. If it only applies to SPV clients, then it doesn't make much difference because these have to trust the miners anyways. I might be slightly confused.

• Since the UTXO set is actually an implementation detail, making it part of the bitcoin protocol seems inelegant and uneccessarily cluttering

by implementation detail, do you mean that the UTXO set is maintained at the client level and is never actually used in the protocol?

yes, that. That's not a very strong reason, though.

[justusranvier]

I think the security of committed UTXO sets might depend on how they are actually implemented.

Maybe it's better to think of a set of transactions which create unspent outputs.

Each transaction in that set can be accompanied by the hashes needed to prove that it was included in the merkle tree of a particular block.

Each bitcoin block that contains a committed set becomes independently verifiable - almost.

If one accepts that they know the hash of the previous block represents a valid block, then any observer can verify that the next block is a valid transformation of the set of transactions which create unspent outputs without knowing any more history than what is contained in the block itself (and its committed set).

In this model, an attacker can not add false transaction to the set without being detected by any client (light or otherwise) which has managed to obtain a correct set of block headers, otherwise the hashes won't match (unless the attacker can also trivially break SHA256).

Now now we're right back to "if an SPV client can manage to connect to a single peer that will tell it the truth, it can't be fooled".

Because each block + committed set is a self-contained proof, even if an attacker with a majority hashpower produces a chain with longer proof of work any honest node can broadcast the proof that one of the links in the attacker's chain is invalid and therefore the entire branch is invalid.

[thezerg]

i have a few questions:

Until yesterday I thought UTXO commitments were a good idea. After the discussion here I oppose it for 2 reasons:

when you say "commitment", you mean by requiring a UTXO hash to be embedded into a block?

yes. I hope I'm using the term correctly.

• It gives additional capabilities to a 51% attacker

you mean that because the UTXO hash is embedded (committed) into blocks, it leaves room to be manipulated?  how so?

thezerg explained an approach and I think the implications are valid here (bolding mine):

If the above is useful, next we create a soft-fork new op code that identifies this as a hash of the UTXO and miners don't accept block with an invalid UTXO.  Miners must put this UTXO hash in every (block number%31 ==0).  Clients don't use the most recent UTXO hash but the prior one.

So now we can have a very small client that is also trustless (well, a single miner would have to find 32 blocks in a row or 51% the network to fool a client).  Small clients could use even older UTXO sets if they are less trusting of the diversity of the miner network.  

The only problem with the above is that today 51%ing the network only lets you get all the newly minted coins and deny the ability of others to spend.  This is a pretty cool feature

But with the above a 51% miner could "mint" new coins (at least from the perspective of the small clients) and remove existing ones by creating a fake UTXO set.  You'd have to remove existing UTXOs or the # of coins would not match the coinbase issue rate.  But if a 51%er ever did this, the non-small clients would see it instantly and so there would presumably be a huge PSA.  So this idea is acceptable within the "weak" validation model...

I'm really no sure about this point after re-reading what thezerg said. If it only applies to SPV clients, then it doesn't make much difference because these have to trust the miners anyways. I might be slightly confused.

Yes, it only applies to thin clients.  A full client calculates the UTXO set and Merkle hash from the complete history, compares it to a new block's UTXO hash and rejects the block if they don't match.

A 51% attack could create a longer chain of malicious UTXO sets than the longest valid chain.  But the full nodes reject the chain because it is not correct (incorrect UTXO hash).  However "light" clients (I'm not calling them SPV because SPV refers to a specific algorithm), can't validate the UTXO hash so they assume that it is correct and therefore use the longest (and incorrect) chain. 

Honestly in practice you would simply code up "light" client that refuses to do any TXNs if there are 2 competing chains that forked > 64 (for example) blocks ago.  Bitcoin's natural forks resolve quickly...

Note that it doesn't have to be all or none either... a client could keep X years of back revision history and just assume if the coin is older than that then it is valid.  This might be very important when Bitcoin is 20+ years old.

• Since the UTXO set is actually an implementation detail, making it part of the bitcoin protocol seems inelegant and uneccessarily cluttering

by implementation detail, do you mean that the UTXO set is maintained at the client level and is never actually used in the protocol?

yes, that. That's not a very strong reason, though.

Once upon a time revision control systems stored each revision as the difference between the prior rev and the next one.  This is bitcoin today.  As files changed hundreds of times it took a long time to "check out" the file because all the revs had to be read and applied.  Then people realized that there is a better way: store the current file and the reverse difference -- that is the changes needed to get BACK to the prior rev.  This made a lot more sense because the current file is what is used 99% of the time.

This idea to store the UTXO Merkle tree hash is the evolution of the network to the back-revision method -- we only need the "current state" because blocks are symmetric.  So technically you are correct; this is redundant data.  But what it actually is a smooth transformation from one structure/protocol to another.

[cypherdoc]

huh.  we got the $DXY and the TLT turning down on top of the Dow Theory non-confirmation we still have in place.  this, plus the break of the RUT below it's support of it's ascending wedge all equals "heads up".  or, i could just be needlessly concerned:

[cypherdoc]

Security researchers are reporting a trend for cybercriminals to go directly after financial institutions instead of their customers. In February, researchers from Kaspersky Lab reported that a gang called Carbanak stole up to $1 billion from banks and other financial institutions in 25 countries after infecting their systems with malware and carefully learning their internal procedures. The primary attack vector used was spear phishing, targeted emails containing malicious attachments.

http://www.pcworld.com/article/2915112/police-breaks-up-cybergang-that-stole-over-15-million-from-banks.html

[cypherdoc]

Yes, it only applies to thin clients.  A full client calculates the UTXO set and Merkle hash from the complete history, compares it to a new block's UTXO hash and rejects the block if they don't match.

how is this practical?  don't UTXO sets differ slightly just like the mempools unconfirmed tx's?

[Kupsi]

Yes, it only applies to thin clients.  A full client calculates the UTXO set and Merkle hash from the complete history, compares it to a new block's UTXO hash and rejects the block if they don't match.

how is this practical?  don't UTXO sets differ slightly just like the mempools unconfirmed tx's?

The UTXO set merkle hash is created out of confirmed transactions.

[justusranvier]

don't UTXO sets differ slightly just like the mempools unconfirmed tx's?

No. The state of the Bitcoin ledger as of each Bitcoin block corresponds to exactly one correct UTXO set.

[cypherdoc]

don't UTXO sets differ slightly just like the mempools unconfirmed tx's?

No. The state of the Bitcoin ledger as of each Bitcoin block corresponds to exactly one correct UTXO set.

then it seems that there is no need to hash the UTXO set into the current block.  we aren't doing that now and it works.

[justusranvier]

then it seems that there is no need to hash the UTXO set into the current block.  we aren't doing that now and it works.

If you put the hash of the UTXO set in the block header, then light clients can obtain a copy of the set from an untrusted source and use the block header to verify that it is correct.

[thezerg]

don't UTXO sets differ slightly just like the mempools unconfirmed tx's?

No. The state of the Bitcoin ledger as of each Bitcoin block corresponds to exactly one correct UTXO set.

then it seems that there is no need to hash the UTXO set into the current block.  we aren't doing that now and it works.

Yes, like I was saying above, it is an optimization.  It transforms the block chain from one that must be processed in one direction (beginning to end) into one that can be read in either direction.  So full clients can still read from beginning to end, but "light" clients can read from end to beginning and essentially make its own tradeoff between confidence level and data storage overhead.   A light client could even read the entire blockchain end to beginning and by doing so gradually become a full client...

What I like most about it is that we could do it today in independent transactions as a proof of concept (but you'd have to trust "us" to not lie about the hash).  However "us" could be multiple independent validators... it could later be turned into a required element in every block to make the trust model similar to what we have with miners today (i.e. not very much trust)...

[rocks]

Right, and now we're back to one of Smooth's points that a network full of nodes that don't personally verify every transaction in each block back to the genesis block (e.g., by relying on UTXO commitments instead) doesn't offer the same level of trustlessness than one that does.

Actually, I wasn't agreeing with his whole concept of just depending on headers and a hash of the UTXO set for a full node. I still think it's important for every new full node to download the entire block chain initially to build a valid UTXO set before discarding  any blocks. But I do like the idea of adding a  hash of the UTXO set into each block to ensure it's integrity.

To expand on this, adding a UTXO hash set simply provides nodes a new additional method to engage with the P2P network. The full history still exists and is independently verifiable to anyone who wishes to do so.

With UTXO hashes, a node can optionally decide what level of trust it requires, and can optionally trust that the P2P network has correctly verified the UTXO set up to a given point to save bandwidth, or not if it wants the full history.

We'd end up with 3 types of nodes (vs. 2 types of nodes today):

1) SPV clients - Thin & lightweight nodes that do not contribute to the network. These need to trust their P2P peers to some degree.

2) Full nodes without full history (i.e. UTXO nodes) - Fully functional nodes that are able to fully participate in the P2P network and validate transactions and new blocks. These need to trust that miners and the P2P network correctly verified their starting UTXO point.

3) Archival nodes with full history (i.e. today's nodes) - Fully functional nodes that are able to fully participate in the P2P network and validate transactions and new blocks. They also valid the fully history of a given chain.

Type 3 nodes (I'm calling archival nodes here) would still exist and verify for everyone that a given chain is valid. Given how public this is, I can't imagine and attack were either the P2P network or the full archival nodes wouldn't be able to flag the attack and kick off the invalid chain. What UTXO hashes provide is the option for nodes to save bandwidth by trusting the longest chain, I'd imagine this trade off is OK for some subset of nodes.

[marcus_of_augustus]

The negative interest rate hall of mirrors is screwing with people's minds enough to wake them up from the Central Bank manipulated price-perception fantasy dream. The monetary CONfidence game may be coming to an end ... feels to me like the Rubicon has been crossed.
http://www.telegraph.co.uk/finance/comment/jeremy-warner/11569329/Jeremy-Warner-Negative-interest-rates-put-world-on-course-for-biggest-mass-default-in-history.html

[rocks]

Until yesterday I thought UTXO commitments were a good idea. After the discussion here I oppose it for 2 reasons:

• It gives additional capabilities to a 51% attacker

I don't think this is true. UTXO hashes would be verified the same as transactions.

For example, today if a 51% attacker tried to insert invalid transactions into a block to create coins, or double spend past transactions, the P2P network would reject such blocks. This is why a 51% attacker can really only attack by selecting specific transactions or refusing to confirm specific transactions.

UTXO hashes would be the same. If a 51% attacker tried to create an invalid UTXO hash, that block would be rejected by the network. It is no different than a 51% attacker inserting an invalid transaction into a block. Both are rejected and a 51% attacker can't do this.

• Since the UTXO set is actually an implementation detail, making it part of the bitcoin protocol seems inelegant and uneccessarily cluttering

I also don't think this is true. As of each block, the bitcoin protocol has a specific and known UTXO set. This isn't an implementation detail but an inherent property of the bitcoin protocol. Future transactions are limited to those in the UTXO set, it is part of the protocol.

[rocks]

Wouldn't the UTXO hash have to be in the header since otherwise it could get pruned?

I don't see why it would have to be. The header is really only for things a node has to have for each block, which is the linkage to the previous block and linkage to it's data package.

You could put it in the block's tree structure. Then if a node needs it, the node could request and verify it. But if the node does not need it (i.e. an SPV client) the node could skip it (along with that block's other info such as the coinbase and transactions).

[hdbuck]

The negative interest rate hall of mirrors is screwing with people's minds enough to wake them up from the Central Bank manipulated price-perception fantasy dream. The monetary CONfidence game may be coming to an end ... feels to me like the Rubicon has been crossed.
http://www.telegraph.co.uk/finance/comment/jeremy-warner/11569329/Jeremy-Warner-Negative-interest-rates-put-world-on-course-for-biggest-mass-default-in-history.html

nothing new, it seems at this point the reality could simply go beyond the fiction. at least for quite a while. just look at varoufakis exiting negociations by the small door.