Utreexo demonstration release

[Ratimov]

There was a release of the first demonstration of Utreexo recently. Utreexo is a new scalability technology for Bitcoin, which can make Bitcoin nodes smaller and faster while keeping the same security and privacy as full nodes.

Lightning Network co-author Thaddeus Dryja has been working on a lightweight full Bitcoin node project since early last year. In the summer of 2019, he posted the code for the Utreexo scaling solution, and last week the development team released the first version of the Utreexo software as part of the MIT Digital Currency Initiative (DCI).

Full nodes verify that new transactions on the Bitcoin blockchain meet protocol requirements and distribute them across the network. However, running a full node requires hundreds of gigabytes of storage. The truncated version of the full node, available in Bitcoin Core, can reduce the minimum volume of stored history and transaction outputs (UTXO) to several gigabytes, but even such an installation requires a complete "pumping" of the blockchain. Therefore, developers have long sought to simplify deployment and make full site capabilities available to more users.

Utreexo determines the size of the "state" of a full node, which shows actual information about who owns how much BTC. Utreexo cuts the minimum size of this state from about four gigabytes to less than a kilobyte.

The current Utreexo code is able to connect to a server, download the blockchain, and verify all the transactions and signatures, and do so using hardly any disk space. Instead of a multi-gigabyte database, utreexo stores a single file of a few hundred bytes which represents the state of the blockchain. The client still needs to download the full sized blockchain (in fact, even more downloading is required) but not store it.

There is also a server component which builds the data to send the client. This takes up considerable storage space. In the current software, each client connects directly to a server to download data, but our plan is to change this into a peer-to-peer model where large numbers of nodes, some with data to serve and some without, automatically connect to each other.

The current software is a demonstration of the technology and a proof-of-concept that a wallet can work (it allows users to specify an address to import, and it gives a watch-only wallet balance) but so far does not have a true wallet and cannot make transactions to send or receive money. The software also operates on testnet, the Bitcoin testing network, and is not recommended for use with real money. There are still plenty of known bugs and inefficiencies in the code, but we’re improving it at a rapid pace.


Further development

Going forward, our plan is to modify btcd, bitcoin node software written in go, to use Utreexo instead of its current database. In doing this we can keep most of the software already written, and see how to fit it into a full node that is connected to a mix of clients, some which are using Utreexo and some which aren’t. Getting Utreexo to work with btcd may take a while, but would be exciting as it would fully show the technology operating a real peer to peer full node.

After btcd, we hope to write a specification and to try to get Utreexo capabilities into Bitcoin Core node software. While Utreexo is not a consensus change and doesn’t require a soft fork, it is a significant re-thinking of how Bitcoin works, changing consensus-critical code. It is thus likely to be difficult to get Utreexo code into Bitcoin Core, and with good reason — we want to be very sure to not introduce problems into a system handling so many people’s money. For this reason we’ve decided to try to get Utreexo capability into btcd’s code first, and in the process learn more about how it affects full node operation.

This is a first release, and there will be more coming soon. One interesting capability I hope to have working soon is the ability to verify the blockchain “backwards”, starting from the most recent and progressing back to the genesis block. While this would be extremely difficult to do with normal database-centered Bitcoin software, it’s not such a big change when using Utreexo. I look forward to writing about that new verification setup shortly. In addition, there are many performance improvements in the works.

A big reason for this release is to get people excited about this new technology — especially other open source developers.

Links:

- Utreexo: A dynamic hash-based accumulator
optimized for the Bitcoin UTXO set
- ELI5: Utreexo — A scaling solution
- btcsuite/btcd
- mit-dci/utreexo


Sources:
- https://bits.media/soavtor-lightning-network-opublikoval-pervuyu-versiyu-tekhnologii-utreexo-dlya-zameny-legkikh-klient/
- https://medium.com/mit-media-lab-digital-currency-initiative/utreexo-demonstration-release-a0d87506fd70

[1680122316]

1680122316

[1680122316]

1680122316

[1680122316]

1680122316

[1680122316]

1680122316

[DarkDays]

How does Utreexo compare with the Coda project (https://codaprotocol.com, white paper at https://eprint.iacr.org/2020/352.pdf)?

Is it a similar idea?

I tried to find discussion of this elsewhere, but didn't come up with anything.

I think the biggest attraction for Coda is its lightweight attribute. Supposedly Coda's staking nodes are round about 1953025% lighter than BTC nodes. (1gb compared to 2tb+) - the plus side for Coda being that it is built from the ground up.

For Utreexo, from what I understand, is trying to fit one way of writing a code into another. Its basically trying to create a Merkle forest where a set of characters in a hash can be substituted for something else, and be done so in a way that can be proved faster than it can be hashed. Essentially its a very complicated game of substituting for x in algebra.

Both are completely different and I need to personally look more into Utreexo to understand it further.

[dkbit98]

Utreexo has been getting more attention recently after developer Calvin Kim received additional $150,000 from Bitmex exchange and Human Rights Foundation to improve Bitcoin scaling
with his algorithm that could reduce Bitcoin state size from gigabytes to under a kilobyte.
This will help running full Bitcoin node much easier and faster than it is currently, and I also see the use case of Utreexo for exchanges.

[Shymaa-Arafat]

$150,000 from Bitmex exchange and Human Rights Foundation to improve Bitcoin scaling

Bitmex exchange is understandable, but what does Human Rights foundations have to do with Bitcoin, or Ecosystems in general?

[Pmalek]

Bitmex exchange is understandable, but what does Human Rights foundations have to do with Bitcoin, or Ecosystems in general?

I think this quote from their website will help you to understand their reasons for wanting to improve bitcoin:

In 2020, the Human Rights Foundation launched a fund to support software developers who are making the Bitcoin network more private, decentralized, and resilient so that it can better serve as a financial tool for human rights activists, civil society organizations, and journalists around the world.

Source: https://hrf.org/bitcoin-development-fund

A more private and scalable Bitcoin blockchain will help their own people and journalists to transfer money without giving the government and the elites they oppose an insight and a way to control who deals with whom and why. Check out the short video in the middle of the article in source link.

[NotATether]

A more private and scalable Bitcoin blockchain will help their own people and journalists to transfer money without giving the government and the elites they oppose an insight and a way to control who deals with whom and why. Check out the short video in the middle of the article in source link.

It's not just development for the bitcoin protocol, apparently they're also giving grants to people doing supporting work such as privacy-focused newsletter editors and translators.

I just hope that they have legal terms for grantees so that the money is properly utilized and embezzlement is prevented. It's a lot of money they're throwing out there loosely.

[franky1]

seems this is not a full node(validate all and archive all) but just a UTXO and block verifier wallet

by this i mean. its obviously not going to be a full node as it wont be a relay of blocks nor a seed of historic blockhain data.

a full node is FULL because its
full tx relayer/mempool store
full block propagator
full block verifier
full blockchain archiver

i think topic creator should try to not over promise things and mis-character their lite wallets as 'good as full node' and instead actually advertise their product as a light wallet that has extra checks

come on admit the common sense that debunks the promotion of
"a new full node in a few kilobytes"
there is so many wrongs and so many embellishments

..
edit to answer below
i understand all that. my point is these people wanting to develop new features for bitcoin wallets. end up promoting their wallets as 'nodes as good as a full node' and then they start suggesting things like putting their wallet feature into things like core to replace how a full node works..
this kind of thing would weaken the node topology of the network by having more node users downgrade to wallet users
.. in short a wallet that doesnt store the blockchain or act as a seed for the blockchain for others is not a 'node' but a wallet. yes this wallet offers more features than litewallets. but its still part of the bottom row of the network where litewallets lay where they are leachers of the data network and not seeders

EG running in prunned mode means that that 'node' is no longer a blockchain seed and just a leacher
same with this topics wallet. if more people prune/utreexo than run full nodes. then the network has less blockchain seeds. this especially happens when less tech savvi people THINK they are still doing their part for the network by 'running a node thats as good as a fullnode' when reality is they are running a lite wallet with extra checks

all im saying is if its part of any 'full node' make it absolutely clear that enabling the litewallet stuff its disabling the full node stuff. and not try to advertise/sell it as a feature that is as good/equal to full node capability

[Shymaa-Arafat]

i think topic creator should try to not over promise things and mis-character their lite wallets as 'good as full node' and instead actually advertise their product as a light wallet that has extra checks

.
Utreexo is about stateless nodes, not full nodes not lite nodes
The concept of stateless nodes started to exist in Bitcoin & Ethereum too as a suitable compromise...
In Bitcoin the stateless node keeps just an accumulator of all UTXOs Merkle Tree(the root of the tree), and contact a bridge server to get the witness (proof) of each UTXO they need to verify( the siblings to the UTXO path to the root so that they can extract it's hash from the root value and verify its correctness)
https://mobile.twitter.com/ArafatShymaa/status/1398300471505436675
-Similarly inform the bridge of new UTXOs with their hash to be inserted in the tree
-ie all in all, only the bridge server stores everything and they contact it,
https://mobile.twitter.com/ArafatShymaa/status/1402161644902436865
 most of the research is about reducing the amount of contact data either by
* making the tree as balanced as possible (less no of siblings to extract from the root value),
*or by trying to make the seemed proofs have a lot of common nodes(for example, we r super lucky if they're all in one subtree so we need only one or two sibling paths to the root, the rest is part of the data itself)
*and by caching along with the above two( from the bridge memory to ur node memory)
.
If I'm not explaining it we'll, u CA still watch the first 6 mins from the video if the paper "Merkle Trees Optimized for Stateless Clients in Bitcoin" for a brief intact explaination
https://t.co/rEUWQiqM7Y?amp=1

»»»» Update
It seems they r really planning to get it into full nodes as a future work maybe with this grant

https://github.com/btcsuite/btcd
.
.
.
After btcd, we hope to write a specification and to try to get Utreexo capabilities into Bitcoin Core node software. While Utreexo is not a consensus change and doesn’t require a soft fork, it is a significant re-thinking of how Bitcoin works, changing consensus-critical code. It is thus likely to be difficult to get Utreexo code into Bitcoin Core, and with good reason — we want to be very sure to not introduce problems into a system handling so many people’s money. For this reason we’ve decided to try to get Utreexo capability into btcd’s code first, and in the process learn more about how it affects full node operation.

[oryhp]

Utreexo is about stateless nodes, not full nodes not lite nodes

AFAIK, the whole point of Utreexo is to have a fully validating node that requires very little storage, but comes with slightly higher throughput cost. Parallel initial block download and other things are just good side effects.

[NotATether]

Utreexo is about stateless nodes, not full nodes not lite nodes

AFAIK, the whole point of Utreexo is to have a fully validating node that requires very little storage, but comes with slightly higher throughput cost. Parallel initial block download and other things are just good side effects.

It's great that Utreexo's going to give everyone a smaller block state, but has anyone stopped and thought that if everyone's storing the compact block state, then who's going to store the full-sized block state? I don't think this question has been asked before. Since the Utreexo needs a server, coding this client, space-saving functionality into Bitcoin core the server will require it to talk to another server that has the actual blocks.

Although we already have pruned nodes, I wonder if this will result in a new "super-pruned" node setting being implemented that merely uses utreexo Merkle forests and other stuff.

It's not the silver bullet for increasing transaction speeds (a la block size increase - the full nodes won't be able to keep up), because just as a network where everyone's running pruned nodes can't function, neither can a network running only utreexo clients. The utreexo server would be hardcoded in the Bitcoin Core client in the full-node configuration.