Can a Bitcoin node build a partial UTXO set from recent N blocks?

[cao123456]

I am investigating whether a storage-constrained Bitcoin node could retain only the most recent (N) blocks, construct a partial UTXO set from those blocks, and use that partial UTXO set to independently validate newly received transactions.

The basic idea would be:

recent (N) blocks → construct a partial UTXO set → validate a new transaction locally if all of its input UTXOs are available in that partial set.

However, I see a potential bootstrap problem.

Suppose the node starts with only the most recent (N) blocks. Some transactions in the earliest retained blocks may spend outputs created before the retained window. Without the earlier UTXO state, the node cannot fully validate those inputs.

If the node therefore cannot fully validate those blocks, can it safely treat the outputs created by those blocks as valid UTXOs and progressively construct a partial UTXO set from them?

More generally:

Is it technically feasible for a Bitcoin node to construct a trustworthy partial UTXO set using only the most recent (N) blocks, without first obtaining a validated UTXO state at the beginning of the retained window?

If all inputs of a newly received transaction are present in such a partial UTXO set, would that be sufficient for the node to independently validate that transaction, or would the incomplete validation history of the partial UTXO set prevent this?

How is this different from the way a Bitcoin wallet maintains information about its own UTXOs? A wallet does not maintain the complete global UTXO set, so what provides the trust basis for the wallet's view of its own unspent outputs?

If this approach is technically possible, why is recent-block-based partial UTXO construction not commonly used by lightweight Bitcoin nodes today? What are the main technical or security limitations?

I am particularly interested in the trust/validation issue rather than merely the storage savings.

[LoyceV]

Is it technically feasible for a Bitcoin node to construct a trustworthy partial UTXO set using only the most recent (N) blocks, without first obtaining a validated UTXO state at the beginning of the retained window?

No.

If all inputs of a newly received transaction are present in such a partial UTXO set, would that be sufficient for the node to independently validate that transaction

No. Someone could create Bitcoin out of thin air and you would just think the UTXO wasn't in your partial UTXO set.

[TheButterZone]

This exact dilemma is why Floresta and Mandacaru exist. They solve the exact problem you are trying to solve (running a node on a storage-constrained device) but without compromising the validation history.
Instead of guessing or truncating the UTXO set based on a time window, Floresta uses Utreexo, a dynamic hash-based accumulator.

* The global UTXO set (millions of entries, tens of gigabytes) is compressed into a compact set of roots (under 1 Kilobyte).
* When a Floresta node receives a new transaction, the sending peer attaches a cryptographic proof showing that the UTXOs being spent exist inside those tiny roots.
* This allows a mobile phone running Mandacaru to independently validate transactions from Genesis onward without actually holding the gigabytes of data that a traditional node requires.

Furthermore, projects like Mandacaru utilize assumeutreexo (similar to Core's assumevalid). It allows the node to bootstrap instantly by hardcoding a cryptographically signed snapshot of the Utreexo accumulator roots at a specific recent block height, bypassing the initial history securely while maintaining full validation going forward.

Mandacaru discussion here: https://bitcointalk.org/index.php?topic=5579479.0

[nc50lc]

How is this different from the way a Bitcoin wallet maintains information about its own UTXOs? A wallet does not maintain the complete global UTXO set, so what provides the trust basis for the wallet's view of its own unspent outputs?

No UTXO set is kept locally by those lightweight wallets because they can't verify alone without connecting to a full node.

For comparison;
Compared to SPV wallets?
It's totally different because those still relies on full nodes to verify while only keeping the block headers to see if those nodes are providing it the correct data.

Or compared to the new clients that utilizes "Compact Block Filters"?
BIP157 Specs should explain it better: github.com/bitcoin/bips/blob/master/bip-0157.mediawiki#specification

[ABCbits]

However, I see a potential bootstrap problem.

Suppose the node starts with only the most recent (N) blocks. Some transactions in the earliest retained blocks may spend outputs created before the retained window. Without the earlier UTXO state, the node cannot fully validate those inputs.

There's bigger problem. If new/fresh node literally only download most recent N blocks, how can it verify those block actually part of longest chain and input of each TX isn't created from UTXO that never exist or already spend?

[Cricktor]

I think it could be possible with UTXO set snapshots, but this is not something that you can obtain trustlessly, at least I've no idea how.  And the node would've to rebuild the full UTXO set starting with the snapshot and extending it from there on to reconstruct the full UTXO set which in my opinion is mandatory to trustlessly verify if a coin is spendable or not. As others already said, you can't do this with only a partial UTXO set.

Only thing that pops up in my head is: those could be published e.g. by Core devs and independently verified by others, secured with hash checksums and digital signature which everyone could verify. Something for much later when a complete IBD would take really really long time even with good hardware and resources.

I also think that part of the RPC infrastructure is already somewhat in place with RPCs dumptxoutset and loadtxoutset. I don't know if those RPCs are completely usable. Never used them, so far.

[NotATether]

I also think that part of the RPC infrastructure is already somewhat in place with RPCs dumptxoutset and loadtxoutset. I don't know if those RPCs are completely usable. Never used them, so far.

Bitcoin Core would need to have a specific RPC for dumping the entire historical outset to a file, or just building it asynchronously in a file and then have the RPC copy it somewhere.

Then there would need to be some kind of signature in the output to prove the outset authenticity, along with metadata, followed by another RPC command to verify such a signature's authenticity independently for the tx outset to be trusted.

Also, each outset would have to include its associated address and public key and spending script.

This could be an interesting idea for a BIP. I might even write one... 

[LoyceV]

This could be an interesting idea for a BIP. I might even write one...

Isn't that what Boost Your Bitcoin Node Sync With UTXO Snapshots is for?

[NotATether]

Isn't that what Boost Your Bitcoin Node Sync With UTXO Snapshots is for?

Possibly, but the tx outsets will be generated directly by Core, and it will no longer have to trust user-generated input. We all know the risks of downloading the blockchain from random sources. That's why I proposed adding a simple signature to the tx outset that confirms it had not been tampered with.

The private key can simply be a sha256 of each of the block hashes concatenated together, up to the latest chain tip.

[d5000]

If the node therefore cannot fully validate those blocks, can it safely treat the outputs created by those blocks as valid UTXOs and progressively construct a partial UTXO set from them?

You would have to trust the block where you started the download (let's call it block X) and the current block where the download ended (let's call it block Y), i.e. the chain tip you downloaded. Trusting block Y of course implies trusting block X too and all intermediate blocks too.

In practical terms this could be possible, say you lookup the blockhash of block X and Y manually in several block explorers, on two consecutive days, and then mark the block as trustworthy and only then start using the client -> the probability of being gamed would be negligible. This would be afaik more safe than the assumeUTXO method Core uses for background validation, where you use only one source for an UTXO snapshot, and this source could be not only malicious, but also have been hacked or manipulated in some way. You could alternatively use a method like assumeUTXO and compare the snapshots from different sources.

(Of course, assumeUTXO is different from your idea: the whole blockchain is validated, but in the background while the node is already "somewhat" usable, so any manipulation would be detected quite fastly.)

But without this trust assumption, your idea is impossible.

Is it technically feasible for a Bitcoin node to construct a trustworthy partial UTXO set using only the most recent (N) blocks, without first obtaining a validated UTXO state at the beginning of the retained window?

LoyceV is correct from my point of view if you don't trust any block, but if you trust the blocks X and Y, you could construct such a partial UTXO set. The problem however would be that this set would be very small if N isn't a large part of the blockchain.

You could only integrate UTXOs into this set which are present completely in one of the N blocks you downloaded, i.e. block X and after. And for the blocks after Y, you cannot assume the transactions with UTXOs which are not in the blocks N are valid. You can of course "legalize" a new stretch of the chain after some time if you check the blockhashes again.

The advantage over SPV clients would be mainly privacy, not so much security. And an approach close to the Monero client "Feather" -- a light client where you download all recent blocks where transactions of your wallet could be located -- would yield the same advantages. (Edit: Ginger/Wasabi afaik do not achieve such a high level of privacy because the filtered blocks could still be processed by chain analysis.)

If all inputs of a newly received transaction are present in such a partial UTXO set, would that be sufficient for the node to independently validate that transaction, or would the incomplete validation history of the partial UTXO set prevent this?

Again only if you trust the blocks X and Y.

If this approach is technically possible, why is recent-block-based partial UTXO construction not commonly used by lightweight Bitcoin nodes today? What are the main technical or security limitations?

Apart from the trust assumption, the small partial UTXO set would limit the usability of that node severely OR require a high number of blocks (e.g. a third of the blockchain) to be validated, and as I wrote before, it would only be safe if you check the block X / Y at least twice in time, so your node would not immediately be fully usable. assumeUTXO looks like a way better alternative.

[Mia Chloe]

~snip

Well I think the trust assumption is the key limitation here. Once you trust X and Y the node can build a partial UTXO set from the retained range but you're kinda effectively shifting the security boundary rather than removing it. I think the bigger issue is that most UTXOs being spent in those recent blocks were created before X so without a validated state at X you kinda cannot independently establish their existence or validity.

So technically feasible yes but it doesn't really give you the security properties of a fully validating node. Assumed UTXO is more practical because the snapshot is only kinda like a shortcut to initial usability while full validation still works toward the same independently verified state.