Blockchain Keyframing

[No_2]

I had an idea a while back and can’t see any major flaws with it from a technical perspective it so thought I’d get feedback here.

In an attempt to reduce blockchain size would it be possible to introduce ‘keyframe’ blocks to a blockchain which summarise the blockchain’s current valid transactions all in one block?

This block would have different validation rules to normal blocks, but in essence it summarise all the currently valid transactions across the entire blockchain.

So to pick a number – every 500 blocks a keyframe block is generated (about 3.4 days for BTC).

This means nodes only concerned with the current transaction balances need only store at most 500 blocks back to the last keyframe block. Nodes which wish to see the entire transaction history store the entire blockchain.

[azeteki]

By 'currently valid' I'm assuming you mean unspent outputs.

Two problems I can see right now;

Firstly, the UTXO set is rather large. There are roughly ten million unspent transaction outputs. Would you store txid and output only, or the entire transaction? Even the former case could take up hundreds of megabytes.

Secondly, how do you propose that a client verifies the UTXO set it receives is valid? Currently we achieve a UTXO set by cumulatively verifying from the genesis block.

And an issue rather than a problem;

The reference client already stores a full UTXO database. There is no need for this to propagate over the network. After bootstrapping it would be technically possible for bitcoin-core to discard older blocks. It just doesn't do that at present.

Basically, bitcoind is already producing its' own 'keyframe block', to use your terminology. If we didn't care about syncing new users up, or the possibility of database corruption, we could erase everything behind a reasonable reorg threshold today.

[No_2]

By 'currently valid' I'm assuming you mean unspent outputs.

Two problems I can see right now;

Firstly, the UTXO set is rather large. There are roughly ten million unspent transaction outputs. Would you store txid and output only, or the entire transaction? Even the former case could take up hundreds of megabytes.

Yes, unspent outputs, that's the term I should have used. My objective here was to make the blockchain being used smaller for devices such as smart phones etc rather than relying on a trusted node, e.g. many mobile wallet providers give a view of the blockchain to the wallet client from a central sever that stores an entire copy of the blockchain, from what I understand.

In this instance I'm suggesting storing the entire transaction I guess (not thought this one through fully) as if it is in a previous block then trust that that transaction is valid is implicit (based on its depth in the blockchain).

Secondly, how do you propose that a client verifies the UTXO set it receives is valid? Currently we achieve a UTXO set by cumulatively verifying from the genesis block.

What security does a smart phone wallet employ when it gets its view of the blockchain via its central server? This is a questions as I don't fully know the answer.

My thinking was that if you're using a keyframe block then at least you know all the nodes who've propagated it have checked its validity. Perhaps having it as a side chain may work better.

And an issue rather than a problem;

The reference client already stores a full UTXO database. There is no need for this to propagate over the network. After bootstrapping it would be technically possible for bitcoin-core to discard older blocks. It just doesn't do that at present.

This is an issue of convenience versus file size and bandwidth. Perhaps this idea is infeasible or there are better possible solutions but right now for me at least I know the current blockchain size is causing problems on some wallets.

When you say "discarding older blocks" are you talking about just storing the UTXOs in the database?

[gmaxwell]

My objective here was to make the blockchain being used smaller for devices such as smart phones etc rather than relying on a trusted node, e.g. many mobile wallet providers give a view of the blockchain to the wallet client from a central sever that stores an entire copy of the blockchain, from what I understand.

There are hosted wallets, but the true mobile wallets like "android wallet" use SPV mode see section 8 of the bitcoin whitepaper.  SPV wallets do not trust the nodes they query except for some denial of service/privacy issues, and have basically no storage requirements and very low bandwidth (far below your keyframes), and they do not use a centralized server (they use the bitcoin p2p network). SPV nodes security involves additional assumptions about miner honesty.

What you're describing is a pretty radical change in the security model. In Bitcoin we do not trust the data we are given at all, we verify it. The 'keyframes' you'd send there would need to be trusted, if one bogusly claimed to reassign ownership of a bunch of coin— no one relying on that data could tell (at least not mechanically). Assuming they are mined it would basically be the SPV security assumption set, but less efficient for a mobile wallet (since they'd have to transfer a full 'keyframe').

It's perfectly reasonable to relax the security assumptions for some applications, mobile wallets— for example— but thats why we have SPV mode already.

[No_2]

There are hosted wallets, but the true mobile wallets like "android wallet" use SPV mode see section 8 of the bitcoin whitepaper.  SPV wallets do not trust the nodes they query except for some denial of service/privacy issues, and have basically no storage requirements and very low bandwidth (far below your keyframes), and they do not use a centralized server (they use the bitcoin p2p network).

What you're describing is a pretty radical change in the security model. In Bitcoin we do not trust the data we are given at all, we verify it. The 'keyframes' you'd send there would need to be trusted, if one bogusly claimed to reassign ownership of a bunch of coin— no one relying on that data could tell (at least not mechanically).

It's perfectly reasonable to relax the security assumptions for some applications, mobile wallets— for example— but thats why we have SPV mode already.

I didn't realise SPV mode had been implemented, this is much nicer.

While network nodes can verify transactions for themselves, the simplified method can be fooled by an attacker's fabricated transactions for as long as the attacker can continue to overpower the network. One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block, prompting the user's software to download the full block and alerted transactions to confirm the inconsistency.

I'm guessing here that Satoshi is referring to an attack that creates another Merkel tree with the same values that matches the Merkel tree within a given block, substituting in a transaction in place of the 'real' one and then feeding it to the node requesting it via a Sybil attack to make it look like the real version on the network? Have I understood that correctly?

"One strategy to protect against this would be to accept alerts from network nodes when they detect an invalid block" - does the Bitcoin network currently do this?

Yes, the keyframe idea would involve trust so therefore breaks the current security model; I would therefore class it as flawed. Thanks.

[gmaxwell]

There aren't any current alerts for fraud.  The original idea in the paper was a bit immature— the problem is that a DOS attacker could cheaply force SPV nodes into the very expensive operation of fetching the whole blocks.

Subsequently we've come up with some more refined versions of the idea where, with a few additions to the data blocks contain we'd be able to extract compact proofs that a block is invalid, preventing that DOS attack, but they've not been implemented yet.