[BIP][Draft] SPV improvement for light wallet

[zhouqi_cn]

Request for discussion

SPV : Simplified Payment Verification (The original SPV idea was coming from Satoshi Nakamoto's paper)

Code:

BIP:
Title: SPV improvement for light wallet
Author: Zhou Qi, Bither Team
Status: Pre-draft
Type: Standards Track
Created: 

SPV improvement for light wallet.

SPV mode of Bitcoin light wallet
With the growth of Bitcon blockchain data (>30GB), normal users should use light wallet for convenience.

Current implementation of the SPV
SPV mode stores only the most recent block headers. When a user uses the light wallet for the first time, the light wallet will sync with the Bitcoin nodes from a staring point of block header.
That means wallet developers should prepare the starting point header in wallet's installation package. For little data synchronisation, developers have to upgrade the packages periodically even only for the starting point updating.
That is definitely an issue.

How to solve this issue
We suggest that the bitcoind should add a simple service to get a block hash by block number. This improvement is very easy for Bitcoin core protocol, but will help SPV mode light wallet's a lot.

BIP details:
getblockhash

The getblockhash message contains only one uint32_t field that means block height to indicate the needed block.

The Bitcoin node which receives the getblockhash information may send inv message. inv message contains only a block hash with the point height.

The inv message is supported, we only need to support the getblockhash message.

In bitcoind's rpc command, getblockhash has already been implemented, so we think that bitcoind supporting this communication is feasible.

Specific node's communication can be as the following:

• 1. Connecting to the node, and get version message of the node. The version message will including its current maximum block height n to represent.
• 2. Calculate the block height m to represent.

The code logic of light wallet can be like:

Code:

	if n % 2016 < 100
		m = n - (n % 2016) - 2016 
	else 
		m = n - (n % 2016)

• 3. Send getblockhash message
• 4. Receive inv message. So we get the starting point block's hash.
• 5. Send getheaders message. (The getheaders message only need block hash).
• 6. Receive block header information and circulation 5,6 until sync to the latest block.

Implementation of the analysis

Safety
We ensure that the number of the block header taken at least 100. Light client verifies blocks at least 100, while also avoiding the risk of blockchain branching to some extent brought about.


Zhou Qi
Bither Team

[zhouqi_cn]

I will modify this topic later.
Currently, only for discussion.

[grau]

Syncing headers from genesis on is very fast, it can be accomplished in less than 160 round trips of getheaders, delivering around 22 MB data.

You can drastically cut even that down by picking your own checkpoint instead of the genesis, say block 300000.
Hardwiring such checkpoint hash into an SPV client is not for purists, but is safe in practice. You could roll the checkpoint with new releases of the software by 10000s of blocks.

SPV has security limits, such that this would not be significant compared to them.

[zhouqi_cn]

Syncing headers from genesis on is very fast, it can be accomplished in less than 160 round trips of getheaders, delivering around 22 MB data.

You can drastically cut even that down by picking your own checkpoint instead of the genesis, say block 300000.
Hardwiring such checkpoint hash into an SPV client is not for purists, but is safe in practice. You could roll the checkpoint with new releases of the software by 10000s of blocks.

SPV has security limits, such that this would not be significant compared to them.

Thank you for your reply!

In SPV client,  the purpose for getting block headers is to verify the following blocks. We think more than 100 block headers would be safe enough, and it may not improve security much by getting more block headers. In fact most SPV clients don't store that many block headers. The extra block headers can be a waste of time and space.

We propose this BIP to let users of SPV clients download less data and use bitcoin safely enough. Hardwiring such checkpoint hash is not good enough.

Zhou Qi
Bither Team

[Mike Hearn]

Well, I guess most SPV wallets today use bitcoinj at least (though the competition is finally heating up thank goodness!). And that stores 5000 headers.

I'm not convinced that this is an important problem right now. It's totally sufficient to refresh the checkpoints every couple of months, and most wallets tend to push software updates more frequently than that anyway. What you're proposing to do is effectively make the checkpoint be selected by a random p2p peer instead of the wallet developers. It could work, but it'd be less secure, and for not much visible improvement in convenience or decentralisation.

[zhouqi_cn]

Well, I guess most SPV wallets today use bitcoinj at least (though the competition is finally heating up thank goodness!). And that stores 5000 headers.

I'm not convinced that this is an important problem right now. It's totally sufficient to refresh the checkpoints every couple of months, and most wallets tend to push software updates more frequently than that anyway. What you're proposing to do is effectively make the checkpoint be selected by a random p2p peer instead of the wallet developers. It could work, but it'd be less secure, and for not much visible improvement in convenience or decentralisation.

Dear Mike Hearn,

Yes, it is not an important problem right now, but still worth improving:
1. Of course the developers can update the SPV checkpoint file each time when building installation package, but if the developers find that the wallet is stable enough and currently do not need any new features, they still need to upgrade it periodly only for updating checkpoint file. Is that really a good idea?
2. With very little improvement of bitcoind (already supported in RPC, only need to support for peers communication), SPV wallets can easily get checkpoint data from Bitcoin P2P peers, and they can verify the data as they validate other things from peers. That does not mean less secure, and it can be as secure as other data's verification. Also depending on P2P network and validating the data by the peer itself, is more decentralisation than the old solution.

Thanks for your answering, I really appreciate your help.

Zhouqi

Bither Team

[zhouqi_cn]

I put this BIP onto github.com. I upgrade this document's format match the bip0001 requirement.

https://github.com/bither/bips/blob/master/spv.md

[songchenwen]

Well, I guess most SPV wallets today use bitcoinj at least (though the competition is finally heating up thank goodness!). And that stores 5000 headers.

I'm not convinced that this is an important problem right now. It's totally sufficient to refresh the checkpoints every couple of months, and most wallets tend to push software updates more frequently than that anyway. What you're proposing to do is effectively make the checkpoint be selected by a random p2p peer instead of the wallet developers. It could work, but it'd be less secure, and for not much visible improvement in convenience or decentralisation.

As he said this will be easy to implement for it's already in the RPC and to download less data will be helpful to SPV client. I don't see any point in turning this down.

[keeshux]

SPV mode stores only the most recent block headers. When a user uses the light wallet for the first time, the light wallet will sync with the Bitcoin nodes from a staring point of block header.

The getblockhash message contains only one uint32_t field that means block height to indicate the needed block.

bitcoinj and other SPV clients infer the starting block using the fast catch-up timestamp concept, that is typically the timestamp of the oldest key in the wallet or -in other words- the timestamp before which we know no relevant transactions exist. The FCU is totally blockchain agnostic so there should be a way to translate the timestamp to an effective checkpoint block instead. Using a block height instead of a hash is of no help to me, we don't know in advance which height to start at anyway.

[zhouqi_cn]

SPV mode stores only the most recent block headers. When a user uses the light wallet for the first time, the light wallet will sync with the Bitcoin nodes from a staring point of block header.

The getblockhash message contains only one uint32_t field that means block height to indicate the needed block.

bitcoinj and other SPV clients infer the starting block using the fast catch-up timestamp concept, that is typically the timestamp of the oldest key in the wallet or -in other words- the timestamp before which we know no relevant transactions exist. The FCU is totally blockchain agnostic so there should be a way to translate the timestamp to an effective checkpoint block instead. Using a block height instead of a hash is of no help to me, we don't know in advance which height to start at anyway.

In this BIP, I want to improve the way to find the first block header we are going to store in the client. What you are saying is since which block we may want the transactions in it. In another word, I'm talking about which block to start header downloading. While you are talking about which block to start body downloading, which means header downloading should end in the mean time. We are not talking about the same problem here.

[keeshux]

In this BIP, I want to improve the way to find the first block header we are going to store in the client. What you are saying is since which block we may want the transactions in it. In another word, I'm talking about which block to start header downloading. While you are talking about which block to start body downloading, which means header downloading should end in the mean time. We are not talking about the same problem here.

If you read carefully I moved the problem to the FCU simply because it's pointless to set a checkpoint without considering the FCU, unless you intend to support raw monitors that only observe upcoming blocks and transactions.

I don't see any advantage for SPV wallets with some history, whereas a protocol message providing a (timestamp -> checkpoint_hash) mapping would be much more useful for both monitors and wallets. By checkpoint_hash I don't mean "the first block with transactions", rather a checkpoint block the way you meant in your previous posts.

Example: "hey peer, I want to start syncing from 'Jan 01, 2012' (or from now on). I don't have any idea about what the hash of the first block to start sync from could be, can you give it to me?"

[zhouqi_cn]

In this BIP, I want to improve the way to find the first block header we are going to store in the client. What you are saying is since which block we may want the transactions in it. In another word, I'm talking about which block to start header downloading. While you are talking about which block to start body downloading, which means header downloading should end in the mean time. We are not talking about the same problem here.

If you read carefully I moved the problem to the FCU simply because it's pointless to set a checkpoint without considering the FCU, unless you intend to support raw monitors that only observe upcoming blocks and transactions.

I don't see any advantage for SPV wallets with some history, whereas a protocol message providing a (timestamp -> checkpoint_hash) mapping would be much more useful for both monitors and wallets. By checkpoint_hash I don't mean "the first block with transactions", rather a checkpoint block the way you meant in your previous posts.

Example: "hey peer, I want to start syncing from 'Jan 01, 2012' (or from now on). I don't have any idea about what the hash of the first block to start sync from could be, can you give it to me?"

For now, FCU is only use for decide to use getblock message or getheader message which means download block's transaction or not. But we still need to  know the first block's hash, and get the first block's block time to decide with FCU.

OK, may be you think why not peer direct give us a block hash that before a point time? I think it may be a future improve for Bitcoin, and may be another BIP.

[keeshux]

Again: a height-to-hash translator might be useful in general, still it's no use for SPV wallets.

[songchenwen]

Again: a height-to-hash translator might be useful in general, still it's no use for SPV wallets.

If you think FCU is important to SPV client, you should consider knowing where to start and end FCU is also important. Since you already know where to end it, it does not make sense that you think knowing where to start is not important.

[gmaxwell]

I can very very cheaply create 100 difficulty 1 blocks and feed them to your client and claim that they're the tip of the best chain. You'd know no better.

I don't think compromising the security of SPV wallets further is worth an unnoticeable small improvement.

A zero trust compression of the past headers with log(n) scaling is possible too, and will hopefully be deployed for other applications. Because of the above point it is not very interesting for SPV, but if it were deployed in any case using it would be much better than having an unverified and unverifiable dependence on peers.

[zhouqi_cn]

I can very very cheaply create 100 difficulty 1 blocks and feed them to your client and claim that they're the tip of the best chain. You'd know no better.

I don't think compromising the security of SPV wallets further is worth an unnoticeable small improvement.

A zero trust compression of the past headers with log(n) scaling is possible too, and will hopefully be deployed for other applications. Because of the above point it is not very interesting for SPV, but if it were deployed in any case using it would be much better than having an unverified and unverifiable dependence on peers.

Dear gmaxwell:

Thanks for your suggestion. I will read and try to understand the link you post.

Zhou Qi
Bither Team

[keeshux]

Again: a height-to-hash translator might be useful in general, still it's no use for SPV wallets.

If you think FCU is important to SPV client, you should consider knowing where to start and end FCU is also important. Since you already know where to end it, it does not make sense that you think knowing where to start is not important.

Oh my.. it's getting clearer that you two didn't understand a word of what I said. I'm giving up, good luck!

[songchenwen]

Again: a height-to-hash translator might be useful in general, still it's no use for SPV wallets.

If you think FCU is important to SPV client, you should consider knowing where to start and end FCU is also important. Since you already know where to end it, it does not make sense that you think knowing where to start is not important.

Oh my.. it's getting clearer that you two didn't understand a word of what I said. I'm giving up, good luck!

Wish you good luck too. Bye.