If you don't want any transactions at all, just use getheaders.

I don't think you can perfectly partition blocks using filtered blocks, as there are several matching criteria, and one match is enough to include a transaction. So if you send disjunct Bloom filters (nHashes=1, and disjunct bitsets to each peer), you'll still get doubles. It is perhaps an interesting extension to BIP37 to support such partitioning, especially for larger blocks.

However, one case that is explicitly (and intentionally) supported by BIP37 is requesting blocks (full-node wise) with the transactions already known to the peer filtered out. So contrary to what Mike says, I'm pretty sure you can have a full node that uses BIP37 to fetch blocks, and save download bandwidth using that.

I agree completely. But it still has nothing to do with your logic of deciding what to fetch and how to validate it. It's just using a different protocol to do it.

It doesn't, as far as I know. It asks for "up 500 blocks starting at hash X", where X is the last known block.


Using Bloom filtering may not be entirely viable yet, I'll have to check. The big changes is first downloading and validating headers, and then downloading and validating the blocks itself. IMHO, it's the only way to have a sync mechanism that is both fast, stable and understandable (I have no doubt that there are other emchanisms that share two of those three properties...).


What protocol is used to actually fetch blocks is pretty much orthogonal to the logic of deciding what to fetch, and how to validate it, IMHO.

Not sure what you mean by "as deep as possible". We always send getdata starting at whatever block we already know. The reason for starting from early blocks and moving forward is because validation is done is stages, and at each point as much as possible is already validated (as a means to prevent DoS attacks, mostly). As most checks can only be done when you have the entire chain of blocks from genesis to the one being verified, you need them more or less in order.


That's not true, we only ask for each block once (and retry after a timeout), but it is done to a single peer (not to all, and not balanced across nodes). That's a known badness, but changing isn't trivial, because of how validation is done.

There is one strategy however that's pretty much accepted as the way to go, but of course someone still has to implement it, test it, ... and it's a pretty large change. The basic idea is that downloading happens in stages as well, where first only headers are fetched (using getheaders) in a process similar to how getblocks is done now, only much faster of course. However, instead of immediately fetching blocks, wait until a long chain of headers is available and verified. Then you can start fetching individual blocks from individual peers, assemble them, and validate as they are connected to the chain. The advantage is that you already know which chain to fetch blocks from, and don't need to infer that from what others tell you.


The Bitcoin Foundation has only been around for a year or so, and they don't control development. They pay Gavin's salary, but other developers are volunteers that work on Bitcoin in their free time.


BIP37 actually introduced a way to fetch parts of blocks, and it can be used to fetch a block with just the transactions you haven't heard about, so it avoids resending those that have already been transferred as separate transactions (though I don't know of any software that uses this mechanism of block fetching yet; once BIP37 is available on more nodes, I expect it will be). Any other system which requires negotiating which transactions to send adds latency to block propagation time, so it's not necessarily the improvement you want.


There are many ideas about how to improve historic block download. I've been arguing for a separation between archive storage and fresh block relaying, so nodes could be fully verifying active nodes on the network without being required to provide any ancient block to anyone who asks. Regarding moving to other protocols, there is the bootstrap.dat torrent, and there's recently been talk about other mechanism on the bitcoin-development mailinglist.

They don't know the originating IP; only the IP that first relayed it to them.

Adding more flag bits is certainly possible, but we can extend the length of the serialized i value in that case, so the only thing that needs to change is the serialization format.


I like the simplicity of a single existing construct that operates natively on 512 bits. Yes, separate SHA256 calls would work too, but are less elegant IMHO. I don't mean to say it would be less secure, and this is just bike shedding.

If smartcards have a problem with SHA512, they'll have a huge problem with ECDSA.


That provability argument is interesting, but can you come up with use cases? I'm also interested in knowing the complexity of iddo's zero-knowledge proofs that could avoid this.

I'm in the middle about this.

One the one hand, I do understand the reason for a change like this. If miners become (short-term) selfish/rational, and start implementing behaviour like the one implemented by this patch, I will completely be in favor of making it the default - as anything else would indeed just be a false sense of security. It would remove the "best-effort attempt" by the network to secure 0-conf transactions, and force people to find real solutions for fast transactions, instead of relying on behaviour that cannot be guaranteed. And perhaps that is inevitable anyway. And looking at Bitcoin as an experiment, it's so much more interesting to aim for a system that doesn't require specific parties to act in the best interest (especially non-rationally) of the community anyway...

On the other hand, it's not true that doing double-spending 0-conf transactions is trivial now. Not particularly hard, but not as easy as whistling in a telephone either. So in that sense, the network right now does provide some 0-conf security. Not much, and nothing guaranteed, but it works, and there is economic infrastructure that relies on it. I don't think they're even wrong about relying on it, if they're willing to eat the costs from damage caused by it (on the other hand, perhaps few can actually estimate the risk correctly, and will get hurt when their business becomes profitable enough to attack...). Anyway, what I'm getting at is that this is a drastic change if we'd put it in mainline, and drastic changes may be more damaging than the actual problem we're solving. Perhaps we need time to outgrow relying on 0-conf security.

So: I don't know. I like the idea of aligning miner's short-term self interest (even if they don't exploit that right now) with the policy implemented by the reference client. But perhaps not immediately...

Just as a FYI, the C secp256k1 library I'm writing has support for creating recoverable signatures, and doing key recovery (though not the - admittedly weird - serialization that Bitcoin uses for it).

I still haven't seen a compelling argument for being able to change chaincode independently of keys. As far as I'm concerned, they constitute an somewhat-less-secret overflow entropy buffer for keys, and are thus intimately linked with them.

Yes, I understand there are use cases, but they seem far-fetched and either assume it is hard to set up a secure communication channel between trusted parties (which I think in practice is easy), or integrating them more tightly into multisig schemes (which I don't like; scripts/addresses are a layer strictly above key derivation, imho; in that sense I also dislike using the term "address" for identifying keys - it should only be called address when you're talking about the script template associated with a particular key, with the intention of receiving coins on it).

Maybe at some point there will be a use case important enough, and we can still standardize a system for "updating" a BIP32 chain with a new chaincode, though I don't like the complexity of reasoning about the security of that.

Unless someone comes up with a use case that it worth supporting (and hopefully doesn't require pretty much rewriting it from scratch...), I'm going to announce the current specification as final on the conference next week.

Non-release builds carry that warning (and this is just release candidate).

Where does the CScript from if you don't know the other keys in the first place? P2SH transactions do not contain the actual (public) keys anyway, so whatever you do, you need some metadata about the P2SH script in your wallet. I don't see the problem with extending this with the individual BIP32 public parent keys (if you need to be able to access the entire chain of multisig addresses in the first place).

I imagined constructing multisig addresses of a set of (potentially completely independent) BIP32 chains, using equal indices in each, in lockstep.

In my opinion, BIP32 is mostly about generating sequences of keys, which can either be used directly as pay-to-pubkeyhash addresses, or combined using multisig into a P2SH address. Isn't that both more simple than trying to put BIP32 on top of multisig, and no reason for having the chaincodes be independent?

Tomorrow we're crossing 2⁷⁰, by the way

If an attacker needs to guess a key, there is nothing to worry about. The keyspace is way too large for that.

If an attacker has access to your wallet/backup/passphrase/... in a way that grants him access to one of the keys, he very likely has access to all keys.

There is one small security difference between deterministic and randomly-generated wallet keys: if someone manages to copy the keys from the second, he cannot wait (long) before stealing, as the coins tend to move to newer addresses (i.e., it becomes "unstolen" over time).

Also, there are plans to implement deterministic wallets for the reference client too, as the advantages for backup safety far outweigh the security risks.

This is nonsense. The bootstrap isn't validated in any way - it's just a way to get blocks to your client, and the blocks are verified before accepting them.

Bitcoin - at least as implemented by the reference client today - is a zero-trust system: every rule that can be validated is validated by your local node upon receiving data from other nodes. If you want to apply this principle to history, it means you have to see history. There is no way around it: bootstrapping a full node requires either feeding it the entire history, or trusting someone else to give you the current state that resulted from that history.

That does not mean everyone needs to keep the entire chain around, it just has to be available enough so that new nodes can get it (once).

There is an alternative, with almost as good security properties, namely if we could somehow encode a hash of the current state inside blocks (in the coinbase, for example). That would mean someone bringing up a new code could just download the historic headers, verify the proof-of-work in it, download the current state from somewhere, and verify it against the latest block. This would allow him to only require "SPV-level" trust in the past (maybe up to some point ago he considers safe), and then continue full validation from there.

The reason for specifying that I_L >= n isn't allowed is indeed to guarantee uniformity. However, as this has such a ridiculously small chance to ever occur, I don't want whatever rule is used to deal with it to be complex to implement.

Also, the reason for using the same construction (k_i = k_par + I_L) for type-1, is to 1) make the code to implement it more uniform 2) not give a performance advantage to type-1. I agree that strictly from a security point of view k_i = I_L is just as good.

Listunspent fetches from the wallet, not from the (global) UTXO set.

Hey iddo & thanke,

I love how thoroughly you've been discussing this, though I must admit I haven't read the through entire discussion.

I've updated the specification to use addition instead of multiplication, but now I'd like to make the BIP32 specification final soon.

So my questions are:

• Is there a use case to allow updating keys without updating chain codes, that's worth breaking the current spec for?
• Is there a reason to disallow secret derivation after public derivation?

If not, I'd like the current version to be final.

Grau: The thread has progressed quite a bit since your comment, but I agree it makes sense to have a set of "guidelines for wallet behaviour" that will make working with these wallets easier, though in no way required.