A bit of criticism on how the bitcoin client does it

[piotr_n]

Actually, if you look at this article it even clearly advises:

To create the block locator hashes, keep pushing hashes until you go back to the genesis block. After pushing 10 hashes back, the step backwards doubles every loop

Yeah, you are only at block 236k+, so just keep pushing all the hashes, starting from the top, until you reach the genesis block - a brilliant idea

What's wrong with this? It sounds to me like you don't understand the purpose of the locators.

Again: this wastes a hell lot of bandwidth, generating lots of traffic in the network each time a new block is mined.

I believe that I understand the purpose of the locators very well - and that is why my implementation always sends only one locator... which surely does not point to a genesis block, while I am already at #236475. The only reason to send a locator pointing to a genesis block, in such a case, would be to "recover" from 4+ years long lasting fork... so whoever does that, I think he doesn't understand the purpose of the locators.

[1714770519]

1714770519

[1714770519]

1714770519

[1714770519]

1714770519

[Mike Hearn]

You don't send a full locator every time a new block is mined. Nodes are supposed to send a getdata and only do a getblocks if what they get back is an orphan.

No, the increase in bandwidth wouldn't faze me. 10GB per day would be 300GB per month and I get 500 on the cheapest plan available via BitVPS, which costs hardly anything. But regardless, in future I anticipate lower bandwidth usage because nodes will become more stable. Right now a lot of regular users end up installing Bitcoin-Qt and sucking down the chain without really intending to run a full node, because they don't know about SPV clients. After we get a new MultiBit release out, I'm hoping we can point people towards that instead and the amount of chain that is served to new users will go down.

[Zeilap]

Again: this wastes a hell lot of bandwidth, generating lots of traffic in the network each time a new block is mined.

No. That's not how it works.

I believe that I understand the purpose of the locators very well - and that is why my implementation always sends only one locator

If you do this after the last checkpoint, you're in for a surprise. Good luck.

[piotr_n]

You don't send a full locator every time a new block is mined. Nodes are supposed to send a getdata and only do a getblocks if what they get back is an orphan.

Of course I don't.
But I am saying that this is what I get from a bunch of nodes running the official client, often after a new block has been mined.
I have no idea why they do it, but feel free to explain it to me..

Again: this wastes a hell lot of bandwidth, generating lots of traffic in the network each time a new block is mined.

No. That's not how it works.

Are you saying that I am sending all these screwed up "getblocks" to myself, from all the different IPs - or are you saying that I am lying about what I see?
Or maybe I'm just crazy and what I see is not real...

If you do this after the last checkpoint, you're in for a surprise. Good luck.

Thanks, but I don't use any checkpoints, so I don't think I'm going to need any luck here.

It's very simple:
1) measure a time difference between now and when you received your last block - divide it by whatever period you like, to get a number (I use minutes)
2) then go back the chain, starting from the head, as many blocks up, as the number from point 1 was
3) pass the hash of the block you have reached in point 2 to "getblocks" - and voila.
Using such a simple way, you can recover from basically any fork, as long as it isn't longer than a couple of hundreds blocks. And if a fork would happen o be longer, I will surely have enough time to develop an improvement before it's actually needed

[Mike Hearn]

They were probably on a fork or had missed a block. I don't know why else it would happen - there might be a bug, if you spot such a thing let us know.

[piotr_n]

They were probably on a fork or had missed a block. I don't know why else it would happen - there might be a bug, if you spot such a thing let us know.

You mean: let you know, even more than I have been trying to let you know so far?
Sorry, no - I think pushing it even more would be just rude and completely unproductive.

[Mike Hearn]

What I meant is, if you believe it's a bug and not intended behaviour, then feel free to investigate and figure out the cause. It's expected that remote nodes will sometimes send you block locators when new blocks are mined (if it's an orphan). I didn't see any evidence that isn't the cause here.

As we've already pointed out, it doesn't seem like many people are complaining about bandwidth usage at the moment. That's why it's not been worked on. If they were, then the best improvement available right now is full-match Bloom filters. Perhaps optimising addr/inv broadcasts might improve things too, it's a bit hard to know exactly where all the bandwidth goes right now as the software lacks instrumentation.

[piotr_n]

Oh, I just figured out what I was doing wrong handling these "getblocks" requests with 29 different locator hashes up to the genesis block.

Originally I had thought that the only limit, for building the "inv" reply, should be either reaching the hash_stop or the max 500 records.
But now I figured that I should have ignored the remaining locators after matching on a first one from the list - which almost always points to the end of my tree, so in most cases the reply should be just empty, and not 500 hashes long.
At least that's how satoshi does it..
So after fixing this in my code, the upload bandwidth usage looks indeed much better - and the whole idea of 29 locator hashes makes much more sense, as well.

Anyway, sorry - my bad.
Mike's node might be indeed in a data center