bitcoinj bit by bitcoin flood protection

[noa]

I took bitcoinj on a test drive today. Firstly, I would like to say that I'm really happy about the project. I think that having a second independent bitcoin protocol implementation is really valuable, especially since reference implementation code base have a quite steep learning curve.

However, when trying to start up the included PingService it fails to download the initial block chain from the bitcoin peer running on localhost. When i start up the PingService it starts to try to download the blocks from that it has gotten previously. After a few tens of blocks received the peer on localhost:8333 closes the TCP connection, and writes something along the lines of:

socket send flood control disconnect (1380543 bytes)

in the it's debug.log.

I'll look into working around this tomorrow, but thought I should bring this up here first.

What would be the reasonable thing for a client to do to not be seen as flooding by the bitcoin peer? The method I'm thinking about is to split the list of block hashes returned as a response to the getblocks message in several chunks and send a few separate getdata requests with a delay timer.

cheers
noa

Version info:
bitcoin client 0.3.20.1 BETA, OSX, no special settings
bitcoinj r51

[dave.gz2ba]

Found the same, and agree on splitting up the requests.

Having this complete faster than a single peer's rate-limit is a pretty good reason to support multiple peers and do the load-balancing across peers, then across time if still necessary.  I was thinking of doing this anyway, to cut down on an initial time required for block-chain fast-forwarding.

Question is, do apps implement the multi-peer management or would bitcoinj consider including this at some point?

[Cdecker]

I opensource BitDroid-Network, but once [mike]'s implementation hit there wasn't much interest in it so I stopped developing it. One of the main advantages is that I do in fact allow multiple connections and splitting the requests over multiple connections is a simple matter of implementing an actionlistener. Did I mention it uses non-blocking IO which allows for several hundred connections on a computer or dozens on an android phone?

[Gavin Andresen]

0.3.20.1 -maxsendbuffer was too small for the initial block download-- you were probably just unlucky and connected to a 0.3.20.1 node.   Connect to somebody running either 0.3.20.2 or an earlier release and you won't run into that problem (does bitcoinj re-connect if disconnected during block download?)

[noa]

(does bitcoinj re-connect if disconnected during block download?)

Oh, if it's just a buggy version of the mainline client that triggers this it's probably not needed to modify bitcoinj to work around it.

bitcoinj doesn't re-connect currently. The I/O thread dies and the main thread continues indefinitely waiting for the blockchain to be downloaded.

/noa

[Mike Hearn]

Yes, I reported that bug to Gavin. It's not a BitCoinJ problem. Nothing can download the block chain successfully from those nodes. They are basically broke. I wonder if it's worth an alert broadcast actually, but I suspect that functionality is pretty useless given the number of headless nodes there must be out there.

[Cdecker]

I wonder whether my problem with my own implementation applies here. I seem to get disconnected when asking other peers for their known peers:

Code:

323275 [main] INFO Received /205.185.123.216: null of type OUTGOING_CONNECTION_TYPE
323276 [main] Sent /205.185.123.216: VersionMessage[proto=31700] of type VERSION_TYPE
323454 [main] Received /205.185.123.216: VersionMessage[proto=32002] of type VERSION_TYPE
323454 [main] Sent /205.185.123.216: VersionMessage[proto=31700] of type VERSION_TYPE
323454 [main] Sent /205.185.123.216: VerackMessage[] of type VERACK_TYPE
323763 [main] Received /205.185.123.216: VerackMessage[] of type VERACK_TYPE
323763 [main] Sent /205.185.123.216: net.bitdroid.network.messages.GetAddrMessage@1128f5a of type GET_ADDR_TYPE
326227 [main] Received /205.185.123.216: AddrMessage[addresses=1000 1.148.88.47:8333[Services=1] 1.152.220.152:8333[Services=1] 2.36.83.47:8333[Services=1] 2.68.109.202:8333[Services=1] 2.92.120.37:8333[Services=1] 2.94.192.18:8333[Services=1] 2.102.32.14:8333[Services=1] 2.138.178.76:8333[Services=1] 8.3.224.32:8333[Services=1] 8.18.115.2:8333[Services=1] 12.43.161.87:8333[Services=1] 12.47.114.47:8333[Services=1] 12.71.233.34:8333[Services=1] 24.0.228.170:8333[Services=1] 24.2.12.155:8333[Services=1] 24.2.243.57:8333[Services=1] 24.4.139.99:8333[Services=1] 24.4.172.88:8333[Services=1] 24.4.216.211:8333[Services=1] 24.4.250.252:8333[Services=1] 24.5.171.161:8333[Services=1] 24.9.187.70:8333[Services=1] 24.16.124.154:8333[Services=1] 24.21.68.223:8333[Services=1] 24.21.181.152:8333[Services=1] 24.22.98.156:8333[Services=1] 24.22.180.66:8333[Services=1] 24.36.99.165:8333[Services=1] 24.49.39.196:8333[Services=1] 24.53.130.151:8333[Services=1] 24.60.18.78:8333[Services=1] 24.60.85.3:8333[Services=1] 24.61.104.245:8333[Services=1] 24.61.145.229:8333[Services=1] 24.63.35.214:8333[Services=1] 24.63.86.32:8333[Services=1] 24.67.16.168:8333[Services=1] 24.72.80.236:8333[Services=1] 24.73.232.230:8333[Services=1] 24.73.252.170:8333[Services=1] 24.83.9.72:8333[Services=1] 24.84.30.183:8333[Services=1] 24.90.16.243:8333[Services=1] 24.90.236.191:8333[Services=1] 24.98.29.224:8333[Services=1] 24.99.250.75:8333[Services=1] 24.117.24.237:8333[Services=1] 24.117.176.129:8333[Services=1] 24.126.59.67:8333[Services=1] 24.127.235.108:8333[Services=1] 24.128.175.1:8333[Services=1] 24.130.12.194:8333[Services=1] 24.130.130.192:8333[Services=1] 24.131.81.110:8333[Services=1] 24.137.73.230:8333[Services=1] 24.138.120.7:8333[Services=1] 24.143.242.65:8333[Services=1] 24.145.31.10:8333[Services=1] 24.145.247.228:8333[Services=1] 24.158.184.45:8333[Services=1] 24.161.186.32:8333[Services=1] 24.165.162.72:8333[Services=1] 24.182.67.94:8333[Services=1] 24.183.25.189:8333[Services=1] 24.208.56.50:8333[Services=1] 24.209.183.157:8333[Services=1] 24.215.154.30:8333[Services=1] 24.216.232.164:8333[Services=1] 24.222.40.193:8333[Services=1] 24.224.153.118:8333[Services=1] 24.224.176.192:8333[Services=1] 24.228.26.241:8333[Services=1] 24.236.255.210:8333[Services=1] 24.240.34.29:8333[Services=1] 24.242.210.31:8333[Services=1] 24.247.43.212:8333[Services=1] 24.248.196.235:8333[Services=1] 24.250.247.240:8333[Services=1] 24.251.117.46:8333[Services=1] 24.253.235.179:8333[Services=1] 38.100.222.91:8333[Services=1] 41.3.197.175:8333[Services=1] 41.132.7.80:8333[Services=1] 41.146.207.59:8333[Services=1] 41.181.55.138:8333[Services=1] 41.241.13.220:8333[Services=1] 41.241.35.200:8333[Services=1] 46.0.111.62:8333[Services=1] 46.4.89.172:8333[Services=1] 46.9.6.86:8333[Services=1] 46.15.188.87:8333[Services=1] 46.59.16.42:8333[Services=1] 46.59.17.182:8333[Services=1] 46.158.71.166:8333[Services=1] 46.158.218.38:8333[Services=1] 0.0.0.0:0[Services=0] 0.0.0.0:0[Services=0] ... lots and lots more of 0.0.0.0:0[Services=0] ... ] of type ADDR_TYPE
326368 [main] INFO net.bitdroid.monitor.BitcoinMultiConnectionTest - Received /205.185.123.216: null of type DISCONNECTED_TYPE

Directly afterwards I have to disconnect because I lost sync (probably I'm just noticing late that I was disconnected and I'm reading only 0-bytes). Strangely enough it seems to work with some peers but some others will just disconnect.

Is it the flood protection hitting me or do I have something seriously wrong?

[Mike Hearn]

If it's flood protection you can see a message in the logs stating that it was that.

[Cdecker]

It's rather strange, when connecting to the local client it all works nicely, it's only when trying to connect to remote clients, which means I cannot check it as easily.

Not asking for the addresses helps and most connections will survive
Still trying to reproduce the error

Edit: sorry for hijacking the thread, I'll create a new one if needed ^^

[Mike Hearn]

You can also just look at the version number of the node. If it's between the versions that had the bug, there's your issue.