How does "getheaders" to work?

[yky_sh]

These day , i want to get block header infomation, but, i don't get it success, so, I have to ask for your help.
After I connect one node, my step as follow:
step 1:i send "version" message to node, this step is success, i recieve "version" message from node, but , after recieve "version" message, i can recieve "verack" message, "inv" message, "addr" message, and i don't send any control message or data message. the node send these message initiatively。
step 2: after i recieve "version" message, i send "verack" message, but , the node no response.
step 3: i send "getheaders" messge, but , the node no response.

The "getheaders" message payoad
Field Size    Description    Data type    Comments
4    version    uint32_t    the protocol version
1+    hash count    var_int    number of block locator hash entries
32+    block locator hashes    char[32]    block locator object; newest back to genesis block (dense to start, but then sparse)
32    hash_stop    char[32]    hash of the last desired block header; set to zero to get as many blocks as possible (2000)

version, i get the version of"version" message from node, and compare my version 70002, which is low, i use it;
hash count , i set 1;
block locator hashes, i choose the genesis block hash
hash_stop, i set 0;

message hex:
sendMsg len:93
header:
0xf9, 0xbe, 0xb4, 0xd9, 0x67, 0x65, 0x74, 0x68, 0x65, 0x61, 0x64, 0x65, 0x72, 0x73, 0x00, 0x00,
0x45, 0x00, 0x00, 0x00, 0xee, 0x40, 0x68, 0xa4,
payload:
0x72, 0x11, 0x01, 0x00,
0x01,
0x6f, 0xe2, 0x8c, 0x0a, 0xb6, 0xf1, 0xb3, 0x72, 0xc1, 0xa6, 0xa2, 0x46, 0xae, 0x63, 0xf7, 0x4f, 0x93, 0x1e, 0x83,
0x65, 0xe1, 0x5a, 0x08, 0x9c, 0x68, 0xd6, 0x19, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

i don't understand , why node don't reply the "getheaders" message, which step i was wrong?

[1715205920]

1715205920

[1715205920]

1715205920

[1715205920]

1715205920

[1715205920]

1715205920

[achow101]

The Genesis block isn't a block that you can request, it is hard coded into the software and never enters the database. Try a different block.

[yky_sh]

The Genesis block isn't a block that you can request, it is hard coded into the software and never enters the database. Try a different block.

I test a different hash, but the node don't reply. not only "getheaders" message,  I send "getaddr" message, the node don't reply. the node only reply "version" message.
i found after i send "version" message, i will recieve "version" ,"verack", "inv"......"inv", "addr" message, sometimes, i will recieve "version" ,"verack","ping", "addr" message. then, i send any type message, the node don't reply.

[achow101]

The Genesis block isn't a block that you can request, it is hard coded into the software and never enters the database. Try a different block.

I test a different hash, but the node don't reply. not only "getheaders" message,  I send "getaddr" message, the node don't reply. the node only reply "version" message.
i found after i send "version" message, i will recieve "version" ,"verack", "inv"......"inv", "addr" message, sometimes, i will recieve "version" ,"verack","ping", "addr" message. then, i send any type message, the node don't reply.

Did you complete the handshake; it seems that you did not. You first send the version message. Then you wait for both a verack from the node and the node's version message. Then you send a verack to the node. And then you can send your other messages.

[yky_sh]

The Genesis block isn't a block that you can request, it is hard coded into the software and never enters the database. Try a different block.

I test a different hash, but the node don't reply. not only "getheaders" message,  I send "getaddr" message, the node don't reply. the node only reply "version" message.
i found after i send "version" message, i will recieve "version" ,"verack", "inv"......"inv", "addr" message, sometimes, i will recieve "version" ,"verack","ping", "addr" message. then, i send any type message, the node don't reply.

Did you complete the handshake; it seems that you did not. You first send the version message. Then you wait for both a verack from the node and the node's version message. Then you send a verack to the node. And then you can send your other messages.

Thanks a lot. yesterday, i tested many nodes, i found some nodes send "version", then "verack", then "ping", last "addr" message, after they finish to send these messages, they don't reply any kinds of messages. but if the node can send "inv" message initiatively， it can response other message, such as "getaddr" message, the node can reply "addr" message. So, i don't know which kind of nodes i can download full of blocks or headers, can I identity this node according which message in wiki protocol? wiki link: https://en.bitcoin.it/wiki/Protocol_documentation

[achow101]

Thanks a lot. yesterday, i tested many nodes, i found some nodes send "version", then "verack", then "ping", last "addr" message, after they finish to send these messages, they don't reply any kinds of messages.

I think you should use this: https://bitcoin.org/en/developer-reference#constants-and-defaults as your reference. It pretty much explains everything.

Anytime you want to connect to a node, you must complete the handshake. The handshake consists of you sending the peer a version message and the peer sending you a version message. Upon you receiving the version message, you must return a verack. When the peer receives your version message, he must send a verack. Only when both peers have received verack messages in response to the version messages can any other messages be sent.

Many times if you don't respond to a message even though your protocol version indicates you should, the connection will be dropped. If you receive a ping message, you must respond with a pong.

but if the node can send "inv" message initiatively， it can response other message, such as "getaddr" message, the node can reply "addr" message.

It depends on the protocol and software version. IIRC Bitcoin Core 0.13+ will not allow you to request a block or transaction unsolicited unless the peer has sent you an inv first.

So, i don't know which kind of nodes i can download full of blocks or headers, can I identity this node according which message in wiki protocol?

No, that is not the way to identify those services. In the version message, there is a section for the services. It is an 8 byte bitfield. If the first bit of that bitfield is set, then the node can send blocks and headers; it is a full node. If that bit is not set, then it is not a full node.


Are you sure that you are sending the getheaders message correctly?


Edit: Your checksum hash is wrong. Did you do a hash of the actual bytes or did you just hash the hex string? I'm pretty sure that it should be

Code:

0xf5fcbcad

[yky_sh]

Thanks a lot. yesterday, i tested many nodes, i found some nodes send "version", then "verack", then "ping", last "addr" message, after they finish to send these messages, they don't reply any kinds of messages.

I think you should use this: https://bitcoin.org/en/developer-reference#constants-and-defaults as your reference. It pretty much explains everything.

Anytime you want to connect to a node, you must complete the handshake. The handshake consists of you sending the peer a version message and the peer sending you a version message. Upon you receiving the version message, you must return a verack. When the peer receives your version message, he must send a verack. Only when both peers have received verack messages in response to the version messages can any other messages be sent.

Many times if you don't respond to a message even though your protocol version indicates you should, the connection will be dropped. If you receive a ping message, you must respond with a pong.

but if the node can send "inv" message initiatively， it can response other message, such as "getaddr" message, the node can reply "addr" message.

It depends on the protocol and software version. IIRC Bitcoin Core 0.13+ will not allow you to request a block or transaction unsolicited unless the peer has sent you an inv first.

So, i don't know which kind of nodes i can download full of blocks or headers, can I identity this node according which message in wiki protocol?

No, that is not the way to identify those services. In the version message, there is a section for the services. It is an 8 byte bitfield. If the first bit of that bitfield is set, then the node can send blocks and headers; it is a full node. If that bit is not set, then it is not a full node.


Are you sure that you are sending the getheaders message correctly?


Edit: Your checksum hash is wrong. Did you do a hash of the actual bytes or did you just hash the hex string? I'm pretty sure that it should be

Code:

0xf5fcbcad

Thanks for your detail answer。
1, My steps to complete the handshake as follow, but hash the hex maybe wrong.  Is the checksum hash256 the payload twice?
step        me                     node
1            version      -->
2                           <--      version
3                           <--      verack
4            verack      -->            
if the node send ping, I reply pong, the payload is the same as ping.

but some nodes don't care your reply, they only send "version", "verack", "ping", "addr" message orderly. I am sure when i connect these node, i follow the steps to complete the handshake.

2, In the version message, there are three sections of services, “services", "addr_recv services", and "addr_trans services". which is the services you refer to?

Thanks a lot.

[achow101]

Thanks for your detail answer。
1, My steps to complete the handshake as follow, but hash the hex maybe wrong.  Is the checksum hash256 the payload twice?

You have to sha256 hash the sha256 hash (so two hashes) of the payload. This is all done as binary data, not as a hex string.

step        me                     node
1            version      -->
2                           <--      version
3                           <--      verack
4            verack      -->            
if the node send ping, I reply pong, the payload is the same as ping.

but some nodes don't care your reply, they only send "version", "verack", "ping", "addr" message orderly. I am sure when i connect these node, i follow the steps to complete the handshake.

Interesting. Can you provide the user agent strings of those boxes? If you don't send a version message, then verack should not be sent. If you haven't sent a verack, ping and addr should not be sent.

2, In the version message, there are three sections of services, “services", "addr_recv services", and "addr_trans services". which is the services you refer to?

Thanks a lot.

"services". It refers to the services that the sending node is advertising, which is what you want to know.

[yky_sh]

Thanks for your detail answer。
1, My steps to complete the handshake as follow, but hash the hex maybe wrong.  Is the checksum hash256 the payload twice?

You have to sha256 hash the sha256 hash (so two hashes) of the payload. This is all done as binary data, not as a hex string.

step        me                     node
1            version      -->
2                           <--      version
3                           <--      verack
4            verack      -->            
if the node send ping, I reply pong, the payload is the same as ping.

but some nodes don't care your reply, they only send "version", "verack", "ping", "addr" message orderly. I am sure when i connect these node, i follow the steps to complete the handshake.

Interesting. Can you provide the user agent strings of those boxes? If you don't send a version message, then verack should not be sent. If you haven't sent a verack, ping and addr should not be sent.

2, In the version message, there are three sections of services, “services", "addr_recv services", and "addr_trans services". which is the services you refer to?

Thanks a lot.

"services". It refers to the services that the sending node is advertising, which is what you want to know.

my version message:
sendMsg len:131
0xf9, 0xbe, 0xb4, 0xd9, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x0, 0x0, 0x0, 0x0, 0x0,
0x6b, 0x0, 0x0, 0x0, 0xd9, 0x9c, 0xe5, 0x9, 0x7c, 0x11, 0x1, 0x0, 0x1, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0, 0x57, 0xa6, 0xe0, 0xe6, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff,
0xd1, 0x7e, 0x69, 0x7, 0x8d, 0x20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0, 0xff, 0xff, 0x3a, 0x20, 0x40, 0x9, 0x5, 0xb5,
0x25, 0x8, 0x55, 0xca, 0x19, 0xa0, 0x39, 0x5, 0x15, 0x2f, 0x62, 0x69, 0x74, 0x63, 0x6f, 0x69,
0x6e, 0x2d, 0x73, 0x65, 0x65, 0x64, 0x65, 0x72, 0x3a, 0x30, 0x2e, 0x30, 0x31, 0x2f, 0x0, 0x0,
0x0, 0x0, 0x1, send success, len:131

node version message
payload msg:
  0x7c  0x11  0x1  0x0  0x5  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0xd3  0xe0  0xa6  0x57
  0x0  0x0  0x0  0x0  0x1  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0
  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0xff  0xff  0x3d  0xab  0x74  0x1e  0x93  0xfe
  0x5  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0
  0x0  0x0  0x0  0xff  0xff  0xd1  0x7e  0x69  0x7  0x20  0x8d  0x1b  0x8b  0x29  0x4c
  0x18  0x6c  0xaa  0xdb  0x10  0x2f  0x53  0x61  0x74  0x6f  0x73  0x68  0x69  0x3a  0x30
  0x2e  0x31  0x32  0x2e  0x30  0x2f  0x9a  0x78  0x6  0x0  0x1
Start Decode:version:70012
mVersion:70012
services:0x5,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
timeStamp:1470554323
recv services:0x1,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
recvIp:0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0xff,0xff,0x3d,0xab,0x74,:0x1e,0x93,0xfe,
recv services:0x5,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
recvIp:0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0xff,0xff,0xd1,0x7e,0x69,:0x7,0x20,0x8d,
nonce:0x1b,0x8b,0x29,0x4c,0x18,0x6c,0xaa,0xdb,
user_agent:0x2f,0x53,0x61,0x74,0x6f,0x73,0x68,0x69,0x3a,0x30,0x2e,0x31,0x32,0x2e,0x30,0x2f,strUserAgent:/Satoshi:0.12.0/|

Peer Block Height:424090

node verack message:
strCommand:verack
verack msg:msgLen:24
0xf9,0xbe,0xb4,0xd9,0x76,0x65,0x72,0x61,0x63,0x6b,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x5d,0xf6,0xe0,0xe2,

node ping message:
strCommand:ping
ping msg len:32
0xf9,0xbe,0xb4,0xd9,0x70,0x69,0x6e,0x67,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
0x8,0x0,0x0,0x0,0x88,0xda,0x1c,0xf5,0x9b,0x13,0x28,0x99,0xf5,0x46,0x15,0x60,

node addr message:
msgCommand:addr
addr msg:msgLen:55
0xf9,0xbe,0xb4,0xd9,
0x61,0x64,0x64,0x72,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,
0x1f,0x0,0x0,0x0,
0xc4,0x9c,0x92,0x9a,
0x1,0xd3,0xe0,0xa6,0x57,0x5,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0xff,0xff,0xd1,0x7e,0x69,0x7,0x20,0x8d,

First , after i connected, I sent "version" message, and i didn't do anything more. then, I received four messages,  in this stage,  i didn't send any messages.
so, it is very strange.

[hhanh00]

During handshake, both parties send their version message. They can cross since the communication is duplex.
step        me                     node
1            version      -->
2                           <--      version

Then the other node received 1) and replied with verack. You did the same with your verack.

3                           <--      verack
4            verack      -->             

Now that you have handshaked, the communication is in the normal phase. The other node is continuously broadcasting some of its state. So you should expect to receive inv, addr, tx, at any time. In addition, the other node can also send you a command like 'ping'.

The protocol isn't request/response. It is closer to a publish/subscribe. By connecting, you have subscribed to relay transactions and blocks and the other node will send you messages for that.

In short, you'll have an easier time if you organize your node around a message handler because messages can be unsolicited (at least not directly), or you can have a message with multiple responses (getblocks). Basically, be ready to receive anything and throw away the stuff you don't handle or need. If you want something, request it explicitly.