|
Title: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: Coding Enthusiast on August 01, 2017, 05:18:45 AM This is the response I get after sending the Version message:
(this is from WireShark: Code: Frame 8: 180 bytes on wire (1440 bits), 180 bytes captured (1440 bits) on interface 0 My confusion is in two parts: 1. Why is it sending the response (the returned Version message) in 2 TCP Segments instead of 1? And will it always be header separate from message body? 2. Why is Verak message inside the same response and will it always be like this meaning multiple messages inside one response? (Have not yet gotten around testing other messages) Title: Re: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: achow101 on August 01, 2017, 05:24:29 AM 1. Why is it sending the response (the returned Version message) in 2 TCP Segments instead of 1? And will it always be header separate from message body? Because some optimizations were made where the header is queued to be sent first, then the payload if there is a payload. This results in two calls to the kernel, but it should still be received as one message chunk even though sent separately.2. Why is Verak message inside the same response and will it always be like this meaning multiple messages inside one response? (Have not yet gotten around testing other messages) Verack should not be part of the same response. It may be sent at the same time though, but multiple messages should not be in the same TCP packets.Title: Re: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: Coding Enthusiast on August 01, 2017, 06:15:25 AM I'm new to TCP protocol so my terminology may be a bit wrong.
I am receiving the response over the same open socket but over 2 different calls (2 different buffers). 1 header 24 byte then 1 body variable byte [This was actually a source of big confusion for me] Then looking deeper at WireShark I saw that line I shared above showing 2 different Segments. This is what Wireshark shows (The text in red is added on photo): https://i.imgur.com/DKbHg50.jpg https://i.imgur.com/NOEzcEs.jpg Also in my code this is the hex of the total buffer I receive. It contains both messages: Quote f9beb4d976657273696f6e0000000000660000008a94bbe97f1101000d00000000000000131b80590000000 0010000000000000000000000000000000000ffff05ef60c22acf0d000000000000000000000000 0000000000ffff23a7f609208df65ca61c4a15f201102f5361746f7368693a302e31342e312f374 d070001f9beb4d976657261636b000000000000000000005df6e0e2 Title: Re: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: theymos on August 01, 2017, 06:50:40 AM Upper-layer protocols like Bitcoin experience a TCP socket as a stream of bytes. So from a node's perspective, it writes to an outgoing stream per peer in a manner like "versionVerackAddrInvBlock...", with messages just stuck together, and a similar incoming stream per peer. The fact that things are actually broken up into packets by TCP isn't generally something that users of TCP have to worry about (except sometimes for optimization reasons). A Bitcoin message can be broken into multiple TCP packets (if it's too large for one), and a TCP packet can contain multiple Bitcoin messages (if Bitcoin Core happens to send two Bitcoin message very close together so that the OS can batch them). If you're interested in Bitcoin rather than TCP, then you shouldn't really worry about this: just stick all of the TCP payloads together.
Try writing a simple TCP server and client in C or something using the basic syscalls. It's pretty simple, and it'll give you more understanding of how it works from the perspective of a TCP user. Title: Re: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: Coding Enthusiast on August 01, 2017, 03:15:23 PM If you're interested in Bitcoin rather than TCP, then you shouldn't really worry about this: just stick all of the TCP payloads together. Try writing a simple TCP server and client in C or something using the basic syscalls. It's pretty simple, and it'll give you more understanding of how it works from the perspective of a TCP user. I'm actually interested in both and kind of learning everything at the same time! Your comment helped a lot as a matter of fact. I've changed my MessageManager class which is supposed to handle these received byte arrays and now it looks for the Magic in messages and based on first 24 byte of header and the PayloadSize inside of it, I decide how much of the packet is remaining or how to generally go on from there. Now everything works fine and life is beautiful again after 1 week of struggling. Thanks. :) Title: Re: Is there any reason why Messages are sent in 2 TCP Segments instead of 1? Post by: Andre_Goldman on August 02, 2017, 05:29:25 PM If you're interested in Bitcoin rather than TCP, then you shouldn't really worry about this: just stick all of the TCP payloads together. Try writing a simple TCP server and client in C or something using the basic syscalls. It's pretty simple, and it'll give you more understanding of how it works from the perspective of a TCP user. I'm actually interested in both and kind of learning everything at the same time! Your comment helped a lot as a matter of fact. I've changed my MessageManager class which is supposed to handle these received byte arrays and now it looks for the Magic in messages and based on first 24 byte of header and the PayloadSize inside of it, I decide how much of the packet is remaining or how to generally go on from there. Now everything works fine and life is beautiful again after 1 week of struggling. Thanks. :) sounds to me that it is TCP Segmentation Offload. Quote Most modern operating systems support some form of network offloading, where some network processing happens on the NIC instead of the CPU. Normally this is a great thing. It can free up resources on the rest of the system and let it handle more connections. https://wiki.wireshark.org/CaptureSetup/Offloading Nice observation, thanks for your post. It let me thinking about OS detection and TCP/IP stack fingerprinting. |
