Bitcoin Forum
August 17, 2022, 02:08:39 PM *
News: Latest Bitcoin Core release: 23.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: ZMQ raw block questions  (Read 76 times)
tiagocs
Newbie
*
Offline Offline

Activity: 28
Merit: 24


View Profile
November 12, 2020, 10:27:27 PM
 #1

I have two questions regarding ZMQ raw block notifications:

1 - The default high watermark for bitcoin core is 1000. If I understand correctly this means that up to 1000 raw blocks can be stored in memory (send buffer), which could translate to approximately 2GB, if there is a subscriber who forgets to read the notifications. This may lead to a crash due to insufficient memory in many machines, so it would be safer to set it to 100 and that would still allow the notification subscriber to be ~14h late in reading them, correct?

2 - It seems like raw block notifications are not published twice for the same height. An issue was just published on the bitcoin repository, but there are no comments yet: https://github.com/bitcoin/bitcoin/issues/20376 . Is this expected behavior or a bug?
1660745319
Hero Member
*
Offline Offline

Posts: 1660745319

View Profile Personal Message (Offline)

Ignore
1660745319
Reply with quote  #2

1660745319
Report to moderator
1660745319
Hero Member
*
Offline Offline

Posts: 1660745319

View Profile Personal Message (Offline)

Ignore
1660745319
Reply with quote  #2

1660745319
Report to moderator
1660745319
Hero Member
*
Offline Offline

Posts: 1660745319

View Profile Personal Message (Offline)

Ignore
1660745319
Reply with quote  #2

1660745319
Report to moderator
icon
Automate your trading like a pro. Lightning fast execution.
Start trading
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
NotATether
Legendary
*
Offline Offline

Activity: 966
Merit: 3466


aka. The Underboss


View Profile WWW
November 20, 2020, 08:44:12 AM
 #2

Late answer because it took me some time to figure out how ZeroMQ works.

1 - The default high watermark for bitcoin core is 1000. If I understand correctly this means that up to 1000 raw blocks can be stored in memory (send buffer), which could translate to approximately 2GB, if there is a subscriber who forgets to read the notifications. This may lead to a crash due to insufficient memory in many machines, so it would be safer to set it to 100 and that would still allow the notification subscriber to be ~14h late in reading them, correct?

Well this will only be a problem if you subscribe to ZMQ notifications directly because there are RPC calls that can fetch the blocks that you missed. So ZMQ won't send you the missing blocks but you can pass the last block hash you received from ZMQ to the getblock RPC call, it'll give you the height which you can increment and pass to getblockhash to get the others, stopping once you reach the block hash in the new ZMQ notification.

2 - It seems like raw block notifications are not published twice for the same height. An issue was just published on the bitcoin repository, but there are no comments yet: https://github.com/bitcoin/bitcoin/issues/20376 . Is this expected behavior or a bug?

You got the answer in the Github issue, if the new block at that height was already in the active chain, then there is no notification, but you can subscribe to the "sequence" topic as a workaround to get notified again for the height, discerning the information from block connections and disconnection messages.

You currently need a node running a development version of bitcoin core to use "sequence" as a new release hasn't been made since the PR that merged it.

https://bitcoin.stackexchange.com/questions/91678/is-it-possible-to-catch-up-missing-blocks-with-zeromq-if-script-was-down-for-a-w
https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md#remarks

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!