Bitcoin Forum
April 26, 2024, 09:53:55 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: ZMQ raw block questions  (Read 83 times)
tiagocs (OP)
Newbie
*
Offline Offline

Activity: 28
Merit: 24


View Profile
November 12, 2020, 10:27:27 PM
 #1

I have two questions regarding ZMQ raw block notifications:

1 - The default high watermark for bitcoin core is 1000. If I understand correctly this means that up to 1000 raw blocks can be stored in memory (send buffer), which could translate to approximately 2GB, if there is a subscriber who forgets to read the notifications. This may lead to a crash due to insufficient memory in many machines, so it would be safer to set it to 100 and that would still allow the notification subscriber to be ~14h late in reading them, correct?

2 - It seems like raw block notifications are not published twice for the same height. An issue was just published on the bitcoin repository, but there are no comments yet: https://github.com/bitcoin/bitcoin/issues/20376 . Is this expected behavior or a bug?
1714168435
Hero Member
*
Offline Offline

Posts: 1714168435

View Profile Personal Message (Offline)

Ignore
1714168435
Reply with quote  #2

1714168435
Report to moderator
1714168435
Hero Member
*
Offline Offline

Posts: 1714168435

View Profile Personal Message (Offline)

Ignore
1714168435
Reply with quote  #2

1714168435
Report to moderator
1714168435
Hero Member
*
Offline Offline

Posts: 1714168435

View Profile Personal Message (Offline)

Ignore
1714168435
Reply with quote  #2

1714168435
Report to moderator
Activity + Trust + Earned Merit == The Most Recognized Users on Bitcointalk
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714168435
Hero Member
*
Offline Offline

Posts: 1714168435

View Profile Personal Message (Offline)

Ignore
1714168435
Reply with quote  #2

1714168435
Report to moderator
1714168435
Hero Member
*
Offline Offline

Posts: 1714168435

View Profile Personal Message (Offline)

Ignore
1714168435
Reply with quote  #2

1714168435
Report to moderator
NotATether
Legendary
*
Offline Offline

Activity: 1582
Merit: 6687


bitcoincleanup.com / bitmixlist.org


View Profile WWW
November 20, 2020, 08:44:12 AM
 #2

Late answer because it took me some time to figure out how ZeroMQ works.

1 - The default high watermark for bitcoin core is 1000. If I understand correctly this means that up to 1000 raw blocks can be stored in memory (send buffer), which could translate to approximately 2GB, if there is a subscriber who forgets to read the notifications. This may lead to a crash due to insufficient memory in many machines, so it would be safer to set it to 100 and that would still allow the notification subscriber to be ~14h late in reading them, correct?

Well this will only be a problem if you subscribe to ZMQ notifications directly because there are RPC calls that can fetch the blocks that you missed. So ZMQ won't send you the missing blocks but you can pass the last block hash you received from ZMQ to the getblock RPC call, it'll give you the height which you can increment and pass to getblockhash to get the others, stopping once you reach the block hash in the new ZMQ notification.

2 - It seems like raw block notifications are not published twice for the same height. An issue was just published on the bitcoin repository, but there are no comments yet: https://github.com/bitcoin/bitcoin/issues/20376 . Is this expected behavior or a bug?

You got the answer in the Github issue, if the new block at that height was already in the active chain, then there is no notification, but you can subscribe to the "sequence" topic as a workaround to get notified again for the height, discerning the information from block connections and disconnection messages.

You currently need a node running a development version of bitcoin core to use "sequence" as a new release hasn't been made since the PR that merged it.

https://bitcoin.stackexchange.com/questions/91678/is-it-possible-to-catch-up-missing-blocks-with-zeromq-if-script-was-down-for-a-w
https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md#remarks

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!