Bitcoin Node Block Propagation Delays / Connectivity Issues

[BitcoinCanSaveUsAll]

  Hello all, my bitcoin node is experiencing extreme block propagation delays. On average 1 to 4 blocks behind what blockstream dot info shows. I'm also noticing that my node has also been unable to maintain connections very long with many other nodes as it used to.

Some notable items of the issue I’ve been seeing are:

\- According to bitnodes dot io  my node is only intermittently reachable.

\- These symptoms / results have been persistent both from behind my router using IPV4 as well as directly connected to cable modem using IPV6 connected to the RJ45 port.

\- I've done virus checks, reinstalled the WIFI card in device manager, disabled my anti-virus

\- I've even downgraded bitcoin core to several versions thinking it may have been a bug that I didn't notice initially in the most recent release but no luck there either (down to V 18.x) with reboots in between.

\- I've performed  SCF /scannow (Windows 10 machine) and repaired some corrupted files

\- uninstalled avast antivirus

\- sidestepped the router and plugged directly into the cable modem

\- Scanned laptop with windows defender after updating definitions.

I've had my node running for well over a year and a half and just noticed this issue this week (although it may have been going on for the last few weeks).  I'm really starting to believe it's my ISP fighting bitcoin or maybe even something worse.

Either way I'm downloading another node on another machine and will test it to rule out my hardware on my machine that has been having the issue and hope to rule this out later today. In the meantime if any of you who are also running a node can confirm that you are receiving blocks at the normal speed it would be appreciated. Note that when doing so don’t just rely on the time stamp in the console view of the bitcon core info section! Be sure to test it by either using the GETBLOCKCOUNT command in the console and/or reviewing the historical log of your node to see what time your node actually received the updated tip (most recent block hash). The console window on my node shows the correct timestamps but the node itself isn’t receiving the blocks until sometimes 30+ minutes after they’re propagated to the network.

Thanks in advance for your feedback!

[ranochigo]

Make sure you allow incoming connection and configure your router/firewall properly

It isn't necessary to allow incoming and I doubt OP would fix his issue with this. As long as there are peers, OP's client should be functioning fine.

It certainly isn't normal to have your client getting new blocks so long after it starts getting propagated (propagation should only take several minutes at the worst case scenario). Have you done any hardware diagnostics? Your debug.log should provide some clues at least.

[BitcoinCanSaveUsAll]

Hello everyone and thank you for replying to my thread. I just wanted to try to clear up a few different items that I feel may not have been addressed in my original post. Firstly I should point out that I had previously been running this node on the same device, a Windows laptop or close to if not slightly more than two years time. During that time I had port forwarding enabled Port 8333 and everything had been working just fine where I would get many simultaneous connections sometimes more than 80 and I never had any issues with block propagation whatsoever. Unfortunately that all changed within the last 2 weeks or so when I started to notice that the inbound connections to my node were attempting to be established and then being disconnected repeatedly.

After extensive troubleshooting and installing a full copy of Bitcoin core on a different laptop, I was having the same issues. I even went so far as to call the ISP who told me that they were not blocking that Port although I don't even know if they knew with a port was at the level one support at Comcast which was very frustrating because they refused to get me over to the next level of support. Either way I'm starting to suspect that my ISP, Comcast, is using some type of traffic shaping system to disallow Communications to my node. I should also point out that when I disable the port forwarding on my router and only allow one-way communication that the block propagation operates normal. At least that's what it appears to be so far. I think I really do need to enable the tor system to run the node because it appears that Comcast is actively fighting inbound connections to my node. 

If anyone has a step-by-step guide for enabling Bitcoin core node on windows 10 over tor I would really appreciate it. This is all new to me and I think that it may be a bit tricky so any pointers there would be appreciated. Also if you have any other suggestions or ideas as to what might have caused this issue please let me know. The last thing I would ask is that if there are any other folks it here running nodes on US ISPs to check to see if when running a full node accepting inbound and outbound connections if you two are experiencing an issue.

Thanks all!

[bob123]

If anyone has a step-by-step guide for enabling Bitcoin core node on windows 10 over tor I would really appreciate it.

You can find a small guide here: https://en.bitcoin.it/wiki/Tor.
If you want to accept incoming connections via tor (what i guess you do want, otherwise you wouldn't ask for this but just use it without incoming connections), you need to run a hidden service.
Information on how to do this together with core on linux, can be found here: https://en.bitcoin.it/wiki/Setting_up_a_Tor_hidden_service.

It will be slightly different setting this up on windows (e.g. you don't need to figure out which user is running bitcoind).

I guess most people use core/tor on linux. But those two sites should be pointing in the right direction.

[DaveF]

Some things to check.

How is the connectivity otherwise? Speedtests? Pings (long term not just 4 pings but leave it running for a bit and see if any drop)

Can you run a traceroute / mtr to other nodes that you are connected to? Are there any glaring issues?

What plan are you on with comcast? They supposedly took off data caps but that does not mean that they are not playing some other stupid throttling game elsewhere.

Can you plug the node into another connection someplace else / with another service? 4G hotspot for an hour? If it still has issues someplace else its the node, if it goes away it's your connection.

-Dave

[BitcoinCanSaveUsAll]

Hi Bob,

First of all that is my new favorite avatar you have!  Secondly thanks for replying. I have been able to get my tor setup but as you mentioned it only seems to work in the one direction as I don't currently have a hidden service up and running.  I think I will try to look into that deeper as it appears that I will need to do that one way or the other.

Hi Dave,

To answer your questions:

1) I haven't noticed any issues with my normal browsing through Comcast. 


2) My plan currently doesn't have any limits.  As you said they have disabled them for Corona virus although even with them before it was never an issue for me.

3) I have loaded bitcoin core node on another computer completely on the same WIFI and it too experienced the same symptoms.

All,

Here's some more interesting pieces I've come across going down this rabbit hole. After posting this initially I realized that my VPN provider (PIA) allows for port forwarding on some gateways so I've gone ahead and enabled it and opened up port forwarding on my router with those forwarded ports and added the port= flag in the bitcoin.conf file and it seems to be working fine that way so far.

That all said however I cannot seem to find a bitcoin wallet app that I can use on my android devices in which I can specify my IP address of my node and this port is non standard port from the VPN (i.e. not 8333).  I did try using  Green wallet but for some strange reason every time I connect to my node it flags the wallets as misbehaving nodes.  I wonder why that would be? I've tried this from android wallets behind several different VPN gateways and even without using a VPN and same results every single time.  Node misbehaving.

That all said, if you have any bitcoin wallet apps which you can recommend other than Green that are open source and allow you to specify your own node to maintain your own sovereignty, I'd love to know of it.  Keep in mind that in addition to specifying your own IP address I would need to tell it the port as it's not 8333.  Thanks again all for your feedback!   

[DaveF]

Not sure what you are looking to do with a mobile wallet, but most (all?) mobile wallets you can't connect to a node directly you have to connect to a SPV (electrum / electrumX) server.

I don't know your budget / tech experience but I have been recommending https://mynodebtc.com/ even the free version has an electrum server.

-Dave
 

[ranochigo]

1) I haven't noticed any issues with my normal browsing through Comcast. 

2) My plan currently doesn't have any limits.  As you said they have disabled them for Corona virus although even with them before it was never an issue for me.

3) I have loaded bitcoin core node on another computer completely on the same WIFI and it too experienced the same symptoms.

My guess is that Comcast has been selectively blocking connections to try to conserve bandwidth for other users. Throttling is not uncommon for applications that are somewhat bandwidth intensive. That being said, could you share your debug.log? This would be able to confirm the theory.

That all said however I cannot seem to find a bitcoin wallet app that I can use on my android devices in which I can specify my IP address of my node and this port is non standard port from the VPN (i.e. not 8333).  I did try using  Green wallet but for some strange reason every time I connect to my node it flags the wallets as misbehaving nodes.  I wonder why that would be? I've tried this from android wallets behind several different VPN gateways and even without using a VPN and same results every single time.  Node misbehaving.

The problem is not with your VPN. Starting with Bitcoin Core 0.19, bloom filter has been disabled by the client by default. In Bitcoin.conf, insert the following line: peerbloomfilters=1

Restart your client and try connecting again.

[BitcoinCanSaveUsAll]

Hello.  Yes I had heard that was something that changed in a recent version with respect to peerbloomfilters so I did enable it as you said in the bitcoin.conf file to peerbloomfileters=1  and then went back over my PIA VPN with port forwarding. 

I then attempted to connect again through an android device with my node running the new bitcoin.conf file and yet my node still banded the IP address my device that I was trying to connect to it.  Very strange.  Either way I think there may be a few different issues going on here.  1) Comcast fighting my node and 2) something wrong with either Green wallet or bitcoin core nodes banning the wallet when specifying a SPV node IP address. 

Either way I think that there needs to be a lot more bitcoin wallets which allow you to run your own nodes as the validating source based on whatever port you want.  It's scary to think about how few options there are out there.

[BitcoinCanSaveUsAll]

Hi Dave, thanks for recommendation.  I have always used my node (for nearly 2 years now) running on my windows machine and connecting to it via the bitcoin wallet app on android by specifying my nodes IP address and using it to send and receive bitcoin completely independent of any other organization which is, in my opinion the whole point of bitcoin (independence from any institution, financial autonomy etc).  Something has changed within the last few days / weeks which put a stop to my ability to do this.  I know that this isn't something with the individual computer I was running (as it's also happening on a new install on a separate machine).  I also know it's not my wifi router as I sidestepped it and went directly to the cable modem with the same issues.  And to me that just leaves the ISP.  My concern is that if the ISPs here in the US are starting to fight our ability to run our own nodes / our own ability to have financial autonomy then they are directly attacking what bitcoin is supposed to be all about to begin with. I hope there are options out there to combat their attacks and if so that they can shared with the community.  I also hope they're not too complex as if so then this would dissuade folks from running their own nodes and delay / attenuate mass adoptions for bitcoin in my opinion.

[ranochigo]

I then attempted to connect again through an android device with my node running the new bitcoin.conf file and yet my node still banded the IP address my device that I was trying to connect to it.  Very strange.  Either way I think there may be a few different issues going on here.  1) Comcast fighting my node and 2) something wrong with either Green wallet or bitcoin core nodes banning the wallet when specifying a SPV node IP address. 

Hmm. It could be a problem with how your wallet interacts with the node. Misbehaving node is likely a problem with your client rather than your ISP. It could also be that your banscore is too high and the node would just automatically reject the wallet. To clarify, which android wallet are you using?

You can try whitelisting your IP by putting whitelist=(IP ADDRESS), replacing IP address with the public IP address on your phone.

Either way I think that there needs to be a lot more bitcoin wallets which allow you to run your own nodes as the validating source based on whatever port you want.  It's scary to think about how few options there are out there.

Electrum is probably the only effective one as most wallets are phasing out bloom filter at the moment and there isn't any defined standards yet.

[DaveF]

If you have your phone on the same lan as the PC can you connect to it on the private IP?

Can you tracerout from your phone to your public cable IP?

Or, can you borrow a neighbors / friends connection for a quick test? If it does the same thing in another location it points to your setup. If it works it points to your location.
If comcast was blocking it for everyone there would most likely be more people posting about it.

Other then that I have no idea.

-Dave

[BitcoinCanSaveUsAll]

Hello ranochigo,

Thank you for your reply.  It certainly could be an issue with the wallet.  It was actually the first time I've tried to use Green wallet so I don't really know if it had ever worked before. I had always used the regular bitcoin wallet for android previously and it worked flawlessly for years by simply putting my IP address into it while having my home router using port forwarding opening ports 8333.   Also I've never adjusted my ban score at all and I've never had to whitelist anything before so something has certainly changed recently here..I guess I'm just trying to figure out what.

As far as Electrum goes though, don't you have to use some kind of electrum personal server (i.e. not your own node? ) I guess I just don't understand why you wouldn't be able to use your own personal node.  I mean that's the point is how I thought it was supposed to be.  


Hi again Dave,

I've tested the following setups:  1) running my node with my legacy configuration which involved port forwarding on port 8333.  With this implementation I can no longer connect to my node using bitcoin wallet on android and blocks are taking an extremely long time to propagate to my node.  I'm also noticing a lot of disconnects. 

2) I've tested my node using port forwarding from my VPN provider (PIA).  With this configuration the block propagation issue seems to be resolved but the port that is forwarded to me is generated randomly (not 8333) and the bitcoin wallet app I had previously used does not allow you to specify a port..  I therefore tried to use the Green wallet which does get through the my node in this configuration but my node then immediately bans it regardless if my android device is also behind a VPN or not.  I've also tested trying to connect to this node with a friends android device using green but it's IP was also immediately banned.

3) my node running tor (without hidden service). This allows me to eliminate the block propagation delay but as far as I know I am unable to connect to the node without a hidden service running which I'm not sure exactly how to do. 

[ranochigo]

As far as Electrum goes though, don't you have to use some kind of electrum personal server (i.e. not your own node? ) I guess I just don't understand why you wouldn't be able to use your own personal node.  I mean that's the point is how I thought it was supposed to be.  

Electrum has a few advanced features that Bitcoin Core doesn't support. You can run a personal Electrum server which take information from your Bitcoin Core node.

-snip-

Could you paste a copy of your debug.log, it would really be a huge help in terms of identifying the exact issue. Just use pastebin.com and send the link here.

[BitcoinCanSaveUsAll]

Ah ok I didn't realize that about Electrum.  I will have to research that further.

Unfortunately my log file is nearly 10 M  and apparently the limits are 512 kilobytes for that site.  Do you have any other suggestions on how to share it?  Thanks

[ranochigo]

Ah ok I didn't realize that about Electrum.  I will have to research that further.

Unfortunately my log file is nearly 10 M  and apparently the limits are 512 kilobytes for that site.  Do you have any other suggestions on how to share it?  Thanks

You don't have to share the full logs. Just the lines for the previous few days (from when you're experiencing the problems) will do.

[BitcoinCanSaveUsAll]

There is a ton of data just over the past few days but I've been able to past from the 24th forward for you.  It can be found here:

https://pastebin.com/CQnLiW5C

Just know that this was with a lot of troubleshooting that involved various port specifications, bitcoin core downgrades and upgrades etc. 

[BitcoinCanSaveUsAll]

Hey guys..kind of embarrassing but I have to tell you.  I double checked the configuration file and realized I misspelled the peerbloomfilters=1 command initially.  After correcting it and then putting it behind my PIA VPN with port forwarding I was able to correct the Green wallet and it is now able to use my SPV with that configuration file. Thank you all for your input.  This resolves that issue but it still appears as though my ISP (Comcast / Xfinity) is actively blocking my node from communicating in the clear for inbound and outbound connections.  Either way I do appreciate all your help!