bitcoin core not making outbound connections or taking hours to make 2 or 3 outb

[73blazer]

I've had a full node running for years, in and out (8333 open).
Also have a testnet full node running as well (18333 open).

Last weekend my mainchain full node stop syncing and kept saying orphaned block in the debug log.
I stopped it, starting it with rescan, no sale. Stopped it, deleted the last few blocks in it's directory and re-started with reindex.
That took daaaaaayyyyssss, until today actually, over 7 days.
So it finally synced, and I noticed during the last few hours of sync, it only had 3 outbound connections.
I tried picking some random nodes from bitnodes and manually adding them via addnode command, all I get is this:

socket recv error Connection reset by peer (104)


2018-02-04 16:22:09 socket recv error Connection reset by peer (104)
2018-02-04 16:22:09 socket recv error Connection reset by peer (104)
2018-02-04 16:22:09 connect() to [2001:0:9d38:6abd:281e:3ae8:af4a:9717]:8333 failed: Network is unreachable (101)
2018-02-04 16:22:09 socket recv error Connection reset by peer (104)
2018-02-04 16:22:09 socket recv error Connection reset by peer (104)
2018-02-04 16:22:09 socket recv error Connection reset by peer (104)
 or
2018-02-04 16:20:17 socket recv error Connection reset by peer (104)
2018-02-04 16:20:17 socket recv error Connection reset by peer (104)
2018-02-04 16:20:18 connect() to 51.15.169.94:8333 failed after select(): Connection refused (111)
2018-02-04 16:20:18 socket recv error Connection reset by peer (104)
2018-02-04 16:20:19 socket recv error Connection reset by peer (104)


I could manually telnet to those ip's I chose port 8333 and I get a connection

I did have a gekko stick miner that runs solo against this node (lottery) . I suspect it thinks it found something because the first messages were mining against orphaned block. It had been running for over a year.  I stopped this before I restarted in rescan/reindex mode and it hasn't been on since.
I shut my node down and restarted again after sync and now it won't make any outbound connections. it's almost as if my node has been marked as banned or something??
Is there a way to tell if my node is banned? Or why it cannot make any outbound conns? I suspect after time it will pick up those 3 I had after sync and before I shut it down, but..what the heck is happening here? I've no idea why it suddenly stopped/hung on a certain block and now it takes hours to make an outbound connection.
I've mode no changes on my firewall and the  testnet chain node is running just fine the whole time.
It's core 0.15.1

[73blazer]

I tired removing peers.dat and restarting.
Doesn't seem to help.
I just watch the inbound/outbound numbers flash from 0 to 1 and back to 0 .
What can I do?!?!?!?!?

[achow101]

Can you post the entirety of your debug.log file?

[Jet Cash]

I was about to start a thread about this, as I've got a similar problem.

I've got a node running under Windows 10. The node is v,0.15.1 and it is uncustomised. I run it over a variety of connections - public WiFi, tethered to a mobile, and over broadband in a private house. All give much the same result. I can only make outbound connections, and I often have trouble finding peers. Those that are found can have ping rates varying between low nn ms and high nn,nnn ms , and often I have to sync using just 2 or 3 peers. When I look at the traffic activity I notice that I have long gaps with no activity.

I've been wondering if the problem was caused by a recent Windows upgrade which seems to have messed around with its firewall. I'm thinking of reviving my old Linux node, and moving to that. Do you think that will make much difference?

[achow101]

I've been wondering if the problem was caused by a recent Windows upgrade which seems to have messed around with its firewall.

It's possible that changes to Windows Defender has caused some issues. IIRC Windows Defender will scan your network traffic for anything malicious and it may be confusing Bitcoin traffic for malicious traffic and block it thus causing connection loss.

[73blazer]

Can you post the entirety of your debug.log file?

Here's the log since I restarted after it did manage to sync this morning (at that time it had 3 outbound and it had been running for a week). Then I stopped, moved old debug.log to backup and restarted. It's been several hours and still no outbound connections. I did stop it once in here and removed peers.dat and restarted. That was about an hour ago.
http://snyderworld.org/misc/debug.log

BTW, mines running on ubuntu.

[achow101]

Here's the log since I restarted after it did manage to sync this morning (at that time it had 3 outbound and it had been running for a week). Then I stopped, moved old debug.log to backup and restarted. It's been several hours and still no outbound connections. I did stop it once in here and removed peers.dat and restarted. That was about an hour ago.
http://snyderworld.org/misc/debug.log

Do you have any logs of when this began happening?

Try connecting to my node: 73.191.37.221

[73blazer]

Here's the log since I restarted after it did manage to sync this morning (at that time it had 3 outbound and it had been running for a week). Then I stopped, moved old debug.log to backup and restarted. It's been several hours and still no outbound connections. I did stop it once in here and removed peers.dat and restarted. That was about an hour ago.
http://snyderworld.org/misc/debug.log

Do you have any logs of when this began happening?

Try connecting to my node: 73.191.37.221

It appears not, It looks as though it starts a new debug.log every time you start, so I lost the one where it actually started happening.
I do remember looking at it and it was just spewing messages to the effect that I was mining against and orpahned block. I shut the miner off and it was sitting at 0 outbound and 0 inbound connections. I then restarted it in rescan, 2 days later, nothing, restarted it in reindex after removing a few blocks from the block dir and it took until this morning to sync and it had only 3 outbound. I shut it off then and tried to restart normally, 3-4hours now, no outbound conns.

I tried connecting to your node. No sale. Mine is 71.13.92.62 do I show up in your banned list? I get the feeling that's what's happening. I'm almost inclined to just shut it off for a week and try restarting it then.

I can connect to it via telnet

Code:

root@impalaubu:/bitcoin/data# telnet 73.191.37.221 8333
Trying 73.191.37.221...
Connected to 73.191.37.221.
Escape character is '^]'.
hello
^D
Connection closed by foreign host.
root@impalaubu:/bitcoin/data#

(I dunno the protocol so no idea what to type, but it connects and waits for me to type ...)
So my firewall or my isp is not blocking.

[achow101]

It appears not, It looks as though it starts a new debug.log every time you start, so I lost the one where it actually started happening.

If the log is larger than 10 MB, it will truncate it at 10 MB.

I tried connecting to your node. No sale. Mine is 71.13.92.62

Can you try again? I enabled more verbose logging on my node so I can see what's actually going on (I forgot to enable this earlier).

I can connect to it via telnet

Interesting..

Edit: Connecting with telnet does not necessarily mean that your node should be able to connect. Your ISP maybe doing some packet inspection and dropping the packets when it detects that it is related to Bitcoin, but letting packets for telnet through.

[73blazer]

Can you try again? I enabled more verbose logging on my node so I can see what's actually going on (I forgot to enable this earlier).

I tried again, a couple times.
i see it appear for a split second as a peer then it goes away.

I'm using: addnode 73.191.37.221:8333 onetry   ... my understanding is that will try to connect once, if it can't it doesn't retain it as a peer to keep trying later.

[achow101]

I tried again, a couple times.
i see it appear for a split second as a peer then it goes away.

This is what I see in the log:

Code:

2018-02-04 19:00:29 Added connection to 71.13.92.62:4829 peer=53
2018-02-04 19:00:29 connection from 71.13.92.62:4829 accepted
2018-02-04 19:00:29 socket recv error Connection reset by peer (104)
2018-02-04 19:00:29 disconnecting peer=53
2018-02-04 19:00:29 Cleared nodestate for peer=53

This means that the disconnection is happening either on your machine or somewhere in between your machine and mine. I suspect it may be your ISP.

You could also try reinstalling Bitcoin Core but I don't think that will do much.

I'm using: addnode 73.191.37.221:8333 onetry   ... my understanding is that will try to connect once, if it can't it doesn't retain it as a peer to keep trying later.

That's correct.

[Jet Cash]

I've been wondering if the problem was caused by a recent Windows upgrade which seems to have messed around with its firewall.

It's possible that changes to Windows Defender has caused some issues. IIRC Windows Defender will scan your network traffic for anything malicious and it may be confusing Bitcoin traffic for malicious traffic and block it thus causing connection loss.

That's what I suspected. I think I'll wait until I'm using Linux again before any more investigation.

[73blazer]

This is what I see in the log:

Code:

2018-02-04 19:00:29 Added connection to 71.13.92.62:4829 peer=53
2018-02-04 19:00:29 connection from 71.13.92.62:4829 accepted
2018-02-04 19:00:29 socket recv error Connection reset by peer (104)
2018-02-04 19:00:29 disconnecting peer=53
2018-02-04 19:00:29 Cleared nodestate for peer=53

This means that the disconnection is happening either on your machine or somewhere in between your machine and mine. I suspect it may be your ISP.

You could also try reinstalling Bitcoin Core but I don't think that will do much.

This is what I see after restarting with some more detailed log options:

Code:

root@impalaubu:/bitcoin/data# cat debug.log | egrep '(191.37.221|peer=220)'
2018-02-04 19:19:41 trying connection 73.191.37.221:8333 lastseen=0.0hrs
2018-02-04 19:19:41 Added connection to 73.191.37.221:8333 peer=220
2018-02-04 19:19:41 sending version (102 bytes) peer=220
2018-02-04 19:19:41 send version message: version 70015, blocks=507613, us=[::]:0, them=73.191.37.221:8333, peer=220
2018-02-04 19:19:41 disconnecting peer=220
2018-02-04 19:19:41 Cleared nodestate for peer=220

Well,I appreciate your help. I was actually hoping I was banned. Haha. I'll try and debug it further on myfirewall and see what the heck is happening, I may also try to reset the ISP's modem/router but it's "supposed" to just be a passthrough. It's a business class line and has been rock solid for many years with no blocking anything that I ever noticed, I've got many VPN's to customers, my own voip, email, webservices and some applications customers use with lots of weirdo ports and alot of bandwidth at times, never has it been an issue.

[73blazer]

Oh, f-me. I found the issue. MY firewall was dropping it, considering it as an intrusion all of a sudden for some reason. I could see this only after turning on some more detailed loggin on my firewall.
Hmmmm...

Code:

*Packet number: 1841*
Header Values:
 Bytes captured: 90, Actual Bytes on the wire: 90
Packet Info(Time:02/04/2018 14:46:45.816):
 in:X0*(interface), out:--, DROPPED, Drop Code: 126(IDP detection DROP_IP_IDP_RESET_CONNECTION), Module Id: 25(network), (Ref.Id: _7160_txGsIboemfJqQlu), 1:3)
Ethernet Header
 Ether Type: IP(0x800), Src=[00:0c:29:77:dc:aa], Dst=[18:b1:69:16:06:d4]
IP Packet Header
 IP Type: TCP(0x6), Src=[192.168.168.81], Dst=[73.191.37.221]
TCP Packet Header
 TCP Flags = [ACK,PSH,], Src=[39796], Dst=[8333], Checksum=0x3442
Application Header
 Not Known
Value:[0]
Hex and ASCII dump of the packet:
 18b16916 06d4000c 2977dcaa 08004500 004cdeea 40004006 *..i.....)w....E..L..@.@.*
 832bc0a8 a85149bf 25dd9b74 208dfa48 168ab683 1a108018 *.+...QI.%..t ..H........*
 00e53442 00000101 080a07f0 c86e7797 9e44f9be b4d97665 *..4B.........nw..D....ve*
 7273696f 6e000000 00006600 0000dc9b 33bf              *rsion.....f.....3.      *

I made an exception for any src port where destination port=8333 and it instantly started making outbounds again.
Holy cow. Now I just need to dig a bit further and find which signature in the intrusion detection signatures to disable as I think that will be better than an exclusion rule. Those signatures update automatically (like virus definitions) from the firewall manufacture (sonicwall in this case).

[DrSha]

Same issue here recent download core latest version took over a week to download blockchain always limited peers connecting to pass the blockchain.
Should be connecting to a few I get 2 peers maximum with ridiculous ping times.