Bitcoin Forum
May 07, 2024, 10:22:20 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Blockchain Erratic Sync Speed - Why?  (Read 1424 times)
Cubic Earth (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1018



View Profile
October 02, 2014, 06:24:13 PM
 #1

The blockchain synchronizes quickly when Bitcoin Core is first started, but after some time, maybe an hour or so, it slows to a crawl.  I can’t find the bottle neck.

This behavior is occurring under good testing conditions.


Computer: MacBook Pro 3.1, 2.4 Ghz Core 2 Duo, 4 GB RAM, 250GB SSD

OS: Mavericks 10.9.5 - Clean Install.  Bitcoin Core 9.3 is the only non-packaged software.

Internet: 25 / 5 mbps cable (comcast).

All settings are essentially stock.  I turned on the software firewall, then turned it back off.  Filevault 2 has been activated.

Description:  I did a clean install of the OS last night on a brand new hard drive.  I encrypted the drive and immediately proceeded to download Bitcoin Core 9.3 to test it out, and to test out the current blockchain total download time.

The client has been running for about 7 hours now, and currently has 16 inbound and 8 outbound connections, but it seems like nothing is happening. The block count in the debug window hasn’t budged in 5 minutes

CPU is 80% idle.  RAM is only about 50% full.  Network - 20 KB /s.

Then, as I am writing this, the data rate flowing into the computer surges to ~1.0 MB/s, the CPU starts processing, and the block count is flying upwards.  It is currently requesting blocks from September, 2011 era.

Now, just a couple of minutes and 1,000 blocks later, the computer is receiving only ~50 KB/s, the CPU is idle again, and the block count is not increasing, not even slowly.  It is stuck at block 143,882.

Next I quit and restart Bitcoin Core.  Almost instantly blocks start processing again, and downloading at 500 - 1,000 KB/s.  Within a minute or two, it has 1 inbound connection and 8 outbound.

I have observed the same thing happening on other Macs running 10.9.5.

Does anyone have a good explanation for this behavior?  It almost seems like my client just happens to be connected to 8 peers, none of which are willing to share any blockchain data.  If that is the case, I am surprised it wouldn’t do a more active search for sources of fast blocks.
1715120540
Hero Member
*
Offline Offline

Posts: 1715120540

View Profile Personal Message (Offline)

Ignore
1715120540
Reply with quote  #2

1715120540
Report to moderator
1715120540
Hero Member
*
Offline Offline

Posts: 1715120540

View Profile Personal Message (Offline)

Ignore
1715120540
Reply with quote  #2

1715120540
Report to moderator
"Bitcoin: mining our own business since 2009" -- Pieter Wuille
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1715120540
Hero Member
*
Offline Offline

Posts: 1715120540

View Profile Personal Message (Offline)

Ignore
1715120540
Reply with quote  #2

1715120540
Report to moderator
1715120540
Hero Member
*
Offline Offline

Posts: 1715120540

View Profile Personal Message (Offline)

Ignore
1715120540
Reply with quote  #2

1715120540
Report to moderator
dserrano5
Legendary
*
Offline Offline

Activity: 1974
Merit: 1029



View Profile
October 02, 2014, 06:33:00 PM
 #2

Recently I synchronized a new node from scratch and observed a similar behaviour. The debug.log was filled with entries "UpdateTip" then suddenly it stopped and some time later it started again. My theory (out of thin air) is that it stops when the connection to the node serving blocks to us is interrupted. I don't know what condition makes the daemon restart the process again.
Cubic Earth (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1018



View Profile
October 02, 2014, 10:59:19 PM
 #3

I controlled some variables and the problem still persisted.

I set still client that is still bootstrapping to connect=192.168.1.100, that being the static local address of another computer running a full node.  It certainly pulled in the blocks fast, at about 1 MB/s.  That only for three minutes or so before the sync ground to a halt.  I restarted the program, and sure enough, another solid run of data transfer, maybe for six or seven minutes, then it stopped.

Here is a graph of my CPU.  It shows about 30 minutes of history.  Bitcoin was the only thing running.  I quit the program twice and restarted twice over the course of the graph.  It let the computer idle for a minute in between.  Starting from the left, about 20% into the graph, there is a short lull, followed by the biggest chunk of CPU thus far.  That was when I restarted the client.  Then there is a break in CPU effort, after the client has started, but before it was crunching blocks.  Then there is a period of block processing, before it reverts to a very low intensity state, with not much syncing.

After ~8 minutes of mostly idling, I restated again (the middle point of the graph).  The same pattern repeats, though this time it processed blocks for ~7 minutes or so.




I even unplugged the WAN cable.  Each client showed one connection, one outbound, one inbound.  They were just talking to each other.  Again, the syncing went fast for a few minutes, then mostly stopped.

I have seen basically the same behavior just running the client stock, connecting to larger bitcoin network, and running it modified, to only connect to local hosts.
cr1776
Legendary
*
Offline Offline

Activity: 4032
Merit: 1301


View Profile
October 02, 2014, 11:35:39 PM
 #4

Do you have "app nap" turned off? (In Finder, do a "get info" and then make sure you have prevent app nap checked).
grue
Legendary
*
Offline Offline

Activity: 2058
Merit: 1431



View Profile
October 02, 2014, 11:43:12 PM
 #5

I'm guessing slow peer (bitcoin only syncs with one peer at a time, and doesn't always chooses the fastest) or busy HDD

It is pitch black. You are likely to be eaten by a grue.

Adblock for annoying signature ads | Enhanced Merit UI
Cubic Earth (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1018



View Profile
October 03, 2014, 12:15:52 AM
 #6

Do you have "app nap" turned off? (In Finder, do a "get info" and then make sure you have prevent app nap checked).

App nap was not disabled.  But now it is.  Thanks.  I was plugged in the whole time, it would be slightly strange for process to be throttled in such circumstances, but I suppose there is the issue of heat buildup, and maintaining UI responsiveness in the foreground.  I'll see if it makes a difference.

I'm guessing slow peer (bitcoin only syncs with one peer at a time, and doesn't always chooses the fastest) or busy HDD

I guessed that as well.  That is why I isolated my systems form the internet, and made them connect to each other.  Both have fast SSDs.  The bottle neck really should have been the CPU in the bootstrapping machine.

Cubic Earth (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1018



View Profile
October 03, 2014, 05:10:00 AM
 #7

Disabling AppNap seems to have solved the problem.  Thanks for the tip.

With AppNap throttling enabled, bootstrapping would have taken weeks.

I am guessing there is a way for software to tell the OS not to throttle it with AppNap.  I could see not including such an 'extra' in Bitcoin Core for security and stability reasons.  Making it clear to the user that AppNap can severely limit the bootstrapping process I think would be a good idea.  not sure where the warning should go.
tl121
Sr. Member
****
Offline Offline

Activity: 278
Merit: 252


View Profile
October 09, 2014, 12:07:49 AM
 #8

I'm guessing slow peer (bitcoin only syncs with one peer at a time, and doesn't always chooses the fastest) or busy HDD

Most likely it's a slow peer.  If this happens it may last for hours if the connection to the slow peer stays up.  You can "kick it" to get it going faster by shutting down Bitcoin-Qt and restarting it.  This has usually worked for me if I have been off-line for a few days and am trying to catch up.

If you are downloading from scratch you might try the bootstrap.dat torrent, particularly if you already have experience with torrents and have a bittorrent client installed on your computer.  See https://bitcointalk.org/index.php?topic=145386.0
cr1776
Legendary
*
Offline Offline

Activity: 4032
Merit: 1301


View Profile
October 09, 2014, 01:32:58 AM
Last edit: October 09, 2014, 02:19:24 AM by cr1776
 #9

Glad that helped.

I believe it can be programmatically disabled.  I will look into that.

Edit:
I went to look at the code and it appears someone added some os x specific code in the plist to disable app nap a few days ago. So perhaps this will resolve it in a future update.


Disabling AppNap seems to have solved the problem.  Thanks for the tip.

With AppNap throttling enabled, bootstrapping would have taken weeks.

I am guessing there is a way for software to tell the OS not to throttle it with AppNap.  I could see not including such an 'extra' in Bitcoin Core for security and stability reasons.  Making it clear to the user that AppNap can severely limit the bootstrapping process I think would be a good idea.  not sure where the warning should go.
Newar
Legendary
*
Offline Offline

Activity: 1358
Merit: 1001


https://gliph.me/hUF


View Profile
October 09, 2014, 04:50:35 PM
 #10

[...] You can "kick it" to get it going faster by shutting down Bitcoin-Qt and restarting it.  [...]

From my experience you also have to delete peers.dat for a successful "kick".

OTC rating | GPG keyid 1DC91318EE785FDE | Gliph: lightning bicycle tree music | Mycelium, a swift & secure Bitcoin client for Android | LocalBitcoins
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!