Bitcoin Forum
December 10, 2018, 10:08:54 AM *
News: Latest Bitcoin Core release: 0.17.0 [Torrent].
 
   Home   Help Search Login Register More  
Poll
Question: How long did it take to sync bitcoin core block data?
Few hours - 0 (0%)
Less than a day - 1 (14.3%)
Few days - 3 (42.9%)
Week - 2 (28.6%)
Few weeks - 1 (14.3%)
Total Voters: 7

Pages: [1]
  Print  
Author Topic: [POLL] How long to sync on an initial core install  (Read 118 times)
d4n13
Full Member
***
Offline Offline

Activity: 210
Merit: 101


“Create Your Decentralized Life”


View Profile
January 09, 2018, 02:30:13 AM
Merited by OgNasty (1)
 #1

Performing a fresh install on a very wimpy computer.  It's syncing now and will look to take between 1-2 weeks to sync.  I know this is way outside the bell curve and I'm far far below the minimum install requirements, so... Wanted to know how long a propper install takes to perform.  I'll give the form for the reply data, if anyone cares to fill it out:

Quote
Rules: Fresh installs with no pre seeded block data or levelDB data

  • Duration: Number of hours it took to sync.
  • Date: When was the sync started?
  • Version: Bitcoind or Bitcoin-QT version?
  • Headless: Running Bitcoind or Bitcoin-QT?
  • Pruning: Pruning enabled?  If so, what argument was given?
  • dbcache: dbcache limit enabled?  If so, what argument was given?
  • maxmempool: maxmempool limit enabled?  If so, what argument was given?
  • bandwidth: What network bandwidth was the system capable of?
  • DiskIO: What disk bandwidth was the system capable of?
  • CPU: What CPU type / count?
  • Memory: How much memory (RAM) was installed?
  • OS: What OS was installed?

Here's my my config (I know... its wimpy and well below the minimum requirements)

  • Duration: 336 hours (projected), aka: 14 days, aka: 2 weeks
  • Date: Jan-6 2018
  • Version: v0.15.1
  • Headless: Yes, bitcoind
  • Pruning: -prune=20480
  • dbcache: -dbcache=155
  • maxmempool: -maxmempool=5
  • bandwidth: 500 Mbit/s
  • DiskIO: 100 MB/s
  • CPU: 0.5 vCPU, GCE f1-micro VM with 0.5 vCPU (shared CPU)
  • Memory: 0.6 GB
  • OS: Unbuntu 17.10 (artful)

1544436534
Hero Member
*
Offline Offline

Posts: 1544436534

View Profile Personal Message (Offline)

Ignore
1544436534
Reply with quote  #2

1544436534
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1544436534
Hero Member
*
Offline Offline

Posts: 1544436534

View Profile Personal Message (Offline)

Ignore
1544436534
Reply with quote  #2

1544436534
Report to moderator
1544436534
Hero Member
*
Offline Offline

Posts: 1544436534

View Profile Personal Message (Offline)

Ignore
1544436534
Reply with quote  #2

1544436534
Report to moderator
jnano
Member
**
Offline Offline

Activity: 241
Merit: 18


View Profile
January 09, 2018, 11:33:45 PM
 #2

A different OS, and you practically have no RAM, but if you could spare more RAM, see if putting the chainstate directory on a RAM drive helps:
https://bitcointalk.org/index.php?topic=2291857.0
d4n13
Full Member
***
Offline Offline

Activity: 210
Merit: 101


“Create Your Decentralized Life”


View Profile
January 15, 2018, 07:00:51 AM
 #3

How big is the chainstate database?  I know it changes everyday, but I can't get a clear answer.

I hear the uncompressed UTXO database (as of today) is around 4 GB, and that the chainstate (compressed) is around 2 GB.  Does this sound right?

LoyceV
Legendary
*
Online Online

Activity: 1330
Merit: 2327


Self-made Legendary!


View Profile WWW
January 15, 2018, 09:22:36 AM
 #4

I've tested this a while back:

    Duration: maybe 20 hours
    Date: say 2 months ago
    Version: Bitcoin Core v0.15.1.0
    Headless: Running Bitcoin Core
    Pruning: Pruning enabled for the test, -prune=550
    dbcache: unchanged default options
    maxmempool: unchanged default options
    bandwidth: standards DSL, kinda irrelevant for this test
    DiskIO: very high, I saved this to a ramdrive (/run/shm)
    CPU: 3 year old Intel i3 laptop
    Memory: 12 GB, up to 50%
    OS: 64 bit Linux

In my experience, there's 2 bottlenecks for Bitcoin Core: total RAM for file cache, and access time to your disk.

For practical use, I now have my chainstate-directory on an ssd, and my blocks-directory on a (much bigger) hdd.
I assume you're using cloud hosting considering your specs. If you want this to be faster, you could add more ram/cpu during syncing.

jnano
Member
**
Offline Offline

Activity: 241
Merit: 18


View Profile
January 15, 2018, 08:04:41 PM
 #5

How big is the chainstate database?  I know it changes everyday, but I can't get a clear answer.
With Bitcoin Core's native storage format, mine is about 3,780,000,000 bytes as of a few days ago.

Trying solid LZMA compression on 2% of the data, I got a reduction of 25% with solid compression, and 19% without. With ZIP compression (32KB blocks) I get 15%. Is there any RAMdrive software with a configurable compression block size?

d4n13
Full Member
***
Offline Offline

Activity: 210
Merit: 101


“Create Your Decentralized Life”


View Profile
January 15, 2018, 08:37:40 PM
 #6

...
    Duration: maybe 20 hours
...
    Pruning: Pruning enabled for the test, -prune=550
...
    DiskIO: very high, I saved this to a ramdrive (/run/shm)
...
    OS: 64 bit Linux
...

Yep... saw your post on /run/shm and tried the same.  Spun up a full size server (2xCore 13GB mem) and sync too 20.16 hrs out of ramdisk.

achow101
Moderator
Legendary
*
Offline Offline

Activity: 1610
Merit: 1789


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
January 16, 2018, 06:57:28 AM
 #7

It will sync faster and have less disk I/O if you increase the dbcache. This requires more RAM though. Raising the dbcache to anything larger than 8000 MB (-dbcache=8000 is large enough to store the whole thing in memory and then flush right at the end so there's significantly less disk I/O then. Of course you will need plenty of RAM in order to set the dbcache to that high.

jnano
Member
**
Offline Offline

Activity: 241
Merit: 18


View Profile
January 16, 2018, 05:48:19 PM
 #8

Compared with version 0.15.0.1, did 0.15.1 change in any way the caching or disk I/O logic?
d4n13
Full Member
***
Offline Offline

Activity: 210
Merit: 101


“Create Your Decentralized Life”


View Profile
January 20, 2018, 06:08:14 AM
 #9

It will sync faster and have less disk I/O if you increase the dbcache. This requires more RAM though. Raising the dbcache to anything larger than 8000 MB (-dbcache=8000 is large enough to store the whole thing in memory and then flush right at the end so there's significantly less disk I/O then. Of course you will need plenty of RAM in order to set the dbcache to that high.

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?

achow101
Moderator
Legendary
*
Offline Offline

Activity: 1610
Merit: 1789


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
January 21, 2018, 03:27:25 AM
 #10

Compared with version 0.15.0.1, did 0.15.1 change in any way the caching or disk I/O logic?
No.

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?
That's the on disk sized which is compressed. The in memory size is much larger, ~7-8GB

d4n13
Full Member
***
Offline Offline

Activity: 210
Merit: 101


“Create Your Decentralized Life”


View Profile
January 23, 2018, 10:20:16 PM
 #11

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?
That's the on disk sized which is compressed. The in memory size is much larger, ~7-8GB
[/quote]

So as a WAG, I'll just double what the on-disk size reported from the report.  Won't be precise, but better than nothing I suppose.

Thx

jrian
Jr. Member
*
Offline Offline

Activity: 39
Merit: 4


View Profile
January 31, 2018, 11:18:57 AM
 #12

    Duration: 8-12 hours
    Date: 2017-2018
    Version: latest
    Headless: Yes, bitcoind
    Pruning: no
    bandwidth: 1000 Mbit/s
    CPU: XEONs
    Memory: 8-128
    OS: Unbuntu/Debian
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!