[POLL] How long to sync on an initial core install

[d4n13]

Performing a fresh install on a very wimpy computer.  It's syncing now and will look to take between 1-2 weeks to sync.  I know this is way outside the bell curve and I'm far far below the minimum install requirements, so... Wanted to know how long a propper install takes to perform.  I'll give the form for the reply data, if anyone cares to fill it out:

Rules: Fresh installs with no pre seeded block data or levelDB data

• Duration: Number of hours it took to sync.
• Date: When was the sync started?
• Version: Bitcoind or Bitcoin-QT version?
• Headless: Running Bitcoind or Bitcoin-QT?
• Pruning: Pruning enabled?  If so, what argument was given?
• dbcache: dbcache limit enabled?  If so, what argument was given?
• maxmempool: maxmempool limit enabled?  If so, what argument was given?
• bandwidth: What network bandwidth was the system capable of?
• DiskIO: What disk bandwidth was the system capable of?
• CPU: What CPU type / count?
• Memory: How much memory (RAM) was installed?
• OS: What OS was installed?

Here's my my config (I know... its wimpy and well below the minimum requirements)

• Duration: 336 hours (projected), aka: 14 days, aka: 2 weeks
• Date: Jan-6 2018
• Version: v0.15.1
• Headless: Yes, bitcoind
• Pruning: -prune=20480
• dbcache: -dbcache=155
• maxmempool: -maxmempool=5
• bandwidth: 500 Mbit/s
• DiskIO: 100 MB/s
• CPU: 0.5 vCPU, GCE f1-micro VM with 0.5 vCPU (shared CPU)
• Memory: 0.6 GB
• OS: Unbuntu 17.10 (artful)

[1714944818]

1714944818

[1714944818]

1714944818

[jnano]

A different OS, and you practically have no RAM, but if you could spare more RAM, see if putting the chainstate directory on a RAM drive helps:
https://bitcointalk.org/index.php?topic=2291857.0

[d4n13]

How big is the chainstate database?  I know it changes everyday, but I can't get a clear answer.

I hear the uncompressed UTXO database (as of today) is around 4 GB, and that the chainstate (compressed) is around 2 GB.  Does this sound right?

[LoyceV]

I've tested this a while back:

    Duration: maybe 20 hours
    Date: say 2 months ago
    Version: Bitcoin Core v0.15.1.0
    Headless: Running Bitcoin Core
    Pruning: Pruning enabled for the test, -prune=550
    dbcache: unchanged default options
    maxmempool: unchanged default options
    bandwidth: standards DSL, kinda irrelevant for this test
    DiskIO: very high, I saved this to a ramdrive (/run/shm)
    CPU: 3 year old Intel i3 laptop
    Memory: 12 GB, up to 50%
    OS: 64 bit Linux

In my experience, there's 2 bottlenecks for Bitcoin Core: total RAM for file cache, and access time to your disk.

For practical use, I now have my chainstate-directory on an ssd, and my blocks-directory on a (much bigger) hdd.
I assume you're using cloud hosting considering your specs. If you want this to be faster, you could add more ram/cpu during syncing.

[jnano]

How big is the chainstate database?  I know it changes everyday, but I can't get a clear answer.

With Bitcoin Core's native storage format, mine is about 3,780,000,000 bytes as of a few days ago.

Trying solid LZMA compression on 2% of the data, I got a reduction of 25% with solid compression, and 19% without. With ZIP compression (32KB blocks) I get 15%. Is there any RAMdrive software with a configurable compression block size?

[d4n13]

...
    Duration: maybe 20 hours
...
    Pruning: Pruning enabled for the test, -prune=550
...
    DiskIO: very high, I saved this to a ramdrive (/run/shm)
...
    OS: 64 bit Linux
...

Yep... saw your post on /run/shm and tried the same.  Spun up a full size server (2xCore 13GB mem) and sync too 20.16 hrs out of ramdisk.

[achow101]

It will sync faster and have less disk I/O if you increase the dbcache. This requires more RAM though. Raising the dbcache to anything larger than 8000 MB (-dbcache=8000 is large enough to store the whole thing in memory and then flush right at the end so there's significantly less disk I/O then. Of course you will need plenty of RAM in order to set the dbcache to that high.

[jnano]

Compared with version 0.15.0.1, did 0.15.1 change in any way the caching or disk I/O logic?

[d4n13]

It will sync faster and have less disk I/O if you increase the dbcache. This requires more RAM though. Raising the dbcache to anything larger than 8000 MB (-dbcache=8000 is large enough to store the whole thing in memory and then flush right at the end so there's significantly less disk I/O then. Of course you will need plenty of RAM in order to set the dbcache to that high.

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?

[achow101]

Compared with version 0.15.0.1, did 0.15.1 change in any way the caching or disk I/O logic?

No.

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?

That's the on disk sized which is compressed. The in memory size is much larger, ~7-8GB

[d4n13]

Only data I can find on the UTXO set shows it's closer to 4GB instead of 8GB.  Am I looking at the on-disk size (compressed) compared to the in-memory size?

That's the on disk sized which is compressed. The in memory size is much larger, ~7-8GB
[/quote]

So as a WAG, I'll just double what the on-disk size reported from the report.  Won't be precise, but better than nothing I suppose.

Thx

[jrian]

    Duration: 8-12 hours
    Date: 2017-2018
    Version: latest
    Headless: Yes, bitcoind
    Pruning: no
    bandwidth: 1000 Mbit/s
    CPU: XEONs
    Memory: 8-128
    OS: Unbuntu/Debian