Slow synchronisation -

[chaintool]

Hello,

Using Bitcoin-QT 11.2 (64 bit) on a Macbook air (I3 quad  core 1.8Ghz / 4GM RAM). I have got to week 19 (block 374185) and Bitcoin-QT appears to only want to synchronise 1 week per day now! I have already been running none stop for over two weeks to get this far. I looks like there are sometimes 1 peer in and 8 out, my block chain is 52.9 GB, The CPU is around 15% on Bitcoin-QT? Any advice on how to speed this up will be much appreciated.

[1715505593]

1715505593

[1715505593]

1715505593

[1715505593]

1715505593

[1715505593]

1715505593

[1715505593]

1715505593

[1715505593]

1715505593

[XERES]

you can download the blockchain manual as a bootstrap.dat. there should be one on the forum, if not try the bitcoin website

[ranochigo]

Hello,

Using Bitcoin-QT 11.2 (64 bit) on a Macbook air (I3 quad  core 1.8Ghz / 4GM RAM). I have got to week 19 (block 374185) and Bitcoin-QT appears to only want to synchronise 1 week per day now! I have already been running none stop for over two weeks to get this far. I looks like there are sometimes 1 peer in and 8 out, my block chain is 52.9 GB, The CPU is around 15% on Bitcoin-QT? Any advice on how to speed this up will be much appreciated.

Does your peers have a ping and have a block height that is higher than yours? What is the network graph speed? 8 peers in and 1 peer out is normal and good. Most of the time, the bottleneck is usually the CPU/HDD/Network speed. With a slow HDD and network speed, it would cause a significant bottleneck. CPU is likely not the issue here.

you can download the blockchain manual as a bootstrap.dat. there should be one on the forum, if not try the bitcoin website

Bootstrap.dat doesn't help a lot with headers-first synchronization. It gets the blocks from many peers simultaneously while verifying it. Using bootstrap.dat would in fact slow down the synchronization by a lot.

[-XXIII-]

(block 374185)

That block was made in September 2015, google shows me this happened during that time:

...the UK-based exchange released the private keys to hundreds of addresses containing small amounts of bitcoin. Thousands of transactions have flooded the network as users seek to send the bitcoin held by these addresses to their own wallets. CoinWallet has indicated that it intends to give away $48,000 in free bitcoin using this method. At the time of writing, there were over 70,000 unconfirmed transactions waiting to be processed by the network.

https://bitcoinmagazine.com/articles/bitcoin-businesses-take-steps-prepare-coinwallets-september-stress-test-1441917829

Would something like this cause the blockchain download process to take so long?

[cr1776]

you can download the blockchain manual as a bootstrap.dat. there should be one on the forum, if not try the bitcoin website

This is a bad idea and no longer useful with the last 3+ versions.

You said you were on a Macbook Air.  Have you checked the sleep settings?  Which version of OS X are you running?

[ranochigo]

(block 374185)

That block was made in September 2015, google shows me this happened during that time:

...the UK-based exchange released the private keys to hundreds of addresses containing small amounts of bitcoin. Thousands of transactions have flooded the network as users seek to send the bitcoin held by these addresses to their own wallets. CoinWallet has indicated that it intends to give away $48,000 in free bitcoin using this method. At the time of writing, there were over 70,000 unconfirmed transactions waiting to be processed by the network.

https://bitcoinmagazine.com/articles/bitcoin-businesses-take-steps-prepare-coinwallets-september-stress-test-1441917829

Would something like this cause the blockchain download process to take so long?

Definitely. Bitcoin blocks contains transactions which the user needs to verify and/or download. With larger transaction volumes, the size would be higher with more transaction per block and hence making the synchronization time longer. You would realise that synchronization is relatively fast at the start and gets slower towards the end.

*Edit: Saw it wrongly, it would take significantly more time but probably not as much as the OP.

[shorena]

(block 374185)

That block was made in September 2015, google shows me this happened during that time:

...the UK-based exchange released the private keys to hundreds of addresses containing small amounts of bitcoin. Thousands of transactions have flooded the network as users seek to send the bitcoin held by these addresses to their own wallets. CoinWallet has indicated that it intends to give away $48,000 in free bitcoin using this method. At the time of writing, there were over 70,000 unconfirmed transactions waiting to be processed by the network.

https://bitcoinmagazine.com/articles/bitcoin-businesses-take-steps-prepare-coinwallets-september-stress-test-1441917829

Would something like this cause the blockchain download process to take so long?

Definitely. Bitcoin blocks contains transactions which the user needs to verify and/or download. With larger transaction volumes, the size would be higher with more transaction per block and hence making the synchronization time longer. You would realise that synchronization is relatively fast at the start and gets slower towards the end.

If the CPU is only at 15% its not the issue though. Its either connection (slow / bad peers) or HDD speed.

[cr1776]

(block 374185)

That block was made in September 2015, google shows me this happened during that time:

...the UK-based exchange released the private keys to hundreds of addresses containing small amounts of bitcoin. Thousands of transactions have flooded the network as users seek to send the bitcoin held by these addresses to their own wallets. CoinWallet has indicated that it intends to give away $48,000 in free bitcoin using this method. At the time of writing, there were over 70,000 unconfirmed transactions waiting to be processed by the network.

https://bitcoinmagazine.com/articles/bitcoin-businesses-take-steps-prepare-coinwallets-september-stress-test-1441917829

Would something like this cause the blockchain download process to take so long?

Definitely. Bitcoin blocks contains transactions which the user needs to verify and/or download. With larger transaction volumes, the size would be higher with more transaction per block and hence making the synchronization time longer. You would realise that synchronization is relatively fast at the start and gets slower towards the end.

If the CPU is only at 15% its not the issue though. Its either connection (slow / bad peers) or HDD speed.

Perhaps. 

But perhaps it is going to sleep when he is not using it and then when he is looking at the Activity Monitor it is slowing down background processes.  With 0.11.2, it shouldn't happen with App Nap (he can verify in Activity Monitor under Energy), but it could depending on which version of OS X he is running.  ;-)

I agree, it should be a much higher percentage than that but if it is being slowed in the background, that could explain it. 

All MacBook Airs since 2011 have SSDs (iirc) and so that should be plenty fast. If it is an older model, then that would be useful to know.

I'd also be curious to know:
1. Disk space available?  e.g. if it is a 128GB SSD, he could be running low on disk space and that could be an issue.
2. How many peers show up?

[chaintool]

Hi y'all,

Thanks so much for your suggestions and observations. My problem is bigger now, after weeks of downloading and verifying I have a check sum error in my data base;

2016-02-05 15:56:26 Keys: 101 plaintext, 0 encrypted, 101 w/ metadata, 101 total
2016-02-05 15:56:26  wallet                   10ms
2016-02-05 15:56:26 init message: Activating best chain...
2016-02-05 15:57:49 UpdateTip: new best=00000000000000000d710f3876a7b235aab173042cfe77ff7d0d1972bc279465  height=382098  log2_work=83.560277  tx=91028107  date=2015-11-05 01:57:40 progress=0.917517  cache=42.9MiB(4399tx)
2016-02-05 15:58:49 LevelDB read failure: Corruption: block checksum mismatch
2016-02-05 15:58:49 Corruption: block checksum mismatch
2016-02-05 15:58:56 Error reading from database: Database corrupted
2016-02-05 21:20:12

So 57GB later, I'm drifting down shit creek without a paddle!

DiskUtil checks out 100%, SDD is OK.

I have been trying to run a full node on a Raspberry PI2, this SDD/blockchain was going to boot strap it as it would not load the whole blockchain native on an HCSD card.

I'm beginning to wonder if this tech is stable, scalable and intended for domestic HW?

very disillusioned...

G*

[2112]

Hi y'all,

Thanks so much for your suggestions and observations. My problem is bigger now, after weeks of downloading and verifying I have a check sum error in my data base;

So 57GB later, I'm drifting down shit creek without a paddle!

DiskUtil checks out 100%, SDD is OK.

I have been trying to run a full node on a Raspberry PI2, this SDD/blockchain was going to boot strap it as it would not load the whole blockchain native on an HCSD card.

I'm beginning to wonder if this tech is stable, scalable and intended for domestic HW?

very disillusioned...

G*

MacBook Airs were never really reliable hardware. In addition, their lightness promotes the abuse. The most common errors are:

1) not waiting for the proper OS shutdown: start a shutdown and then rapidly close the lid before shutdown completes
2) not waiting for the proper Bitcoin-Qt shutdown: unfortunately it closes all its windows before the data is fully committed to disk
3) DiskUtil never in the history of Mac OS's did the proper/full test of disk, it is only rather superficial file system integrity check

Compounded with the fact that Bitcoin-Qt has almost no internal error checking and uses the educational toy database engine you are where you are: your own bank where bank manager decided that running backups to Time Machine is an unnecessary expense.

[Jet Cash]

Can you delete later block files to step back a bit on the download, or does core keep a tag of where it finished last time?

[achow101]

Can you delete later block files to step back a bit on the download, or does core keep a tag of where it finished last time?

Neither. Deleting any of the block files will result in a full reindex, which although faster than a resync, still can take a significant amount of time.

[chaintool]

Hi Y'all,

I think the main point has been missed, it may be that regular home computers can no longer run a full bitcoin node. So much for a public, democratic, none elitist,  de-centralised distribution scheme!

My Machine has pretty powerful duel core 64 bit CPU;

  Model Name:   MacBook Air
  Model Identifier:   MacBookAir5,2
  Processor Name:   Intel Core i5
  Processor Speed:   1,8 GHz
  Number of Processors:   1
  Total Number of Cores:   2
  L2 Cache (per Core):   256 KB
  L3 Cache:   3 MB
  Memory:   4 GB

When Satoshi Nakamoto started Bitcoin he mined his first coins (lots of them) on a home PC, I think it should be still possible to run a full node on my machine.

There are no capacity issues with CPU / RAM / Network bandwidth @ 65Mps.

I tried Bitcoind twice on my Raspberry PI2 and failed. I tried Bitcoin-QT twice on my laptop and failed, once using a 64GB SD card and once with an external USB3 SDD.

On a healthy laptop surly I should be able to download and install the wallet software and let it synchronise. I don't mind it takes a few days running 24/7 but to fail after many weeks, still being 13 weeks from current is unacceptable.

I sent my crash report to apple and GitHub...

G*

[2112]

Hi Y'all,

I think the main point has been missed, it may be that regular home computers can no longer run a full bitcoin node. So much for a public, democratic, none elitist,  de-centralised distribution scheme!

My Machine has pretty powerful duel core 64 bit CPU;

  Model Name:   MacBook Air
  Model Identifier:   MacBookAir5,2
  Processor Name:   Intel Core i5
  Processor Speed:   1,8 GHz
  Number of Processors:   1
  Total Number of Cores:   2
  L2 Cache (per Core):   256 KB
  L3 Cache:   3 MB
  Memory:   4 GB

When Satoshi Nakamoto started Bitcoin he mined his first coins (lots of them) on a home PC, I think it should be still possible to run a full node on my machine.

There are no capacity issues with CPU / RAM / Network bandwidth @ 65Mps.

I tried Bitcoind twice on my Raspberry PI2 and failed. I tried Bitcoin-QT twice on my laptop and failed, once using a 64GB SD card and once with an external USB3 SDD.

On a healthy laptop surly I should be able to download and install the wallet software and let it synchronise. I don't mind it takes a few days running 24/7 but to fail after many weeks, still being 13 weeks from current is unacceptable.

I sent my crash report to apple and GitHub...

G*

No, mate, it is you who missed the point.

Bitcoin was never for a man on the Clapham omnibus who just stepped into an electronics store and purchased some random imported electronics crap.

Bitcoin was always for a technically curious person capable of assembling a reliable hardware and discerning and following the reasonable advice like "disable App Nap in Mac OSX".

The barrier to entry of being your own bank is no longer financial but intellectual. Bitcoin doesn't require the latest technological toy of chattering classes riding that proverbial Clapham omnibus. The history of computing is full of them: BBC Micro, Sinclair ZX Spectrum and man others. Now they sell Raspbery Pi to that segment.

Bitcoin is known to work reasonably well on a reasonably sensible home computer like my old Windows XP machine with original Clawhammer AMD 64 from 2004 or the old Mac Mini Server from 2009.

It is always interesting and educational to read on this forum the experiences of the new entrants to the Bitcoin ecosystem. Some are quick on the uptake and nearly immediately learn to be self-sufficient. Other are just capital donors who will incessantly complain about not enough handholding.

You'll need to either learn how to use a computer or pay somebody who will run your computers for you. This is a 21st century version of egalitarianism in the knowledge society.