Bitcoin Forum
December 11, 2016, 12:40:53 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Downloading the blockchain  (Read 2035 times)
Mushoz
Hero Member
*****
Offline Offline

Activity: 686


Bitbuy


View Profile WWW
November 30, 2011, 10:10:36 PM
 #1

Hey guys,

I just installed Bitcoin on a new computer and moved my wallet.dat and started the client so it could begin downloading the chain. At the beginning it was downloading blocks really fast, but now that it is at 80% it's slowing down to a crawl with loads of HDD activity. Why is that? Why is it slowing down so much at the end? Are the blocks that much bigger? And why is it causing so much HDD activity? AFAIK the blockchain was only about 700ish MB last time I checked, so I'm not really sure why the HDD is a bottleneck when downloading the chain over a 25mb/s internet connection. Thank you very much in advance for explaining my questions Smiley

www.bitbuy.nl - Koop eenvoudig, snel en goedkoop bitcoins bij Bitbuy!
1481416853
Hero Member
*
Offline Offline

Posts: 1481416853

View Profile Personal Message (Offline)

Ignore
1481416853
Reply with quote  #2

1481416853
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481416853
Hero Member
*
Offline Offline

Posts: 1481416853

View Profile Personal Message (Offline)

Ignore
1481416853
Reply with quote  #2

1481416853
Report to moderator
1481416853
Hero Member
*
Offline Offline

Posts: 1481416853

View Profile Personal Message (Offline)

Ignore
1481416853
Reply with quote  #2

1481416853
Report to moderator
1481416853
Hero Member
*
Offline Offline

Posts: 1481416853

View Profile Personal Message (Offline)

Ignore
1481416853
Reply with quote  #2

1481416853
Report to moderator
Gabi
Legendary
*
Offline Offline

Activity: 1050


View Profile
December 01, 2011, 05:59:24 PM
 #2

Quote
Are the blocks that much bigger?
Yup

Quote
And why is it causing so much HDD activity?
Dunno but it's normal, same happens here

Quote
when downloading the chain over a 25mb/s internet connection.
Yup but you don't download blocks at 25mb/s, but only at the speed as the peers you are connected to send them to you, so much slower.
SgtSpike
Legendary
*
Offline Offline

Activity: 1344



View Profile
December 01, 2011, 06:03:48 PM
 #3

Hey guys,

I just installed Bitcoin on a new computer and moved my wallet.dat and started the client so it could begin downloading the chain. At the beginning it was downloading blocks really fast, but now that it is at 80% it's slowing down to a crawl with loads of HDD activity. Why is that? Why is it slowing down so much at the end? Are the blocks that much bigger? And why is it causing so much HDD activity? AFAIK the blockchain was only about 700ish MB last time I checked, so I'm not really sure why the HDD is a bottleneck when downloading the chain over a 25mb/s internet connection. Thank you very much in advance for explaining my questions Smiley
The earlier blocks only have a single transaction - the transaction to generate a new bitcoin address.  Laters blocks have hundreds, sometimes even thousands of transactions.  So yes, they're going to take a lot longer to download.
SaintFlow
Sr. Member
****
Offline Offline

Activity: 415


for the love of water


View Profile
December 01, 2011, 06:10:30 PM
 #4

Sorry to threatjack,

Can someone tell me how long it now takes to download the entire blockchain if you are new to this
on average.

Will this become something of a scalability issue to bitcoin in the future?


account resurrected
the lurking ends
SgtSpike
Legendary
*
Offline Offline

Activity: 1344



View Profile
December 01, 2011, 06:14:49 PM
 #5

My estimate is 4-24 hours, depending on your connection speed and the connection speed of the nodes you are connected to.

Yes, it is a scalability issue, but it will be solved with lite clients that don't store the full blockchain.
ovidiusoft
Sr. Member
****
Offline Offline

Activity: 252


View Profile
December 01, 2011, 06:34:16 PM
 #6

My estimate is 4-24 hours, depending on your connection speed and the connection speed of the nodes you are connected to.
Yes, it is a scalability issue, but it will be solved with lite clients that don't store the full blockchain.

My way of doing it is to connect to a few fallback nodes ( https://en.bitcoin.it/wiki/Fallback_Nodes#IPv4_Nodes ). For example, btcnode.novit.ro would be a good choice for people in Romania or a few hops away (Europe, Russia), it is located in Bucharest and has 100 Mbit bandwidth, soon (read: next year) to be upgraded to 1 Gbit.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
December 01, 2011, 06:35:57 PM
 #7

Also be aware that even w/ unlimited nodes and unlimited bandwidth it takes some time to validate each block (and every transaction in the block) and write it to the database.  The client (properly) assumes anything the network tells it can't be trusted until the data is validated against the portion of block chain already validated. In essence is is rebuilding the entire transaction history of Bitcoin from the genesis block forward.
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
December 01, 2011, 11:55:16 PM
 #8

You don't need "fallback nodes", as every connection is a node. The best thing to speed downloading is to have lots of connections, such as properly setting up port forwarding. 30+ connections instead of eight is normal.

You can also look at the CPU usage while downloading; unless a CPU core is maxed to 100%, CPU usage likely isn't the limiting factor. Disk access is also an order of magnitude faster than the average high-speed connection.

My direct-download blockchain thread was un-stickied, it seems, but you can find it in the newbie forum.

ovidiusoft
Sr. Member
****
Offline Offline

Activity: 252


View Profile
December 02, 2011, 10:59:24 AM
 #9

You don't need "fallback nodes", as every connection is a node. The best thing to speed downloading is to have lots of connections, such as properly setting up port forwarding. 30+ connections instead of eight is normal.

Not necessarily. Due to the random way you get connections, it's possible that a significant portion of these are on slow connections or unstable and the client will work a lot on finding another node to send you those blocks, and so on.

Plus, it always pissed me off that on every Bitcoin client startup I had to wait tens of seconds (sometimes minutes) until it would start downloading. This is worse if a lot of time passed since you shutdown (most of the IPs in your local cache are no longer available).
racerguy
Sr. Member
****
Offline Offline

Activity: 271


View Profile
December 04, 2011, 05:50:14 PM
 #10

http://eu1.bitcoincharts.com/blockchain/
download that it's much faster, and current to within 24hrs.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
December 05, 2011, 05:35:00 AM
 #11

http://eu1.bitcoincharts.com/blockchain/
download that it's much faster, and current to within 24hrs.

Yes, this is good advice. In fact, I couldn't even get the client to download the blockchain. Once I downloaded the nightly from bitcoincharts it updated fine.
gmaxwell
Staff
Legendary
*
Offline Offline

Activity: 2030



View Profile
December 05, 2011, 05:54:45 AM
 #12

AFAIK the blockchain was only about 700ish MB last time I checked, so I'm not really sure why the HDD is a bottleneck when downloading the chain over a 25mb/s internet connection. Thank you very much in advance for explaining my questions Smiley

Bitcoin is a system designed so that you don't have to trust anyone— so your client doesn't trust your peers to give you valid blocks, it doesn't even trust that the miners followed the rules, it processes each one of them performing all the transaction processing the bitcoin system does on its own to validate that the rules of the bitcoin system have been respected and that a couple of big miners didn't just quietly agree to give themselves a million coin out of thin air.

Of course, it's possible to build bitcoin end user software that doesn't do all this checking— bitcoinj for example, doesn't even need to keep a copy of the historical transactions— but then it can't also participate as a full peer in the network. It has to have trust— that there aren't sizable miners out there willing to insert complete rubbish into the blockchain (or a non-trivial fork of it).

The hard disk activity is because the later blocks have many more transactions and for each transaction your system must find the transactions that paid into it to make sure the figures add up... the software then does synchronous database writes to make sure that the data isn't lost— a good thing during the normal processing, but kinda silly during the initial synchup.  I expect future versions will speed this up quite a bit, though initial sync on a full node will always be somewhat slow.

The actual downloading time is almost never an issue for users— its the validation thats slow. Though there still are a lot of pre-0.3.24 nodes out there which slow the syncup because they randomly hang up on peers asking for a couple of the more recent blocks at once.

Contrary to the claims in posts upthread, for almost everyone the disk IO is the bottleneck. If you run your node on a ramdisk or a very fast SSD you can get CPU to be the bottleneck.  You'd probably have to be on dialup before the network would be the bottleneck.

fivemileshigh
Full Member
***
Offline Offline

Activity: 136


View Profile
December 09, 2011, 11:34:08 PM
 #13

How does one connect to a particular node from the list at

https://en.bitcoin.it/wiki/Fallback_Nodes ?

 I'm assuming the " -addnode=ip " switch is for windows users?

I'm on OSX 10.6.8

thanks!
racerguy
Sr. Member
****
Offline Offline

Activity: 271


View Profile
December 10, 2011, 07:12:14 AM
 #14

How does one connect to a particular node from the list at

https://en.bitcoin.it/wiki/Fallback_Nodes ?

 I'm assuming the " -addnode=ip " switch is for windows users?

I'm on OSX 10.6.8

thanks!


add that line into your bitcoin.conf file, wherever that is on a mac.
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
December 11, 2011, 11:41:16 PM
 #15

How does one connect to a particular node from the list at

https://en.bitcoin.it/wiki/Fallback_Nodes ?

 I'm assuming the " -addnode=ip " switch is for windows users?

I'm on OSX 10.6.8

thanks!


Code:
$ /Applications/Bitcoin.app/Contents/MacOS/bitcoin --help

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!