Block chain size/storage and slow downloads for new users

[CoinCidental]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I helped a friend download the blockchain a couple of days ago and set up a node, it only took 2 days to download and about 38 GB iirc on a 1tb external drive

Might have been faster to use the torrent but we were not in a hurry anyway and the laptop was around 7 years old  so probably the cpu would have kept us from going much faster

I think most people could spare 38GB at the price drives are now, for the people who can't there is always multibit

By the time he can fill that 1tb drive, it maybe several years down the line and storage and bandwidth will be much cheaper than now

[ranochigo]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I helped a friend download the blockchain a couple of days ago and set up a node, it only took 2 days to download and about 38 GB iirc on a 1tb external drive

Might have been faster to use the torrent but we were not in a hurry anyway and the laptop was around 7 years old  so probably the cpu would have kept us from going much faster

I think most people could spare 38GB at the price drives are now, for the people who can't there is always multibit

By the time he can fill that 1tb drive, it maybe several years down the line and storage and bandwidth will be much cheaper than now

It should probably be around the same speed since there is headers-first synchronization. Just out of curiousity, did it took 2 actual days to download and what version are you using? Unless you have weak CPU and the external drives becomes a bottleneck, it should take slightly less than 1 day max.

Torrent version is quite far behind so they can still take sometime to download the remaining blocks.

[shorena]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I helped a friend download the blockchain a couple of days ago and set up a node, it only took 2 days to download and about 38 GB iirc on a 1tb external drive

Might have been faster to use the torrent but we were not in a hurry anyway and the laptop was around 7 years old  so probably the cpu would have kept us from going much faster

I think most people could spare 38GB at the price drives are now, for the people who can't there is always multibit

By the time he can fill that 1tb drive, it maybe several years down the line and storage and bandwidth will be much cheaper than now

It should probably be around the same speed since there is headers-first synchronization. Just out of curiousity, did it took 2 actual days to download and what version are you using? Unless you have weak CPU and the external drives becomes a bottleneck, it should take slightly less than 1 day max.

Torrent version is quite far behind so they can still take sometime to download the remaining blocks.

Dont overestimate a 2008 system. 1000 USD got you for example an Asus X83VM-X1[1], with 2 cores, 4 Gigs of DDR2(!) ram and 320Gig HDD. That might not sound slow, but thats several CPU and 2 RAM generations old. I recently had a blackout and had to reindex the blockchain on a 2009 desktop AMD tripple core, similar HDD and RAM (only DDR3). I have no exact numbers, but I had to let it run over night, so I guess somewhere between 12 and 16 hours. Even worse it might be a netbook

I agree that the torrent no longer makes sense unless you have to sync several systems and have low bandwith. Thought even in that case you could just sync the machine with the fastest CPU/storrage first and let the rest connect exclusivly to it via LAN.

[1] http://www.cnet.com/products/asus-x83vm-x1-core-2-duo-p8400-2-26-ghz-14-1-inch-tft/

[CoinCidental]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I helped a friend download the blockchain a couple of days ago and set up a node, it only took 2 days to download and about 38 GB iirc on a 1tb external drive

Might have been faster to use the torrent but we were not in a hurry anyway and the laptop was around 7 years old  so probably the cpu would have kept us from going much faster

I think most people could spare 38GB at the price drives are now, for the people who can't there is always multibit

By the time he can fill that 1tb drive, it maybe several years down the line and storage and bandwidth will be much cheaper than now

It should probably be around the same speed since there is headers-first synchronization. Just out of curiousity, did it took 2 actual days to download and what version are you using? Unless you have weak CPU and the external drives becomes a bottleneck, it should take slightly less than 1 day max.

Torrent version is quite far behind so they can still take sometime to download the remaining blocks.

Dont overestimate a 2008 system. 1000 USD got you for example an Asus X83VM-X1[1], with 2 cores, 4 Gigs of DDR2(!) ram and 320Gig HDD. That might not sound slow, but thats several CPU and 2 RAM generations old. I recently had a blackout and had to reindex the blockchain on a 2009 desktop AMD tripple core, similar HDD and RAM (only DDR3). I have no exact numbers, but I had to let it run over night, so I guess somewhere between 12 and 16 hours. Even worse it might be a netbook

I agree that the torrent no longer makes sense unless you have to sync several systems and have low bandwith. Thought even in that case you could just sync the machine with the fastest CPU/storrage first and let the rest connect exclusivly to it via LAN.

[1] http://www.cnet.com/products/asus-x83vm-x1-core-2-duo-p8400-2-26-ghz-14-1-inch-tft/

It wasn't a netbook thk fck lol
It was a hp pavilion 17" entertainment laptop but spec was quite weak compared to modern equipment :
Core 2 duo @ 2.26 with 4gb  of ddr3 and a 500Gb sat and  1tb external seagate
We put Bitcoin on the external because he's going to let it run as a node until it dies and the 1tb drive should last few years

[Clint]

Threads like yours is helping all of us, but also tells me how complicated btc really is. Thanks for the new update though! 

[TransaDox]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

Check your privilege.

[Meuh6879]

can you de-stick this thread ?

v0.10.x solve the slow network diffusion ... only SLOW blockchain creation is from CPU and HARD DRIVE SPEED issues.

Core 2 Duo 2,9GHz = 3 days to recreate the blockchain.

[ranochigo]

can you de-stick this thread ?

v0.10.x solve the slow network diffusion ... only SLOW blockchain creation is from CPU and HARD DRIVE SPEED issues.

Core 2 Duo 2,9GHz = 3 days to recreate the blockchain.

This isn't much of an issue. Most people at least would have I3 or I5, if they don't have that, SPV clients would be a better way to go. But still, a Pentium g3258 is quite powerful while being under 80USD. The main problem is that he is talking about using a external HDD which may be a bottleneck and we don't really know his internet speed yet.

[Borisz]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I think there are a lot of students around and there money is an issue. Plus, if you need some special hardware for bitcoin (e.g. 1TB HDD) then it can be somewhat "exclusive" (In terms of running a full node, of course). As said, I stopped my full node from my laptop when the blockchain went around 30GB-ish. For me, that is a lot of space.  If I can't keep it up 24/7 (or close), not much point anyway.

[CoinCidental]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I think there are a lot of students around and there money is an issue. Plus, if you need some special hardware for bitcoin (e.g. 1TB HDD) then it can be somewhat "exclusive" (In terms of running a full node, of course). As said, I stopped my full node from my laptop when the blockchain went around 30GB-ish. For me, that is a lot of space.  If I can't keep it up 24/7 (or close), not much point anyway.

how is 30 odd GB a lot of space ? what age of computer do you have ?

[wlefever]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I think there are a lot of students around and there money is an issue. Plus, if you need some special hardware for bitcoin (e.g. 1TB HDD) then it can be somewhat "exclusive" (In terms of running a full node, of course). As said, I stopped my full node from my laptop when the blockchain went around 30GB-ish. For me, that is a lot of space.  If I can't keep it up 24/7 (or close), not much point anyway.

how is 30 odd GB a lot of space ? what age of computer do you have ?

That could easily be 1/3 of his total space on a laptop. I do understand what you're saying though, and as the blockchain size increases we will continue to see larger amounts of storage offered cheaper. 

[CoinCidental]

i just checked the bitcoin folder and its currently  41.8GB   

It makes no differnce to me as i have a few 1TB drives


what laptop would 30 GB  be a third  of the hdd thesedays ? i rememeber trying to buy a replacement 120GB HDD about 2-3 years ago
and the people in the shop laughed and said we dont sell those anymore (the smallest they had was 320 or 500GB even back then )

[shorena]

i just checked the bitcoin folder and its currently  41.8GB  

It makes no differnce to me as i have a few 1TB drives


what laptop would 30 GB  be a third  of the hdd thesedays ? i rememeber trying to buy a replacement 120GB HDD about 2-3 years ago
and the people in the shop laughed and said we dont sell those anymore (the smallest they had was 320 or 500GB even back then )

Exchange SSD for HDD and you will see plenty of "new" hardware that is limited to 100odd GB of disk space[1]. My laptop (not its not running a full node) "only" has 240 GB and with a collections of different ISOs and other large files I can easily imagine that bitcoin core would not fit any longer over the years. That beeing said, I dont see any reason to run a full node on my laptop, but others might see this differently.

[1] Hi, apple.

[Borisz]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

I think there are a lot of students around and there money is an issue. Plus, if you need some special hardware for bitcoin (e.g. 1TB HDD) then it can be somewhat "exclusive" (In terms of running a full node, of course). As said, I stopped my full node from my laptop when the blockchain went around 30GB-ish. For me, that is a lot of space.  If I can't keep it up 24/7 (or close), not much point anyway.

how is 30 odd GB a lot of space ? what age of computer do you have ?

I was dealing with bitcoin issues from an old linux laptop of mine with a HDD of 80 GB and I would like to do other things as well. I consider the (now) 41 GB even on a 500 GB disk a lot. I have other files to store apart from my tiny Bitcoin stash.

Also, now with the proposed gradual block size increase the more transactions happen, the larger the blocksize will be. I wonder how long this "be your own bank" can be sustainable before some other measures need to be introduced that wouldn’t require everyone to hold the full blockchain, just say the last X GB.

[TransaDox]

Also, now with the proposed gradual block size increase the more transactions happen, the larger the blocksize will be. I wonder how long this "be your own bank" can be sustainable before some other measures need to be introduced that wouldn’t require everyone to hold the full blockchain, just say the last X GB.

I agree. The mindset that tries to fill technology as it arrives should be resisted. We should be viewing this as a minimal resource system so that you don't necessarily need expensive 1st world infrastructure to participate.

My opinion is probably on the other extreme to those that think throwing TB of disk space is a non-issue but my sentiment is valid, I think. We should be trying to put our energies into making Bitcoin resource agnostic, as far as it can be, and have design goals around low footprint, low bandwidth with the aim of being able to run full nodes over things like radio. The current mindset of a "huge blockchain is OK" is an anathema to a decentralized system pushing more and more control into infrastructure-rich people and societies and I would like to see a reversal of this.

[jorjito25]

um why are people complaining about blockchain size I mean 1tb drives aren't that expensive

Yeah, I kind of agree. Standard HDD in normal desktop computers is lager than 1 TB, and for laptops there are even 2TB SSDs (even if expensive).

The technology moves forward as does Butcoin.

[momore]

Thanks for sharing this useful info. It's very helpful for me.

[CoinCidental]

Also, now with the proposed gradual block size increase the more transactions happen, the larger the blocksize will be. I wonder how long this "be your own bank" can be sustainable before some other measures need to be introduced that wouldn’t require everyone to hold the full blockchain, just say the last X GB.

I agree. The mindset that tries to fill technology as it arrives should be resisted. We should be viewing this as a minimal resource system so that you don't necessarily need expensive 1st world infrastructure to participate.

My opinion is probably on the other extreme to those that think throwing TB of disk space is a non-issue but my sentiment is valid, I think. We should be trying to put our energies into making Bitcoin resource agnostic, as far as it can be, and have design goals around low footprint, low bandwidth with the aim of being able to run full nodes over things like radio. The current mindset of a "huge blockchain is OK" is an anathema to a decentralized system pushing more and more control into infrastructure-rich people and societies and I would like to see a reversal of this.

We
I get your point about the blockchain growing too large to easily work  with  but I also agree with the other guy who said sooner or later its going to be infeasible for Everyone to have the whole blockchain and I hope the lead developers are close to  a solution  so  future users will have the option of storing a certain x amount of blockchain
The most recent 5 or 10GB etc would be fine imo or maybe 2 versions of the official client so power users can still run their own "bank" if they want to and a lite version similar to multibit that would enable secure use  without the whole chain

At the moment I don't think it's a problem   (42GB) but Bitcoin is growing exponentially and they're will have to be lite options for half of the world that don't have the latest cpu and terabytes of storage and a fibre optic Internet connection to get it all in a reasonable amount of time

The more Bitcoin takes off the more people will be looking at new  solutions to the problem but there are solutions already and perhaps we are worrying about this too much at the moment since  there is already a lot of alternative wallets to store your Bitcoin even today without needing the whole chain

[TransaDox]

There are a number of issues that I think get rolled into one "its too big" which really shouldn't be.

A client doesn't require a lot of CPU or memory or disk space to confirm transactions or signatures. The hardware requirements are pretty much the same now as they were when Satoshi had his brainwave. This aspect can be achieved on Raspberry Pi and even some micro controllers. You don't need huge clusters of supercomputers either now or in the future to confirm transactions and signatures.

To be able to confirm transactions and sigs, however, you need to be able to trust your current information about the blockchain. This is where things are ballooning.

The Bitcoin Core software is naive, in software terms, in that assumes the host computer is for the sole purpose of running the application. It requires this, not for operation, but for pre-processing the block-chain only. It will max out bandwidth trying to catch up, hammer the CPU to verify blocks and use a huge chunk of a disk drive without options to move the data or offload any or all of those facets. The software needs to be more cooperative and smarter in the way it stores and reassembles the block chain and it is my opinion that bittorrent, as has been adopted by the games industry for large scale distributions for this exact reason,  is the way forward there where the data can be downloaded directly to a NAS, piece by piece in reverse order with the option of how far back you want to go.

I also see that there is a real need for a ACID compliant database - after all, this is money we are talking about  - which would relegate to the history books, the godawful verifying every time you start the application (SQLite is the first choice here, I think). The latter would also enable adhoc verifications rather than the completely linear one of the hash-table method. (e.g. pick tx 1, 9 and 50 produce a hash then ask another client what they make it).

I would like to see these kinds of things. Perhaps not exactly, but in that general direction. All I see at the moment though is less and less hardware being capable of running the software and more and more centralization to those who own the infrastructure. The end result will be online web services that you access through a browser and the banks will be those services.

[TransaDox]

Drives are pretty much cheap now, but I think a huge problem is still bandwidth caps. 40GB is about two times the bandwidth cap I get where I am right now.

[sarcasm]
Bandwidth's pretty much cheap now; I have unlimited 100 mb/sec. Lets increase the required bandwidth because its only waiting a couple of minutes longer.
[/sarcasm]

I get fed up with the "I'm alright Jack cos I have a 10 tonne tower of hard drives." attitude. It's not a problem FOR YOU and it's cheap FOR YOU and you can do anything you like to your computer because you don't have an IT department in your bedroom.