Bitcoin Forum
February 21, 2019, 07:20:22 PM *
News: Latest Bitcoin Core release: 0.17.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2] 3 »  All
  Print  
Author Topic: Whole Bitcoin Core 0.15 blockchain database on Google Drive  (Read 2061 times)
LoyceV
Legendary
*
Offline Offline

Activity: 1400
Merit: 2951


Self-made Legendary!


View Profile WWW
October 29, 2017, 05:36:46 PM
 #21

"PRUNED TO a RAM drive" - that's the key to your success,as I understand it.

Try to download the full 160Gb (not pruned) blockchain from the start, then tell us how much time it has taken you
Of course that's the key, but my point is still valid: it all depends on your PC.
Downloading to an SSD is much faster than downloading to a hdd.

That being said, my test also showed downloading a pruned chain on ramdrive did less blocks per minute than downloading blocks to my full chain on hdd. It took 70 seconds for 11 hours of blocks, that's about 1 block per second (see my test).
I don't really want to spend the time to do a full test, but from this small test, I extrapolate I can download a full blockchain to my hdd in 2-3 days.

I have no experience on Windows though, especially a system low on RAM can take much longer.

1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
Your Bitcoin transactions
The Ultimate Bitcoin mixer
made truly anonymous.
with an advanced technology.
Mix coins
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
1550776822
Hero Member
*
Offline Offline

Posts: 1550776822

View Profile Personal Message (Offline)

Ignore
1550776822
Reply with quote  #2

1550776822
Report to moderator
Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 29, 2017, 06:02:13 PM
 #22

"PRUNED TO a RAM drive" - that's the key to your success,as I understand it.

Try to download the full 160Gb (not pruned) blockchain from the start, then tell us how much time it has taken you
Of course that's the key, but my point is still valid: it all depends on your PC.
Downloading to an SSD is much faster than downloading to a hdd.

That being said, my test also showed downloading a pruned chain on ramdrive did less blocks per minute than downloading blocks to my full chain on hdd. It took 70 seconds for 11 hours of blocks, that's about 1 block per second (see my test).
I don't really want to spend the time to do a full test, but from this small test, I extrapolate I can download a full blockchain to my hdd in 2-3 days.

I have no experience on Windows though, especially a system low on RAM can take much longer.

All of the time all of my own main components (CPU, RAM, download bandwidth) were busy at 2-5% (peak loads 10-12%). My ONLY possible bottleneck could have been the hdd which was a pretty fast 7200 rpm, but still a usual hard drive (not SSD).
But! Trying to figure out why the download was so slow at a moment I copied all Bitcoin Core 0.14.2 (already downloaded) data directories into a fast USB 3.0 128 Gb flash drive.  The syncing process did become a little bit faster, but not significantly!
So, I am still sure the bottleneck can be anywhere but never inside my PC or my internet channel.

I assume it was insufficient number of other WIDE UPLOAD bandwidth, accepting incoming connections full Bitcoin Core nodes.



Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
LoyceV
Legendary
*
Offline Offline

Activity: 1400
Merit: 2951


Self-made Legendary!


View Profile WWW
October 29, 2017, 07:06:14 PM
 #23

All of the time all of my own main components (CPU, RAM, download bandwidth) were busy at 2-5% (peak loads 10-12%). My ONLY possible bottleneck could have been the hdd which was a pretty fast 7200 rpm, but still a usual hard drive (not SSD).
But! Trying to figure out why the download was so slow at a moment I copied all Bitcoin Core 0.14.2 (already downloaded) data directories into a fast USB 3.0 128 Gb flash drive.  The syncing process did become a little bit faster, but not significantly!
So, I am still sure the bottleneck can be anywhere but never inside my PC or my internet channel.
There was someone else earlier (see my test-link) having the same problem on Windows, and he eventually moved his pruned blockchain to a RAM-drive. That solved his problem.
If you haven't checked the number of read/write-actions to your hdd, you can't rule it out from being the bottleneck.

I'm no export on Windows, but as far as I know most flash drives don't use write cache (to be safe when you suddenly unplug it).

Quote
I assume it was insufficient number of other WIDE UPLOAD bandwidth, accepting incoming connections full Bitcoin Core nodes.
That's not what I experienced. A very simple way to test it: when you start your download, and blocks are still small, there's much less checking per block. At that point I download at the maximum speed my internet connection can deliver. It only slows down later, when blocks require more checking.

Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 29, 2017, 08:03:12 PM
 #24


Yes, I've been using windows (10, x64) Bitcoin Core version.
But I have always intentionally enabled write cash for removable drives

Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 30, 2017, 12:53:36 AM
 #25

Why would anyone trust them with something that poses a threat to their future ambitions.
Friend, what kind of threat can you see here?
I assure you, Google Drive would never replace my files, or do other similar meanness)
So, download safely (if you haven't started your full Bitcoin Core node yet) and don't worry about anything you mentioned

First, you didn't understand about how a signature of the file directory is the only verification.

Now, again, you stand on a position of perceived threats, and assure people as to that they should TRUST google.

But you make these assertions with respect to Bitcoin, which is defined as a peer to peer method of exchanging money WITHOUT TRUST.

I'm seeing a little conflict there.

Opinion?
Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 30, 2017, 04:50:01 AM
 #26

But you make these assertions with respect to Bitcoin, which is defined as a peer to peer method of exchanging money WITHOUT TRUST.

I'm seeing a little conflict there.

Opinion?

I suggest a little flexibility of mind and common sense.

The main idea is to fully realize the great importance of a greater number (the more the better) of Bitcoin Core full nodes in the network.

And to start as many full nodes as possible, as quickly as possible, and to exclude the trust factor, you may simply instruct your Bitcoin Core wallet application to just re-index the downloaded database. Yes, it is a pretty long process too, but it is still very much faster than downloading the full blockchain (as I've known windows systems seem to be more conserned about this)

Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
cellard
Legendary
*
Offline Offline

Activity: 1288
Merit: 1204


View Profile
October 30, 2017, 05:49:01 PM
 #27

I like the initiative of making Bitcoin Core nodes set up faster, but unfortunately, I don't think it's safe to download the Bitcoin blockchain from anywhere in the world but through the client itself as it validates it.

If you download it from a 3rd party, you risk that the blockchain has been somehow modified. The only way to know they are the exact same files as the ones being delivered by the client itself is through a checksum of sorts, but you would need to trust someone else to do that too.

Maybe im too paranoid, but I would rather go through the annoying process of downloading it myself on the Bitcoin Core client even if it's slower.

Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 30, 2017, 05:57:25 PM
 #28

I like the initiative of making Bitcoin Core nodes set up faster, but unfortunately, I don't think it's safe to download the Bitcoin blockchain from anywhere in the world but through the client itself as it validates it.

If you download it from a 3rd party, you risk that the blockchain has been somehow modified. The only way to know they are the exact same files as the ones being delivered by the client itself is through a checksum of sorts, but you would need to trust someone else to do that too.

Maybe im too paranoid, but I would rather go through the annoying process of downloading it myself on the Bitcoin Core client even if it's slower.

You are certainly right on the whole.

Maybe that post might be a solution? See here:

I have tested the archive, so you may download safely.
Just playing devil's advocate here..

That's great that you've "tested the archive"... But how can we "test" you? A "member" with 70 odd posts... Huh

What guarantee do we have that this Blockchain is "legit"... Other than you word?
As far as I understand, you can just install Bitcoin Core and -rescan the database:
Code:
bitcoin -rescan # or any Windows equivalent
Assuming you download a legit Bitcoin Core, it will verify all blocks on your disk and you can be absolutely sure you have a legit blockchain.

That being said, I wouldn't use this myself, I'm totally fine letting Bitcoin Core download it's own blocks.

Secondly, I wrote a great number of posts in  https://bitcointalk.org/index.php?topic=2303869 explaining :
how long I had to be waiting (weeks) till my Bitcoin Core 0.14.2 fully synced itself at last. And the bottleneck was NOT in any way related to my PC or internet bandwidth.
As a test, I recently downloaded a full blockchain (pruned to a RAM drive) in less than 24 hours, on a several years old PC with an average internet connection. Given the fact that I can do this, your bottleneck must be related to your PC or internet connection.


Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 30, 2017, 06:20:09 PM
Last edit: October 30, 2017, 06:38:57 PM by Spendulus
 #29

.....
Secondly, I wrote a great number of posts in  https://bitcointalk.org/index.php?topic=2303869 explaining :
how long I had to be waiting (weeks) till my Bitcoin Core 0.14.2 fully synced itself at last. And the bottleneck was NOT in any way related to my PC or internet bandwidth.


Maybe there's a shitload of viruses slowing your system down as they inject themselves into everything that's outward bound?

Smiley
Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 30, 2017, 06:43:19 PM
 #30

Maybe there's a shitload of viruses slowing your system down as they inject themselves into everything that's outward bound?
Smiley

I have been in IT technologies for years (database programming, system administration, etc.), so I assure you my computers, smartphones, etc all are absolutely free from any viruses or malware of any sort

Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
onurgozupek
Sr. Member
****
Offline Offline

Activity: 672
Merit: 284

https://earn.com/onurgozupek


View Profile WWW
October 30, 2017, 09:16:15 PM
 #31

It's a good idea to keep a copy of the Bitcoin blockchain on a cloud drive and it will definitely be faster to download the whole chain rather than synchronising it from nodes.

In crypto me trust Smiley
ZipReg
Member
**
Offline Offline

Activity: 324
Merit: 66


View Profile WWW
October 30, 2017, 10:58:10 PM
 #32

I am a bitcoin noob, however, I am not a best practice data retention noob Smiley  so the first thing I did after getting my bitcoin core install fully synced today? I compressed it and backed it up.

The whole 160gb compressed down to 116gb in 25 dvd sized file library. It took like 4 hours to compress.

You never know when you might need to recover due to unforeseen events / failure.

As far as storing it on google, I wouldn't personally use them.

I would also note there is a test function (t in 7z) that will confirm the integrity of an archive, but this does not confirm the integrity of the contents inside have not been tampered with before compression, only that the archive itself is intact.

7z t archive.name

Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 31, 2017, 02:40:00 AM
 #33

Maybe there's a shitload of viruses slowing your system down as they inject themselves into everything that's outward bound?
Smiley

I have been in IT technologies for years (database programming, system administration, etc.), so I assure you my computers, smartphones, etc all are absolutely free from any viruses or malware of any sort

Mathematical proofs of the integrity of data are required.

Please stop with the "Trust me" game.

Nobody is fooled by it.
Erelas
Full Member
***
Offline Offline

Activity: 280
Merit: 102


View Profile WWW
October 31, 2017, 02:58:23 AM
 #34

Caveat, I'll never be fluent in crypto anything to hold my own, but...

The initiative shown by the OP in making the blockchain available like this (trusting Google or not) is admirable, sometimes you just have to do something.

Reading thru most of the comments, especially those that mention "trust / trustless" always kind of makes me grin, the mathematics behind the blockchain is the only thing that is trustless about cryptocurrency, the rest is very much based on trust.  Maybe that's just me being picky over semantics, meh.

DeepOnion    ▬▬  Anonymous and Untraceable  ▬▬    ENJOY YOUR PRIVACY  •  JOIN DEEPONION
▐▐▐▐▐▐▐▐   ANN  Whitepaper  Facebook  Twitter  Telegram  Discord    ▌▌▌▌▌▌▌▌
Get $ONION  (✔Cryptopia  ✔KuCoin)  |  VoteCentral  Register NOW!  |  Download DeepOnion
Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 31, 2017, 12:08:18 PM
 #35

Caveat, I'll never be fluent in crypto anything to hold my own, but...

The initiative shown by the OP in making the blockchain available like this (trusting Google or not) is admirable, sometimes you just have to do something.

Reading thru most of the comments, especially those that mention "trust / trustless" always kind of makes me grin, the mathematics behind the blockchain is the only thing that is trustless about cryptocurrency, the rest is very much based on trust.  Maybe that's just me being picky over semantics, meh.

Okay, so this phraseology....pulled from a standard known download location for Core....

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
You MUST verify the integrity of this software before running it.

Should be changed to...

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
Don't worry about the integrity of the software, trust us.
Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
October 31, 2017, 02:46:32 PM
 #36


My archive consists of the blockchain database data only.

The Bitcoin Core 0.15.0.1 wallet application should be downloaded exclusively(!)  from the official site:

https://bitcoin.org/en/download

Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
LoyceV
Legendary
*
Offline Offline

Activity: 1400
Merit: 2951


Self-made Legendary!


View Profile WWW
October 31, 2017, 03:14:06 PM
 #37

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
You MUST verify the integrity of this software before running it.

Should be changed to...

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
Don't worry about the integrity of the software, trust us.
Anybody who knows anything about security would never say "trust me".
You must have heard this phrase before:

Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 31, 2017, 08:06:52 PM
 #38

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
You MUST verify the integrity of this software before running it.

Should be changed to...

You can also use Bitcoin Core as a very secure Bitcoin wallet.
Latest stable version: 0.15.0 (September 2017) [BitTorrent]
Don't worry about the integrity of the software, trust us.
Anybody who knows anything about security would never say "trust me".
You must have heard this phrase before:


That Bitcoin (and many other potential blockchain applications) are trustless protocols is a key concept.

Here I simply apply this principle to the issue of downloading a database.
Spendulus
Legendary
*
Offline Offline

Activity: 2156
Merit: 1139



View Profile
October 31, 2017, 08:11:59 PM
 #39


My archive consists of the blockchain database data only.

The Bitcoin Core 0.15.0.1 wallet application should be downloaded exclusively(!)  from the official site:

https://bitcoin.org/en/download


And THEN the signature checked, to validate the downloaded copy, before installation.
Bestcoin-fan
Member
**
Offline Offline

Activity: 153
Merit: 10


View Profile
November 01, 2017, 05:07:24 AM
 #40


My archive consists of the blockchain database data only.

The Bitcoin Core 0.15.0.1 wallet application should be downloaded exclusively(!)  from the official site:

https://bitcoin.org/en/download


And THEN the signature checked, to validate the downloaded copy, before installation.

Tell me please, exactly what should I do (brief yet step-by-step guide) to provide you with the signatures for the database I uploaded to the Google drives?

Donate (SegWit BTC) 3KdUnrmCvFSRNFErTb2m4RYvWiuJpLCrUZ
Pages: « 1 [2] 3 »  All
  Print  
 
Jump to:  

Bitcointalk.org is not available or authorized for sale. Do not believe any fake listings.
Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!