Bitcoin Forum
October 23, 2020, 06:08:10 AM *
News: Latest Bitcoin Core release: 0.20.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)  (Read 212 times)
LoyceV
Legendary
*
Offline Offline

Activity: 2016
Merit: 7076


Thick-Skinned Gang Leader


View Profile WWW
August 21, 2019, 09:07:14 AM
Merited by pooya87 (1), tbct_mt2 (1), aplistir (1)
 #1

First: It's NOT recommended to download the blockchain any other way than having Bitcoin Core download it on it's own. Don't do it! Seriously, don't!

Still reading? Well, you'd better know what you're doing, you're on your own now Cheesy

Download a pruned blockchain
I've seen several topics about downloading and pruning Bitcoin Core. It can take a long time, and you'll need to download about 220 GB before it's done. After pruning, you'll only have a few GB left.
Since I was playing around with a VPS anyway, I now create daily snapshots of the "blocks" and "chainstate" directories. Download the latest pruned Bitcoin blockchain, unpack it to the right directory, and start your own Bitcoin Core (I'm using v0.18.0.0, older than this may not work). Configure it to prune the blockchain and you're good to go.
It should connect, and update the blocks from there. Once running, you won't need to update it from my server anymore.
This should make it possible to get a pruned and up-to-date Bitcoin Core running in an hour instead of a couple of days (or weeks on some systems).

Which directory to use (source: stackexchange.com)
Quote
Linux:
Code:
~/.bitcoin/
MacOS:
Code:
~/Library/Application Support/Bitcoin/
Windows:
Code:
%APPDATA%\Bitcoin

While my VPS is still syncing, I'd like to start some discussion about this:
How risky is this?
You have to trust not only me, but also my VPS and shared hosting too, and I can't make any guarantees about the last 2. If someone downloads my 550 blocks and his own Bitcoin Core connects and downloads the most recent blocks from there, is that enough to be fairly certain it's on the real chain?

On Reddit, theymos wrote this 3 years ago:
This is massively insecure. Bitcoin Core trusts its block database files absolutely. /u/nullc has said that it is not particularly unlikely that a maliciously-modified block database could be used for arbitrary code execution. And even if that's not possible, all sorts of more obvious evil could be done, such as allowing the provider of the block database to create a special killswitch transaction which forks everyone who used his block database, or having everyone who used his block database think that he actually owns 22 million BTC.

Nobody should ever receive block database files from untrusted sources.

Also see: https://en.bitcoin.it/wiki/Data_directory#Transferability
So again: don't do it!

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1603433290
Hero Member
*
Offline Offline

Posts: 1603433290

View Profile Personal Message (Offline)

Ignore
1603433290
Reply with quote  #2

1603433290
Report to moderator
1603433290
Hero Member
*
Offline Offline

Posts: 1603433290

View Profile Personal Message (Offline)

Ignore
1603433290
Reply with quote  #2

1603433290
Report to moderator
Carlton Banks
Legendary
*
Offline Offline

Activity: 2884
Merit: 2349



View Profile
August 21, 2019, 09:15:29 AM
Merited by Foxpup (1)
 #2

yep, don't do this

Vires in numeris
aliashraf
Hero Member
*****
Offline Offline

Activity: 1148
Merit: 898

Always remember the cause!


View Profile WWW
August 21, 2019, 01:21:38 PM
Last edit: August 21, 2019, 01:32:17 PM by aliashraf
 #3

op,
Obviously getting rid of the headache of bitcoin bootstrap is a very important topic but surprisingly it is an old story ...
Quote from: aliashraf
@eurekafag AFAIK is the first person who said something about it, july 2010(!), he used the term snapshotting (it is why I used it above, to show my respect). The topic got no attention but another user, @Bytecoin rephrased it two days later and posted a more comprehensive proposal.

Satoshi Nakamoto was still around and he never made a comment regarding this, Gavin Andersen didn't get it, neither @Theymos, ... just 2 and a half pages of non-productive discussions. Obviously in mid 2010 there were few blocks, few UTXOs and so many other problems and priorities.

Almost one year later, july 2011, Gregory Maxwell, made a contribution to this subject he basically proposed something that later was termed, UTXO Commitment, it was Merkle era, people were excited about the magical power of Merkle Trees and Maxwell proposed maintaining a Merkle Hash Tree of UTXO by full nodes that enables them to spot an unspent output efficiently while miners include the root of such a tree in coinbase transaction (later others proposed including it directly in block header) this way, 'lite clients' would be able to ask for proof of any tx input as being committed to the UTXO Merkle root included in latest blocks.  
Months ago, when I was writing the above paragraphs, I was somehow confused supposing it would need a hard fork to be implemented which is not true.
Actually, it is very easy to have such a UTXO commitment feature in bitcoin with a very small piece of code and a soft-fork as legacy nodes could continue confirming blocks and won't fork out.

I've contributed to this subject even more and proposed an even softer approach for implementing this idea. According to my approach, we don't need to enforce an upgrade for a majority of full nodes. We need a majority of good UTXO actors against the malicious ones:

Suppose we have convinced like 10% of miners to join our UTXO commitment upgrade such that we have like 50 good UTXO signals every 100 blocks, now it will be enough to have less than 10 bad UTXO signals in the same window of time (i.e. 440 null signals). Upgraded nodes could consider forking from blocks with malicious signals after they get a majority of like 450 consistent signals in last 500 blocks.

UTXO commitment has been discussed and developed a lot more after G Maxwell has proposed and abandoned it, but what bothers me is how it has been undermined in bitcoin community and boosted in BCH. They have done a lot more about it in bitcoin cash and I'm just like  Shocked Huh

             ▄██▄
   ▄██▄      ▀█▀▀     ▄██▄
   ▀██▀▄  ▄▄█████▄▄  ▐███▀
       ███████████████
      ████████▀▄▄▄▀████
 ▄▄  ▐███▀▄▀██▄▀▀▀▄█████  ▄▄
████▀█████▄███▀▀█████ ██▀████
 ▀▀  ▐███▄███ ██ ████ █▌  ▀▀
      ▀████▄██▄▄███▀▄█▀
    ▄▄ █▀██████▀▄▄▄█▀█ ▄▄
   ████▀   ▀▀▀█▀▀▀   ▐████
    ▀▀       ▄██▄      ▀▀
             ▀██▀
⟩ ⟩ ⟩             ▄▄▄
  ▄▄▄▄▄▄▄▄▄▄█   █▄
 █           ▀▀▀  █
 ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀
▄▀▀ ▄▄▄▄▄▄▄▄▄▄▄▄ ▀▀▄
█ ▄▀ ▄▄▄▄▄▄▄    ▀█ █
█ █ █       █    █ ▄
█ █ ▄▀▀▀▀▀▀▄▄    █ █
█ █ ▀▄▄▄▄▀▀▄▄▀▀▄ █ █
█ █ █   █  ██  █ █ █
█ █ ▄▀▀▀▀▄▄▀▀▄▄▀ █ █
█ ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ █
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
⟩ ⟩ ⟩       ▄████▄  ▄████▄
      ████████████████
      ████████████████
       ██████████████
        ▀██████████▀
██        ▀██████▀        ██
██▌   ▄            ▄   ▐██
███  ███▄          ▄███  ███
▀███▄ ▀███▄      ▄███▀ ▄███▀
  ▀████████      ████████▀
     ▀████▀      ▀████▀
     ▄   ▄▄      ▄▄   ▄
     ▀█████      █████▀
Carlton Banks
Legendary
*
Offline Offline

Activity: 2884
Merit: 2349



View Profile
August 21, 2019, 03:43:50 PM
 #4

UTXO commitment has been discussed and developed a lot more after G Maxwell has proposed and abandoned it, but what bothers me is how it has been undermined in bitcoin community


you clearly aren't familiar with the parable of the hare and the tortoise

  • hare and tortoise race
  • hare begins race too fast
  • hare takes nap, overconfident of win
  • tortoise wins while hare asleep


please don't reply, you have a bad reputation

Vires in numeris
aliashraf
Hero Member
*****
Offline Offline

Activity: 1148
Merit: 898

Always remember the cause!


View Profile WWW
August 21, 2019, 03:51:45 PM
 #5

UTXO commitment has been discussed and developed a lot more after G Maxwell has proposed and abandoned it, but what bothers me is how it has been undermined in bitcoin community


you clearly aren't familiar with the parable of the hare and the tortoise

  • hare and tortoise race
  • hare begins race too fast
  • hare takes nap, overconfident of win
  • tortoise wins while hare asleep


please don't reply, you have a bad reputation
Joking? Tortoises lose nowadays, wake-up and don't say anything more, besides your worse reputation, you are trolling right now.

             ▄██▄
   ▄██▄      ▀█▀▀     ▄██▄
   ▀██▀▄  ▄▄█████▄▄  ▐███▀
       ███████████████
      ████████▀▄▄▄▀████
 ▄▄  ▐███▀▄▀██▄▀▀▀▄█████  ▄▄
████▀█████▄███▀▀█████ ██▀████
 ▀▀  ▐███▄███ ██ ████ █▌  ▀▀
      ▀████▄██▄▄███▀▄█▀
    ▄▄ █▀██████▀▄▄▄█▀█ ▄▄
   ████▀   ▀▀▀█▀▀▀   ▐████
    ▀▀       ▄██▄      ▀▀
             ▀██▀
⟩ ⟩ ⟩             ▄▄▄
  ▄▄▄▄▄▄▄▄▄▄█   █▄
 █           ▀▀▀  █
 ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀
▄▀▀ ▄▄▄▄▄▄▄▄▄▄▄▄ ▀▀▄
█ ▄▀ ▄▄▄▄▄▄▄    ▀█ █
█ █ █       █    █ ▄
█ █ ▄▀▀▀▀▀▀▄▄    █ █
█ █ ▀▄▄▄▄▀▀▄▄▀▀▄ █ █
█ █ █   █  ██  █ █ █
█ █ ▄▀▀▀▀▄▄▀▀▄▄▀ █ █
█ ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ █
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
⟩ ⟩ ⟩       ▄████▄  ▄████▄
      ████████████████
      ████████████████
       ██████████████
        ▀██████████▀
██        ▀██████▀        ██
██▌   ▄            ▄   ▐██
███  ███▄          ▄███  ███
▀███▄ ▀███▄      ▄███▀ ▄███▀
  ▀████████      ████████▀
     ▀████▀      ▀████▀
     ▄   ▄▄      ▄▄   ▄
     ▀█████      █████▀
tbct_mt2
Hero Member
*****
Offline Offline

Activity: 1036
Merit: 585


First 100% Liquid Stablecoin Backed by Gold


View Profile
August 21, 2019, 04:01:37 PM
 #6

Yes, don't do this!

Around less than one year ago, when I searched for bitcoin wallets (that is my first time to use bitcoin wallets that I can control private key). I firstly began with bitcoin core wallet, and as OP wrote, my computer was unable to fully synchronize with bitcoin network, due to hardware storage capacity.
It was also slow down my computer operating speed, so I gave up, and search more.
Finally, I found good threads on Electrum (in forum), and used Electrum by now.

aplistir
Full Member
***
Offline Offline

Activity: 373
Merit: 175



View Profile
August 21, 2019, 04:30:27 PM
 #7

Brilliant idea  Grin

You do not need to have daily snapshots of it though. Even 6 months old snapshot would speed up the sync considerably.

Would be even better if you could make a pruned and zipped blockchain available, get a hash of it, and some other users verify that the pruned blockchain is correct. (how?)
Then new downloaders can trust with even higher certainty that it has not been tampered with.




My Address: 121f7zb2U4g9iM4MiJTDhEzqeZGHzq5wLh
aliashraf
Hero Member
*****
Offline Offline

Activity: 1148
Merit: 898

Always remember the cause!


View Profile WWW
August 21, 2019, 06:49:01 PM
 #8

I'd rather run SPV client and connect to lots of full nodes rather than do this Tongue

UTXO commitment has been discussed and developed a lot more after G Maxwell has proposed and abandoned it, but what bothers me is how it has been undermined in bitcoin community and boosted in BCH. They have done a lot more about it in bitcoin cash and I'm just like  Shocked Huh

It's not undermined, but generally people aren't interested to use UTXO commitment because actual full node (which validate and store all blocks) is still needed and higher degree of trust required.
I've been debating this __higher degree of trust__ thing at least two times with Andrew and Greg and found no reason behind their arguments. If they haven't moved this thread from technical sub-forum I'd include a full mathematical analysis right here to prove there is no trust issue (I mean a practical issue). Briefly speaking, there is no rational attack scenario for generating a large number of blocks committing to a poisoned UTXO. To be more specific there is always a number n for the pruned chain length that guarantees full security from a game-theoretic point of view. For current bitcoin blockchain, a UTXO with 2000 block commitments is practically safe for wallets that are handling 1 billion dollars worth of assets! The only possible problem would be Sybill attack which could be easily mitigated by introducing a lower bound for difficulty when bootstrapping or including checkpoints. It can also be included in the protocol itself: require more "work" instead of more number of commitments. easy.

As of your comment about the need for full nodes, let's make it crystal clear: A pruned node IS actually a full node! The only shortage they have is their inability to help with the current sync strategy in bitcoin which is absurd anyway.

Quote
On a different note, have you heard about Neutrino? It's quite interesting protocol for SPV/light wallet which offer some degree of privacy.
Yes, I know about client-side filtering. It helps with privacy for spv wallets. Alternatively, I'm thinking of eliminating spv wallets completely.

             ▄██▄
   ▄██▄      ▀█▀▀     ▄██▄
   ▀██▀▄  ▄▄█████▄▄  ▐███▀
       ███████████████
      ████████▀▄▄▄▀████
 ▄▄  ▐███▀▄▀██▄▀▀▀▄█████  ▄▄
████▀█████▄███▀▀█████ ██▀████
 ▀▀  ▐███▄███ ██ ████ █▌  ▀▀
      ▀████▄██▄▄███▀▄█▀
    ▄▄ █▀██████▀▄▄▄█▀█ ▄▄
   ████▀   ▀▀▀█▀▀▀   ▐████
    ▀▀       ▄██▄      ▀▀
             ▀██▀
⟩ ⟩ ⟩             ▄▄▄
  ▄▄▄▄▄▄▄▄▄▄█   █▄
 █           ▀▀▀  █
 ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀
▄▀▀ ▄▄▄▄▄▄▄▄▄▄▄▄ ▀▀▄
█ ▄▀ ▄▄▄▄▄▄▄    ▀█ █
█ █ █       █    █ ▄
█ █ ▄▀▀▀▀▀▀▄▄    █ █
█ █ ▀▄▄▄▄▀▀▄▄▀▀▄ █ █
█ █ █   █  ██  █ █ █
█ █ ▄▀▀▀▀▄▄▀▀▄▄▀ █ █
█ ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ █
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
⟩ ⟩ ⟩       ▄████▄  ▄████▄
      ████████████████
      ████████████████
       ██████████████
        ▀██████████▀
██        ▀██████▀        ██
██▌   ▄            ▄   ▐██
███  ███▄          ▄███  ███
▀███▄ ▀███▄      ▄███▀ ▄███▀
  ▀████████      ████████▀
     ▀████▀      ▀████▀
     ▄   ▄▄      ▄▄   ▄
     ▀█████      █████▀
pooya87
Legendary
*
Online Online

Activity: 2156
Merit: 3204


Remember tonight for it's the beginning of forever


View Profile
August 22, 2019, 04:57:27 AM
 #9

this topic may not be appropriate in the beginners and help board. the warning part is OK but the walk-through followup doesn't belong here. it is best be posted in project development board since the only usage i can think of for this is mostly for developers trying to test things and don't need trust or security.

ETFbitcoin
Legendary
*
Offline Offline

Activity: 1666
Merit: 2549

Use SegWit and enjoy lower fees.


View Profile WWW
August 22, 2019, 08:42:21 AM
Merited by aliashraf (1)
 #10

I've been debating this __higher degree of trust__ thing at least two times with Andrew and Greg and found no reason behind their arguments. If they haven't moved this thread from technical sub-forum I'd include a full mathematical analysis right here to prove there is no trust issue (I mean a practical issue). Briefly speaking, there is no rational attack scenario for generating a large number of blocks committing to a poisoned UTXO. To be more specific there is always a number n for the pruned chain length that guarantees full security from a game-theoretic point of view. For current bitcoin blockchain, a UTXO with 2000 block commitments is practically safe for wallets that are handling 1 billion dollars worth of assets! The only possible problem would be Sybill attack which could be easily mitigated by introducing a lower bound for difficulty when bootstrapping or including checkpoints. It can also be included in the protocol itself: require more "work" instead of more number of commitments. easy.

I should specify the higher degree of trust depends on how they implement UTXO commitment. Few implementation (sometimes under different name) i've seen are ridiculous, such have central or master node as UTXO source.

The UTXO commitment idea itself isn't bad and i don't mind if Bitcoin protocol/client adopt UTXO commitment feature (if done right).
But personally, i prefer use it to allow client to use their Bitcoin while waiting all blocks are downloaded & verified.

aliashraf
Hero Member
*****
Offline Offline

Activity: 1148
Merit: 898

Always remember the cause!


View Profile WWW
August 22, 2019, 10:17:58 AM
 #11

The UTXO commitment idea itself isn't bad and i don't mind if Bitcoin protocol/client adopt UTXO commitment feature (if done right).
But personally, i prefer use it to allow client to use their Bitcoin while waiting all blocks are downloaded & verified.
Koodos  Smiley
Very good strategy, I can imagine a lazy synchronization process for extending the chain height in reverse order as far as the user wishes and she is capable of affording resources for this.

this topic may not be appropriate in the beginners and help board. the warning part is OK but the walk-through followup doesn't belong here. it is best be posted in project development board since the only usage i can think of for this is mostly for developers trying to test things and don't need trust or security.
Actually, it was opened in the right board, mods moved it here apparently because of the "Don't do this!" thing.

             ▄██▄
   ▄██▄      ▀█▀▀     ▄██▄
   ▀██▀▄  ▄▄█████▄▄  ▐███▀
       ███████████████
      ████████▀▄▄▄▀████
 ▄▄  ▐███▀▄▀██▄▀▀▀▄█████  ▄▄
████▀█████▄███▀▀█████ ██▀████
 ▀▀  ▐███▄███ ██ ████ █▌  ▀▀
      ▀████▄██▄▄███▀▄█▀
    ▄▄ █▀██████▀▄▄▄█▀█ ▄▄
   ████▀   ▀▀▀█▀▀▀   ▐████
    ▀▀       ▄██▄      ▀▀
             ▀██▀
⟩ ⟩ ⟩             ▄▄▄
  ▄▄▄▄▄▄▄▄▄▄█   █▄
 █           ▀▀▀  █
 ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀
▄▀▀ ▄▄▄▄▄▄▄▄▄▄▄▄ ▀▀▄
█ ▄▀ ▄▄▄▄▄▄▄    ▀█ █
█ █ █       █    █ ▄
█ █ ▄▀▀▀▀▀▀▄▄    █ █
█ █ ▀▄▄▄▄▀▀▄▄▀▀▄ █ █
█ █ █   █  ██  █ █ █
█ █ ▄▀▀▀▀▄▄▀▀▄▄▀ █ █
█ ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ █
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
⟩ ⟩ ⟩       ▄████▄  ▄████▄
      ████████████████
      ████████████████
       ██████████████
        ▀██████████▀
██        ▀██████▀        ██
██▌   ▄            ▄   ▐██
███  ███▄          ▄███  ███
▀███▄ ▀███▄      ▄███▀ ▄███▀
  ▀████████      ████████▀
     ▀████▀      ▀████▀
     ▄   ▄▄      ▄▄   ▄
     ▀█████      █████▀
LoyceV
Legendary
*
Offline Offline

Activity: 2016
Merit: 7076


Thick-Skinned Gang Leader


View Profile WWW
August 22, 2019, 12:04:41 PM
 #12

You do not need to have daily snapshots of it though. Even 6 months old snapshot would speed up the sync considerably.
Making one zip-file per day is not a problem. I've already set it up in a cronjob.

Quote
Would be even better if you could make a pruned and zipped blockchain available, get a hash of it, and some other users verify that the pruned blockchain is correct. (how?)
I tried that before, but the files on disk (such as "blk01117.dat") are different on other systems. It would have been easy to create a checksum if those files would be exact copies all the time.

this topic may not be appropriate in the beginners and help board. the warning part is OK but the walk-through followup doesn't belong here. it is best be posted in project development board since the only usage i can think of for this is mostly for developers trying to test things and don't need trust or security.
Good point, for developers it could be convenient to quickly have a pruned node running. Although some would argue that's what the Testnet is for.
I posted this in Project Development, but a Mod seemed to disagree.

Actually, it was opened in the right board, mods moved it here apparently because of the "Don't do this!" thing.
I wanted to make very clear this isn't supposed to be used for real funds, but I also believe in freedom to make your own choices.

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!