Bitcoin Forum
September 22, 2018, 10:08:02 PM *
News: ♦♦ New info! Bitcoin Core users absolutely must upgrade to previously-announced 0.16.3 [Torrent]. All Bitcoin users should temporarily trust confirmations slightly less. More info.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: [Guide] Split Bitcoin blockchain between SSD & HDD and speed up the initial sync  (Read 117 times)
actuallytwolamas
Member
**
Offline Offline

Activity: 70
Merit: 10

Lama


View Profile WWW
July 22, 2018, 11:07:02 AM
Merited by Jet Cash (5), hase0278 (1)
 #1

So, for a very long time, I wanted to run a full node myself to contribute to the bitcoin network. I had a very low space on SSD, but enough space on HDD. However, I read that downloading the whole blockchain using only HDD could be problematic as it might take days, and probably never fully catch up with the network, but I still wanted to try it myself.
I installed the Bitcoin core client, and tried to sync the blockchain only to HDD, but after 10 minutes found out that the expected sync time would be 4 days, which is way too much.
So, I decided that I need to split the blockchain between SSD and HDD. After splitting, the expected sync time dropped to 9 hours, so in 15 hours I finally had a fully synced blockchain on my PC.

How I did that? Here are the steps I followed:
 1. Download and install the Bitcoin core client from the official website. https://bitcoin.org/en/download
 2. After installation, you choose where the Bitcoin Core is going to be stored (wallet, and whole blockchain). Choose HDD (in my case it's 'F:/ disk') and press OK.
 3. Let the client run for a few minutes until it creates necessary folders and downloads all necessary files. Close the client (wait until it fully closes).
 4. Create a new folder in your SSD (in my case it is 'C:\Users\user\Bitcoin')
 5. Go back to HDD where your wallet & blockchain is located. Backup the whole folder in case something goes wrong. From there you will need two folders: 'chainstate' and 'blocks\index'. Copy and paste them into SSD folder you created ('C:\Users\user\Bitcoin' in my case) and delete those folders from HDD.
 6. Press Shift + Right mouse on Bitcoin folder in HDD and choose 'Open command window here'.



 7. Type: 'mklink /j chainstate C:\Users\user\Bitcoin\chainstate' and press 'Enter'. You should see the message 'Junction created for chainstate <<===>> C:\Users\user\Bitcoin\chainstate'.
 8. Do the same with second folder, type: 'mklink /j blocks\index C:\Users\user\Bitcoin\blocks\index'.
 9. Now both folders should appear in your HDD disk and it should look like this.



 10. And we're done, start Bitcoin core client and let it sync.

Folders in SSD will take about 3-4GB of space and the whole blockchain(~170GB) will stay in HDD.
1537654082
Hero Member
*
Offline Offline

Posts: 1537654082

View Profile Personal Message (Offline)

Ignore
1537654082
Reply with quote  #2

1537654082
Report to moderator
1537654082
Hero Member
*
Offline Offline

Posts: 1537654082

View Profile Personal Message (Offline)

Ignore
1537654082
Reply with quote  #2

1537654082
Report to moderator
1537654082
Hero Member
*
Offline Offline

Posts: 1537654082

View Profile Personal Message (Offline)

Ignore
1537654082
Reply with quote  #2

1537654082
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1537654082
Hero Member
*
Offline Offline

Posts: 1537654082

View Profile Personal Message (Offline)

Ignore
1537654082
Reply with quote  #2

1537654082
Report to moderator
Jet Cash
Legendary
*
Offline Offline

Activity: 1008
Merit: 1204


I don't merit Spambies


View Profile WWW
July 22, 2018, 11:50:40 AM
 #2

Very interesting experiment. I'll try this on a very old notebook that I've got, and I'll see if it is viable for such a low spec machine.

Kunzaii
Jr. Member
*
Offline Offline

Activity: 103
Merit: 2


View Profile
July 22, 2018, 12:18:21 PM
 #3

So, for a very long time, I wanted to run a full node myself to contribute to the bitcoin network. I had a very low space on SSD, but enough space on HDD. However, I read that downloading the whole blockchain using only HDD could be problematic as it might take days, and probably never fully catch up with the network, but I still wanted to try it myself.
I installed the Bitcoin core client, and tried to sync the blockchain only to HDD, but after 10 minutes found out that the expected sync time would be 4 days, which is way too much.
So, I decided that I need to split the blockchain between SSD and HDD. After splitting, the expected sync time dropped to 9 hours, so in 15 hours I finally had a fully synced blockchain on my PC.

How I did that? Here are the steps I followed:
 1. Download and install the Bitcoin core client from the official website. https://bitcoin.org/en/download
 2. After installation, you choose where the Bitcoin Core is going to be stored (wallet, and whole blockchain). Choose HDD (in my case it's 'F:/ disk') and press OK.
 3. Let the client run for a few minutes until it creates necessary folders and downloads all necessary files. Close the client (wait until it fully closes).
 4. Create a new folder in your SSD (in my case it is 'C:\Users\user\Bitcoin')
 5. Go back to HDD where your wallet & blockchain is located. Backup the whole folder in case something goes wrong. From there you will need two folders: 'chainstate'and 'blocks\index'. Copy and paste them into SSD folder you created ('C:\Users\user\Bitcoin' in my case) and delete those folders from HDD.
 6. Press Shift + Right mouse on Bitcoin folder in HDD and choose 'Open command window here'.



 7. Type: 'mklink /j chainstate C:\Users\user\Bitcoin\chainstate' and press 'Enter'. You should see the message 'Junction created for chainstate <<===>> C:\Users\user\Bitcoin\chainstate'.
 8. Do the same with second folder, type: 'mklink /j blocks\index C:\Users\user\Bitcoin\blocks\index'.
 9. Now both folders should appear in your HDD disk and it should look like this.



 10. And we're done, start Bitcoin core client and let it sync.

Folders in SSD will take about 3-4GB of space and the whole blockchain(~170GB) will stay in HDD.


Great guide! Thanks for sharing.
Helped me at least to get the full node running Cheesy

★ PRiVCY ➢ Own Your Privacy! ➢ Best privacy crypto-market! ★
✈✈✈[PoW/PoS]✅[Tor]✅[Airdrop]✈✈✈ (https://privcy.io/)
hase0278
Hero Member
*****
Online Online

Activity: 868
Merit: 535



View Profile
July 22, 2018, 12:30:24 PM
 #4

Thanks for the guide but can this be used also for splitting bitcoin blockchain between SSD and an external HDD or will it only work on internal hard drives? I've wanted to do it because I want to save space as much as possible on internal storage spaces to lessen lag on my Personal Computer when using it daily while running a full node myself.

        ▄▄███████████▄▄
     ▄██▀▀           ▀▀██▄
   ▄█▀   ▄▄█████████▄▄   ▀█▄
  █▀  ▄███████▌ █▌ █████▄  ▀█
 █▀  ██  ██         ▀█████  ▀█
▐█  ██▀▀█████   ▄▄▄   █████  █▌
█▌ ▐██▄▄████▌  ████   █████▌ ▐█
█  ███  ████        ▄███████  █
█▌ ▐█▀▀████▌   ▄▄▄  ▀██████▌ ▐█
▐█  █▄▄████   ████   ▐█████  █▌
 █▄  ████           ▄█████  ▄█
  █▄  ▀████  ███  ██████▀  ▄█
   ▀█▄   ▀▀█████████▀▀   ▄█▀
     ▀██▄▄           ▄▄██▀
        ▀▀███████████▀▀
bitcore.
...The Future Is Now...
▄▄▄▄▄▄▄▄    ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄     ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄

        ██
██████
        ██
        ██████
        ██
██████
        ██
        ██████
        ██
██████
                 ▄▄██
 ▄██████████████████████▄
█▀         ▄▄██▀▀   ██  ▀█
█▄     ▄▄██▀▀        ██  █
█▀█████████████████████████▄
█                          ▀█
█                           █
█              ▄████▄       █
█             ██▀  ▀██      █
█            ▐█▌    ▐████████
█             ██▄  ▄██      █
█              ▀████▀       █
█                           █
█▄                         ▄█
 ▀█████████████████████████▀
.
iOs
Android

        ██
        ██████
        ██
██████
        ██
        ██████
        ██
██████
        ██
        ██████


     ▄▄█████████▄▄
   ▄███▀▀     ▀▀███▄
  ███             ███
 ███               ███
▐██   ▐█▄   ▄███▄   ██▌
██▌    ███▄██████▀  ▐██
██▌    ▐████████    ▐██
▐██     ▐██████     ██▌
 ███   ▀█████▀     ███
  ███             ███
   ▀███▄▄     ▄▄███▀
     ▀▀█████████▀▀


     ▄▄█████████▄▄
   ▄███▀▀     ▀▀███▄
  ███             ███
 ███       ▄███    ███
▐██       █████     ██▌
██▌       ███       ▐██
██▌     ███████     ▐██
▐██       ███       ██▌
 ███      ███      ███
  ███             ███
   ▀███▄▄     ▄▄███▀
     ▀▀█████████▀▀


     ▄▄█████████▄▄
   ▄███▀▀     ▀▀███▄
  ███             ███
 ███         ▄▄█▌  ███
▐██      ▄▄█████    ██▌
██▌   ▄████████     ▐██
██▌  ▐████████      ▐██
▐██    ▀▀████       ██▌
 ███     ▀██       ███
  ███             ███
   ▀███▄▄     ▄▄███▀
     ▀▀█████████▀▀
Website
ANN
Block Explorer
Jet Cash
Legendary
*
Offline Offline

Activity: 1008
Merit: 1204


I don't merit Spambies


View Profile WWW
July 22, 2018, 12:56:51 PM
 #5


Great guide! Thanks for sharing.
Helped me at least to get the full node running Cheesy

Why did you mess up a decent thread by quoting the whole of the opening post. I put you on ignore for that really annoying habit, so no merits from me in future. I bet you haven't even started a node in the hour or so between the start of the thread and your post.

actuallytwolamas
Member
**
Offline Offline

Activity: 70
Merit: 10

Lama


View Profile WWW
July 22, 2018, 12:58:06 PM
 #6

Thanks for the guide but can this be used also for splitting bitcoin blockchain between SSD and an external HDD or will it only work on internal hard drives? I've wanted to do it because I want to save space as much as possible on internal storage spaces to lessen lag on my Personal Computer when using it daily while running a full node myself.

I didn't tested myself with an external drive, but it should work. Parts in the guide with HDD change to your external drive, and you should be ok. I had a thought maybe there could be a problem with symlinks to an external drive when you plug it off from PC, but tested it now and symlinks stay in place after you plug it off and plug in your drive to the PC.
withanias
Newbie
*
Offline Offline

Activity: 31
Merit: 0


View Profile
July 29, 2018, 02:49:07 AM
 #7

You can also improve read speeds if you use hardware RAID 1 (disk mirroring) where each disk is on its controller. On Windows, I usually save all the blockchain files inside a VHDx file
actuallytwolamas
Member
**
Offline Offline

Activity: 70
Merit: 10

Lama


View Profile WWW
July 30, 2018, 05:39:08 AM
 #8

You can also improve read speeds if you use hardware RAID 1 (disk mirroring) where each disk is on its controller. On Windows, I usually save all the blockchain files inside a VHDx file

Could you explain how exactly you using VHDx format for the Bitcoin core client?
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!