Bitcoin Forum
April 24, 2024, 01:23:37 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: [Guide] Split Bitcoin blockchain between SSD & HDD and speed up the initial sync  (Read 308 times)
actuallytwolamas (OP)
Member
**
Offline Offline

Activity: 69
Merit: 20

Lama


View Profile WWW
July 22, 2018, 11:07:02 AM
Last edit: July 22, 2018, 02:39:44 PM by actuallytwolamas
Merited by Welsh (6), Jet Cash (5), vapourminer (3), Husna QA (1), hase0278 (1)
 #1

So, for a very long time, I wanted to run a full node myself to contribute to the bitcoin network. I had a very low space on SSD, but enough space on HDD. However, I read that downloading the whole blockchain using only HDD could be problematic as it might take days, and probably never fully catch up with the network, but I still wanted to try it myself.
I installed the Bitcoin core client, and tried to sync the blockchain only to HDD, but after 10 minutes found out that the expected sync time would be 4 days, which is way too much.
So, I decided that I need to split the blockchain between SSD and HDD. After splitting, the expected sync time dropped to 9 hours, so in 15 hours I finally had a fully synced blockchain on my PC.

How I did that? Here are the steps I followed:
 1. Download and install the Bitcoin core client from the official website. https://bitcoin.org/en/download
 2. After installation, you choose where the Bitcoin Core is going to be stored (wallet, and whole blockchain). Choose HDD (in my case it's 'F:/ disk') and press OK.
 3. Let the client run for a few minutes until it creates necessary folders and downloads all necessary files. Close the client (wait until it fully closes).
 4. Create a new folder in your SSD (in my case it is 'C:\Users\user\Bitcoin')
 5. Go back to HDD where your wallet & blockchain is located. Backup the whole folder in case something goes wrong. From there you will need two folders: 'chainstate' and 'blocks\index'. Copy and paste them into SSD folder you created ('C:\Users\user\Bitcoin' in my case) and delete those folders from HDD.
 6. Press Shift + Right mouse on Bitcoin folder in HDD and choose 'Open command window here'.



 7. Type: 'mklink /j chainstate C:\Users\user\Bitcoin\chainstate' and press 'Enter'. You should see the message 'Junction created for chainstate <<===>> C:\Users\user\Bitcoin\chainstate'.
 8. Do the same with second folder, type: 'mklink /j blocks\index C:\Users\user\Bitcoin\blocks\index'.
 9. Now both folders should appear in your HDD disk and it should look like this.



 10. And we're done, start Bitcoin core client and let it sync.

Folders in SSD will take about 3-4GB of space and the whole blockchain(~170GB) will stay in HDD.
1713921817
Hero Member
*
Offline Offline

Posts: 1713921817

View Profile Personal Message (Offline)

Ignore
1713921817
Reply with quote  #2

1713921817
Report to moderator
1713921817
Hero Member
*
Offline Offline

Posts: 1713921817

View Profile Personal Message (Offline)

Ignore
1713921817
Reply with quote  #2

1713921817
Report to moderator
There are several different types of Bitcoin clients. The most secure are full nodes like Bitcoin Core, but full nodes are more resource-heavy, and they must do a lengthy initial syncing process. As a result, lightweight clients with somewhat less security are commonly used.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1713921817
Hero Member
*
Offline Offline

Posts: 1713921817

View Profile Personal Message (Offline)

Ignore
1713921817
Reply with quote  #2

1713921817
Report to moderator
1713921817
Hero Member
*
Offline Offline

Posts: 1713921817

View Profile Personal Message (Offline)

Ignore
1713921817
Reply with quote  #2

1713921817
Report to moderator
Jet Cash
Legendary
*
Offline Offline

Activity: 2688
Merit: 2449


https://JetCash.com


View Profile WWW
July 22, 2018, 11:50:40 AM
 #2

Very interesting experiment. I'll try this on a very old notebook that I've got, and I'll see if it is viable for such a low spec machine.

Offgrid campers allow you to enjoy life and preserve your health and wealth.
Save old Cars - my project to save old cars from scrapage schemes, and to reduce the sale of new cars.
My new Bitcoin transfer address is - bc1q9gtz8e40en6glgxwk4eujuau2fk5wxrprs6fys
hase0278
Hero Member
*****
Offline Offline

Activity: 882
Merit: 544


View Profile
July 22, 2018, 12:30:24 PM
 #3

Thanks for the guide but can this be used also for splitting bitcoin blockchain between SSD and an external HDD or will it only work on internal hard drives? I've wanted to do it because I want to save space as much as possible on internal storage spaces to lessen lag on my Personal Computer when using it daily while running a full node myself.
Jet Cash
Legendary
*
Offline Offline

Activity: 2688
Merit: 2449


https://JetCash.com


View Profile WWW
July 22, 2018, 12:56:51 PM
 #4


Great guide! Thanks for sharing.
Helped me at least to get the full node running Cheesy

Why did you mess up a decent thread by quoting the whole of the opening post. I put you on ignore for that really annoying habit, so no merits from me in future. I bet you haven't even started a node in the hour or so between the start of the thread and your post.

Offgrid campers allow you to enjoy life and preserve your health and wealth.
Save old Cars - my project to save old cars from scrapage schemes, and to reduce the sale of new cars.
My new Bitcoin transfer address is - bc1q9gtz8e40en6glgxwk4eujuau2fk5wxrprs6fys
actuallytwolamas (OP)
Member
**
Offline Offline

Activity: 69
Merit: 20

Lama


View Profile WWW
July 22, 2018, 12:58:06 PM
 #5

Thanks for the guide but can this be used also for splitting bitcoin blockchain between SSD and an external HDD or will it only work on internal hard drives? I've wanted to do it because I want to save space as much as possible on internal storage spaces to lessen lag on my Personal Computer when using it daily while running a full node myself.

I didn't tested myself with an external drive, but it should work. Parts in the guide with HDD change to your external drive, and you should be ok. I had a thought maybe there could be a problem with symlinks to an external drive when you plug it off from PC, but tested it now and symlinks stay in place after you plug it off and plug in your drive to the PC.
withanias
Copper Member
Newbie
*
Offline Offline

Activity: 59
Merit: 0


View Profile
July 29, 2018, 02:49:07 AM
 #6

You can also improve read speeds if you use hardware RAID 1 (disk mirroring) where each disk is on its controller. On Windows, I usually save all the blockchain files inside a VHDx file
actuallytwolamas (OP)
Member
**
Offline Offline

Activity: 69
Merit: 20

Lama


View Profile WWW
July 30, 2018, 05:39:08 AM
 #7

You can also improve read speeds if you use hardware RAID 1 (disk mirroring) where each disk is on its controller. On Windows, I usually save all the blockchain files inside a VHDx file

Could you explain how exactly you using VHDx format for the Bitcoin core client?
jnano
Member
**
Offline Offline

Activity: 301
Merit: 74


View Profile
June 18, 2019, 06:48:38 PM
Merited by vapourminer (1)
 #8

Also related to this, it's possible to temporarily use a RAM drive:
https://bitcointalk.org/index.php?topic=2291857.msg23439471#msg23439471
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!