1TB micro SD card for storing blockchain, any reason not to do this?

[zabzob]

Hey all, the 1TB SSD on my Thinkpad T590 (running a full node) is running out of space and the blockchain will outgrow it within the next few months. Because of current high SSD prices, I'm looking for a cheaper alternative to upgrading to a 2TB SSD. Since the laptop has an SD card reader, I'm wondering whether storing the blockchain on a 1TB microSD card would create any issues?

[LoyceV]

I expect this to work fine. But: only put blocks on the SD card, not chainstate! Depending on your RAM, Bitcoin Core will write terabytes of data to chainstate, and SD cards don't handle that very well. The blocks directory is written only once.

[zabzob]

I expect this to work fine. But: only put blocks on the SD card, not chainstate! Depending on your RAM, Bitcoin Core will write terabytes of data to chainstate, and SD cards don't handle that very well. The blocks directory is written only once.

Thanks so much, I'll do it this way

[nc50lc]

Thanks so much, I'll do it this way

I take that you're already aware that it requires --blocksdir arg or config to tell Core where the new blocks directory path is?
If not, I recommend setting it in your bitcoin.conf file.

Anyways, what happened to your other option which is HDD? It can be plugged externally.
I saw you mentioned this in your other reply a few months ago.

[zabzob]

I take that you're already aware that it requires --blocksdir arg or config to tell Core where the new blocks directory path is?
If not, I recommend setting it in your bitcoin.conf file.

I wasn't aware of this, but was going to try to figure it out or ask here once I have the sd card. Thanks for saving me the trouble, I'll edit the config file as you explain when the time comes. It will probably be a month or two before I migrate the blocks file.

Anyways, what happened to your other option which is HDD? It can be plugged externally.
I saw you mentioned this in your other reply a few months ago.

I considered this option, but would rather avoid using an external device, since the laptop moves around frequently. I would have gone this way had the sd option not been feasible. I might still do it if an external hdd is much cheaper than a 1tb sd card. There's a relatively cheap sd from Sabrent that looks reliable.

[LoyceV]

I take that you're already aware that it requires --blocksdir arg or config to tell Core where the new blocks directory path is?

I prefer to deal with this on a filesystem level (on Linux), so I just use a symlink as blocks directory. Or you can just mount the SD card as blocks directory directly.

[zabzob]

I take that you're already aware that it requires --blocksdir arg or config to tell Core where the new blocks directory path is?

I prefer to deal with this on a filesystem level (on Linux), so I just use a symlink as blocks directory. Or you can just mount the SD card as blocks directory directly.

Thanks, I'll consider this option as well

[Bitcoin Smith]

SD card slots are available in the old model laptops, and I assume their R/W speed is less compared to SSD or even HDD!

Why not just use an HDD that is reliable compared to an SD, but you said Micro SD, which is even possible to insert in a laptop without the SD card reader? That is going to take months to complete sync.

[nc50lc]

Why not just use an HDD that is reliable compared to an SD, but you said Micro SD, which is even possible to insert in a laptop without the SD card reader? That is going to take months to complete sync.

This is why it's only the blocksdir that'll be moved out of the internal SSD.
And T590 has an integrated SD Card reader, it's an SD Card slot on its side.

If it's about sync speed, OP obviously already has his node synced based from the post's context,
it's just he needs to upgrade his 1TB SSD since it wont have enough space for the blockchain a few years from now.
The main issue is if he'll need to rescan or build other databases which will be slower than HDD.

I take that you're already aware that it requires --blocksdir arg or config to tell Core where the new blocks directory path is?

I prefer to deal with this on a filesystem level (on Linux), so I just use a symlink as blocks directory. Or you can just mount the SD card as blocks directory directly.

Personally, I prefer the native way of how Bitcoin Core manages its directories.

[LoyceV]

SD card slots are available in the old model laptops, and I assume their R/W speed is less compared to SSD or even HDD!

Some SD cards are faster than some HDDs, but none of that matters much for one-time writing.

Why not just use an HDD that is reliable compared to an SD

Why are you asking things OP already answered?

I considered this option, but would rather avoid using an external device

but you said Micro SD, which is even possible to insert in a laptop without the SD card reader?

That completely depends on the laptop, I've seen them with all kinds of slots.

That is going to take months to complete sync.

With the right hardware, it can be done within a day.

The main issue is if he'll need to rescan or build other databases which will be slower than HDD.

For raw read speed, I wouldn't be surprised if a (quality) SD card is faster than an equal size HDD. Those disks are limited by their spinning speed, my 2 TB 2.5" laptop drive doesn't get above 100 MB/s.

Personally, I prefer the native way of how Bitcoin Core manages its directories.

The main reason I don't do that, is laziness Creating a symlink is faster than reading application-specific documentation.

[ABCbits]

OP, have you checked average cost of 2TB microSD and 2TB SSD on place where you live? Both of them share some same component, so i notice price of both SD and microSD card have risen a lot in past year. If the price gap is relative small, replacing your internal SSD with 2TB ones probably makes more sense.

but you said Micro SD, which is even possible to insert in a laptop without the SD card reader?

Since OP mentioned type of his laptop, it would be great if you checked specification of that laptop before writing your reply.

Despite the redesign, the port selection has not changed significantly compared with the Lenovo ThinkPad T580. The full-size SD card-slot has been removed in favor of a microSD slot.

[zabzob]

OP, have you checked average cost of 2TB microSD and 2TB SSD on place where you live? Both of them share some same component, so i notice price of both SD and microSD card have risen a lot in past year. If the price gap is relative small, replacing your internal SSD with 2TB ones probably makes more sense.

It would be a 1TB microSD, whereas if I replace the SSD I'd have to get a 2TB one because I run a lot of other apps on the laptop. 1TB microSD should keep me going for long enough to wait for SSD prices to eventually come back down. But now that I think of it, maybe even the 1TB microSD card wold run out of space too quickly. However, with no OS or any other data other than blockchain on it, it should last at least another year.

A 2tb external HDD is not out of the question, even though it would not be the most convenient for me.

[Cricktor]

I have an old Lenovo Thinkpad T520 where my 1TB SSD ran out-of-space with a non-pruned Bitcoin Core node. I added another 1TB SSD which I had spare with an adaptor and moved block data to the second SSD in the optical drive bay.

See my own topic about it:
Moving Bitcoin Core's block storage to other storage media - blocksdir option

This is the current space usage on that system (I believe I left ~1% reserved blocks on /dev/sdb1):

Code:

$ df
Filesystem                1K-blocks      Used Available Use% Mounted on
tmpfs                       1625284      4316   1620968   1% /run
/dev/mapper/vgubuntu-root 941533576 168004964 763935332  19% /
tmpfs                       8126404         0   8126404   0% /dev/shm
tmpfs                          5120         4      5116   1% /run/lock
efivarfs                         56        28        24  54% /sys/firmware/efi/efivars
/dev/sda2                   1684592    409988   1240720  25% /boot
/dev/sda1                    523248      6232    517016   2% /boot/efi
/dev/sdb1                 960302096 827059208 123458896  88% /mnt/ssdplus
tmpfs                       1625280      4768   1620512   1% /run/user/1000
$ df -h
Filesystem                 Size  Used Avail Use% Mounted on
tmpfs                      1,6G  4,3M  1,6G   1% /run
/dev/mapper/vgubuntu-root  898G  161G  729G  19% /
tmpfs                      7,8G     0  7,8G   0% /dev/shm
tmpfs                      5,0M  4,0K  5,0M   1% /run/lock
efivarfs                    56K   28K   24K  54% /sys/firmware/efi/efivars
/dev/sda2                  1,7G  401M  1,2G  25% /boot
/dev/sda1                  511M  6,1M  505M   2% /boot/efi
/dev/sdb1                  916G  789G  118G  88% /mnt/ssdplus
tmpfs                      1,6G  4,7M  1,6G   1% /run/user/1000

After 10 months blocksdata storage rose from ~716GiB to now 789GiB, that's about 73GiB increase. Remaining storage on that 1TB SSD might last me on average for another ~16 months from now.

Expect similar figures with your 1TB MicroSD card. I'm not sure if SSD prices will return to more "normal" values within the next 16 months. Fortunately I have one or two 2TB SSDs spare which I bought when prices were "normal".

[LoyceV]

Remaining storage on that 1TB SSD might last me on average for another ~16 months from now.
Expect similar figures with your 1TB MicroSD card.

I haven't tested it, but I expect it to be possible to symlink some of the old files in blocks to a directory on another partition. You have 80% free on your root partition, that would last you many years if you can spread out your blockchain.
A bash oneliner should be able to do that for thousands of files (after shutting down Bitcoin Core). If it doesn't work, you may need to download all blocks again.

[ABCbits]

A bash oneliner should be able to do that for thousands of files (after shutting down Bitcoin Core). If it doesn't work, you may need to download all blocks again.

If OP or other reader actually consider such approach, they probably also should consider using folder or disk level compression first. On windows, AFAIK you can simply enable it for a folder or drive from file explorer. But on linux, you either use fuze-zip (folder level) or use btrfs or ZFS filesystem (i don't know how good or stable those)

[LoyceV]

If OP or other reader actually consider such approach, they probably also should using folder or disk level compression.

I'd expect block data to not compress very well. Let's test it on 42 random files:

Code:

mkdir /tmp/blocks
cd ~/.bitcoin/blocks
cp `ls *dat | shuf | head -n42` /tmp/blocks
cd /tmp
tar -cvvf - blocks | pbzip2 -z -9 > blocks.tar.bz2
tar -cvvf - blocks | pigz > blocks.tar.gz
du -sch blocks*
rm -r blocks*

Code:

3040    blocks
2515    blocks.tar.bz2
2497    blocks.tar.gz

Those 42 random blocks took 17% less space after compression. For disk level compression, I expect slightly lower compression rates. I probably wouldn't do it myself, but if it means a 1 TB disk can last two more years, it could actually be worth it.