Best Hard Drive for Bitcoin Node?

[HCP]

I have some good news and some bad news.
Good news is that three days passed after I replaced cables/port and my Seagate drive is working like a charm, so it looks like I don't have to purchase new drive after all.
I left it running 24/7, every parameters are looking very good and there are no issues with crashes or bad sectors, CrystalDiskInfo is showing healthy drive.

One thing which I'm fairly sure wasn't mentioned as yet... have you checked the actual voltages coming out of the PSU?

There are some utilities that might be able to display these values... but the most accurate way is to check using a multimeter and direct measuring the values.

A number of years ago, I had a PSU that was "dying" and was "occasionally" only providing ~10-11V on the the 12V rail... probably didn't help that I had a stupid number of HDDs installed in a stupid RAID array^[1]

But I digress, the "less than ideal" voltage meant that I was getting weird issues with a drive "disappearing" randomly and/or straight up data corruption and r/w errors etc.

At first I suspected it might be the drive, but it came up clean in all tests... I changed cables etc... still got weird stuff happening. It wasn't until I checked the voltages and noticed that the 12V rail was sometimes dropping down to values nowhere near 12V that I was able to finally figure it out and solve the issue.

New PSU... "proper" (and stable) voltages... and the drive worked perfectly after that.


As for the "best hard drive for a node" question... I'm fairly happy with the blocks on (1TB) HDD and "chainstate" symlinked to SSD solution that I've been using.

Not sure what it would be like with an IBD, but I've got 2 copies of the blockchain at the moment... so I don't anticipate it being an issue. Even when I set up my Pi node using one of the "cheap" USB3.0 1TB Seagate external HDDs (firstly with RaspiBlitz and then Umbrel), I was able to shortcut that process quite substantially by simply copying all the required data from my other node instead of having to "suffer" through a full sync.



[1] - RAID10 with a bunch of old NAS drives I got gifted by a friend... because I could (Mobo had a built in RAID controller)... I had never used RAID before and felt like experimenting with stuff... and then realised I had too much data installed to "undo" it all

[1714851494]

1714851494

[dkbit98]

One thing which I'm fairly sure wasn't mentioned as yet... have you checked the actual voltages coming out of the PSU?

I haven't checked voltages until now, but changing the cables and port worked very well and I don't have any disconnects after that.
One of the things I initially did was to disable hard drive going to sleep in power settings, and now I am monitoring voltage showing 12V correctly.

Getting back to shingled SMR slow drives, one of the interesting things I saw for improving speed for Hard Drives if you have supporting processor is using software called AMD StoreMI that combines SSD and HDD into drive, problem is that this software is only for Windows OS.

[HCP]

Getting back to shingled SMR slow drives, one of the interesting things I saw for improving speed for Hard Drives if you have supporting processor is using software called AMD StoreMI that combines SSD and HDD into drive, problem is that this software is only for Windows OS.

Only for Windows and, as noted, is actually AMD only... it requires a specific chipset:

If you have an AMD X570, PRO 565, B550, A520, 400 Series, X399, TRX40 or WRX80 motherboard, you can download StoreMI for free.

Meanwhile, Intel have something similar with their "Optane" system, but that involves buying an extra bit of hardware. It too is Windows only.


Both systems effectively seem to be creating "Hybrid" drives... so you get large storage on the HDD, combined with faster speeds by caching the frequently accessed data on the SSD. There is a fairly decent comparison of the two here: https://appuals.com/intel-optane-vs-amd-storemi-what-is-the-difference/

[kano]

...
Getting back to shingled SMR slow drives, one of the interesting things I saw for improving speed for Hard Drives if you have supporting processor is using software called AMD StoreMI that combines SSD and HDD into drive, problem is that this software is only for Windows OS.

Used to have one of these long ago, a seagate, that didn't require special code, the SSD cache was in the standard sata HDD interface.
This of course did speed it up.

... until it failed ... then decided that wiping such a HDD of all it's data could also be problematic ...

Never got another one due to both of those issues.

[HCP]

Yeah... I haven't really seen any full hardware "hybrid" drives for a while... The were a thing when SSD's were a lot more expensive, but as SSDs slowly came down in price, it seemed hybrid drives just kinda disappeared before they really became a thing.

[dkbit98]

Only for Windows and, as noted, is actually AMD only...

I have supporting chipset and someone told me there is a way for making this work even on Linux OS.
Here is video review by Level1Techs explaining this better than me:
https://youtu.be/wbl2dYgjMQ4

Meanwhile, Intel have something similar with their "Optane" system, but that involves buying an extra bit of hardware. It too is Windows only.

You can combine and use StoreMI together with Intel Optane for full performance, as if that software was made for supporting Optane, maybe AMD did it to piss off Intel

... until it failed ... then decided that wiping such a HDD of all it's data could also be problematic ...

Failed how, and what exactly failed? Seagate HDD or unified drive made by StoreMI?

Yeah... I haven't really seen any full hardware "hybrid" drives for a while... The were a thing when SSD's were a lot more expensive, but as SSDs slowly came down in price, it seemed hybrid drives just kinda disappeared before they really became a thing.

Hybrid drives are junk, that is why nobody buy them anymore and they are not advertised.
StoreMI is creating unified layered drive with tiered storage that is much better solution and different tech compared to hybrid drives.

[DaveF]

Hybrid drives are junk, that is why nobody buy them anymore and they are not advertised.
StoreMI is creating unified layered drive with tiered storage that is much better solution and different tech compared to hybrid drives.

They served a purpose for a VERY short period of time. They gave a speed boost to spinning dives while keeping the cost down.
If SSD prices did not fall as quickly as they did then we would probably still be seeing them.

Because for the most part, I only cared about speed or size, so they did nothing for me, I did not look too closely. BUT, I remember seeing them mostly at the OEM level, not retail.

-Dave

[kano]

For a node, it doesn't matter too much what you use unless you care a lot about performance.
SSDs do not have a long life due to rewrite limitations vs HDDs.
SSDs are typically about 7 times faster than HDDs.
Adding a USB interface to either type of disk will of course slow it down.

However, a node without a wallet is simply a copy of data that is mirrored 10's of thousands of times around the internet.
So if the disk dies, go get a new one, then resync the node.

As mentioned already, 1TB should last you quite a while.

This is of course assuming you don't have a wallet on the node.

[DaveF]

...
SSDs do not have a long life due to rewrite limitations vs HDDs.
SSDs are typically about 7 times faster than HDDs.
...

It's a bit closer to 10x now. Basing on consumer 'retail' grade vs consumer 'retail' grade. And commercial vs commercial

The lifespan is an interesting question. With a lot of writes there is a finite lifespan to an SSD.
BUT in something for just holding node & wallet data. It's probably going to last a lot longer then a spinning drive. For the most part blockchain data once written is not written again and the little bit of data that other files rewrite is tiny.

In the general view, it's the proper tool for a job. Do you need speed or the ability to do a ton of data movement all day every day, do you need long term storage or

Saying one is better then the other is like saying car X is better then car Y without knowing what the driver needs it for.

-Dave

[HCP]

Yeah... aside from IBD, using an HDD (on both my PC-based and Pi-based nodes) isn't really an issue. It's certainly "cheaper".

I'm sure we'll get to a point where the price difference between HDD and SSD becomes so negligible that you may as well just get an SSD... but, for my budget (and location), we're not quite there yet SSDs are still around 2-2.5x more expensive than an equivalent sized HDD

[kano]

Yeah... aside from IBD, using an HDD (on both my PC-based and Pi-based nodes) isn't really an issue. It's certainly "cheaper".

I'm sure we'll get to a point where the price difference between HDD and SSD becomes so negligible that you may as well just get an SSD... but, for my budget (and location), we're not quite there yet SSDs are still around 2-2.5x more expensive than an equivalent sized HDD

There is also the issue of data retention.
SSDs don't typically hold the data for as long HDDs do.
Writing the block data (blocks/ directory) is usually a one off write, never to be written again, so how long will that data last?
Wallet is possibly a similar issue, certainly areas of it and also if you are hodling.

From some technical reading I've done, since I use RAID1, 2 disks, different batch, on all my disks everywhere,
I feel the need to do a double resync of data about every 2 years or so on SSD, and about every 5 years or so on HDD.
i.e. resync one to the other, then the other back to the first one - to do a full rewrite of all the data on both disks.
YMMV

[LoyceV]

There is also the issue of data retention.
SSDs don't typically hold the data for as long HDDs do.

I didn't know that They should write a big warning on the packaging!
I searched a bit, and it turns out data retention gets a lot worse when the drive is either powered off, or stored at a high temperature. And, to my surprise, enterprise drives are much worse at data retention than consumer drives:

High-end consumers who might be tempted by enterprise-level NAND drives need to pay attention to the brief unpowered data retention times — in this case, buying an enterprise drive really might not be the best choice for a system that remains unpowered for any length of time.

[dkbit98]

The lifespan is an interesting question. With a lot of writes there is a finite lifespan to an SSD.
BUT in something for just holding node & wallet data. It's probably going to last a lot longer then a spinning drive. For the most part blockchain data once written is not written again and the little bit of data that other files rewrite is tiny.

There are no moving mechanical parts in SSD drives, you can track condition of drive with good accuracy, and if you enable enough reserved free space for over provisioning, you would significantly increase lifespan of your drive.
SSD's are quiet, not producing any noise, and they are usually heating less than old HDD drives, this can also increase their life.

SSDs don't typically hold the data for as long HDDs do.

It probably depends on type of memory and controller used in SSD drives, but I heard that big data centers are using some combination of SSD and HDD drives.
HDD are using for files not moving a lot and moving files are usually stored on SSD drives.

High-end consumers who might be tempted by enterprise-level NAND drives need to pay attention to the brief unpowered data retention times — in this case, buying an enterprise drive really might not be the best choice for a system that remains unpowered for any length of time.

So this issue is mostly related with higher temperatures, meaning you need to have proper cooling system and ventilation in your computer.
I liked conclusion in that extremetech article - no single storage type is perfect, and always back up important data

[cryptomaxsun]

On one computer, I have a full node  bitcoin running on an old 2009 child disk WD6400AAKS 640 Gb.
Operating time of almost 50,000 hours.
The initial synchronization takes a long time, but then it works 24/7 without any problems, I will spit so as not to jinx it.
So you can run a full node on the old disk.