hdd cache software, to allow running block storage on less costly disk

[15horses1donkey]

Hi - has anyone used Primocache or other caching software to speed access to a HDD-based block store?

I've used the trial and it substantially increased the time to load the windows bitcoin client.
It also seems to reduce wear of unnecessary writes to the primary SSD I use, which has the client and some indexes on it. I decided to buy the full package, as it's useful for more than only bitcoin. Is there other software that runs reliably that is a substantial improvement on the native windows caching systems, that makes use of resources like RAM to reduce wear to components like HDDs and SSDs, when using a bitcoin client?

This is really important if your systems are highly multipurpose and multi-functional, to reduce e waste and other pollution, and make better use of limited resources so that more people can benefit.

I assume that with POS there is less need to run a full node, but sometimes it's useful. Thoughts on this would be valued.

Version I've used is 4.2.0

[LoyceV]

I'm not sure if you're advertising this or asking a serious question. I'll assume the latter.

a substantial improvement on the native windows caching systems

Disk caching should be handled by your OS. If you want to improve Windows' caching, switch to Linux!

reduce wear of unnecessary writes to the primary SSD I use

Let's not have third party software decide what's necessary to write to your disk. Increase dbcache to 4096 MB, and let Bitcoin Core do it's job.

[joniboini]

Most of these cache software are paid btw, I don't think you'd need to do that if you just want to run a node. Just use Linux as mentioned above. I use this disk caching thing for gaming, but wouldn't really use it for anything else especially if it is as sensitive as running a node where you are online 24/7.

[DaveF]

From the server world, software RAM cache is not worth it. Either do it properly with a controller card with cache and and a battery to keep the information in the RAM in case of power outage OR just go for a SSD. Not saying it will not work, just that it's not going to work as well as something that was designed to do it.

Without knowing the specs of the rest of the system it's tough to give good advice as to what would help performance.

-Dave

[NotATether]

Hi - has anyone used Primocache or other caching software to speed access to a HDD-based block store?

Software caching works by disabling important safety checks such as write buffers and read buffers which prevent data loss in the event of an unexpected system shutdown such as from power failure. It's better to just insert a second HDD in your system and run them on RAID0.

[LoyceV]

Software caching works by disabling important safety checks such as write buffers and read buffers which prevent data loss in the event of an unexpected system shutdown such as from power failure.

That's the same when your OS handles the caching. Power failures can be handled by a UPS (or laptop). Or just take the risk: when I'm writing data to a slow USB stick, it takes several minutes to sync before I can unmount it. I like it, because without write cache, writing small files would take much longer. If it corrupts it's file system for any reason, I just format the stick and copy back my data.
But as I said, I wouldn't use third party software for this.

reduce wear of unnecessary writes

I often use tmpfs for this: I can spare up to a few GB of RAM, and use this when I update my Trust/Merit data. Every week, it saves me from writing millions of files to disk.

[NotATether]

Software caching works by disabling important safety checks such as write buffers and read buffers which prevent data loss in the event of an unexpected system shutdown such as from power failure.

That's the same when your OS handles the caching. Power failures can be handled by a UPS (or laptop). Or just take the risk: when I'm writing data to a slow USB stick, it takes several minutes to sync before I can unmount it. I like it, because without write cache, writing small files would take much longer. If it corrupts it's file system for any reason, I just format the stick and copy back my data.

That's a suitable configuration for read/writing small files but it is no good for running I/O intensive sofware like Bitcoin Core on because without a buffer, blocks can easily get corrupted (considering that most nodes are desktops or servers), forcing you to redownload the blockchain again.

[dkbit98]

Hi - has anyone used Primocache or other caching software to speed access to a HDD-based block store?

I tried Promocache and similar ramdisk software several years ago, but I don't think this would be very much useful for bitcoin blockchain.
If you are using HDD they are now so cheap that it makes no sense to care so much about disk cost, and you can even buy dirt cheap used HDD with 100% health.
You can even buy 1TB SSD drive that is more expensive but they have much bigger TBW or  total bytes written.
For example Samsung Evo 860 has 600 TBW or 5 years warranty, that is more than enough for bitcoin blockchain.

[NotFuzzyWarm]

Good lord. HDD cache software is still being sold?
Um, it pretty much became a useless bit of kit in the late 1980's once the various OS's started to mature and began to handle disk access more intelligently...

[DaveF]

Good lord. HDD cache software is still being sold?

Such oil snake software (which have small impact or has unmentioned danger, while also collect your personal data) still has place today. IMO the worst offender would be game/FPS booster.

Going to disagree with you both a very very little bit on this. 99% of the time you are right.

For the most part for it's crap and does nothing. However, you do have to put in a 'but'.
On older systems, with slower hard drives, and yes there are a ton of machines out there with 5400RPM drives, it does help.
And keep in mind some OEMs made PCs with 1TB 5400 RPM drives with no onboard cache to meet a price point.
Today, 4GB sticks of DDR3 RAM are very cheap to the point that a lot of IT people just give it away since it has no real use at all anymore.

So in the situation of old PC, slow crap drive, free extra RAM. Yes it does help.
And you don't have to buy a better drive or clone your old drive or do anything like that.

Like I said very limited good comes from it, but not zero at all times.

-Dave

[PawGo]

I have no idea why one would like to "buy" a dedicated software for that purpose (on the other hand I do understand that someone wants to sell it); the idea of ramdisks is old as hell. Some used it just for temporary folder, but if you have more RAM, why not to use it for other "parts" of software system. OF course there are always extra tasks (and problems) to do, like initial loading and unloading back to hdd, danger related to unexpected crash of system and losing data stored in memory etc etc. Personally I have never used ramdisk for any sensitive data.

By the way, I understand that we have progress in hardware development, we have faster disks, memory and processors, but it does not mean that all 10 year old computers shoud go to bin. 10 years ago people worked on computers and were able to do their daily tasks...

[LoyceV]

Thanks for mentioning this. I know existence of tmpfs, but always seems to forget it when i need to process lots of small files.

I used to symlink /tmp to /dev/shm, but that fails when programs (such as sort) write large amounts of data to /tmp. I once synced Bitcoin Core (pruned) entirely on a ramdrive. It's great for performance, but less desirable for the rest of my system.

Personally I have never used ramdisk for any sensitive data.

I do the exact opposite: whenever possible, I only use my ramdisk for sensitive data.

[PawGo]

Personally I have never used ramdisk for any sensitive data.

I do the exact opposite: whenever possible, I only use my ramdisk for sensitive data.

I see your point, maybe I should say more precisely - not for data I do not want to lose. I mean - I would not use ramdisk for saving draft of document I edit, you know what I mean.
As a place which makes the traces of work to disappear - I understand that approach.

Regarding the main question - as far as I know blocks does not need to be stored on a fast drive, the problem is with indexes.