Bitcoin Knots Update -> SSD Daily TBW Skyrockets

[vxyz123456]

Below I have included an image of a chart which overlays what software my node was running on top of the system's cumulative gigabyte/terabyte written value on the SSD. The OS is Xubuntu on an 8th gen Intel Celeron with a SATA SSD.

https://i.imgur.com/KHnpk9X.png

Whenever I update to a version of Bitcoin Knots greater than 28.1, the disk writes skyrocket. Previously disk writes were at just under 20 GB / day, now they are at 230 GB / day, over a tenfold increase. I believe something about the new node version is causing the indexer (I'm using Fulcrum) to work extra hard or something.

I have no idea why. Can anybody assist? Why would using a newer version of Bitcoin knots cause so many more disk writes? I'm running on an SSD and I don't want to kill it prematurely.

Is there some setting I can tweak to prevent this? What changed between 28.1 and 29.3 that would cause the indexer to do so many extra disk writes?

I asked a similar question a while ago, but about the indexer electrumX (rather than the node version), as I tried it initially and found it was doing high disk writes. That was the reason I switched to Fulcrum:
https://bitcointalk.org/index.php?topic=5501257.0

Other relevant system info:
- Fulcrum 1.11.0.
- Xubuntu 24.04 LTS
- 8 GB of RAM
- Intel 8th gen Celeron CPU (2 cores, 2 threads).
- With Fulcrum running and Knots 29.3 20260508 going, the task manager only shows 2.0 / 8 GB of RAM in use, so I don't think this is a SWAP issue.

[LoyceV]

I asked a similar question a while ago ~
https://bitcointalk.org/index.php?topic=5501257.0

Your problem is the same as it was 2 years ago:

- 8 GB of RAM

- With Fulcrum running and Knots 29.3 20260508 going, the task manager only shows 2.0 / 8 GB of RAM in use, so I don't think this is a SWAP issue.

The problem isn't swap, it's chainstate. If it doesn't fit in RAM, it needs to be largely rewritten for every new block. Two years ago it was 12 GB, now it's 13 GB.

What changed between 28.1 and 29.3 that would cause the indexer to do so many extra disk writes?

I can't tell you this, I've never used Bitcoin Knots. But RAM would be the first thing to upgrade.

[ABCbits]

- With Fulcrum running and Knots 29.3 20260508 going, the task manager only shows 2.0 / 8 GB of RAM in use, so I don't think this is a SWAP issue.

The problem isn't swap, it's chainstate. If it doesn't fit in RAM, it needs to be largely rewritten for every new block. Two years ago it was 12 GB, now it's 13 GB.

For additional information, UTXO/chainstate size passed 8GB in middle of 2023[1]. That's why the total write suddenly skyrocket. It's when Ordinal gaining popularity, so Fulcrrum also works harder to index more address and UTXO.

[1] https://statoshi.info/d/000000009/unspent-transaction-output-set?orgId=1&from=now-5y&to=now&timezone=browser&refresh=10m

[nc50lc]

I asked a similar question a while ago ~

Your problem is the same as it was 2 years ago:

- 8 GB of RAM

I'm not sure if it's actually his ram.
Because in the graph that he provided, the average disk usage reduced again when he downgraded his client to the unaffected version and then spiked again after another upgrade.
If it's hardware, any of those versions should exhibit similar performance issues.

@OP how accurate are those client version labels in the graph?

Why would using a newer version of Bitcoin knots cause so many more disk writes? I'm running on an SSD and I don't want to kill it prematurely.

I'm not using Knots to help you accurately but those versions have BIP110 activated, perhaps it has something to do with its new consensus.

[LoyceV]

I'm not sure if it's actually his ram.
Because in the graph that he provided, the average disk usage reduced again when he downgraded his client to the unaffected version and then spiked again after another upgrade.

Even the lower value of 20 writes per day is high, with sufficient RAM I'd expect it to be a lot less.

I'm not using Knots to help you accurately but those versions have BIP110 activated, perhaps it has something to do with its new consensus.

Could it be they flush dbcache to disk a lot more frequently in the newer version?

[vxyz123456]

I asked a similar question a while ago ~

Your problem is the same as it was 2 years ago:

- 8 GB of RAM

I'm not sure if it's actually his ram.
Because in the graph that he provided, the average disk usage reduced again when he downgraded his client to the unaffected version and then spiked again after another upgrade.
If it's hardware, any of those versions should exhibit similar performance issues.

@OP how accurate are those client version labels in the graph?

Why would using a newer version of Bitcoin knots cause so many more disk writes? I'm running on an SSD and I don't want to kill it prematurely.

I'm not using Knots to help you accurately but those versions have BIP110 activated, perhaps it has something to do with its new consensus.

Those labels are perfectly accurate, down to the day. I know because I remember when I switched versions.

Your observation is correct, as soon as I downgrade back to 28.1, the writes are reduced significantly, back to their prior trend (20 GB/day).

Perhaps it is related to the new consensus rules, but I don't really understand why that would be related to this. I'd assume it would be some other change that is causing the indexer to work harder. But I don't know what.

Also, regarding this comment by LoyceV:

"Even the lower value of 20 writes per day is high, with sufficient RAM I'd expect it to be a lot less."

Keep in mind that this is total system disk writes, not only the node, but also the indexer (Fulcrum). The indexer is responsible for most of those. Would the indexer be writing less if the system had more RAM?

I can test this with 16 GB of RAM next weekend, but I doubt this is the problem, considering the task manager isn't showing high RAM utilization and I'm not aware of what the newer versions of Knots would be doing that would require so much more.