LevelDB reliability?

[misterbigg]

Could this be of use? It is designed for high performance and reliability on SSDs
https://github.com/ripple/rippled/tree/develop/src/beast/beast/nudb

[1715057353]

1715057353

[1715057353]

1715057353

[1715057353]

1715057353

[1715057353]

1715057353

[1715057353]

1715057353

[2112]

Could this be of use? It is designed for high performance and reliability on SSDs
https://github.com/ripple/rippled/tree/develop/src/beast/beast/nudb

It doesn't matter. The real answer is simple: the storage layers (plural!) need to be abstracted. No single engine could meet all needs.

[watashi-kokoto]

People like Gregory Maxwell, Mike Hearn or Peter Todd are getting paid to pretend to not understand.

Let's not discuss what they pretend, or critize some hobby project
Could you please elaborate, what major problems for Bitcoin could be caused by  devs failing to understand utxo db, blockchain spool storage or other aspects

[misterbigg]

Could this be of use? It is designed for high performance and reliability on SSDs
https://github.com/ripple/rippled/tree/develop/src/beast/beast/nudb

It doesn't matter. The real answer is simple: the storage layers (plural!) need to be abstracted. No single engine could meet all needs.

Of course. I did that for my first task at Ripple. You're telling me that Bitcoin Core still has not abstracted the storage layer?

[2112]

Could this be of use? It is designed for high performance and reliability on SSDs
https://github.com/ripple/rippled/tree/develop/src/beast/beast/nudb

It doesn't matter. The real answer is simple: the storage layers (plural!) need to be abstracted. No single engine could meet all needs.

Of course. I did that for my first task at Ripple. You're telling me that Bitcoin Core still has not abstracted the storage layer?

How could I know what they really did? I feel I need to copy-paste something I wrote nearly 4 years ago under "[POLL] Multi-sig or scalability--which is more pressing?".

Gentle reminder to the other bitcoin developers: it is generally best not to feed trolls.  Use the ignore button.

“If you sit in on a poker game and don’t see a sucker, get up. You’re the sucker.”

http://quoteinvestigator.com/2011/07/09/poker-patsy/

Anyway, here's the example of how open source poker is being played.

A whale of a player spends big bucks developing a secret database engine. After 5 years this player takes one of the earliest branches and open sources it. Lets call that branch LevelDB. Suckers jump on it, spend money and energy to develop some basic tools like statistics gathering and query optimization. Then the whale brings the cold deck to the table, which gives him an instant 5 years of leadtime. It looks like that:

Then, a month later, the main devs decided to switch to compressed public keys which requires a whole new wallet format for Armory.  I was crushed.

I'm no Google insider or anything like that. But I used to knew the people who played with the current Googlers; and I'm broadly familiar with the level of skill involved.

Before you spend to much time at the keyboard and mouse please do see the old David Mamet's movie "House of games". Remember the line:

“it was only business … nothing personal.”

[fluffypony]

Could this be of use? It is designed for high performance and reliability on SSDs
https://github.com/ripple/rippled/tree/develop/src/beast/beast/nudb

It doesn't matter. The real answer is simple: the storage layers (plural!) need to be abstracted. No single engine could meet all needs.

That's precisely what we did with Monero. We abstracted our blockchain access subsystem out into a generic blockchainDB class, and then build out implementations for LMDB (default, preferred) and for BerkeleyDB (failover).

Of course, there exists a risk that a portion of the network may be forked off if in an edge-case between implementations, but that's no different to a subset being forked off the network because they're running 32-bit whatever in a 64-bit world.

[smaxz]

so is there no way to get my backed up blockchains/wallets on 4.8.3 to work with current core?

i've managed to compile and link the archived datasets from oracle but core wont build past leveldb.. figured it was something i did wrong.

[gmaxwell]

That's precisely what we did with Monero. We abstracted our blockchain access subsystem out into a generic blockchainDB class,

Thats exactly how core has been done for years.

Though we don't consider it acceptable to have 32bit and 64 bit hosts fork with respect to each other, and so prefer to not take risks there!

[hyc]

That's precisely what we did with Monero. We abstracted our blockchain access subsystem out into a generic blockchainDB class,

Thats exactly how core has been done for years.

Though we don't consider it acceptable to have 32bit and 64 bit hosts fork with respect to each other, and so prefer to not take risks there!

Monero is using LMDB's VL32 support on 32bit systems, which gives it identical data format and capacity of 64bit builds. There's no difference between a 32bit or 64bit host in the blockchain.

[2112]

That's precisely what we did with Monero. We abstracted our blockchain access subsystem out into a generic blockchainDB class,

Thats exactly how core has been done for years.

Though we don't consider it acceptable to have 32bit and 64 bit hosts fork with respect to each other, and so prefer to not take risks there!

This cannot be right. The Satoshi Bitcoin client always stored blockchain as the plain flat files. The database engines were used only for indexing into those files. The recent LevelDB-based backend worsened the situation by explicitly storing (also in plain files) the some precomputed data to undo the transaction confirmations in case of a reorganization.

The proper way to modularize the storage layers is to completely encapsulate all data and functions to access blockchain, without the crossing the abstraction layers inside the upper-level code.

I stress that "storage layers" need to be plural. The mempool is also a storage layer and also needs to be properly abstracted. In a proper, modular implementation the migration of transactions between those storage layers (for unconfirmed and confirmed transactions) will ideally require setting one field in the transaction attributes, e.g. bool confirmed;.