NuDB: A Key/Value Store For Decentralized Systems

[misterbigg]

I developed this embedded database for Ripple:

NuDB: A Key/Value Store For Decentralized Systems
https://github.com/vinniefalco/Beast/tree/master/beast/nudb

Fetch performance is outstanding, and remains constant no matter how big the database. It uses an on-disk hash table so the database needs to be on an SSD drive. It has journaling so it resists corruption on power failure. It supports POSIX and Win32 native APIs.

LevelDB, and RocksDB turned out to be bad solutions for us because as the database grows the performance goes down. And those databases require increasing amounts of memory. So NuDB is designed specifically for our use case.

Could Bitcoin or other related software benefit from this? If there is sufficient interest I might put it in its own repository. The code is headers-only (like boost), except for the unit tests.

[samson]

Well it looks interesting, how does it compare to sqlite for a few million records ?

[misterbigg]

Well it looks interesting, how does it compare to sqlite for a few million records ?

I'm not sure a comparison against SQLite is meaningful. SQLite offers indexes and structured data, NuDB is only a key/value storage system - you can only look up values by their key and nothing else.

Beyond that, you need to be more specific. A few million records of what? Inserts? Fetches?

Or are you asking about the performance of the database after a few million records are already present? Because NuDB does not slow down at all, regardless of how large the database gets. It offers constant performance.

[misterbigg]

No answer?

[tacotime]

These types of DBs are popular lately, for example, btcwallet now uses boltdb, while Monero uses LMDB. They're all kind of similar: full ACID semantics w/ MVCC, data stored on disk in B+ trees, async read/update with scheduling and transitory state patches for reads, multiple namespaces, nested buckets, etc.

AFAIK, bitcore is moving towards adopting more DB agnostic architecture, so in the future you can plug and play whatever crazy DB type you want.

[samson]

No answer?

Nothing specific, I just find it interesting. Certain useful datasets can be key/value for something I'm working on.

I'm using sqlite with simple (but potentially large) recordsets based on a unique key at the moment so I'm open to new interesting things.