libbitcoin

[w1R903]

This is great work.  I'm looking forward to trying out the python API.  Thank you.

[1714839019]

1714839019

[1714839019]

1714839019

[1714839019]

1714839019

[1714839019]

1714839019

[genjix]

Thanks.

I'm disconnecting from the forums for a few days to focus. I can be contacted using my email on the front of bitcoin.org

[zwierzak]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

[Matt Corallo]

And what about noSQL databases?

NoSQL is entirely a buzzword and has no meaning whatsoever.  Asking if they will support NoSQL is like asking if they will support about 30 different databases with completely different structures and APIs.

Everyone says that they are faster and more scalable than SQL.

SQL isnt the problem here, its ACID.  If you throw ACID out the window, there are a ton of things you can do to increase performance, which is the reason some of the NoSQL solutions can benchmark so well.  If you are doing something with financial software, not using ACID is about as stupid as you can get (for the actual financial stuff, maybe for the blockchain where you have a bit more leeway and are only updating once every ~10 minutes anyway...)

Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

I can't speak for xelister, but I dont think supporting JSON matters here as its a whole new lib anyway so whether or not the original bitcoin client supports JSON or not is fairly irrelevant.

[Xephan]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

BAD idea. NoSQL is only faster because it doesn't do all that ACID checking that transaction-safe RDBMS does. While it's perfectly fine for data that doesn't need to be consistent i.e. nobody's going to particularly care if half the world saw your twit #156483 and the other half only get updated to #156482 for the next 2 minutes. But for financial transaction, having inconsistent data and no referential integrity is malpractice and asking outright to be scammed.

[vector76]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

[Matt Corallo]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of...

[vector76]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of...

Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state.  Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB.

I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.

[Matt Corallo]

Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state.  Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB.

I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.

Well if the BDB transaction fails, there will be no pointers to the extra crap at the end of the flat file and subsequent blocks will be added later, so all that would happen would be a bit of extra space would be wasted (in theory).  Not quite ACID but not quite leaving the database in an "inconsistent" state.
Anyway, I think weve overtaken this thread enough...

[genjix]

No, don't worry. The conversation is informative. I have no problems if people want to talk about tangential topics here

I uploaded a postgresql dump of the database: https://bitcointalk.org/index.php?topic=38246.0

[David M]

Love your work genjix.

Without downloading the postgresql dump, can you please post the DDL for your schema here or provide just a dump of the schema.

I am very interested in creating a full data model (with ERD's, key discovery, propositions etc...) for Bitcoin using the Relational Model.
After that is created, I would like to help create a transition model for existing accounting/monetary based software packages to incorporate Bitcoins....

I am willing to provide schema's for all major DBMS packages (Mysql, SQL Server, Oracle, DB2 and Postgress)

I am a DBA/modeller with 25 years experience and quite frankly do not have the time ATM to read and collate the schema from the raw code.

[mizerydearia]

Hey, genjix here, posting under someone else's account

It's in the repo, https://gitorious.org/libbitcoin/libbitcoin/blobs/master/bitcoin.sql

[John Tobey]

Monitoring this thread as I build a g++ capable of building libbitcoin.

Meditating on "block-chain verifier subsystem."

[genjix]

Image on how branch deletion works:

[genjix]

Big milestone reached! libbitcoin is the one of the first alternative implementations of the bitcoin protocol (along with node-bitcoin-p2p) to do full blockchain validation! Exciting milestone. It's taken me around 6 hours to download and validate the blockchain.

Here's a sample of validation times at 130k blocks in milliseconds:

112	22	97	316	0
38	407	1795	375
116	51	116	358
81	481	49	325
87	107	18	9
579	56	65	7
2329	98	25	103
8	4	291	155
147	140	185	55
87	15	31	67
12	47	28	16
12	34	92	0

Also I wrote a development statement for libbitcoin (also in the OP).

The Zen of libbitcoin
Readability over speed.
Beauty over convenience.
Simplicity over complexity.
Architected, not hacked.
Flat, not nested.
Explicit, not implicit.
Errors should be loud.
Never is better than right now.
Now is better than never.
Be flexible and configurable.
Build houses from bricks, software from modules.
Stability is god.

Random bitcoin trivia:

First transaction is in block 170

First standard transaction in block 728

The first difficulty change was at block 32256

Blocks 91842 and 91812, and, 91880 and 91722 contain the same duplicate coinbase transaction. The miners can only use that transaction once.
http://blockexplorer.com/block/00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec
http://blockexplorer.com/block/00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721

2016 (amount of blocks for difficulty re-calculation) corresponds to exactly 2 weeks of 10 minute ideal blocks. 210k (amount of blocks for drop in mining reward) only roughly corresponds to 4 years.

There was a forced fork in the blockchain a while back around 710k blocks when someone DoS bitcoin's scripting system using an excessive number of OP_CHECKSIGs. That's why IsStandard() exists today.

Block 142312 was the first block to have a non-standard coinbase. I think this was one of luke-jr's blocks possibly since it follows his pattern of instant spend.

block 546 spends a tx within the same block: 6b0f8a73a56c04b519f1883e8aafda643ba61a30bd1439969df21bea5f4e27e2

[fellowtraveler]

Congratulations! And great work.


We're getting closer to a high-level interface?

[d33tah]

The Zen sounds pretty much like Python's.

[mizerydearia]

The Zen sounds pretty much like Python's.

Code:

The Zen of Python                                                      The Zen of libbitcoin

Beautiful is better than ugly.                                         Readability over speed.
Explicit is better than implicit.                                      Beauty over convenience.
Simple is better than complex.                                         Simplicity over complexity.
Complex is better than complicated.                                    Architected, not hacked.
Flat is better than nested.                                            Flat, not nested.
Sparse is better than dense.                                           Explicit, not implicit.
Readability counts.                                                    Errors should be loud.
Special cases aren't special enough to break the rules.                Never is better than right now.
Although practicality beats purity.                                    Now is better than never.
Errors should never pass silently.                                     Be flexible and configurable.
Unless explicitly silenced.                                            Build houses from bricks, software from modules. 
In the face of ambiguity, refuse the temptation to guess.              Stability is god.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Yeah, that's why I said "pretty much".

I'm just providing immediate access to both to compare them more easily...

[d33tah]

Yeah, that's why I said "pretty much".

[Bitcoin Oz]

Good work genjix. Congrats.