w1R903
|
|
August 15, 2011, 02:08:57 PM |
|
This is great work. I'm looking forward to trying out the python API. Thank you.
|
4096R/F5EA0017
|
|
|
genjix (OP)
Legendary
Offline
Activity: 1232
Merit: 1076
|
|
August 15, 2011, 05:16:59 PM |
|
Thanks.
I'm disconnecting from the forums for a few days to focus. I can be contacted using my email on the front of bitcoin.org
|
|
|
|
zwierzak
Newbie
Offline
Activity: 24
Merit: 0
|
|
August 16, 2011, 08:58:32 PM |
|
And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.
|
|
|
|
Matt Corallo
|
|
August 16, 2011, 09:07:30 PM |
|
And what about noSQL databases?
NoSQL is entirely a buzzword and has no meaning whatsoever. Asking if they will support NoSQL is like asking if they will support about 30 different databases with completely different structures and APIs. Everyone says that they are faster and more scalable than SQL.
SQL isnt the problem here, its ACID. If you throw ACID out the window, there are a ton of things you can do to increase performance, which is the reason some of the NoSQL solutions can benchmark so well. If you are doing something with financial software, not using ACID is about as stupid as you can get (for the actual financial stuff, maybe for the blockchain where you have a bit more leeway and are only updating once every ~10 minutes anyway...) Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.
I can't speak for xelister, but I dont think supporting JSON matters here as its a whole new lib anyway so whether or not the original bitcoin client supports JSON or not is fairly irrelevant.
|
|
|
|
Xephan
Newbie
Offline
Activity: 42
Merit: 0
|
|
August 16, 2011, 09:12:38 PM |
|
And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.
BAD idea. NoSQL is only faster because it doesn't do all that ACID checking that transaction-safe RDBMS does. While it's perfectly fine for data that doesn't need to be consistent i.e. nobody's going to particularly care if half the world saw your twit #156483 and the other half only get updated to #156482 for the next 2 minutes. But for financial transaction, having inconsistent data and no referential integrity is malpractice and asking outright to be scammed.
|
|
|
|
vector76
Member
Offline
Activity: 70
Merit: 18
|
|
August 16, 2011, 10:07:21 PM |
|
And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.
Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.
|
|
|
|
Matt Corallo
|
|
August 16, 2011, 10:29:16 PM |
|
Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.
The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of...
|
|
|
|
vector76
Member
Offline
Activity: 70
Merit: 18
|
|
August 17, 2011, 12:22:53 AM |
|
Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.
The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of... Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state. Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB. I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.
|
|
|
|
Matt Corallo
|
|
August 17, 2011, 01:14:16 AM |
|
Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state. Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB.
I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.
Well if the BDB transaction fails, there will be no pointers to the extra crap at the end of the flat file and subsequent blocks will be added later, so all that would happen would be a bit of extra space would be wasted (in theory). Not quite ACID but not quite leaving the database in an "inconsistent" state. Anyway, I think weve overtaken this thread enough...
|
|
|
|
genjix (OP)
Legendary
Offline
Activity: 1232
Merit: 1076
|
|
August 20, 2011, 01:18:28 AM |
|
No, don't worry. The conversation is informative. I have no problems if people want to talk about tangential topics here I uploaded a postgresql dump of the database: https://bitcointalk.org/index.php?topic=38246.0
|
|
|
|
David M
|
|
August 21, 2011, 11:25:10 PM |
|
Love your work genjix.
Without downloading the postgresql dump, can you please post the DDL for your schema here or provide just a dump of the schema.
I am very interested in creating a full data model (with ERD's, key discovery, propositions etc...) for Bitcoin using the Relational Model. After that is created, I would like to help create a transition model for existing accounting/monetary based software packages to incorporate Bitcoins....
I am willing to provide schema's for all major DBMS packages (Mysql, SQL Server, Oracle, DB2 and Postgress)
I am a DBA/modeller with 25 years experience and quite frankly do not have the time ATM to read and collate the schema from the raw code.
|
|
|
|
|
John Tobey
|
|
September 06, 2011, 04:41:01 AM |
|
Monitoring this thread as I build a g++ capable of building libbitcoin.
Meditating on "block-chain verifier subsystem."
|
|
|
|
genjix (OP)
Legendary
Offline
Activity: 1232
Merit: 1076
|
|
September 23, 2011, 09:44:37 PM |
|
Image on how branch deletion works:
|
|
|
|
genjix (OP)
Legendary
Offline
Activity: 1232
Merit: 1076
|
|
October 02, 2011, 06:42:19 AM Last edit: December 29, 2011, 07:00:27 AM by genjix |
|
Big milestone reached! libbitcoin is the one of the first alternative implementations of the bitcoin protocol (along with node-bitcoin-p2p) to do full blockchain validation! Exciting milestone. It's taken me around 6 hours to download and validate the blockchain. Here's a sample of validation times at 130k blocks in milliseconds: 112 | 22 | 97 | 316 | 0 | 38 | 407 | 1795 | 375 | 116 | 51 | 116 | 358 | 81 | 481 | 49 | 325 | 87 | 107 | 18 | 9 | 579 | 56 | 65 | 7 | 2329 | 98 | 25 | 103 | 8 | 4 | 291 | 155 | 147 | 140 | 185 | 55 | 87 | 15 | 31 | 67 | 12 | 47 | 28 | 16 | 12 | 34 | 92 | 0 |
Also I wrote a development statement for libbitcoin (also in the OP). The Zen of libbitcoinReadability over speed. Beauty over convenience. Simplicity over complexity. Architected, not hacked. Flat, not nested. Explicit, not implicit. Errors should be loud. Never is better than right now. Now is better than never. Be flexible and configurable. Build houses from bricks, software from modules. Stability is god. Random bitcoin trivia:First transaction is in block 170 First standard transaction in block 728 The first difficulty change was at block 32256 Blocks 91842 and 91812, and, 91880 and 91722 contain the same duplicate coinbase transaction. The miners can only use that transaction once. http://blockexplorer.com/block/00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caechttp://blockexplorer.com/block/00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd7212016 (amount of blocks for difficulty re-calculation) corresponds to exactly 2 weeks of 10 minute ideal blocks. 210k (amount of blocks for drop in mining reward) only roughly corresponds to 4 years. There was a forced fork in the blockchain a while back around 710k blocks when someone DoS bitcoin's scripting system using an excessive number of OP_CHECKSIGs. That's why IsStandard() exists today. Block 142312 was the first block to have a non-standard coinbase. I think this was one of luke-jr's blocks possibly since it follows his pattern of instant spend. block 546 spends a tx within the same block: 6b0f8a73a56c04b519f1883e8aafda643ba61a30bd1439969df21bea5f4e27e2
|
|
|
|
fellowtraveler
|
|
October 02, 2011, 07:20:30 AM |
|
Congratulations! And great work.
We're getting closer to a high-level interface?
|
|
|
|
d33tah
Newbie
Offline
Activity: 47
Merit: 0
|
|
October 02, 2011, 08:06:31 AM |
|
The Zen sounds pretty much like Python's.
|
|
|
|
mizerydearia
|
|
October 02, 2011, 12:20:38 PM |
|
The Zen sounds pretty much like Python's.
The Zen of Python The Zen of libbitcoin
Beautiful is better than ugly. Readability over speed. Explicit is better than implicit. Beauty over convenience. Simple is better than complex. Simplicity over complexity. Complex is better than complicated. Architected, not hacked. Flat is better than nested. Flat, not nested. Sparse is better than dense. Explicit, not implicit. Readability counts. Errors should be loud. Special cases aren't special enough to break the rules. Never is better than right now. Although practicality beats purity. Now is better than never. Errors should never pass silently. Be flexible and configurable. Unless explicitly silenced. Build houses from bricks, software from modules. In the face of ambiguity, refuse the temptation to guess. Stability is god. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! Yeah, that's why I said "pretty much".
I'm just providing immediate access to both to compare them more easily...
|
|
|
|
d33tah
Newbie
Offline
Activity: 47
Merit: 0
|
|
October 02, 2011, 12:23:07 PM |
|
Yeah, that's why I said "pretty much".
|
|
|
|
Bitcoin Oz
|
|
October 02, 2011, 12:38:48 PM |
|
Good work genjix. Congrats.
|
|
|
|
|