[IDEA] - Bitcoin-Powered Database

[ripper234]

https://en.bitcoin.it/wiki/Bitcoin-Powered_Database

I would imagine that the above was discussed here before, but I'm not sure ... if so, please point me towards that discussion (in fact, I vaguely recall posting about it myself in the past, but I'm not really sure).

As far I know, the Bitcoin blockchain is pretty much the only data structure that is both global and tamper-proof.

global - There is one global instance of the blockchain (up to forks ... which are then resolved).
tamper-proof - If a block enters the blockchain, after 6+ confirmations this block can't be feasible faked or altered.

A Bitcoin-powered database would be an API over the blockchain, that would expose a subset of the usual CRUD database operations. In fact, it would be an append-only data structure, because nothing can ever be really deleted from the blockchain - so only CREATE and READ operations will be implemented. Object Versioning will be used to emulate updates and deletions.

One major use case for this would an Undeletedable, Auditable Asset Ledger. It bothers me greatly that GLBSE, Mt. Gox, and other trading websites only use an internal database to record assets. As the recent Bitcoinica incident has shown, such records can be tampered with or even completely deleted. A BPD can serve as a verifiable copy of the asset records, that can't be deleted.

[1714003758]

1714003758

[1714003758]

1714003758

[1714003758]

1714003758

[1714003758]

1714003758

[rjk]

With the advent of merged mining, the blockchains of such parallel databases can be secured as well as Bitcoin itself, without needing another alt coin to accompany it.

[casascius]

How would this work fundamentally better, than, say, Bitcoinica publishing a nightly encrypted backup (a dump) that anybody can download (either via http or torrent), and also publishing a PGP signature to commit to its hash on a particular date?

This would be a whole lot simpler and less complicated and trivial for stakeholders to participate in the solution.

If there were a need to demonstrate that prior revisions aren't changing and that only thing happening is new records being added, the backups could simply be an archive file, where the backup of any night n contains identical copies of all the files that were present in the backup of night n-1, plus a new file representing that day's activity.

[ripper234]

With the advent of merged mining, the blockchains of such parallel databases can be secured as well as Bitcoin itself, without needing another alt coin to accompany it.

I wasn't suggesting an alternate coin.

How would this work fundamentally better, than, say, Bitcoinica publishing a nightly encrypted backup (a dump) that anybody can download (either via http or torrent), and also publishing a PGP signature to commit to its hash on a particular date?

This would be a whole lot simpler and less complicated and trivial for stakeholders to participate in the solution.

If there were a need to demonstrate that prior revisions aren't changing and that only thing happening is new records being added, the backups could simply be an archive file, where the backup of any night n contains identical copies of all the files that were present in the backup of night n-1, plus a new file representing that day's activity.

You would still need an agreed, secure way to publish it, and later verify it. If for example they published it on their FTP server, an attacker could  steal their PGP key, and publish an alternate history of nightly backups.

Sure, some clients that actually bothered to download the backups before the attack took place would complain, but how would they prove their version is in fact the correct one? The point in my proposal is having a way to formally prove the balance without relying on anything except a small open source client program, and the blockchain.

Suppose the owners of the exchange are themselves the attackers - they could arrange for many dishonest users (real or sockpuppets) to claim that their version of history is the correct one.

[DeathAndTaxes]

I wouldn't like the idea of massive amount of unrelated information being dumped into the blockchain.  Now w/ merged mining and an alt-chain containing encrypted backups that might be interesting.

[rjk]

With the advent of merged mining, the blockchains of such parallel databases can be secured as well as Bitcoin itself, without needing another alt coin to accompany it.

I wasn't suggesting an alternate coin.

I know, but you would need hash power to keep the central database secure. "Confirmations" are backed by hash power, and not many users are going to leave bitcoin to hash your chain for no monetary benefit to themselves, leaving it wide open to attack resulting in blocks rolled back. Merged mining gives you the benefits of the Bitcoin network hash power with not much additional work needed to make it happen.

NB: My use of the word "central" above means universally accessible, not centralized. It would be decentralized in the same manner as Bitcoin.

[casascius]

You would still need an agreed, secure way to publish it, and later verify it. If for example they published it on their FTP server, an attacker could  steal their PGP key, and publish an alternate history of nightly backups.

By definition, it's encrypted.  Plain FTP/HTTP would be more than plenty.  Their PGP key isn't secret, unless you mean their private key, which wouldn't need to be on an FTP server in the first place.  It could be on a Crypto-Stick for all we know, where it would be totally unstealable.

Sure, some clients that actually bothered to download the backups before the attack took place would complain, but how would they prove their version is in fact the correct one? The point in my proposal is having a way to formally prove the balance without relying on anything except a small open source client program, and the blockchain.

They would provide the signed GPG message where the service provider committed to the hash.  The block chain doesn't prove anything and is the wrong tool for the job.  The block chain proves that data X was known on or before time Y, but an attacker could just as easily be building his bogus data and putting it in the chain.  The right tool for this sort of job is message signing, something we had long before Bitcoin, and in fact something Bitcoin relies upon for its existence.

Suppose the owners of the exchange are themselves the attackers - they could arrange for many dishonest users (real or sockpuppets) to claim that their version of history is the correct one.

Unless all of those sockpuppets are able to sign messages with the service provider's signing key, it probably isn't going to work.  If they could, the mere existence of multiple signatures asserting the validity of differing history records for the same day would be evidence on its face that the key was mismanaged.

[ripper234]

I wasn't suggesting an alternate coin.

I know, but you would need hash power to keep the central database secure. "Confirmations" are backed by hash power, and not many users are going to leave bitcoin to hash your chain for no monetary benefit to themselves, leaving it wide open to attack resulting in blocks rolled back. Merged mining gives you the benefits of the Bitcoin network hash power with not much additional work needed to make it happen.

NB: My use of the word "central" above means universally accessible, not centralized. It would be decentralized in the same manner as Bitcoin.

I was suggesting storing the database on the bitcoin blockchain. No additional hash power is required.

I wouldn't like the idea of massive amount of unrelated information being dumped into the blockchain.  Now w/ merged mining and an alt-chain containing encrypted backups that might be interesting.

I have never understood why people don't like it. This usage pays for itself. I haven't gotten into the details of implementing a PUT operation, but I assume there exists a method of doing this that is resistant to reasonable future changes in the Bitcoin protocol e.g. blockchain pruning. I don't know enough about the details of the existing methods of encoding messages in the blockchain to know whether they are resistant to such changes or not. For sake of discussion, let's assume there is such a method.

If such a method exists, then, nobody has to like or dislike it - it just works. Let the free market speak for itself, and price transactions accordingly. Bitcoin has always had to deal with transaction spam, this is just one form of it ... but any transaction that pays the fees is legitimate, even if it is intended as encoded data.

[ripper234]

Sure, some clients that actually bothered to download the backups before the attack took place would complain, but how would they prove their version is in fact the correct one? The point in my proposal is having a way to formally prove the balance without relying on anything except a small open source client program, and the blockchain.

They would provide the signed GPG message where the service provider committed to the hash.  The block chain doesn't prove anything and is the wrong tool for the job.  The block chain proves that data X was known on or before time Y, but an attacker could just as easily be building his bogus data and putting it in the chain.  The right tool for this sort of job is message signing, something we had long before Bitcoin, and in fact something Bitcoin relies upon for its existence.

Every user of the service knows his on guid upon registration, and can monitor all records with this guid on a daily basis. How can an attacker build a false history for this guid? An attacker could append transactions to my account that weren't authorized by me, but this would be detected immediately (I get a notification every time an object with this guid is written). After the hack becomes known, the approximate attack date can usually be traced, and all transactions after this date are considered invalid. I would have a complete, authentic, verifiable record of all asset transactions up until the moment of attack.

Suppose the owners of the exchange are themselves the attackers - they could arrange for many dishonest users (real or sockpuppets) to claim that their version of history is the correct one.

Unless all of those sockpuppets are able to sign messages with the service provider's signing key, it probably isn't going to work.  If they could, the mere existence of multiple signatures asserting the validity of differing history records for the same day would be evidence on its face that the key was mismanaged.

Sure, the key might have been mismanaged, but we need a way to understand, post-mortem, which version of history is the correct one. Bitcoin is the most secure way to provide a time stamped record of authenticity / commitment. There is no other data structure that is safe from deletion, tampering, and capturing by governments.

[casascius]

I have never understood why people don't like it. This usage pays for itself.

What if I don't use the service?  Does the mere fact that the service is commandeering resources on my hard drive and bandwidth on my internet connection mean I should be happy and that I am benefiting somehow?

None of us really like downloading an ever-growing block chain.  But we tolerate it for now because we want Bitcoin to succeed and trust that solutions to cut the growth of that burden will be forthcoming.  Don't confuse that with a presumed willingness of Bitcoin users to donate their computing resources to anything anybody feels like throwing their way just because they feel they can.  Look to Wuala to crowdsource storage needs.  (they even accept BTC)

[cbeast]

Yeah ripper234, you are onto what I've been planning for a long time. I am planning an art registration business using the blockchain to verify authenticity of original paintings, prints, and other physical works of art.

[ripper234]

What if I don't use the service?  Does the mere fact that the service is commandeering resources on my hard drive and bandwidth on my internet connection mean I should be happy and that I am benefiting somehow?

None of us really like downloading an ever-growing block chain.  But we tolerate it for now because we want Bitcoin to succeed and trust that solutions to cut the growth of that burden will be forthcoming.  Don't confuse that with a presumed willingness of Bitcoin users to donate their computing resources to anything anybody feels like throwing their way just because they feel they can.  Look to Wuala to crowdsource storage needs.  (they even accept BTC)

You will not hold the blockchain on your personal hard drive forever. Eventually it will migrate to dedicated miner servers with strong hard drives.

If this "feature" of Bitcoin, the ability of its users to pay money in exchange for everybody to store their data forever, is a fundemental unsolvable property of the blockchain (in other words, if future-proof methods of encoding data in the chain exist), then you have no choice in the matter - to participate as a full client in the Bitcoin network, you'll have to store this data. I wonder if this is (yet another) Tragedy of the Commons situation ...

The cost of storing something in the chain is fixed - whatever the fee is at the time of encoding. The overall cost to Bitcoin miners over the future could be unbounded (depends how quickly the price of storage/network solutions drops). So this might be an inherent externality in the protocol (because the net benefit to everyone, of storing data in the chain might be negative). However, this does't mean Bitcoin is doomed because it will become everybody's favorite porn hard drive - if Bitcoin's overall net benefit to users is greater than this negative cost (and I suspect it is greater by several orders of magnitude), then Bitcoin will continue buzzing along despite being used to store "irrelevant" data.

[ripper234]

Yeah ripper234, you are onto what I've been planning for a long time. I am planning an art registration business using the blockchain to verify authenticity of original paintings, prints, and other physical works of art.

Yeah, my idea isn't original ... all I propose is to standardize and open source the protocol and client library.

[casascius]

If this "feature" of Bitcoin, the ability of its users to pay money in exchange for everybody to store their data forever, is a fundemental unsolvable property of the blockchain (in other words, if future-proof methods of encoding data in the chain exist), then you have no choice in the matter - to participate as a full client in the Bitcoin network, you'll have to store this data. I wonder if this is (yet another) Tragedy of the Commons situation ...

The minimal storage burden for being a full client in a fully optimized Bitcoin network is to store all unspent transactions (and only the unspent outputs thereof), which is a very small fraction of the total blockchain size, which grows only in proportion to coin ownership becoming "fragmented" (i.e. the total number of unique bitcoin addresses possessing coins).  Any storage beyond that is consumed merely to cryptographically vouch for the integrity of that first data set.  

The only reason to store the entire transaction history is to favor of some highly conservative security assumptions that are important for the proof of concept, but which are impractical to the point of ridicule.  If 85% of the block chain file exists solely for the benefit of the client's ability to verify the block chain isn't tampered with, and I am able to download a block chain digest 15% of the size and there is some other way for me to establish the accuracy/veracity of the unspent transaction set it represents, and my user experience is all the same, I'm likely to opt for the latter.

Imagine if online banking had two options, http and https, and using https required you to wait 8 hours each time you wanted to use online banking.  Nobody would use it.  People would just bank offline, or take their chances on getting attacked.

Nor is anyone going to ever download the block chain - EVER - when it gets to, say 100GB+, other than self-appointed historians who want to make sure the original block chain never dies.  Everyone else not using a hosted service will be downloading pruned or digested block chains.  As the block chain size tends to infinity, the number of volunteer historians will tend to zero.

If only a dwindling number of historians will download the block chain when it gets so big, you're back to the same problem as square one: you're depending on volunteers to do your backups for you.  You may as well let your volunteers back up your data directly via http, so they are only backing up data they have a stake in (something they might actually do out of prudence), rather than every gig of everybody's data who mistakenly thinks the block chain is their own personal closet, toilet, and dumping ground.

[ripper234]

Nor is anyone going to ever download the block chain - EVER - when it gets to, say 100GB+, other than self-appointed historians who want to make sure the original block chain never dies.  Everyone else not using a hosted service will be downloading pruned or digested block chains.  As the block chain size tends to infinity, the number of volunteer historians will tend to zero.

I know "normal users" won't download the blockchain - people will migrate to lightweight/thin clients anyway pretty soon. However, miners will still need to download the chain in order to verify it. As you know, miners are not volunteers, but have an incentive to mine and verify blocks. Miners could delete ancient history, but I'm not sure they will really choose to, as 100GB isn't really that much to keep around on modern hard drives. Moore's Law will help keep the cost of storing the ever-increasing blockchain small, compared to the mining rewards / tx fees.

[casascius]

Mining is not verifying the past chain. Miners need nothing more than the same data - a reliable set of unspent tx outputs - to do their work.  They definitely do not need the full block chain.

[Sukrim]

They definitely DO need the whole block chain, otherwise they might include an invalid/double spent transaction and loose a block, because this block won't be accepted by anyone.

This "reliable" set of unspent transactions can only be produced by someone you trust (Bitcoin = 0 trust to others!) or yourself - and this person needs to have the whole block chain to be reliable in filtering out bad transactions.

[phelix]

sounds like a job for namecoin

[CIYAM]

I think that this idea is something that could be very useful for creating a web-of-trust system where all "rating" entries become a part of the "public" DB (I made a suggestion along these lines to https://bitcointalk.org/index.php?topic=87012).

[cbeast]

I think that this idea is something that could be very useful for creating a web-of-trust system where all "rating" entries become a part of the "public" DB (I made a suggestion along these lines to https://bitcointalk.org/index.php?topic=87012).

I am very much against the "web of trust" as it is only a continuation of elitism. There are other ways to apply trust to Bitcoin, but a universal database isn't one of them. A Ripple type system would provide the trust a lender might need. Ripple will work best in local economies, but still need a way to trade globally. Ripple/Bitcoin exchanges might do that.