|
Title: Blockchain in SQL format Post by: Andrew Vorobyov on September 18, 2011, 05:35:39 PM I would like to export blockchain transactions in SQL/CSV format. What is the best way to do it? Except scraping http://blockexplorer.com to death
Title: Re: Blockchain in SQL format Post by: etotheipi on September 18, 2011, 05:52:59 PM Do you want to maintain a running, up-to-date version of the blockchain in SQL? Or are you looking for a snapshot of the blockchain at a point in time?
The reason I ask is because there's plenty of folks who could use their blockchain management code to give you an SQL snapshot, but if you want a persistent, updating SQL database, you're going to have to do something more creative. Maybe modify and recompile the client to maintain a parallel SQL DB, to be updated when new blockdata is received. Or, as you said, scrape a website/service that offers up-to-the-minute blockchain data. I guess it depends on your application/requirements. Title: Re: Blockchain in SQL format Post by: Andrew Vorobyov on September 18, 2011, 06:42:41 PM I need snapshot and then will continue scraping.
Title: Re: Blockchain in SQL format Post by: etotheipi on September 18, 2011, 06:54:31 PM So I haven't worked much with SQL other than the fact that I know SQLite is pretty convenient in python, and I can read the blockchain from python -- so I should be able to figure out how to put the two together. If you write out explicitly what you want the database/tables to look like, and toss me a couple BTC (upon delivery) I should be able to produce a .sqlite file for you.
For instance, I assume you want two tables, one of block headers, one of transactions. The headers table would have all the core information in it (hashes, difficulty, nonce, etc). The transaction table would have the header hash, but be a little more complicated since each transaction actually has a variable list of inputs and outputs. I suppose you could go a step further and add a table of TxIn objects and a table of TxOut objects, each of which contain the parent Tx hash. If it's not too intense, I can probably figure it out. I already have most of the tools made to do it, I just gotta learn the python-sqlite interface (is simply including hashes of the parent objects enough to consider the DB "relational?") -Eto Title: Re: Blockchain in SQL format Post by: Andrew Vorobyov on September 18, 2011, 07:04:22 PM Something like this. http://awesomescreenshot.com/01aknhyc7. I don't care about normalization
Give me you paypal. I'm out of BTCs Title: Re: Blockchain in SQL format Post by: Andrew Vorobyov on September 18, 2011, 07:17:34 PM Wait a sec I will create DB structure dump for you.
Title: Re: Blockchain in SQL format Post by: Andrew Vorobyov on September 18, 2011, 07:48:27 PM Sent you private message
Title: Re: Blockchain in SQL format Post by: genjix on September 18, 2011, 08:21:09 PM Bitcoin network database dump in postgresql (download link inside)
https://bitcointalk.org/index.php?topic=38246 |