Bitcoin Forum
June 16, 2024, 05:15:07 AM *
News: Voting for pizza day contest
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1]
1  Bitcoin / Bitcoin Discussion / Re: A publicly queryable database of info scraped from the blockchain. on: March 22, 2014, 04:26:12 PM
I want to propose something that would hopefully open up some doors to researchers, analysts, who may not be able to code at this time, but want to process the blockchain for stats, trends, and other info unavailable through stuff like blockchain.info. I'm not looking for another blockexplorer, I want something that people can query for big data.

Does anyone want to help out with this? I plan to import the blockchain into a MySQL database, storing basic block information (output from getblock), and the same for transactions. Also build up a list of addresses related to a transaction.

I expect this would be an unwieldy amount of data, so input from people would be key to this. Also help in supporting this (a dedicated machine/vps that could manage this would be required.. if people donated to keep it floating that would be great.

I have made a start on coding some things would make importing the blockchain a lot quicker - A function to decode a raw transaction instead of asking bitcoind has drastically sped up my parsing times. I need to add some more rules so it recognizes unusual transactions.. it isn't incredibly flexible yet but it recognizes pay-to-pubkey-hash and pay-to-scripthash outputs. I'm tempted to write something that would just parse the blkxxxxx.dat files..

I think having a public querying engine to ask things like 'how many P2SH addresses are being used today, or how has this changed over time?'. Or a visual thing showing how transaction consensus has changed over time. Or other creative questions that I wouldn't think to ask. I know there are a few blockchain forensics/analytics companies popping up.. a community lead project focused on gleaning insight from the block chain would be great though.

Thoughts?
There are quite few open source blockchain parser available on github: blockparser, bitcoin-abe, block-browser. These can dump data into sql. MySQL/PostgreSQL will be fine to handle it but about 60G+ of storage and half to several days to process.
Some of the stats about script types used are available at http://webbtc.com/stats.
Theres is also a up to date sql dump available http://dumps.webbtc.com/bitcoin/
2  Bitcoin / Development & Technical Discussion / Re: What meaningful info people want to see extracted from blockchain? on: February 07, 2014, 11:01:44 PM
So far they've seen all they needed.
Do you?
What tools do you use? (apart from the obvious blockchain.info and blockexplorer.com)
3  Bitcoin / Development & Technical Discussion / What meaningful info people want to see extracted from blockchain? on: February 07, 2014, 09:49:56 PM
I'm a data mining researcher and want to contribute to Bitcoin.
What meaningful info people want to see extracted from blockchain?
4  Bitcoin / Development & Technical Discussion / Re: Bitcoin Computer Science research topics on: February 04, 2014, 03:29:59 PM
A bit of advice: Bitcoin and PoW crypto-currencies like it are unique because to understand them you can't just understand crypto, you also need to understand a bit of economics, game theory, and even politics. Equally, it's very common that ideas that seem technically sound fall flat when analyzed from an economic/social/political point of view; it's easy to make ideas that only work if all the participants are "honest" in some way, which just isn't good enough here.

Thanks for the advice. I'm also talking to a CS research group with PhD candidates and people doing dual-degree in Econ/CS & Econ/Math so I think there can be a good fit with lots of real-world experimental opportunity.
5  Bitcoin / Development & Technical Discussion / Re: Bitcoin Computer Science research topics on: February 04, 2014, 03:20:07 PM
Wow! Many thanks for this. Looks like lots to choose from.

I guess you mean he's studying CS at the undergraduate level.

He's doing 5-year CS program which a combination of 3 years bachelor general CS degree + 2 years graduate CS specialized topic programme.
6  Bitcoin / Development & Technical Discussion / Bitcoin Computer Science research topics on: February 04, 2014, 12:50:31 PM
I've got a friend studying Computer Science and looking for a research topic around bitcoin.
What are the hard-core CS challenges in the bitcoin tech space both in software and hardware?
Any links/pointers would be appreciated.
7  Alternate cryptocurrencies / Pools (Altcoins) / Re: [ANN] profit switching auto-exchanging pool - middlecoin.com on: January 05, 2014, 11:37:55 AM
I had the same problem; h2o explained that was because he wasn´t recording ltc blocks after eu server changed to mine LTC. After that period we experienced an increase in "unexechanged" etc due to those blocks, so everything went back to normal. I think the main problem now is that we are mining ltc on eu server that is not the most profitable coin now and thats why our profits decreased dramatically.  
Ok that make sense, since my rigs were all pointed to EU servers.
Although if I were mining LTC straight, I would expect to get ~2x of yesterday payout from MC (i.e. ~8.5Mhs accepted / 3.2 * 0.029 = 0.077 BTC vs actual 0.038 BTC)
8  Alternate cryptocurrencies / Pools (Altcoins) / Re: [ANN] profit switching auto-exchanging pool - middlecoin.com on: January 05, 2014, 11:27:02 AM
Anyone seeing a period of mining "nothing" yesterday on their middlecoin graph? It looks like for a quarter of day, there was no growth in balances/immatures/unexchanges.

this is just a display issue
Wouldn't that happen to all accounts? And it doesn't, some has flat and some doesn't.
Examples from the top:
No flat : http://middlecoin2.s3-website-us-west-2.amazonaws.com/reports/1QABREeywTMZEGCwRocBdPn37WddpxDAhC.html
With flat : http://middlecoin2.s3-website-us-west-2.amazonaws.com/reports/1DziP6pJTLNyV8xYTNXoD5VcQgU8vSXtks.html
9  Alternate cryptocurrencies / Pools (Altcoins) / Re: [ANN] profit switching auto-exchanging pool - middlecoin.com on: January 05, 2014, 11:21:02 AM
Yeah i got the same problem, so it's nothing to worry about?
My payout yesterday dropped to a third of usual daily amount so I'd like to find out what's the cause.
10  Alternate cryptocurrencies / Pools (Altcoins) / Re: [ANN] profit switching auto-exchanging pool - middlecoin.com on: January 05, 2014, 10:55:23 AM
Anyone seeing a period of mining "nothing" yesterday on their middlecoin graph? It looks like for a quarter of day, there was no growth in balances/immatures/unexchanges.
11  Alternate cryptocurrencies / Altcoin Discussion / Middlecoin inactivity on: January 04, 2014, 11:45:53 PM
Anyone seeing something similar on their middlecoin graph?

It looks like there was a quarter of a day when nothing was mined?!? Balance stays flat, no immatures (green)/unexchanges(blue).
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!