Bitcoin Forum

I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.

Yea I'd like a parser, I would like to develop blockchains for other currencies, but querying addresses is impossible with the current daemons, because they aren't part of your own wallet. A solution like this that simply connects to a daemon and runs would be awesome.

Have you taken a look at BitcoinJ ? You can configure it to use a SQL database via JDBC.

https://github.com/bitcoinj/bitcoinj

I guess you've found the Gyp issues then, a problem on almost every platform, quick fixes are available on linux, unsure about windows.

Take a look at this and notice blockchain64.exe https://github.com/jeffchan/blockchain-parser - I've only used it to produce statistics so far (on *nix), I cannot remember if you can dump everything to SQL or not.

Best of luck.

Yup found the Gyp issues, been begging for help almost everywhere but i get nothing but trolls most the time, so i have decided to take a different approach and just say screw it insight is a lost cause lacking in documentation and support.


I was just looking into bitcoinJ, unfortunately i know jack diddly about Java. I snagged up a copy of netbeans and have been doing a little poking around, but without someone to help me get on the right track i am afraid i would just be spinning my wheels for months which helps nobody.


In the end, it would be great if someone knew of a fix for the GYP crap with insight on windows but i think that ship has sailed. So i am on a quest for something that is already working, or close to working that i can tinker with to make work. 

Just in case, the two most common problems with node-gyp, are:

1. Gyp has a dependency on Python, so you need that installed
2. It calls 'node' and sometimes the node binaries are referenced as 'nodejs'.

I have to defend the BitPay guys and insight a bit, it does work very well I got one up and running here (http://bitmark.co:3000) for bitmark very easily, on Ubuntu 14.04 however.

Yep, got python installed, not sure where to even start to look for mis referenced 'node' listings, i admire your defense and am jealous of your working insight client, but after a week of asking for anything, just a reply would have been nice. You seem to have it under control with linux. i got an old windows 7 laptop i could wipe out and throw ubuntu on i believe. i dont know if they have something like team viewer for linux, but if i could find something do you think by the graces of god you could help an old man get it set up on our network so i could at least finish development while i wait for a a fix to the GYP issue? I would be forever in your debt.

Yes, I will help you :)

Let's first establish what you need, as insight may not be the answer. Do you need a constantly updated block chain database, or to analyse a timestamped dump?

Also, do you require the full chain data, or only certain parts?

Finally, have you ensured that all dependencies are installed, https://github.com/TooTallNate/node-gyp details what you need.

Coinsolidation, after a week of all nighters and only 16 hours sleep, 1,00 gray hairs and 500 cuss words later, you have finally put light at the end of the old guys tunnel.

Yes i need full constantly updated chain data, data that i cannot get without getting banned from other services for querying too much from. Working on algorithms for measuring bitcoin adoption, truly lost or "dead" coins, hourly transaction volume, hourly price points and a ton of other stuff. I am an economics professor who has recently taken an interest in the economics of bitcoin after attending a conference that was held an hour from my home town.

In principal since i cannot have insight on the web server where all my other applications are installed, i am going to need to be able to access insight on the local network. would the standard  Ubuntu 14.04 be able to be configured as such or do i need some sort of server environment. Ideally i would like to be able to code into my applications get XXXXXX data from 192.168.2.XX:3000/api/command.

Best to check this first... If you are doing API commands have you considered writing your applications to communicate with bitcoind's RPC API directly (or even via the command line)? You can also run programs every time a new block is received using the '-blocknotify=executable %s' configuration option, and maintain an index in the daemon of all transactions with the '-txindex' option. This would all be on a secure local bitcoin running on your server, with no duplication of data.

If you are certain that insight is the answer, and require that local visual browser part of it too, then the best way to proceed may be for me to do the process from scratch on a virtual machine and write the instructions for you to repeat.

Taking the latter option will require you to have a network accessible Ubuntu 14.04 machine. This could be a virtual machine inside VirtualBox or VMWare, or a physical machine on your network. You can run any version of Ubuntu, standard or server, one has the easy to use GUI and the other just a command line.

If you wish to proceed with the insight option, then I'd suggesting getting a Ubuntu machine setup, bitcoin downloaded, and to begin downloading the block chain.

As an alternative option if you can work over the internet rather than just local, then you could acquire a ubuntu VPS (digitalocean/aws) or cheap dedicated server and hand me the root access, I'd set it all up for you. If you are in the UK then I know a man who may be able to provide a good VPS for this quickly - I say UK as then network latency would be lower in communications to and from server.

Warm Regards,

Mark

As an after thought, if you can make a local ubuntu machine network accessible and pass me the ssh details, I could just connect up and set everything up for you on that instead.

Hello DTM - I understand that "coinsolidation" was in the middle of helping you with this issue.  Regretfully he has been banned from bitcointalk.org for 3 days. Please check your PM

here's another approach:
http://codesuppository.blogspot.com/2014/01/a-command-line-interface-for-blockchain.html
http://codesuppository.blogspot.com/2013/07/work-in-progress-on-bitcoin-blockchain.html

You are the Bitmark Project Manager, and he was the developer who was spending too much time in the forums?

Use NBitcoin (.NET) see how it works at http://www.codeproject.com/Articles/784519/NBitcoin-How-to-scan-the-Blockchain (http://www.codeproject.com/Articles/784519/NBitcoin-How-to-scan-the-Blockchain)
The only relevant part for what you want to do is the last one : "Manual Scanning".

Github : https://github.com/NicolasDorier/NBitcoin (https://github.com/NicolasDorier/NBitcoin)
Nuget : https://www.nuget.org/packages/NBitcoin/ (https://www.nuget.org/packages/NBitcoin/)

Give  BitcoinJ a trial and you wouldn't regret it.

What about abe.py?