Bitcoin Forum
November 22, 2019, 09:46:41 PM *
News: The forum is 10 years old today!
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Is there a good blockchain parser that is not Linux?  (Read 2537 times)
DTMking
Newbie
*
Offline Offline

Activity: 14
Merit: 0


View Profile
August 22, 2014, 06:02:41 AM
 #1

I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.
1574459201
Hero Member
*
Offline Offline

Posts: 1574459201

View Profile Personal Message (Offline)

Ignore
1574459201
Reply with quote  #2

1574459201
Report to moderator
1574459201
Hero Member
*
Offline Offline

Posts: 1574459201

View Profile Personal Message (Offline)

Ignore
1574459201
Reply with quote  #2

1574459201
Report to moderator
The Bitcoin Forum is turning 10 years old! Join the community in sharing and exploring the notable posts made over the years.
williamj2543
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500

Get ready for PrimeDice Sig Campaign!


View Profile WWW
August 22, 2014, 06:06:15 AM
 #2

Yea I'd like a parser, I would like to develop blockchains for other currencies, but querying addresses is impossible with the current daemons, because they aren't part of your own wallet. A solution like this that simply connects to a daemon and runs would be awesome.

█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
▓▓▓▓▓  BIT-X.comvvvvvvvvvvvvvvi
→ CREATE ACCOUNT 
▓▓▓▓▓
█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
onchain.io
Newbie
*
Offline Offline

Activity: 29
Merit: 0


View Profile WWW
August 22, 2014, 07:40:45 AM
 #3

Have you taken a look at BitcoinJ ? You can configure it to use a SQL database via JDBC.

https://github.com/bitcoinj/bitcoinj
coinsolidation
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250

Bitmark Developer


View Profile WWW
August 22, 2014, 08:00:57 AM
 #4

I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.

I guess you've found the Gyp issues then, a problem on almost every platform, quick fixes are available on linux, unsure about windows.

Take a look at this and notice blockchain64.exe https://github.com/jeffchan/blockchain-parser - I've only used it to produce statistics so far (on *nix), I cannot remember if you can dump everything to SQL or not.

Best of luck.

Bitmark (reputation+money) : Bitmark v0.9.4 (release)
DTMking
Newbie
*
Offline Offline

Activity: 14
Merit: 0


View Profile
August 22, 2014, 08:30:17 AM
 #5

I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.

I guess you've found the Gyp issues then, a problem on almost every platform, quick fixes are available on linux, unsure about windows.

Take a look at this and notice blockchain64.exe https://github.com/jeffchan/blockchain-parser - I've only used it to produce statistics so far (on *nix), I cannot remember if you can dump everything to SQL or not.

Best of luck.

Yup found the Gyp issues, been begging for help almost everywhere but i get nothing but trolls most the time, so i have decided to take a different approach and just say screw it insight is a lost cause lacking in documentation and support.

Have you taken a look at BitcoinJ ? You can configure it to use a SQL database via JDBC.

https://github.com/bitcoinj/bitcoinj

I was just looking into bitcoinJ, unfortunately i know jack diddly about Java. I snagged up a copy of netbeans and have been doing a little poking around, but without someone to help me get on the right track i am afraid i would just be spinning my wheels for months which helps nobody.


In the end, it would be great if someone knew of a fix for the GYP crap with insight on windows but i think that ship has sailed. So i am on a quest for something that is already working, or close to working that i can tinker with to make work. 
coinsolidation
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250

Bitmark Developer


View Profile WWW
August 22, 2014, 08:41:48 AM
 #6

GYP

Just in case, the two most common problems with node-gyp, are:

1. Gyp has a dependency on Python, so you need that installed
2. It calls 'node' and sometimes the node binaries are referenced as 'nodejs'.

I have to defend the BitPay guys and insight a bit, it does work very well I got one up and running here for bitmark very easily, on Ubuntu 14.04 however.

Bitmark (reputation+money) : Bitmark v0.9.4 (release)
DTMking
Newbie
*
Offline Offline

Activity: 14
Merit: 0


View Profile
August 22, 2014, 09:03:42 AM
 #7

GYP

Just in case, the two most common problems with node-gyp, are:

1. Gyp has a dependency on Python, so you need that installed
2. It calls 'node' and sometimes the node binaries are referenced as 'nodejs'.

I have to defend the BitPay guys and insight a bit, it does work very well I got one up and running here for bitmark very easily, on Ubuntu 14.04 however.

Yep, got python installed, not sure where to even start to look for mis referenced 'node' listings, i admire your defense and am jealous of your working insight client, but after a week of asking for anything, just a reply would have been nice. You seem to have it under control with linux. i got an old windows 7 laptop i could wipe out and throw ubuntu on i believe. i dont know if they have something like team viewer for linux, but if i could find something do you think by the graces of god you could help an old man get it set up on our network so i could at least finish development while i wait for a a fix to the GYP issue? I would be forever in your debt.
coinsolidation
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250

Bitmark Developer


View Profile WWW
August 22, 2014, 09:08:28 AM
Last edit: August 22, 2014, 09:24:10 AM by coinsolidation
 #8

Yes, I will help you Smiley

Let's first establish what you need, as insight may not be the answer. Do you need a constantly updated block chain database, or to analyse a timestamped dump?

Also, do you require the full chain data, or only certain parts?

Finally, have you ensured that all dependencies are installed, https://github.com/TooTallNate/node-gyp details what you need.

Bitmark (reputation+money) : Bitmark v0.9.4 (release)
DTMking
Newbie
*
Offline Offline

Activity: 14
Merit: 0


View Profile
August 22, 2014, 09:34:27 AM
 #9

Yes, I will help you Smiley

Let's first establish what you need, as insight may not be the answer. Do you need a constantly updated block chain database, or to analyse a timestamped dump?

Also, do you require the full chain data, or only certain parts?

Finally, have you ensured that all dependencies are installed, https://github.com/TooTallNate/node-gyp details what you need.

Coinsolidation, after a week of all nighters and only 16 hours sleep, 1,00 gray hairs and 500 cuss words later, you have finally put light at the end of the old guys tunnel.

Yes i need full constantly updated chain data, data that i cannot get without getting banned from other services for querying too much from. Working on algorithms for measuring bitcoin adoption, truly lost or "dead" coins, hourly transaction volume, hourly price points and a ton of other stuff. I am an economics professor who has recently taken an interest in the economics of bitcoin after attending a conference that was held an hour from my home town.

In principal since i cannot have insight on the web server where all my other applications are installed, i am going to need to be able to access insight on the local network. would the standard  Ubuntu 14.04 be able to be configured as such or do i need some sort of server environment. Ideally i would like to be able to code into my applications get XXXXXX data from 192.168.2.XX:3000/api/command.
coinsolidation
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250

Bitmark Developer


View Profile WWW
August 22, 2014, 10:08:14 AM
 #10

Yes, I will help you Smiley

Let's first establish what you need, as insight may not be the answer. Do you need a constantly updated block chain database, or to analyse a timestamped dump?

Also, do you require the full chain data, or only certain parts?

Finally, have you ensured that all dependencies are installed, https://github.com/TooTallNate/node-gyp details what you need.

Coinsolidation, after a week of all nighters and only 16 hours sleep, 1,00 gray hairs and 500 cuss words later, you have finally put light at the end of the old guys tunnel.

Yes i need full constantly updated chain data, data that i cannot get without getting banned from other services for querying too much from. Working on algorithms for measuring bitcoin adoption, truly lost or "dead" coins, hourly transaction volume, hourly price points and a ton of other stuff. I am an economics professor who has recently taken an interest in the economics of bitcoin after attending a conference that was held an hour from my home town.

In principal since i cannot have insight on the web server where all my other applications are installed, i am going to need to be able to access insight on the local network. would the standard  Ubuntu 14.04 be able to be configured as such or do i need some sort of server environment. Ideally i would like to be able to code into my applications get XXXXXX data from 192.168.2.XX:3000/api/command.

Best to check this first... If you are doing API commands have you considered writing your applications to communicate with bitcoind's RPC API directly (or even via the command line)? You can also run programs every time a new block is received using the '-blocknotify=executable %s' configuration option, and maintain an index in the daemon of all transactions with the '-txindex' option. This would all be on a secure local bitcoin running on your server, with no duplication of data.

If you are certain that insight is the answer, and require that local visual browser part of it too, then the best way to proceed may be for me to do the process from scratch on a virtual machine and write the instructions for you to repeat.

Taking the latter option will require you to have a network accessible Ubuntu 14.04 machine. This could be a virtual machine inside VirtualBox or VMWare, or a physical machine on your network. You can run any version of Ubuntu, standard or server, one has the easy to use GUI and the other just a command line.

If you wish to proceed with the insight option, then I'd suggesting getting a Ubuntu machine setup, bitcoin downloaded, and to begin downloading the block chain.

As an alternative option if you can work over the internet rather than just local, then you could acquire a ubuntu VPS (digitalocean/aws) or cheap dedicated server and hand me the root access, I'd set it all up for you. If you are in the UK then I know a man who may be able to provide a good VPS for this quickly - I say UK as then network latency would be lower in communications to and from server.

Warm Regards,

Mark

Bitmark (reputation+money) : Bitmark v0.9.4 (release)
coinsolidation
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250

Bitmark Developer


View Profile WWW
August 22, 2014, 10:16:09 AM
 #11

As an after thought, if you can make a local ubuntu machine network accessible and pass me the ssh details, I could just connect up and set everything up for you on that instead.

Bitmark (reputation+money) : Bitmark v0.9.4 (release)
82ndabnmedic
Full Member
***
Offline Offline

Activity: 250
Merit: 100


View Profile
August 22, 2014, 04:48:13 PM
 #12

Hello DTM - I understand that "coinsolidation" was in the middle of helping you with this issue.  Regretfully he has been banned from bitcointalk.org for 3 days. Please check your PM

CISSP | PMP | CEH

Bitmark: Project Manager & PR Coordinator


BTC: 1FEi8MSP3ccoqLah8EcxfGZVHUViEmQfvQ

BTM: bNidDXnRu5fuD8Th7cPFh7jnPdyAhMh7Nr
RagnarDanneskjold
Full Member
***
Offline Offline

Activity: 144
Merit: 100



View Profile
August 24, 2014, 08:59:41 AM
 #13

here's another approach:
http://codesuppository.blogspot.com/2014/01/a-command-line-interface-for-blockchain.html
http://codesuppository.blogspot.com/2013/07/work-in-progress-on-bitcoin-blockchain.html

git  |  | ID
'Bitcoin is the progress toward a society of privacy. The savage’s whole existence is public, ruled by the laws of his tribe. Bitcoin is the process of setting man free from men'
rapport
Full Member
***
Offline Offline

Activity: 157
Merit: 100


View Profile
August 25, 2014, 07:51:47 AM
 #14

Hello DTM - I understand that "coinsolidation" was in the middle of helping you with this issue.  Regretfully he has been banned from bitcointalk.org for 3 days. Please check your PM

You are the Bitmark Project Manager, and he was the developer who was spending too much time in the forums?
Nicolas Dorier
Hero Member
*****
Offline Offline

Activity: 714
Merit: 501


View Profile
August 27, 2014, 11:12:53 AM
 #15

Use NBitcoin (.NET) see how it works at http://www.codeproject.com/Articles/784519/NBitcoin-How-to-scan-the-Blockchain
The only relevant part for what you want to do is the last one : "Manual Scanning".

Github : https://github.com/NicolasDorier/NBitcoin
Nuget : https://www.nuget.org/packages/NBitcoin/

Bitcoin address 15sYbVpRh6dyWycZMwPdxJWD4xbfxReeHe
nicojuritz
Member
**
Offline Offline

Activity: 98
Merit: 10


View Profile
August 28, 2014, 11:53:39 AM
 #16

Give  BitcoinJ a trial and you wouldn't regret it.
wargo
Sr. Member
****
Offline Offline

Activity: 461
Merit: 250

Technician


View Profile WWW
August 29, 2014, 09:16:25 AM
 #17

What about abe.py?
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!