Bitcoin Forum
June 21, 2024, 04:53:03 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1]
1  Bitcoin / Project Development / Looking for an Oracle DBA i can ask a few questions ! on: September 08, 2014, 04:26:01 PM
I am in quick need of someone who is very knowledgeable about Oracle DB's that i can aska  few questions quick for a bitcoin project i am working on. Message me if you can help please. I dont need any work done, just need a few questions answered.
2  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: September 03, 2014, 02:17:35 PM

Sure, that would be awesome. Is this something hosted on one of your servers or something i can host locally? i would really feel bad eating up your server resources running a bunch of complex queries over time.

I have MySQL on our local server but will probably convert it to standard sql using SSMA if it is something we can host locally as all the information we have databased already is housed on the SQL EE server that we have. My web server and sql server have hundreds of gigs of ECC ram and multiple processors so i dont think it will be too taxing resource wise.

I really appreciate your help, if you need some help writing the documentation side of things, although not a developer, i am VERY good at tinkering and figuring things out so as i use it i can help with some documentation and instructions over time. All the software i have now was things i chopped and screwed to make work the way i want over time, i cannot code, but i can read code (Java, PHP, C#, C++) well enough to understand what is going on and how to make it work or how to copy paste a bunch of crap together to get the job done, not much experience with Python yet, but this would be a good learning experience.

I will be releasing the source. It runs locally and connects to a local coin daemon. Db, RPC connections and some misc. parameters are set in a config file.
You should be able to adapt it to use other databases as I used raw SQL queries (through PyMySql) rather then a library like SQLalchemy, but some change in the coding might be involved.

Thanks dreamwatcher,

i will keep my eye out for the source code and get to work on getting it all installed as soon as i get it. Once i get it SQL friendly and working through a few large querys to ensure it works i will send back a branch for standard SQL DB's in the event it proves useful to you in the future.
3  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: September 03, 2014, 11:03:06 AM
Coming late to the thread on this one, but after reading through it sounds interesting.

While not a 'coder', I've got some background in data processing and would be interested in helping out the OP, if help is wanted.  Where are you planning on running the data collector?

baldpope,

I have a pretty substantial amount of old servers and data miners laying around from previous work as a data center manager. i was able to keep a lot of stuff that the companies were throwing away. So for now i am running the data collection locally on my own local network, although i have several static IP's and T1 & T3 lines so i could easily make it accessible remotely. Feel free to message me in your spare time, i am always looking for a helping hand from someone with experience. especially in data processing as that is the bulk of what the project is about.
4  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: September 03, 2014, 01:19:29 AM
The CCE database loader builds a database that has much of that information. The web server portion of CCE limits queries for ease of use and to avoid overloading the server.

I am making a previous version of the loader (3.0) available to barwizi. I want to clean up the code a bit and write some basic instructions, but I can make it available to you also.

The basic version I am providing will build a database with tables for:
blocks, txins, txouts, and address balances. You can use the coin daemons block notify to trigger an update. During the update the loader also checks for orphan blocks within the last 250 blocks and corrects blocks/tx and address balances when an orphan is detected.
It is written in Python and uses Mysql.

If you want to check it out,I will shoot you a PM when ready with the private Github information.

Sure, that would be awesome. Is this something hosted on one of your servers or something i can host locally? i would really feel bad eating up your server resources running a bunch of complex queries over time.

I have MySQL on our local server but will probably convert it to standard sql using SSMA if it is something we can host locally as all the information we have databased already is housed on the SQL EE server that we have. My web server and sql server have hundreds of gigs of ECC ram and multiple processors so i dont think it will be too taxing resource wise.

I really appreciate your help, if you need some help writing the documentation side of things, although not a developer, i am VERY good at tinkering and figuring things out so as i use it i can help with some documentation and instructions over time. All the software i have now was things i chopped and screwed to make work the way i want over time, i cannot code, but i can read code (Java, PHP, C#, C++) well enough to understand what is going on and how to make it work or how to copy paste a bunch of crap together to get the job done, not much experience with Python yet, but this would be a good learning experience.
5  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: September 02, 2014, 12:18:58 PM
I am curious as to why you are trying to parse the block chain file instead of querying the coin daemon for the information

When I wrote the CCE 3 block explorer software, I completely abandoned the ABE style of parsing the block chain files as the daemon will give you all the information you need without having to adjust to the different block structures of various chains.

I might be willing to give you limited license to the CCE 3 database loader, if you would give me a better idea of what you are ultimately trying to do.

Thanks dreamwatcher,

Ultimately i am working on a FULL bitcoin stats website that can pretty much tell you anything anybody would want to know about bitcoin at any given time. There are a lot of sites that give you a specific data set for a specific inquiry but really none that will give you everything all in one place.

Seeing as the one thing that a lot of people i have found ask for but is not readily available at any given website is "a list of all bitcoin addresses with a positive balance"  i wanted to start there. With that information there are several sub sets of info that can be derived from that. Beyond being able to list every bitcoin address with a positive balance, i can then publish a statistic in number form that is easy to track. ( X# of bitcoin addresses in use today) and also (X# of bitcoin addresses in use yesterday) as well as (average # of bitcoin addresses in use this week, month, year, etc.) Also with that i can create an interface (php website) which links every one of those Xmillion addresses to our insight server (like blockchain) so people can view the statistics of each of them.

For now that is Ultimately what i am working on, in the total picture there is much more which i will list below, but if you were only interested in what i want the data i was asking for, that is why.

In overall other statistics that i will work on finding ways to gather in the end game are. Address with the Most TX's for the day, week, month, year, all time. # of addresses that have not been touched in a month, 3 months, 6 months, 1 year, and longest untouched addresses (no TX out).

And in the VERY, VERY, VERY long term, probably the last part of the project, i will work on a way to calculate the average transaction value for the hour, day, week, month, year, and all time. This will be last as it involves the most amount of work, coding, math, and formulation to achieve.

But for now, i just want to start with finding a way to get a list of all addresses with unspent outputs carved down to a .txt file 1 addy per line that i can import into the SQL database, then i will work on a way of updating it with each block with the on block notify feature so it will be a live statistic and output.
6  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: September 01, 2014, 12:50:29 PM
I was talking about bitcoin, and i'm getting a little closer to a solution, i have gotten Insight up and running, and now i just need to figure out how to access the info in Level DB as at face value it seems that that info should already be parsed in there some where.
7  Alternate cryptocurrencies / Altcoin Discussion / Re: Parse the blockchain for all addresses with positive balance on: August 31, 2014, 01:58:35 PM
This is the most commonly used one. (Linux only!)
Instructions are included in the repo.

I have no experience with it, though.


https://github.com/znort987/blockparser

Thanks flatfly, My experience with anything linux is 0 and i wouldn't have the slightest idea on where to begin or how to make it work.
8  Alternate cryptocurrencies / Altcoin Discussion / Parse the blockchain for all addresses with positive balance on: August 30, 2014, 06:44:50 AM
I have been messing around with a ton of blockchain parsers recently and have yet to find one that I CAN GET to spit out a list of all addresses that currently contain bitcoin. I know it can be done because a few people have published it in the past in .txt format but i cannot seem to find the command/application to do it myself. I was wondering if anyone could shed any light on this as i would like to gather this data regularly (daily) to use for a bitcoin stats website that has all kinds of odd info about the blockchain. # of addresses with a balance, # of bitcoin not touched/moved in the past year/2years/etc. and other things of that sort.

If possible, i am looking for one that can output the data into a long list in .txt format like below

1addresswithbalance523789
1anotheraddresswbalance34
1nextwithbalance23478923
1YOUGETTHEPOINT324789

In the end i am going to try and hire someone to code a simple program that just does this then parses each new block #onblocknotify to keep the list updated, but until that day comes i would like to find the fastest way to do it manually.

Please if you know the program that does it, also include the command line to get it to give the desired result as my coding abilities are pretty much chop&screw.
9  Bitcoin / Development & Technical Discussion / Re: Is there a good blockchain parser that is not Linux? on: August 22, 2014, 09:34:27 AM
Yes, I will help you Smiley

Let's first establish what you need, as insight may not be the answer. Do you need a constantly updated block chain database, or to analyse a timestamped dump?

Also, do you require the full chain data, or only certain parts?

Finally, have you ensured that all dependencies are installed, https://github.com/TooTallNate/node-gyp details what you need.

Coinsolidation, after a week of all nighters and only 16 hours sleep, 1,00 gray hairs and 500 cuss words later, you have finally put light at the end of the old guys tunnel.

Yes i need full constantly updated chain data, data that i cannot get without getting banned from other services for querying too much from. Working on algorithms for measuring bitcoin adoption, truly lost or "dead" coins, hourly transaction volume, hourly price points and a ton of other stuff. I am an economics professor who has recently taken an interest in the economics of bitcoin after attending a conference that was held an hour from my home town.

In principal since i cannot have insight on the web server where all my other applications are installed, i am going to need to be able to access insight on the local network. would the standard  Ubuntu 14.04 be able to be configured as such or do i need some sort of server environment. Ideally i would like to be able to code into my applications get XXXXXX data from 192.168.2.XX:3000/api/command.
10  Bitcoin / Development & Technical Discussion / Re: Is there a good blockchain parser that is not Linux? on: August 22, 2014, 09:03:42 AM
GYP

Just in case, the two most common problems with node-gyp, are:

1. Gyp has a dependency on Python, so you need that installed
2. It calls 'node' and sometimes the node binaries are referenced as 'nodejs'.

I have to defend the BitPay guys and insight a bit, it does work very well I got one up and running here for bitmark very easily, on Ubuntu 14.04 however.

Yep, got python installed, not sure where to even start to look for mis referenced 'node' listings, i admire your defense and am jealous of your working insight client, but after a week of asking for anything, just a reply would have been nice. You seem to have it under control with linux. i got an old windows 7 laptop i could wipe out and throw ubuntu on i believe. i dont know if they have something like team viewer for linux, but if i could find something do you think by the graces of god you could help an old man get it set up on our network so i could at least finish development while i wait for a a fix to the GYP issue? I would be forever in your debt.
11  Bitcoin / Development & Technical Discussion / Re: Is there a good blockchain parser that is not Linux? on: August 22, 2014, 08:30:17 AM
I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.

I guess you've found the Gyp issues then, a problem on almost every platform, quick fixes are available on linux, unsure about windows.

Take a look at this and notice blockchain64.exe https://github.com/jeffchan/blockchain-parser - I've only used it to produce statistics so far (on *nix), I cannot remember if you can dump everything to SQL or not.

Best of luck.

Yup found the Gyp issues, been begging for help almost everywhere but i get nothing but trolls most the time, so i have decided to take a different approach and just say screw it insight is a lost cause lacking in documentation and support.

Have you taken a look at BitcoinJ ? You can configure it to use a SQL database via JDBC.

https://github.com/bitcoinj/bitcoinj

I was just looking into bitcoinJ, unfortunately i know jack diddly about Java. I snagged up a copy of netbeans and have been doing a little poking around, but without someone to help me get on the right track i am afraid i would just be spinning my wheels for months which helps nobody.


In the end, it would be great if someone knew of a fix for the GYP crap with insight on windows but i think that ship has sailed. So i am on a quest for something that is already working, or close to working that i can tinker with to make work. 
12  Bitcoin / Development & Technical Discussion / Is there a good blockchain parser that is not Linux? on: August 22, 2014, 06:02:41 AM
I am looking for a good software to parse the bitcoin blockchain so i can turn it into some sort of database in some sort of SQLish fashion. I have burnt out on how big of a POS npm that comes with Node.js is so therefore have to abandon insight-api as a solution.

I need something that is NON-linux as all of the other softwares are windows dependent and it has to run on windows server as that is the web host server we have.

any help would be appreciated, i have grown a few thousand more grey hairs this past week fighting with insight-api and more so NPM.
13  Economy / Service Discussion / Re: Help installing bitcoin Insight-API on: August 22, 2014, 05:09:41 AM
Anyone?
14  Economy / Service Discussion / Help installing bitcoin Insight-API on: August 22, 2014, 02:51:43 AM
Hey guys,

Trying to get insight-api installed on windows server 2012 to host a blockchain.info alternative with open unrestricted access for developers and having a hell of a time along the way.

I have got the repository cloned and installed, node.JS installed, i installed VS to get around all the sdk issues and now am having MAJOR problems with the NPM install.

I would list the debug here but the errors list is INSANELY long and it would take a few hours for anyone to read through them. I am sure that anyone that has tried to install insight has suffered these same problems as i have tried it on 5 different machines with 3 different OS's and gotten the same issues every time.

What i am looking for is someone who has installed it prior or is super familiar with node.js and NPM that has skype and/or teamviewer who would be willing to spare me a few minutes of their time to help me get past the pulling my hair out phase of this en devour. It would be greatly appreciated.

Thanks
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!