Bitcoin Forum
December 11, 2024, 11:34:36 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Get all bitcoin data in a structured manner  (Read 575 times)
ovanwijk (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 2


View Profile
February 06, 2017, 09:12:30 PM
Merited by ABCbits (1)
 #1

First of all hi, I am totally new to bitcoin development. I did do some mining back in the days but now I am doing something new, researching graph theory and then mainly time evolving graphs.

What is better then actually use a live stream of transactions that make the graph between wallets? Well nothing really.

So what I am asking here is some directions on the easiest way to get all bitcoin block and transaction data and stuff it into a database, possibly with live updates.

At first I though of using the bootstrap.dat file and just read it line by line. However any research in the file on google doesnt hit any developer resources (like file encoding & structure) and mainly shows resources telling its use is outdated.

Some of the options I see are:
- Just downloading a wallet/node, let it sync and read all data files (no idea how yet but it must be somewhere on disk).
- I saw there are some python libraries (https://github.com/petertodd/python-bitcoinlib) that I can use to run the RPC interface? (probably end up downloading all data through that?)
- I saw that there is a RPC Cli? Not sure how it works or if it fits my purpose or if it is mainly useful to keep my database up to date.

PS I am quite the experienced programmer but just don't know C++.

Thanks in advance!
achow101
Moderator
Legendary
*
expert
Offline Offline

Activity: 3570
Merit: 6927


Just writing some code


View Profile WWW
February 06, 2017, 09:24:35 PM
Merited by ABCbits (1)
 #2

You should use Bitcoin Core for this. Bitcoin Core has a JSON-RPC interface which is also accessible through a provided bitcoin-cli binary. It also provides some notifier and listener services.

You will probably want to use the ZeroMQ interface provided by Bitcoin Core. You can set it up to give you block hashes, tx hashes, raw blocks, and raw transactions, every time it receives a new block or transaction. This will essentially give you a live stream of all valid transactions and blocks received by Bitcoin Core. Documentation on using ZMQ with Bitcoin Core is available at https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md

IadixDev
Full Member
***
Offline Offline

Activity: 322
Merit: 151


They're tactical


View Profile WWW
February 06, 2017, 11:11:31 PM
 #3

Maybe you can check block explorer code to get the logic, they pretty much do what you are talking about Smiley

ovanwijk (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 2


View Profile
February 07, 2017, 03:24:53 PM
 #4

Thanks I will look into it!
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!