Reading the block chain with a library?

[Xenland]

How do I read the Bitcoin blockchain with an api or library with a programming language like php, python or java?

[1714883960]

1714883960

[1714883960]

1714883960

[1714883960]

1714883960

[ThomasV]

How do I read the Bitcoin blockchain with an api or library with a programming language like php, python or java?

use libbitcoin

[kokjo]

libbitcoin: https://github.com/libbitcoin/libbitcoin

[arby]

libbitoin as i see is in cpp, not php or java, python as the OP mentioned.

I think he is interested in using this on linux, and not using any executables.

I am searching for this also, but have not found anything yet.

What I found that you can reliably use with PHP is:

https://blockchain.info/api

https://blockchain.info/q

To explore the blockchain.

[kokjo]

libbitoin as i see is in cpp, not php or java, python as the OP mentioned.

I thin he is interested in using this on linux, and not using any executables.

swig!

[arby]

libbitoin as i see is in cpp, not php or java, python as the OP mentioned.

I thin he is interested in using this on linux, and not using any executables.

swig!

Ok I will give that a try also, thanks for the info

[ThomasV]

libbitoin as i see is in cpp, not php or java, python as the OP mentioned.

yes, it has a python frontend

[Xenland]

Nice responses mates, libbitcoin should work fine for what i need

[Xenland]

So how do I use this libbitcoin thing, for example something simple like a balance of a particular address, or a list of details of a particular address? (with python)

[kokjo]

So how do I use this libbitcoin thing, for example something simple like a balance of a particular address, or a list of details of a particular address? (with python)

don't know, try read the api... im not a libbitcoin guru.

[Xenland]

Libbitcoin is quite confusing even looking through the source code, are there any alternatives to libbitcoin that could help me query the blockchain?

[etotheipi]

Armory is perfect this.  It is C++ accessed in Python (via SWIG).  You only ever touch python.

I'm not at my development computer now but there is example code on my github page (though many of the examples were created before RAM reduction, they will mostly still work)

Code:

   from armoryengine import *

   print '\n\nLoading Blockchain from:', BLK0001_PATH
   BDM_LoadBlockchainFile()  # optional argument to specify blk0001.dat location
   print 'Done!'

   print '\n\nCurrent Top Block is:', TheBDM.getTopBlockHeader().getBlockHeight()
   TheBDM.getTopBlockHeader().pprint()

   tx = TheBDM.getTxByHash( hex_to_binary('9c633b5689e462ddf3d52a6edc64226cedd1e1749d0b8e2f70cd9550bfa74c72') )
   tx.pprint()

   # Create and register the wallet before the LoadBlockchainFile call, to merge the scans
   # For now, this will rescan the blockchain
   cppWallet = Cpp.BtcWallet()
   cppWallet.addAddress_1_( hex_to_binary('9c633b56899c633b56899c633b56899c633b5689') )  # hash160
   TheBDM.registerWallet(cppWallet)
   TheBDM.scanBlockchainForTx(cppWallet)
   print cppWallet.getSpendableBalance()

   unspentTxOuts = cppWallet.getSpendableTxOutList()
   for utxo in unspentTxOuts:
      utxo.pprintOneLine()
   

(note, this code is notional since I can't test it where I am, but the syntax is nearly correct)

When I get home later, I can post more example code for how to access what you want.  If you tell me what you're trying to do with it, I can write more directed code samples for you.

[Xenland]

Armory is perfect this.  It is C++ accessed in Python (via SWIG).  You only ever touch python.

I'm not at my development computer now but there is example code on my github page (though many of the examples were created before RAM reduction, they will mostly still work)

Code:

   from armoryengine import *

   print '\n\nLoading Blockchain from:', BLK0001_PATH
   BDM_LoadBlockchainFile()  # optional argument to specify blk0001.dat location
   print 'Done!'

   print '\n\nCurrent Top Block is:', TheBDM.getTopBlockHeader().getBlockHeight()
   TheBDM.getTopBlockHeader().pprint()

   tx = TheBDM.getTxByHash( hex_to_binary('9c633b5689e462ddf3d52a6edc64226cedd1e1749d0b8e2f70cd9550bfa74c72') )
   tx.pprint()

   # Create and register the wallet before the LoadBlockchainFile call, to merge the scans
   # For now, this will rescan the blockchain
   cppWallet = Cpp.BtcWallet()
   cppWallet.addAddress_1_( hex_to_binary('9c633b56899c633b56899c633b56899c633b5689') )  # hash160
   TheBDM.registerWallet(cppWallet)
   TheBDM.scanBlockchainForTx(cppWallet)
   print cppWallet.getSpendableBalance()

   unspentTxOuts = cppWallet.getSpendableTxOutList()
   for utxo in unspentTxOuts:
      utxo.pprintOneLine()
   

(note, this code is notional since I can't test it where I am, but the syntax is nearly correct)

When I get home later, I can post more example code for how to access what you want.  If you tell me what you're trying to do with it, I can write more directed code samples for you.

Nice I heard about armory but I thought it was just a GUI client with advanced features. I'll try it out with python see how it is.

[etotheipi]

Here you go!  I just put together a script that not only scans the blockchain, it does something useful!  (it's at the bottom of the post)

• It took 18 seconds for me to scan the whole blockchain from a cold start
• It took 0.2 seconds to collect a list of every difficulty change since the genesis block.
• It took 24 seconds on my system to count 3,532,497 unique addresses in the blockchain! (as of block 176953)

Timings will vary depending on RAM.  If you have a lot of RAM, just about the whole blockchain will be cached from the first scan, and rescans will be nearly instantaneous.  If you want to avoid rescans, make sure everything is added to your wallet and registered with TheBDM.registerWallet before BDM_LoadBlockChain().

The sample python script at the bottom of this post produced the following output (the difficulty changes):

Code:

Collect all difficulty changes...
     Block       Diff   Date
         0        1.0 	2009-Jan-03 01:15pm
     32256        1.2 	2009-Dec-30 01:11am
     34272        1.3 	2010-Jan-11 05:48pm
     36288        1.3 	2010-Jan-25 08:07am
     38304        1.8 	2010-Feb-04 04:43pm
     40320        2.5 	2010-Feb-14 06:52pm
     42336        3.8 	2010-Feb-24 03:41am
     44352        4.5 	2010-Mar-07 08:14pm
     46368        4.6 	2010-Mar-21 06:54pm
     48384        6.1 	2010-Apr-01 07:07am
     50400        7.8 	2010-Apr-12 04:39am
     52416       11.5 	2010-Apr-21 05:52pm
     54432       12.8 	2010-May-04 05:46am
     56448       11.8 	2010-May-19 10:13am
     58464       16.6 	2010-May-29 09:57am
     60480       17.4 	2010-Jun-11 07:26pm
     62496       19.4 	2010-Jun-24 08:27am
     64512       23.5 	2010-Jul-05 09:57pm
     66528       45.4 	2010-Jul-13 04:03am
     68544      181.5 	2010-Jul-16 12:29pm
     70560      244.2 	2010-Jul-26 10:42pm
     72576      352.2 	2010-Aug-05 03:46pm
     74592      511.8 	2010-Aug-15 07:11am
     76608      623.4 	2010-Aug-26 07:13pm
     78624      712.9 	2010-Sep-08 01:04am
     80640      917.8 	2010-Sep-18 10:04pm
     82656     1318.7 	2010-Sep-28 03:58pm
     84672     1378.0 	2010-Oct-12 01:35am
     86688     2149.0 	2010-Oct-21 01:13am
     88704     3091.7 	2010-Oct-30 06:58pm
     90720     4536.4 	2010-Nov-09 07:29am
     92736     6866.9 	2010-Nov-18 01:44pm
     94752     8078.2 	2010-Nov-30 11:37am
     96768    12252.0 	2010-Dec-09 05:20pm
     98784    14484.2 	2010-Dec-21 01:34pm
    100800    16307.4 	2011-Jan-03 12:10am
    102816    18437.6 	2011-Jan-15 09:26am
    104832    22012.4 	2011-Jan-27 03:16am
    106848    25997.9 	2011-Feb-07 11:53pm
    108864    36459.9 	2011-Feb-18 12:15am
    110880    55589.5 	2011-Feb-27 04:59am
    112896    76192.6 	2011-Mar-09 10:25am
    114912    68977.8 	2011-Mar-24 10:39pm
    116928    82345.6 	2011-Apr-05 04:09pm
    118944    92347.6 	2011-Apr-18 03:49am
    120960   109670.1 	2011-Apr-29 10:53pm
    122976   157416.4 	2011-May-09 05:17pm
    124992   244112.5 	2011-May-18 06:04pm
    127008   434877.0 	2011-May-26 02:41pm
    129024   567269.5 	2011-Jun-06 08:25am
    131040   876954.5 	2011-Jun-15 09:49am
    133056  1379192.3 	2011-Jun-24 07:45am
    135072  1563028.0 	2011-Jul-06 04:35pm
    137088  1690895.8 	2011-Jul-19 03:23pm
    139104  1888786.7 	2011-Aug-01 04:11am
    141120  1805700.8 	2011-Aug-15 07:44pm
    143136  1777774.5 	2011-Aug-30 01:15am
    145152  1755425.3 	2011-Sep-13 05:31am
    147168  1689334.4 	2011-Sep-27 06:47pm
    149184  1468195.4 	2011-Oct-13 09:44pm
    151200  1203461.9 	2011-Oct-30 11:42pm
    153216  1192497.8 	2011-Nov-14 01:56am
    155232  1090715.7 	2011-Nov-29 09:20am
    157248  1155038.3 	2011-Dec-12 02:42pm
    159264  1159929.5 	2011-Dec-26 01:43pm
    161280  1250757.7 	2012-Jan-08 01:26pm
    163296  1307728.4 	2012-Jan-21 10:55pm
    165312  1379647.4 	2012-Feb-04 05:32am
    167328  1376302.3 	2012-Feb-18 06:24am
    169344  1496978.6 	2012-Mar-02 03:25am
    171360  1498294.4 	2012-Mar-16 04:09am
    173376  1626553.5 	2012-Mar-29 01:41am
    175392  1577913.5 	2012-Apr-12 12:04pm
Took 0.2 seconds to collect difficulty list

And here is the script itself.  It illustrates a variety of ways you can access block data.  Mainly, scanning for addresses in the blockchain with balances and unspent outputs,  and walking through every TxOut of every Tx of every Block -- we grab the address from ever standard TxOut and add it to a set() object which only allows unique addresses.

Code:

from armoryengine import *

# NOTE:
#     ALL ADDRESSES THROUGHOUT EVERYTHING ARE IN 20-BYTE BINARY FORM (hash160/addr20)
#     Use hash160_to_addrStr() and addrStr_to_hash160() to convert...

print '\n\nCreating a new C++ wallet, add a few addresses...'
cppWallet = Cpp.BtcWallet()
cppWallet.addAddress_1_( hex_to_binary('11b366edfc0a8b66feebae5c2e25a7b6a5d1cf31') )  # hash160
cppWallet.addAddress_1_( addrStr_to_hash160('1EbAUHsitefy3rSECh8eK2fdAWTUbpVUDN') )   # addrStr
cppWallet.addAddress_1_('\x1b~\xa7*\x85\t\x12\xb7=\xd4G\xf3\xbd\xc1\x00\xf1\x00\x8b\xde\xb0') # binary


print 'Addresses in this wallet:'
for i in range(cppWallet.getNumAddr()):
   print '\t', hash160_to_addrStr(cppWallet.getAddrByIndex(i).getAddrStr20())


print '\n\nRegistering the wallet with the BlockDataManager & loading...'
start = RightNow()
TheBDM.registerWallet(cppWallet)
BDM_LoadBlockchainFile()  # optional argument to specify blk0001.dat location
print 'Loading blockchain took %0.1f sec' % (RightNow() - start)


topBlock = TheBDM.getTopBlockHeight()
print '\n\nCurrent Top Block is:', topBlock
TheBDM.getTopBlockHeader().pprint()


# Add new addresses -- will rescan (which will be super fast if you ahve a lot of RAM)
cppWallet.addAddress_1_( hex_to_binary('0cdcd0f388a31b11ff11b1d8d7a9f978b37bc7af') ) 
TheBDM.scanBlockchainForTx(cppWallet)



print '\n\nBalance of this wallet:', coin2str(cppWallet.getSpendableBalance())
print 'Unspent outputs:'
unspentTxOuts = cppWallet.getSpendableTxOutList(topBlock)
for utxo in unspentTxOuts:
   utxo.pprintOneLine()

print '\n\nTransaction history of this wallet:'
ledger = cppWallet.getTxLedger()
for le in ledger:
   le.pprintOneLine()

print '\n\n'
print '-'*80
print 'Now for something completely different...'
start = RightNow()
print '\n\nCollect all difficulty changes...'
prevDiff = 0
for h in xrange(0,topBlock+1):
   header = TheBDM.getHeaderByHeight(h)
   currDiff = header.getDifficulty()
   if not prevDiff==currDiff:
      print str(h).rjust(10),
      print ('%0.1f'%currDiff).rjust(10),
      print '\t',unixTimeToFormatStr(header.getTimestamp())
   prevDiff = currDiff

print 'Took %0.1f seconds to collect difficulty list' % (RightNow()-start)
      
      
print '\n\nCount the number of unique addresses in the blockchain'
start = RightNow()
allAddr = set()
for h in xrange(0,topBlock+1):
   if h%10000 == 0:
      print '\tScanned %d blocks' % h
      
   header = TheBDM.getHeaderByHeight(h)
   txList = header.getTxRefPtrList()
   for tx in txList:
      for nout in range(tx.getNumTxOut()):
         txout = tx.getTxOutRef(nout)
         if txout.isStandard():
            allAddr.add(txout.getRecipientAddr())

print 'Took %0.1f seconds to count all addresses' % (RightNow()-start)
print 'There are %d unique addresses in the blockchain!' % len(allAddr)

This scanning is lower-level than what happens in Armory, so you are mainly using C++ objects.  You can find a full list of all the methods in BlockObjRef.h and BlockUtils.h.  Please ask questions if something isn't clear, or if you want to use some functionality not exemplified here.  

[Sukrim]

If you tell me what you're trying to do with it, I can write more directed code samples for you.

I personally want to get the timestamp of blocks with a certain block height once per day, but didn't get around to looking at either libbitcoin or Armory.
Ideally it would be something like "TheBDM.getHeaderByHeight(12345).getTimestamp()"...

Also for armory I still have to run bitcoind to get the blockchain updated, right?

Edit:
Wow - this is awesome, you manage to answer my posts before I even post them!
Edit2:
And I even guessed both function names correctly to the letter! This is getting creepy...

[etotheipi]

If you tell me what you're trying to do with it, I can write more directed code samples for you.

I personally want to get the timestamp of blocks with a certain block height once per day, but didn't get around to looking at either libbitcoin or Armory.
Ideally it would be something like "TheBDM.getHeaderByHeight(12345).getTimestamp()"...

Also for armory I still have to run bitcoind to get the blockchain updated, right?

To use Armory, you need bitcoind running to receive blockchain updates and send&receive transactions.  For the example code above, you only need to have the blk0001.dat file produced by the bitcoind, but it doesn't need to be running when you run the script.

Edit:  Wow, I must have telepathy.  Glad I could answer your question... before you asked it!

[Xenland]

How does one just query the balance of a Bitcoin address with armory? is there any calculations involved(like looping over tx history and sum up a balance) or can I just call it and print out a simple balance?

Im' sure a lot of programmers could benefit the solution to this question to write their own software to use armory for like making their own Blockchain.info website or w/e.

[etotheipi]

How does one just query the balance of a Bitcoin address with armory? is there any calculations involved(like looping over tx history and sum up a balance) or can I just call it and print out a simple balance?

Im' sure a lot of programmers could benefit the solution to this question to write their own software to use armory for like making their own Blockchain.info website or w/e.

The code was in my example, but I guess my example was too long.  So here's an extraction of it.

Code:

from armoryengine import *

cppWallet = Cpp.BtcWallet()
cppWallet.addAddress_1_( addrStr_to_hash160('1EbAUHsitefy3rSECh8eK2fdAWTUbpVUDN') )   # addrStr
TheBDM.registerWallet(cppWallet)
BDM_LoadBlockchainFile() 
TheBDM.scanBlockchainForTx(cppWallet)

fullBalance = cppWallet.getFullBalance()

print '\n\nBalance of this wallet:', coin2str(fullBalance)

[Xenland]

How does one just query the balance of a Bitcoin address with armory? is there any calculations involved(like looping over tx history and sum up a balance) or can I just call it and print out a simple balance?

Im' sure a lot of programmers could benefit the solution to this question to write their own software to use armory for like making their own Blockchain.info website or w/e.

The code was in my example, but I guess my example was too long.  So here's an extraction of it.

Code:

from armoryengine import *

cppWallet = Cpp.BtcWallet()
cppWallet.addAddress_1_( addrStr_to_hash160('1EbAUHsitefy3rSECh8eK2fdAWTUbpVUDN') )   # addrStr
TheBDM.registerWallet(cppWallet)
BDM_LoadBlockchainFile() 
TheBDM.scanBlockchainForTx(cppWallet)

fullBalance = cppWallet.getFullBalance()

print '\n\nBalance of this wallet:', coin2str(fullBalance)

Oh yeah I noticed that part of your code, I just assumed that there was a quick way of going about this since it takes a few seconds to scan the block chain for one address and I'm looking for a way to scan and update the database for multiple addresses. Does that sound possible or am I stuck with this route you presented me? Thanks for your help btw I appreciate it.

[etotheipi]

Oh yeah I noticed that part of your code, I just assumed that there was a quick way of going about this since it takes a few seconds to scan the block chain for one address and I'm looking for a way to scan and update the database for multiple addresses. Does that sound possible or am I stuck with this route you presented me? Thanks for your help btw I appreciate it.

You can use as many addAddress_1_() calls as you want before a rescan, and the rescan will still take the same amount of time.  Load 20 addresses into your wallet before loading the blockchain, and it will take the same time as if you loaded in 10,000.

After the blockchain is loaded, if you add any new addresses to the wallet, the chain will be re-scanned on the "scanBlockchainForTx()" call, and again, it won't matter how many addresses that is, it'll take the same amount of time.