Try it out: local block explorer program

[jneb]

Hi folks,
I wrote (yet another?) bitcoin program in Python, allowing to explore the local block chain.
See bitbucket.org/jneb/bitcoin.
I hope you like it.

[1714008994]

1714008994

[1714008994]

1714008994

[1714008994]

1714008994

[1714008994]

1714008994

[jneb]

That is what happens if you publish something for the first time.
There was a big bug in my bitcoin program, so that the "more" function didn't work, resulting in a, er, suboptimal user experience.
The program now gives much better help messages, an it still has all these functions.
You get it from bitbucket.org/jneb/bitcoin, click the "get source" button. It is only one file.
A short demo:

prompt>python jbitcoin.py
Welcome to the interactive environment. Type q to exit; ? for help.
|> last
Block 135113, 07-07-11 @ 06:06:56
173 transactions.
Hash:   0000000000000616989acdd24c859c14b046543d437b70ca050f5d9af76729fc
Nonce: 418719447, Bits: 1a0abbcf, giving difficulty 1563028.00
Next hash: ..0000000000    Previous hash: ..e14e960faa
|> more
Target: 0000000000000abbcf0000000000000000000000000000000000000000000000
 0:  1 in,  1 out, hash=..f55686091b, amount=  50.09 fee=-0.09 (135113)
 1:  1 in,  1 out, hash=..05368e940d, amount=   1.23 fee=0.00 (135113)
 2:  3 in,  2 out, hash=..6bf1a4c2ac, amount=  10.20 fee=0.00 (135113)
 3:  5 in,  1 out, hash=..c8bd3ccbdc, amount=  18.55 fee=0.00 (135113)
 4:  1 in,  2 out, hash=..ea0f4c2e98, amount=   1.02 fee=0.00 (135113)
 5:  3 in,  2 out, hash=..e87643305c, amount= 149.18 fee=0.00 (135113)
 6:  2 in,  2 out, hash=..4c7ee141d5, amount=  17.01 fee=0.00 (135113)
 7:  3 in,  2 out, hash=..2b1d4dfa2b, amount=   5.26 fee=0.00 (135113)
 8:  1 in,  2 out, hash=..14dc3c3169, amount=   0.25 fee=0.00 (135113)
 9:  2 in,  1 out, hash=..69893c0f57, amount=   1.11 fee=0.00 (135113)
...
163:  1 in,  2 out, hash=..029d68e619, amount=   2.44 fee=0.00 (135113)
164:  1 in,  2 out, hash=..0f68ff7f6a, amount=   2.17 fee=0.00 (135113)
165:  1 in,  2 out, hash=..d9517a4d63, amount=   1.10 fee=0.00 (135113)
166:  1 in,  2 out, hash=..421387ba95, amount=   0.96 fee=0.00 (135113)
167:  1 in,  2 out, hash=..5f088c70e4, amount=   0.93 fee=0.00 (135113)
168:  1 in,  2 out, hash=..c30227fd64, amount=   0.59 fee=0.00 (135113)
169:  1 in,  2 out, hash=..ae9f0248f4, amount=   0.09 fee=0.00 (135113)
170:  1 in,  2 out, hash=..1da17dd04c, amount=   0.08 fee=0.00 (135113)
171:  1 in,  2 out, hash=..7cb26c6be8, amount=   0.03 fee=0.00 (135113)
172:  2 in,  2 out, hash=..0976a71f0b, amount=   0.03 fee=0.00 (135113)
|>
No more details.
|> 168
Transaction ..c30227fd64 from block 135113: 07-07-11 @ 06:06:56
Inputs: 1, outputs: 2, amount: 0.59, fee: 0.00
(1 unspent)
|> more
Hash: 346930aabab5680d3e75e7b29df397a70eb6537ae30044534937efc30227fd64
 0: In     0.59: from 1L8zFHdgLD6neXEwkQjPtxjEyCezmJaqoM, trx ..5f088c70e4[0]=
 1: Out    0.09: to   1HoSZisiokmGUhaJzCfXnU5UaCYSfUveJv, trx ..ae9f0248f4[0]=
 2: Out    0.50: to   15CKrgamF2CsuVuY7NbUrYhEyaG8kPZPSv (not spent)=
|> 0
Transaction ..5f088c70e4 from block 135113: 07-07-11 @ 06:06:56
Inputs: 1, outputs: 2, amount: 0.93, fee: 0.00
(1 unspent)
|> more
Hash: a351b331c0202470241e6bafc5228c560648ffd56aeb8e7387c7385f088c70e4
 0: In     0.93: from 12Ck3vuPZQLx9QHHySVyAi2ci19WVSWCQX, trx ..421387ba95[0]=
 1: Out    0.59: to   1L8zFHdgLD6neXEwkQjPtxjEyCezmJaqoM, trx ..c30227fd64[0]=
 2: Out    0.34: to   19PsjrzHTgHCCUGci1hdMwfYPHbfdT3GaF (not spent)=
|> 0
Transaction ..421387ba95 from block 135113: 07-07-11 @ 06:06:56
Inputs: 1, outputs: 2, amount: 0.96, fee: -0.00
(1 unspent)
|>
Hash: ddd9bea081c8ef709699ff309402aca625b44084c13deeb581ea7d421387ba95
 0: In     0.96: from 1NUixFvMttKWpCsM1Jh7dxXsBYtJkcTk7r, trx ..d9517a4d63[0]=
 1: Out    0.93: to   12Ck3vuPZQLx9QHHySVyAi2ci19WVSWCQX, trx ..5f088c70e4[0]=
 2: Out    0.03: to   1B3e87Mq114UoYnWUtxRe4ua5oARYNSi4C (not spent)=
|> converge 0
Followed a chain of length 68
Found 66 extra outputs with 66 addresses
|>
Extra outputs that obtained money
  # freq address
  0:  1* 12W8h3KH1Gq925xuJFAHrsc3o4vhiURwBi
  1:  1* 1Cm95xXpiaMpW7iqKUhKcGe6x32gcxviJX
  2:  1* 1FfJ7MUtaBzoSmUQSt4SP1QWwVYVMr5dAh
  3:  1* 1WxNtJXBnFzHo6Kutc3gyZA5qMaGH8SrL
  4:  1* 1CMPmoY5sQMy4eCiFu5BgMnPNSkMAC6Eqa
  5:  1* 17WLcpiQzApEcrZQ81rPaizFHum47Cr4Hr
  6:  1* 1KAKbv5EPvebcbKerCecGTbG2sU65UsX4P
  7:  1* 1FDDLBSKjg9exoEohjKznK6AX1hRMNEG94
  8:  1* 1P8fTpj6YhhGFRPPs7Cqm1Nkri4zm8RVYH
  9:  1* 192tzZib9NCMMqdxivvEysDQdGdrgjqPG9
...
 56:  1* 1FCH1mRzjzYb3KgWkmX3qmdpQRnqqKmJWP
 57:  1* 1PmdQyKsCKJqkjCZKP9QEQnzBLcPga1g1A
 58:  1* 1KRkJ6qwoq4BbyBha45AkGfeugg3P33VF5
 59:  1* 1Nkj5stjKAaZeiCaLojk6GxeJPxpbVpd99
 60:  1* 163CPiZWEgn8vtCagaxsnyW1erHHWqEkf5
 61:  1* 1Kd7YtchtdyWPe6z9qmEibUPBZhsDzij6j
 62:  1* 17xAoZ51LNE8VvL1BNnidrfnfJBAtZjyRS
 63:  1* 1Nog7UK1A3C6UgpimHDZmR2ghrh6oenbQy
 64:  1* 1CSPcRMLmCsj5oagLv6sSbJyhoJmevfZQs
 65:  1* 17CLwhc48apCHh73Na1bWGDB3q5e5Gba8r
|>
Chain:
66: Transaction(1 in, 2 out, hash=..d9517a4d63) from block 135113
67: Transaction(1 in, 2 out, hash=..0f68ff7f6a) from block 135113
68: Transaction(1 in, 2 out, hash=..029d68e619) from block 135113
69: Transaction(1 in, 2 out, hash=..0035d1987a) from block 135113
70: Transaction(1 in, 2 out, hash=..bc24e69a28) from block 135113
71: Transaction(1 in, 2 out, hash=..7f10f8f98e) from block 135113
72: Transaction(1 in, 2 out, hash=..d8f5e5e391) from block 135113
73: Transaction(1 in, 2 out, hash=..45e5a030c1) from block 135113
74: Transaction(1 in, 2 out, hash=..cf0796496b) from block 135113
75: Transaction(1 in, 2 out, hash=..a44a2188e9) from block 135113
...
124: Transaction(1 in, 2 out, hash=..81a97eaff6) from block 135113
125: Transaction(1 in, 2 out, hash=..6af1eaf662) from block 135113
126: Transaction(1 in, 2 out, hash=..c7994ec60d) from block 135113
127: Transaction(1 in, 2 out, hash=..d0c114e3c1) from block 135113
128: Transaction(1 in, 2 out, hash=..994f886b26) from block 135113
129: Transaction(1 in, 2 out, hash=..a059722a82) from block 135113
130: Transaction(1 in, 2 out, hash=..d55e68cf82) from block 135108
131: Transaction(1 in, 2 out, hash=..31aa415886) from block 135107
132: Transaction(1 in, 1 out, hash=..2c316a9e93) from block 135106
133: Transaction(1 in, 1 out, hash=..a1eb554e32) from block 135004
|> 6
Bitcoin address: 1KAKbv5EPvebcbKerCecGTbG2sU65UsX4P
|>
Hash: c7358931ecd341acc86317e950214520a751e74b
public key not known (yet)
|>
All transactions of 1KAKbv5EPvebcbKerCecGTbG2sU65UsX4P by date:
Pubkey search: blockexplorer : address not seen.
<Takes long reading block file from disk: User pressed Ctrl-C>
*** Printing interrupted.
|> quit

I hope you get the idea.
- jneb

[sadpandatech]

pretty nifty little tool actually.

Is there any way you can have it display the total fees associated with a block maybe right next to the total number of transactions?

[jneb]

I am not sure I understand what you mean.
The total fees per block are shown if you enter a block, and type more (or just <CR>) already.
- jneb

[jneb]

Hi,
My program keeps progressing. To whet your appetite, here some interesting statistics:
Highest transaction ever: 432109.87654321 BTC (block 130281)
Smallest: .00000001 (25888 times)

There are 1532089 transactions and 1228772 accounts.
524680 amounts are unspent.
Accounts used only once for storing an amount of money: 217423
Highest number of transactions to one account: 8076 (159FTr7Gjs2Qbj4Q5q29cvmchhqymQA7of)
Second highest: 7843 (the culprit is 1E29AKE7Lh1xW4ujHotoT4JVDaDdRPJnWu)
Highest number of inputs to a transaction: 965 (block 55862)
Highest number of outputs: 2002 (block 134334)
Transactions of BTC .01: 180593 (very popular amount; first block 2812)
There are 275974 transactions with one input and one output.
Accounts that have money stored to then once, and never retrieved:
The block hash values are nicely distributed in the range 0..limit, which means that nobody is fiddling with the block hashes. As an exception, block 0 has a low hash value, but that is explainable.
There is one signature that is incorrectly coded (used non-significant zeroes in the DER encoding).
Most signatures (with 64 bytes data) take 71 bytes, but there are 15 signatures of 69 bytes (because non-significant zeroes are suppressed).
Lowest "seriously looking" account value: 1TDpWBBvGdL8MggaRq1AJ7U36RWsuL, but no money has been withdrawn from it yet. This is one output of a 1999 output transaction in block 134334.
And so on...

Newest feature of the program: a local public key store, so searching for transactions is faster.

See bitbucket.org/jneb/bitcoin: it is a single Python 2.7 file. Installing colorama is recommended.

Hope you like it.
jneb

[bitlotto]

How long does it take to get all tx for an address? Can you specify after a block number to speed it up?
Also do you intend on having filters? Say, list all deposits for an address bigger than xx? or something.

Pretty neat! I've been looking for a local block explorer in case the Internet is down for looking at my lottery stuff! Good work!

[jneb]

How long does it take to get all tx for an address? Can you specify after a block number to speed it up?
Also do you intend on having filters? Say, list all deposits for an address bigger than xx? or something.

Pretty neat! I've been looking for a local block explorer in case the Internet is down for looking at my lottery stuff! Good work!

Thanks for the reply and the ideas!

Be sure to get the new version: I had a memory leak (!! my first one in Python) which is fixed now.
Now for the questions:
Getting a tx for an address is pretty slow, since BitCoin doesn't keep a database of that. The program searches through the entire block file using mmap.find, and to speed it up, it first figures out the public key.
Searching from a given block number is a good idea: will come up in a future version for sure.

Filters are an interesting idea. There is already a bit of that in the new "categorise addresses" command. It would be very slow, though.

- jneb

[webosftw]

I dont get it. What is this supposed to do?

[k]

thanks jneb - seems like a neat little program form how you describe it.

I downloaded it and tried to get it running but failed. I'm a total newbie with python and have very little coding experience in general (some classes in university years ago) so to be honest I really don't know what I'm doing. I was messing around there for a while but am going to give it a rest for tonight, maybe try again tomorrow.

I think I installed the colorama package although I think that's optional.
Next hurdle was

Code:

    from Crypto.Hash import SHA256, RIPEMD160
ImportError: cannot import name SHA256

From some searching I think that means I need to install the pycrypto package - is that right?
Anyway I couldn't get that to install. Tried a few different things. Can't remember exactly. I think the first error was something like couldn't find the vcvarsall.bat - I followed the instructions here http://blog.eddsn.com/2010/05/unable-to-find-vcvarsall-bat/ [edit: screwed up link before]. Seemed to get a bit further in trying to install pycrypto but think I'm falling down with not having the g++ compiler in the right place. Again, I'm totally clueless here, just trying things, googling, trying something else so am probably going about installing/running all these programs the wrong way.
This is all on windows by the way. Been meaning to try out some sort of linux again so might do that again before I attempt this again.

anyway if you or anyone has any pointers on the how to get this program to run that would be much appreciated.
It might be just a case of me trying to run before I can walk. I've only donwloaded python recently as a result of bitcoin and am meaning to learn how to program with it.

thanks again


Update:

think I may have installed the pycrypto package but not 100% sure. It produces a lot of text in the comand line window, don't see any errors but then when I try to run your jbitcoin.py program I still get this error

Code:

    from Crypto.Hash import SHA256, RIPEMD160
ImportError: cannot import name SHA256

[bitlotto]

I dont get it. What is this supposed to do?

All transactions/bitcoins are stored in the blockchain. This allows you to browse all bitcoin accounts and see what went where. Similar to blockexplorer.com I guess.

[bitlotto]

I haven't tried running it yet but I -quickly- looked at the source. What kind of stuff does it do with the wallet? I was thinking it just browsed the blockchain. Pretty interesting! Is the user able to list transactions that they themselves spent to an address for example? Can the user specify where the blockchain and wallet are located?

[jneb]

thanks jneb - seems like a neat little program form how you describe it.

From some searching I think that means I need to install the pycrypto package - is that right? [...]
anyway if you or anyone has any pointers on the how to get this program to run that would be much appreciated.
It might be just a case of me trying to run before I can walk. I've only donwloaded python recently as a result of bitcoin and am meaning to learn how to program with it.

I you run Windows, the simplest way to get PyCrypto is voidspace: http://www.voidspace.org.uk/python/modules.shtml#pycrypto
I put that in the error message now.
- jneb

[jneb]

I haven't tried running it yet but I -quickly- looked at the source. What kind of stuff does it do with the wallet? I was thinking it just browsed the blockchain. Pretty interesting! Is the user able to list transactions that they themselves spent to an address for example? Can the user specify where the blockchain and wallet are located?

The tool is a "watch only" tool, although it allows you to talk to your local bitcoin server and tell it to do stuff.
And yet, you can list you own transactions, and follow the money back to the coin creation or in the other direction to an unspent account.
I didn't put in a way to locate the blockchain and wallet. Good idea; I put it in version 3.5. (Edited: Version 3.5 is out now).
- jneb

[NYConsultant]

Great stuff!  This really helps me fill in some gaps in my understanding of the protocol.

[k]

thanks jneb - seems like a neat little program form how you describe it.

From some searching I think that means I need to install the pycrypto package - is that right? [...]
anyway if you or anyone has any pointers on the how to get this program to run that would be much appreciated.
It might be just a case of me trying to run before I can walk. I've only donwloaded python recently as a result of bitcoin and am meaning to learn how to program with it.

I you run Windows, the simplest way to get PyCrypto is voidspace: http://www.voidspace.org.uk/python/modules.shtml#pycrypto
I put that in the error message now.
- jneb

thanks, got it working. very nice. sent some bitcoins your way

for some reason even though I had a Crypto.hash module after I had installed it yesterday it didn't contain the SHA256 or RIPEMD160 functions. I tried to install it again from the link you gave. Same deal. Then I deleted all the pycrypto related files and re-installed it using the link you gave me and that   worked.

thanks again. have just played around a small bit. know will try to figure out how to use it and figure out the code.

[jneb]

for some reason even though I had a Crypto.hash module after I had installed it yesterday it didn't contain the SHA256 or RIPEMD160 functions.

Yes, there is a difference between Crypto.hash and the full PyCrypto.
Glad you have it working, and thanks for the Bitcoins!
BTW there is a new version (again), with two new commands:
listen: print a summary of the debug file produced by the BitCoin program
script: give the input/output script of a transaction

Hope you like it.

[jneb]

Thanks for the hint, BitLotto.

Check out version 3.7 of the program: listing transactions per address have an instant start now, once the index is made. (Edited: I realised it is not easy to find the first transaction for an address if all you have is the public key position in the file: a bit of redesign is needed, but there is a kludge for now: it is still fast and works in most cases.)

The command "update" will make an index for you: future versions will even make use of a partial index.

I am also working on a "random address function": please allow a few days for delivery.

Reminder: the program is at bitbucket.org/jneb/bitcoin, prerequistes: Python 2.7, PyCrypto and optionally colorama.

(I could use some help making it an executable, BTW.)

[bitlotto]

I'm not sure if anyone else would have a use for this but would it be hard to have a feature where you specify an exact date say : 02032011000000 (mmddyyyyhhmmss or similar) and have it list the first block to be past that date?

[Donald_Norman]

very cool. There's so many ways to expand in this direction

[jneb]

I'm not sure if anyone else would have a use for this but would it be hard to have a feature where you specify an exact date say : 02032011000000 (mmddyyyyhhmmss or similar) and have it list the first block to be past that date?

That would be quite easy. I put it on my todo list.