[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[John Tobey]

This is a Newbies-friendly copy of: https://bitcointalk.org/index.php?topic=22785.0

!!! Announcing Abe, the open-source block chain browser !!!

While theymos' Bitcoin Block Explorer has served us well, there are benefits that can only be had by an open-source version of this critical Bitcoin infrastructure.

• Privacy: you can view information without revealing your interest in it.
• Extensibility: you can add features that you want.
• Trust: you can audit the source code.
• Reliability: you can run it on your own server free of charge.

Abe provides much of Block Explorer's interface, including the list of recent blocks, a search tool, and detailed block, transaction, and address history pages.  Abe can also present multiple currencies such as Bitcoin, Namecoin, Tenebrix, and whatever FooCoin tomorrow may bring.  Abe lets you page back and forth in the block list and presents some new statistics like Average Coin Age.

Abe's interactive performance approaches Block Explorer's, though it is untested under BBE loads.  Abe supports a subset of Block Explorer's API.

Source code on Github: https://github.com/bitcoin-abe/bitcoin-abe
Demonstration site: http://abe.john-edwin-tobey.org/ (a 512MB VPS, can't handle heavy loads)
Requirements: Python and an RDBMS; tested with PostgreSQL, SQLite, MySQL, Oracle, ODBC, and DB2.
Donations: 1PWC7PNHL1SgvZaN7xEtygenKjWobWsCuf (BTC) and NJ3MSELK1cWnqUa6xhF2wUYAnz3RSrWXcK (NMC).

Enjoy!

December 6, 2012: Version 0.7.2 released.
August 31, 2011: Version 0.6 released.
August 16, 2011: Version 0.5 released.
July 15, 2011: See changes since July 4.

[1715184970]

1715184970

[1715184970]

1715184970

[1715184970]

1715184970

[1715184970]

1715184970

[1715184970]

1715184970

[akaihola]

Does the "trim spent outputs" feature in the no-statistics branch mean that you can't see the total amount of BTC received for each account?

[John Tobey]

Does the "trim spent outputs" feature in the no-statistics branch mean that you can't see the total amount of BTC received for each account?

Yes, that information is lost.  All you see is total received minus total sent.  It would be possible to store the total received by each address, but it would only be a lower bound on the real value, since addresses themselves are trimmed when empty.

Any particular use case in mind?

[akaihola]

Does the "trim spent outputs" feature in the no-statistics branch mean that you can't see the total amount of BTC received for each account?

Any particular use case in mind?

Tracking the total amount received for a bunch of addresses.

[ttsda]

I loaded the blockchain to an Abe sqlite DB, but now I want to use Postgres.
Is there a way to convert the sqlite DB to pg, or do I have to rescan the whole blockchain again?

[John Tobey]

I loaded the blockchain to an Abe sqlite DB, but now I want to use Postgres.
Is there a way to convert the sqlite DB to pg, or do I have to rescan the whole blockchain again?

This would be a nice feature but is not supported currently.  If you have good SQL skills and a few hours to spare, I can outline the steps.  But if you can afford to wait for the reload, that will be much easier.

[ttsda]

If you have good SQL skills and a few hours to spare, I can outline the steps.  But if you can afford to wait for the reload, that will be much easier.

Please do! I think I can do it if you give me a few instructions

[John Tobey]

If you have good SQL skills and a few hours to spare, I can outline the steps.  But if you can afford to wait for the reload, that will be much easier.

Please do! I think I can do it if you give me a few instructions

Answered via PM.  If anything useful results from this exercise, one of us will post it.

[mullick]

Any chance this would be easy enough to adapt for a newbie with a tutorial? I have only ever done a hello world and simple calculator in java but get enough to follow simple directions enough. I could offer someone a few Ltc to make a tutorial. I may be way out of my league he but It's worth a shot

[pointychimp]

I keep getting a "RuntimeError: maximum recursion depth exceeded" error. It happens every time I start abe up, and every time I access a page the error shows up in the console. I start abe with the following command:

python -m Abe.abe --config abe-my.conf

and the (censored) contents of abe-my.conf are as follows:

dbtype MySQLdb
connect-args {"user":"*****","db":"*****","passwd":"*****"}
upgrade
host *****
port 2750
datadir = /****/****/.bitcoin


What more info do you need, if any? Is there something in python I have to change?

[John Tobey]

I keep getting a "RuntimeError: maximum recursion depth exceeded" error. It happens every time I start abe up, and every time I access a page the error shows up in the console.

This has been on the to-do list for a long time.  I'll give it another look, but as I recall, it would take the better part of a day to code and test.

First, run Abe once with --rescan in case that solves it.  If you know Python, you can try sys.setrecursionlimit(300000) as suggested here.  (Replace 300000 with a few more than the number of blocks in your chain, or try lower numbers until either success or another kind of crash.)

I think this happens when the block files are "very" out of sequence: when a sequence of 1000 or more blocks all occur before one of their common ancestors.  Could this be the case with your block files?  Maybe this is normal for the latest Bitcoin versions, which I have not tested.  Is there anything unusual about how your block files were created, or did you download the chain again after running Abe the first time?

[bongwater]

Open source block explorer? can you explain what this is?

[macintosh]

lovin this idea, goodluck

[John Tobey]

Open source block explorer? can you explain what this is?

https://en.bitcoin.it/wiki/Block_chain_browser

[pointychimp]

I keep getting a "RuntimeError: maximum recursion depth exceeded" error. It happens every time I start abe up, and every time I access a page the error shows up in the console.

This has been on the to-do list for a long time.  I'll give it another look, but as I recall, it would take the better part of a day to code and test.

First, run Abe once with --rescan in case that solves it.  If you know Python, you can try sys.setrecursionlimit(300000) as suggested here.  (Replace 300000 with a few more than the number of blocks in your chain, or try lower numbers until either success or another kind of crash.)

I think this happens when the block files are "very" out of sequence: when a sequence of 1000 or more blocks all occur before one of their common ancestors.  Could this be the case with your block files?  Maybe this is normal for the latest Bitcoin versions, which I have not tested.  Is there anything unusual about how your block files were created, or did you download the chain again after running Abe the first time?

I don't think I did anything unusual. I think I did copy block files from one of my computers to my server in order to speed up the chain download time. Problem is I don't remember if I did that before running abe for the first time or after. I'm currently letting abe rescan. If this doesn't work, I might just bite the bullet, remove everything from the mySQL database, delete my .bitcoin/ directory, download the blockchain the conventional way, and try again.

Thanks for the help. I'll try to remember to report back if I ever get this solved!

[macintosh]

thank you for this!

[coinpipe]

Thanks for ABE. I've been trying it out with a few different alt-coins. I could do with a hand understanding address_version.

For most coins it seems to be obvious, just the unicode of the first character of an address. But some coins have addresses like this (feathercoin in this example)

6nh62XDjhbqjjit78jfStECNcHz89xbNeq
72HzQ77Xbpg67MadosyWDjd5YZsVHz4Npt

etc...

I've tried settings an address_version of "\u0036" (a unicode '6') and this seems to work fine, however I'm worried that is not right. Is \u0036 ok, or should I do something else?

[John Tobey]

Thanks for ABE. I've been trying it out with a few different alt-coins. I could do with a hand understanding address_version.

For most coins it seems to be obvious, just the unicode of the first character of an address. But some coins have addresses like this (feathercoin in this example)

6nh62XDjhbqjjit78jfStECNcHz89xbNeq
72HzQ77Xbpg67MadosyWDjd5YZsVHz4Npt

etc...

Plugging it into decode_address, for example:

http://explorer.doubloons.net/chain/Doubloons/q/decode_address/6nh62XDjhbqjjit78jfStECNcHz89xbNeq

gives "0e" as the hex version byte for both those addresses, so I would think it is "\u000e".

[coinpipe]

Ah got it! Thanks a bunch.

[pointychimp]

Hey I've been having more luck since starting fresh with abe and the blockchain. I'm trying to get abe to use RPC but it always says either "catch_up_rpc: abort" or "Exception: RPC load failed." The bitcoin client is the first time I've ever seen RPC so I am very unfamiliar with it. It does seem, however to be a better way for abe to update the database instead of reading the blockfiles over and over again.

So my question is this. How do I make it work? I have a ~/.bitcoin/bitcoin.conf which contains the rpc username and password, and told it to listen for rpc on port 8332 (plus other settings). That port is correctly forwarded in my router. In abe.conf I have (among other things)

datadir += [{
        "dirname": "/home/<myname>/.bitcoin",
        "loader": "rpc",    # See the comments for default-loader below.
        "chain": "Bitcoin"
      }]


Is there any more information you need? Is there someway I can tell abe to connect to bitcoind with RPC over a specific port?

Thanks for your help.

Matt