[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[Jouke]

Cool. If it would be possible to connect to multiple nodes, I guess it wouldn't be that hard to build some rudimentary double spent check. If only I had the time

[1714245308]

1714245308

[1714245308]

1714245308

[1714245308]

1714245308

[1714245308]

1714245308

[1714245308]

1714245308

[John Tobey]

The latest commit supports a very crude SVG rendering of the logarithmic hash rate chart.  I do not expect to have time to add basic stuff like grid lines, axis values, and difficulty for a while, but I do hope to see the basic shape of various alt chains someday.

Trying to look like Sipa's famous chart: http://bitcoin.sipa.be/speed-ever.png

Snapshot taken just now: http://john-edwin-tobey.org/images/nethash.png

Live location: http://abe.john-edwin-tobey.org/chain/Bitcoin/q/nethash?format=svg

The implementation uses the nethash API code and should not burden the server any more than /q/nethash.  There is also JSONP support via /q/nethash?jsonp=<function>.

[ttsda]

Hi.
I am getting this error:

Code:

[2013-05-14 10:42:53] >> Opened /opt/litecoin/.litecoin/blk0001.dat
[2013-05-14 10:42:53] >> Exception at 1034257218
[2013-05-14 10:42:53] >> Failed to catch up {'blkfile_offset': 1034252127, 'blkfile_number': 1, 'chain_id': 8, 'loader': None, 'dirname': '/opt/litecoin/.litecoin', 'id': Decimal('1')}
[2013-05-14 10:42:53] >> Traceback (most recent call last):
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 2549, in catch_up
[2013-05-14 10:42:53] >>     store.catch_up_dir(dircfg)
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 2809, in catch_up_dir
[2013-05-14 10:42:53] >>     store.import_blkdat(dircfg, ds, blkfile['name'])
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 2917, in import_blkdat
[2013-05-14 10:42:53] >>     store.import_block(b, chain_ids = chain_ids)
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 1692, in import_block
[2013-05-14 10:42:53] >>     tx['tx_id'] = store.import_tx(tx, pos == 0)
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 2146, in import_tx
[2013-05-14 10:42:53] >>     txin['prevout_hash'], txin['prevout_n'])
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 2470, in lookup_txout
[2013-05-14 10:42:53] >>     (store.hashin(tx_hash), txout_pos))
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 605, in selectrow
[2013-05-14 10:42:53] >>     store.sql(stmt, params)
[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 481, in sql
[2013-05-14 10:42:53] >>     store.cursor.execute(cached, params)
[2013-05-14 10:42:53] >> InternalError: invalid page header in block 61022 of relation base/17556/17801

Do you know what's happening? Did my DB get corrupted?

[John Tobey]

Code:

[2013-05-14 10:42:53] >>   File "Abe/DataStore.py", line 481, in sql
[2013-05-14 10:42:53] >>     store.cursor.execute(cached, params)
[2013-05-14 10:42:53] >> InternalError: invalid page header in block 61022 of relation base/17556/17801

Do you know what's happening? Did my DB get corrupted?

Offhand, it appears to be internal to the database system.  I don't think "block 61022" refers to the blockchain.  I'd try to dump and reload the database, and if that doesn't do it, search the web for "invalid page header in block".

[toolbag]

Hi! I'm trying to get Abe working through RPC instead of reading my blockchain files directly and I'm having problems. I believe I've met all the requirements listed at the end of abe.conf regarding RPC, but when I try to start Abe, I get this:

Code:

$ python -m Abe.abe --config abe.conf
no chain_id
Failed to catch up {'blkfile_offset': 77633101, 'blkfile_number': 100060, 'chain_id': None, 'loader': None, 'dirname': u'/home/user/.bitcoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2553, in catch_up
    raise Exception("RPC load failed")
Exception: RPC load failed
Abe initialized.
Listening on http://192.168.1.2:2750

I see almost no references in the documentation to "chain_id" so I'm really not sure what to do. Google has been no help. I tried renaming abe.sqlite and hoping it would just rebuild the entire database from scratch using RPC and in that case I get this:

Code:

$ python -m Abe.abe --config abe.conf
ddl_implicit_commit=true
create_table_epilogue=''
max_varchar=4294967295
clob_type=CLOB
binary_type=buffer
int_type=str
Created silly table abe_dual
sequence_type=update
limit_style=native
commit
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1, 'loader': u'rpc', 'dirname': u'/home/user/.bitcoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2551, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2716, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2254, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 319, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http://192.168.1.2:2750

Any ideas?

[John Tobey]

Hi! I'm trying to get Abe working through RPC instead of reading my blockchain files directly and I'm having problems. I believe I've met all the requirements listed at the end of abe.conf regarding RPC, but when I try to start Abe, I get this:

Code:

$ python -m Abe.abe --config abe.conf
no chain_id

Hi!  Thanks for trying, and my apologies for the rough edges.  If using an already loaded database, issue the following SQL before restarting:

Code:

$ sqlite3 abe.sqlite
sqlite> UPDATE datadir SET chain_id = 1 WHERE dirname='/home/user/.bitcoin';
sqlite> .quit

I see almost no references in the documentation to "chain_id" so I'm really not sure what to do. Google has been no help. I tried renaming abe.sqlite and hoping it would just rebuild the entire database from scratch using RPC and in that case I get this:

Code:

$ python -m Abe.abe --config abe.conf
ddl_implicit_commit=true
create_table_epilogue=''
max_varchar=4294967295
clob_type=CLOB
binary_type=buffer
int_type=str
Created silly table abe_dual
sequence_type=update
limit_style=native
commit
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1, 'loader': u'rpc', 'dirname': u'/home/user/.bitcoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2551, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2716, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2254, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 319, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http://192.168.1.2:2750

Any ideas?

Thanks for this, it is a bug that I may be able to fix.  But try the update statement.

[toolbag]

Manually setting the chain_id with sqlite3 seems to have worked! Thanks!

[pizzaman1337]

Is there a way to get a list of transactions to/from an address in JSON format? If not, that would be an awesome feature to have.

blockchain.info has an API like this: https://blockchain.info/address/1dice7fUkz5h4z2wPc1wLMPWgB5mDwKDx?format=json

[vongesell]

Getting this error on first import

Code:

> python -m Abe.abe --upgrade --dbtype=sqlite3  --connect-args=abe.sqlite --port 2750 --datadir=../
no chain_id
catch_up_rpc: abort
Opened ../blocks/blk00000.dat
...
...
block_tx 2260 2290
commit
Exception at 521845
Failed to catch up {'blkfile_offset': 521621, 'blkfile_number': 100000, 'chain_id': None, 'loader': None, 'dirname': '../', 'id': 3}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2596, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2850, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 2958, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1811, in import_block
    len(b['transactions']), b['search_block_id']))
  File "Abe/DataStore.py", line 507, in sql
    store._execute(cached, params)
  File "Abe/DataStore.py", line 484, in _execute
    store.cursor.execute(stmt, params)
OverflowError: long too big to convert

.... thats what she said

[vongesell]

Getting this error on first import

Code:

> python -m Abe.abe --upgrade --dbtype=sqlite3  --connect-args=abe.sqlite --port 2750 --datadir=../
no chain_id
...
OverflowError: long too big to convert

As suggested at https://github.com/jtobey/bitcoin-abe/issues/16 RTFM. the README-SQLITE suggests making int's strings and this fixes it:

Code:

python -m Abe.abe --upgrade --dbtype=sqlite3  --connect-args=abe.sqlite --port 2750 --datadir=../ --int-type=str

[K1773R]

can i have a Abe with sha256d chains and scrypt chains? also where is the docu for scrypt based chains?

[toolbag]

Is there any way for Abe to not update the database with the latest transactions? Running in FastCGI mode, most requests take a long time to return a result because Abe is reading the most recent transactions and inserting them into the database. I know I can start in standalone mode, but then what's the point of FastCGI mode at all? Ideally what I'd like to do is have a process that runs in the background loading new transactions, sleeping for a few seconds or minutes, and loading again, forever. And the FastCGI side of things would just be a user interface to the back-end database, basically. Is that possible? Am I missing something obvious?

[John Tobey]

can i have a Abe with sha256d chains and scrypt chains? also where is the docu for scrypt based chains?

Did you try it?  http://explorer.litecoin.net/ appears Abe based, and doesn't LTC use scrypt?  I'd suggest asking them if they had to change anything.  They should give you the explorer source.  (If not, I'd like to know...)

Abe does not (yet) contain any special code for scrypt.  That would be nice to add.

[John Tobey]

Is there any way for Abe to not update the database with the latest transactions? Running in FastCGI mode, most requests take a long time to return a result because Abe is reading the most recent transactions and inserting them into the database. I know I can start in standalone mode, but then what's the point of FastCGI mode at all? Ideally what I'd like to do is have a process that runs in the background loading new transactions, sleeping for a few seconds or minutes, and loading again, forever. And the FastCGI side of things would just be a user interface to the back-end database, basically. Is that possible? Am I missing something obvious?

You are on the right track.  I ought to be typing this into a new "readme-server.txt" file.  Try "datadir=[]" in your FastCGI config to disable loading, and "no-serve" in the loader config to disable serving.

[toolbag]

Thanks, that worked perfectly! Now the only weirdness is that this keeps showing up in my Apache error log every time I search by Bitcoin address (address redacted in example below):

Code:

/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py:484: Warning: Truncated incorrect INTEGER value: '1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
  store.cursor.execute(stmt, params)

Everything still works just fine, though. Thanks again!

[K1773R]

can i have a Abe with sha256d chains and scrypt chains? also where is the docu for scrypt based chains?

Did you try it?  http://explorer.litecoin.net/ appears Abe based, and doesn't LTC use scrypt?  I'd suggest asking them if they had to change anything.  They should give you the explorer source.  (If not, I'd like to know...)

Abe does not (yet) contain any special code for scrypt.  That would be nice to add.

now i did, works out of the box

[diatonic]

If I click the >> link to browse to the beginning of the block chain, python barfs a big purple "division by zero" screen. Happened on a few of the abe installs I looked at. I'm testing mine for Doubloons at http://explorer.doubloons.net/chain/Doubloons.

Any help to identify the problem would be appreciated.

[John Tobey]

If I click the >> link to browse to the beginning of the block chain, python barfs a big purple "division by zero" screen. Happened on a few of the abe installs I looked at. I'm testing mine for Doubloons at http://explorer.doubloons.net/chain/Doubloons.

Any help to identify the problem would be appreciated.

Interesting.  Please update and see if this fixes it.  The genesis block has zero output value, first time I've seen that.  No premining there!

[qdoop]

I don't know were is the correct place to announce this

Testnet3 Blockexplorer
http://pool.qdoop.net:18331/chain/Testnet3

Thanks Tobey

PS.
Most of the people need something really simple and intuitive for browsing the block chain
Any thoughts of a really light minimalistic interface?

[John Tobey]

PS.
Most of the people need something really simple and intuitive for browsing the block chain
Any thoughts of a really light minimalistic interface?

Well, it would be nice to separate Abe more into model/view/controller.  Maybe JSON versions of the UI pages, and implement the UI in JavaScript?

Thanks for the Testnet3 explorer!  I don't know who uses Testnet3 or how to tell them, but I'm glad to know about it.