[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[elcoco]

As my disk is not so big and I don't want to download further block chain data, so I changed the loader of Abe to "blkfile' from 'rpc', and I expected that the database won't increase. However, it seems Abe reloaded the blkfile into the database from the scratch and the block_tx number continued to increase though the block data was not updated. It did re-index the data into database according to the Abe browser that the block number just several thousands.

What's the problem? How can I recover to the right database?

Thanks.

loading from rpc or blkfile, yields the same database information, i would delete the database and reload everything again, or add new space to your server and change the directory where mysql puts the database.

[1713568098]

1713568098

[John Tobey]

Abe is stuck on a block height... when I search for the block after, it gives me 3 choices and if I click on either of them... it crashes with this

Code:

<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'has_feature' 
      args = ("'NoneType' object has no attribute 'has_feature'",) 
      message = "'NoneType' object has no attribute 'has_feature'"

So I guess those are orphans and Abe doesn't know what to do with those (or witch one to chose)

My question would be, am I correct? how can this be fixed?

I don't really have enough information, but it looks as if some code is not specifying the chain where it should.  I think this is a recent bug.  I tried to fix some cases like this here but it is not well tested.  You might try inserting "chain/.../" into the links returned by the search.

In any case, orphan blocks should not get Abe "stuck" so there is something else wrong.

[John Tobey]

As my disk is not so big and I don't want to download further block chain data, so I changed the loader of Abe to "blkfile' from 'rpc', and I expected that the database won't increase. However, it seems Abe reloaded the blkfile into the database from the scratch and the block_tx number continued to increase though the block data was not updated. It did re-index the data into database according to the Abe browser that the block number just several thousands.

What's the problem? How can I recover to the right database?

Thanks.

loading from rpc or blkfile, yields the same database information, i would delete the database and reload everything again, or add new space to your server and change the directory where mysql puts the database.

Yes, the database is equally huge whichever loader you use.  I've started a new branch ("small") where I try to keep the database size down, but I don't know when it will be usable.  Meanwhile, people are saying good things about Insight: https://github.com/bitpay/insight

[elcoco]

Not sure why but anytime I try to use the newest version of the chain explorer with a proof of stake coin I get this when browsing any block


Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 250, in __call__
    handler(page)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 673, in handle_block
    abe._show_block(page, '', None, block_hash=block_hash)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 564, in _show_block
    is_stake_block = is_stake_chain and b['is_proof_of_stake']
KeyError: 'is_proof_of_stake'


I've tried faircoin and community coin, I use policy NovaCoin, it seems to load all the blocks but when browsing any block I get the above error, when accessing addresses and transaction everything works fine.

[esotericizm]

Hey I'm trying to set up Abe with an X11 coin.

In Chain.py I've made the following changes

Code:

class X11Chain(Chain):
    def block_header_hash(chain, header):
        import xcoin_hash
        return xcoin_hash.getPoWHash(header)

class Logicoin(X11Chain):
    def __init__(chain, **kwargs):
        chain.name = 'Logicoin'
        chain.code3 = 'LGC'
        chain.address_version = '\x30'
        chain.script_addr_vers = '\x05'
        chain.magic = '\xf6\xc6\xb6\xd6'
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = 'logicoin.conf'
    datadir_rpcport = 9652
    datadir_p2pport = 9653

and for abe.conf

Code:

default-loader = blkfile
dbtype MySQLdb
connect-args = {"user":"abe","db":"abe", "passwd":"xxx"}
upgrade
port 2750
host 184.82.172.92
datadir += [{
    "dirname": "/home/crypto/.logicoin",
    "loader": "blkfile",
    "chain": "Hirocoin",
    "policy": "X11"
}]

This will allow my to use the explorer but the currency is labeled "Hirocoin", while all transactions and blocks appear to be from Logicoin. If I change "loader": "Logicoin" then it shows 0 blocks and doesn't work at all. Not sure what I've missed?

Here's the explorer in its current form. http://184.82.172.92:2750/

[elcoco]

Hey I'm trying to set up Abe with an X11 coin.

In Chain.py I've made the following changes

Code:

class X11Chain(Chain):
    def block_header_hash(chain, header):
        import xcoin_hash
        return xcoin_hash.getPoWHash(header)

class Logicoin(X11Chain):
    def __init__(chain, **kwargs):
        chain.name = 'Logicoin'
        chain.code3 = 'LGC'
        chain.address_version = '\x30'
        chain.script_addr_vers = '\x05'
        chain.magic = '\xf6\xc6\xb6\xd6'
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = 'logicoin.conf'
    datadir_rpcport = 9652
    datadir_p2pport = 9653

and for abe.conf

Code:

default-loader = blkfile
dbtype MySQLdb
connect-args = {"user":"abe","db":"abe", "passwd":"xxx"}
upgrade
port 2750
host 184.82.172.92
datadir += [{
    "dirname": "/home/crypto/.logicoin",
    "loader": "blkfile",
    "chain": "Hirocoin",
    "policy": "X11"
}]

This will allow my to use the explorer but the currency is labeled "Hirocoin", while all transactions and blocks appear to be from Logicoin. If I change "loader": "Logicoin" then it shows 0 blocks and doesn't work at all. Not sure what I've missed?

Here's the explorer in its current form. http://184.82.172.92:2750/

change the chain to Logicoin in the abe.conf file like this  "chain": "Logicoin", then delete the database and recreate it.

[esotericizm]

Hey I'm trying to set up Abe with an X11 coin.

In Chain.py I've made the following changes

Code:

class X11Chain(Chain):
    def block_header_hash(chain, header):
        import xcoin_hash
        return xcoin_hash.getPoWHash(header)

class Logicoin(X11Chain):
    def __init__(chain, **kwargs):
        chain.name = 'Logicoin'
        chain.code3 = 'LGC'
        chain.address_version = '\x30'
        chain.script_addr_vers = '\x05'
        chain.magic = '\xf6\xc6\xb6\xd6'
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = 'logicoin.conf'
    datadir_rpcport = 9652
    datadir_p2pport = 9653

and for abe.conf

Code:

default-loader = blkfile
dbtype MySQLdb
connect-args = {"user":"abe","db":"abe", "passwd":"xxx"}
upgrade
port 2750
host 184.82.172.92
datadir += [{
    "dirname": "/home/crypto/.logicoin",
    "loader": "blkfile",
    "chain": "Hirocoin",
    "policy": "X11"
}]

This will allow my to use the explorer but the currency is labeled "Hirocoin", while all transactions and blocks appear to be from Logicoin. If I change "loader": "Logicoin" then it shows 0 blocks and doesn't work at all. Not sure what I've missed?

Here's the explorer in its current form. http://184.82.172.92:2750/

change the chain to Logicoin in the abe.conf file like this  "chain": "Logicoin", then delete the database and recreate it.

Tried that many times. Leads to the block explorer not displaying any blocks and I read earlier on that thats the behaviour when it doesn't recognize the chain.

I've made the change and recreated the database so that you can see what I'm talking about. http://184.82.172.92:2750/

[John Tobey]

This will allow my to use the explorer but the currency is labeled "Hirocoin", while all transactions and blocks appear to be from Logicoin. If I change "loader": "Logicoin" then it shows 0 blocks and doesn't work at all. Not sure what I've missed?

Leave "loader" as "blkfile" and "policy" as what it was when you could browse the blocks.  Add "chain":"Logicoin".  If you omit "policy" it defaults to the chain name, but Abe does not recognize "Logicoin" as a policy name.  (Ideally, we should tell Abe/Chain.py about Logicoin.)  If you specify "policy", then Abe uses the chain name for display and search only, so "chain" will not affect the block count or content.

[John Tobey]

Not sure why but anytime I try to use the newest version of the chain explorer with a proof of stake coin I get this when browsing any block


Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 250, in __call__
    handler(page)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 673, in handle_block
    abe._show_block(page, '', None, block_hash=block_hash)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 564, in _show_block
    is_stake_block = is_stake_chain and b['is_proof_of_stake']
KeyError: 'is_proof_of_stake'


I've tried faircoin and community coin, I use policy NovaCoin, it seems to load all the blocks but when browsing any block I get the above error, when accessing addresses and transaction everything works fine.

Sorry, I have been busy, but this looks fixable.  Did you have it working with an older version?

[elcoco]

Not sure why but anytime I try to use the newest version of the chain explorer with a proof of stake coin I get this when browsing any block


Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 250, in __call__
    handler(page)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 673, in handle_block
    abe._show_block(page, '', None, block_hash=block_hash)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 564, in _show_block
    is_stake_block = is_stake_chain and b['is_proof_of_stake']
KeyError: 'is_proof_of_stake'


I've tried faircoin and community coin, I use policy NovaCoin, it seems to load all the blocks but when browsing any block I get the above error, when accessing addresses and transaction everything works fine.

Sorry, I have been busy, but this looks fixable.  Did you have it working with an older version?

yes, it works on the previous version, I'm just gonna downgrade for now.
thanks.

[dasource]

Not sure why but anytime I try to use the newest version of the chain explorer with a proof of stake coin I get this when browsing any block


Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 250, in __call__
    handler(page)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 673, in handle_block
    abe._show_block(page, '', None, block_hash=block_hash)
  File "/home/*/Downloads/bitcoin-abe/Abe/abe.py", line 564, in _show_block
    is_stake_block = is_stake_chain and b['is_proof_of_stake']
KeyError: 'is_proof_of_stake'


I've tried faircoin and community coin, I use policy NovaCoin, it seems to load all the blocks but when browsing any block I get the above error, when accessing addresses and transaction everything works fine.

Sorry, I have been busy, but this looks fixable.  Did you have it working with an older version?

yes, it works on the previous version, I'm just gonna downgrade for now.
thanks.

Did that work? if so what did you downgrade to? Thanks

[bsom]

This might be a silly question, but how would I use Abe functions in my own Python programs.  The web interface is nice and all, but it is slow.  And the command line --query statements aren't exactly fast either.  I'm thinking if I could call a function directly from within my own Python code, instead of parsing web API output, what I'm writing could be so much more efficient and speedy.

For example, if I wanted to get the balance of an address, it would be nice to be able to directly call the function in Abe that calculates the balance.

Does that make sense?

[John Tobey]

This might be a silly question, but how would I use Abe functions in my own Python programs.  The web interface is nice and all, but it is slow.  And the command line --query statements aren't exactly fast either.  I'm thinking if I could call a function directly from within my own Python code, instead of parsing web API output, what I'm writing could be so much more efficient and speedy.

For example, if I wanted to get the balance of an address, it would be nice to be able to directly call the function in Abe that calculates the balance.

Does that make sense?

Yes, check out these programs in the source tree for example code, possibly out of date:

Abe/admin.py
Abe/reconfigure.py
Abe/verify.py
Abe/mixup.py
tools/namecoin_dump.py
contrib/ecdsa.py

[unick]

Hey John,

I just wanted to make sure that this part of the code

Code:

AND cc.in_longest = 1

in this SQL query

Code:

SELECT b.block_hash, b.block_height, b.block_nTime, b.block_num_tx,
                   b.block_nBits, b.block_value_out,
                   b.block_total_seconds, b.block_satoshi_seconds,
                   b.block_total_satoshis, b.block_ss_destroyed,
                   b.block_total_ss
              FROM block b
              JOIN chain_candidate cc ON (b.block_id = cc.block_id)
             WHERE cc.chain_id = 1
               AND cc.block_height BETWEEN 100 AND 200
               AND cc.in_longest = 1
             ORDER BY cc.block_height DESC LIMIT 50

makes sure that the query returns only valid blocks. leaving orphans out.

inversely, if I would be writing

Code:

AND cc.in_longest = 0

I would have be getting only orphans?  is that correct?

[John Tobey]

Hey John,

I just wanted to make sure that this part of the code

Code:

AND cc.in_longest = 1

in this SQL query

Code:

SELECT b.block_hash, b.block_height, b.block_nTime, b.block_num_tx,
                   b.block_nBits, b.block_value_out,
                   b.block_total_seconds, b.block_satoshi_seconds,
                   b.block_total_satoshis, b.block_ss_destroyed,
                   b.block_total_ss
              FROM block b
              JOIN chain_candidate cc ON (b.block_id = cc.block_id)
             WHERE cc.chain_id = 1
               AND cc.block_height BETWEEN 100 AND 200
               AND cc.in_longest = 1
             ORDER BY cc.block_height DESC LIMIT 50

makes sure that the query returns only valid blocks. leaving orphans out.

inversely, if I would be writing

Code:

AND cc.in_longest = 0

I would have be getting only orphans?  is that correct?

Yes.  Abe updates in_longest during reorgs, and 1 means in the current best chain.

[dasource]

Anybody else noticed that when you use "policy" "NovaCoin" or a clone of it that the database stores really random data? Yet the explorer displays it perfectly?

Take a look at this :
http://explorer.information-coin.com/block/5bf9c6ba1f6e893a50141040601bf0fbeddffab6dd03f125c3434c41eed9e7d3

Code:

mysql> select * from block where block_id=5634;
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
| block_id | block_hash                       | block_version | block_hashMerkleRoot             | block_nTime | block_nBits | block_nNonce | block_height | prev_block_id | search_block_id | block_chain_work                       | block_value_in | block_value_out | block_total_satoshis | block_total_seconds | block_satoshi_seconds | block_total_ss       | block_num_tx | block_ss_destroyed |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
|     5634 | =ô÷!€Åˆ·€�aí3ϲ
                            ApÀ½<SÇ­0"œ              |             4 | É-�ãiª¦˜°ý;Imä]Ó€¦÷扂5Tôò,Aý                   |  1398428970 |   503599037 |            0 |         5368 |          5627 |         ¾-™[   |    12512800000 |     12516750410 |       70151186978517 |             1069334 |   9101769833206534328 | 74298551044584278866 |            2 |   1245836932000000 |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
1 row in set (0.00 sec)

I've been able to reproduce this with half a dozen different coins, yet when I change it from NovaCoin to X11 or Sha256 the data in the DB shows as normal....

[kwolniak]

Hi!

First, thanks for great tool. Nice job!

My question:

I have working site after scanning all block (loaded to MySql database). How can I get list of 100 richest addresses? It is possible?

[John Tobey]

Anybody else noticed that when you use "policy" "NovaCoin" or a clone of it that the database stores really random data? Yet the explorer displays it perfectly?

Take a look at this :
http://explorer.information-coin.com/block/5bf9c6ba1f6e893a50141040601bf0fbeddffab6dd03f125c3434c41eed9e7d3

Code:

mysql> select * from block where block_id=5634;
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
| block_id | block_hash                       | block_version | block_hashMerkleRoot             | block_nTime | block_nBits | block_nNonce | block_height | prev_block_id | search_block_id | block_chain_work                       | block_value_in | block_value_out | block_total_satoshis | block_total_seconds | block_satoshi_seconds | block_total_ss       | block_num_tx | block_ss_destroyed |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
|     5634 | =ô÷!€Åˆ·€�aí3ϲ
                            ApÀ½<SÇ­0"œ              |             4 | É-�ãiª¦˜°ý;Imä]Ó€¦÷扂5Tôò,Aý                   |  1398428970 |   503599037 |            0 |         5368 |          5627 |         ¾-™[   |    12512800000 |     12516750410 |       70151186978517 |             1069334 |   9101769833206534328 | 74298551044584278866 |            2 |   1245836932000000 |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
1 row in set (0.00 sec)

I've been able to reproduce this with half a dozen different coins, yet when I change it from NovaCoin to X11 or Sha256 the data in the DB shows as normal....

Are you sure it depends on policy?  Recently I changed Abe to use binary data types by default.  To get the old behavior, configure binary-type=hex.

[dasource]

Anybody else noticed that when you use "policy" "NovaCoin" or a clone of it that the database stores really random data? Yet the explorer displays it perfectly?

Take a look at this :
http://explorer.information-coin.com/block/5bf9c6ba1f6e893a50141040601bf0fbeddffab6dd03f125c3434c41eed9e7d3

Code:

mysql> select * from block where block_id=5634;
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
| block_id | block_hash                       | block_version | block_hashMerkleRoot             | block_nTime | block_nBits | block_nNonce | block_height | prev_block_id | search_block_id | block_chain_work                       | block_value_in | block_value_out | block_total_satoshis | block_total_seconds | block_satoshi_seconds | block_total_ss       | block_num_tx | block_ss_destroyed |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
|     5634 | =ô÷!€Åˆ·€�aí3ϲ
                            ApÀ½<SÇ­0"œ              |             4 | É-�ãiª¦˜°ý;Imä]Ó€¦÷扂5Tôò,Aý                   |  1398428970 |   503599037 |            0 |         5368 |          5627 |         ¾-™[   |    12512800000 |     12516750410 |       70151186978517 |             1069334 |   9101769833206534328 | 74298551044584278866 |            2 |   1245836932000000 |
+----------+----------------------------------+---------------+----------------------------------+-------------+-------------+--------------+--------------+---------------+-----------------+----------------------------------------+----------------+-----------------+----------------------+---------------------+-----------------------+----------------------+--------------+--------------------+
1 row in set (0.00 sec)

I've been able to reproduce this with half a dozen different coins, yet when I change it from NovaCoin to X11 or Sha256 the data in the DB shows as normal....

Are you sure it depends on policy?  Recently I changed Abe to use binary data types by default.  To get the old behavior, configure binary-type=hex.

Hmm, I was sure it was; I will double check.
I have set binary-type=hex in abe.conf but the DB still shows the same; Have tried on two different setups. Deleted the Database and re-created it.

EDIT binary-type=hex (and binary) works fine on a old version of Abe but not on the latest version; always seems to store in binary.

EDIT Was able to fix this by editing SqlAbstraction.py and setting 'val = sql.config.get('binary_type')' to 'val = "hex"'. binary_type is not set in the DB on latest release.

[unick]

I'm trying to set a new block explorer... Abe reads the blockchain fine and loaded the database without a itch.

I get an error when I try to access the block details page.  Any other page loads fine. I use FastCGI

here's the error (last call)

Code:

is_stake_block undefined, is_stake_chain = True, b = {'chain_candidates': [{'chain': <Abe.Chain.AsiaCoin object>, 'in_longest': Decimal('1')}], 'chain_satoshi_seconds': 2517292340726300000000L, 'chain_satoshis': 3342705000000000, 'chain_work': 13618916087790469, 'fees': 57000, 'generated': 5000000000, 'hash': '00000000001930548377e3e15c927d8d79278cb8ba53b1d015468609e3085b13', 'hashMerkleRoot': '36f8065b8c6c95b37c426fbf95bb0bcf98dc04817de8257ffd624d8311cb2603', 'hashPrev': '000000000141a8721757560bb837c6d7be9aa85cf5333bfeaff2602c9d9a1d7a', 'height': Decimal('12441'), ...}
<type 'exceptions.KeyError'>: 'is_proof_of_stake' 
      args = ('is_proof_of_stake',) 
      message = 'is_proof_of_stake'

Never saw this message before, what does it mean?

Thanks

EDIT: from what I understand, Abe test for the block to see if it's proof-of-stake...

EDIT: I used an older version of abe.py and the error went away. Still any clues on why I get this error would be appreciated