[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[unick]

I'm trying to set a new block explorer... Abe reads the blockchain fine and loaded the database without a itch.

I get an error when I try to access the block details page.  Any other page loads fine. I use FastCGI

here's the error (last call)

Code:

is_stake_block undefined, is_stake_chain = True, b = {'chain_candidates': [{'chain': <Abe.Chain.AsiaCoin object>, 'in_longest': Decimal('1')}], 'chain_satoshi_seconds': 2517292340726300000000L, 'chain_satoshis': 3342705000000000, 'chain_work': 13618916087790469, 'fees': 57000, 'generated': 5000000000, 'hash': '00000000001930548377e3e15c927d8d79278cb8ba53b1d015468609e3085b13', 'hashMerkleRoot': '36f8065b8c6c95b37c426fbf95bb0bcf98dc04817de8257ffd624d8311cb2603', 'hashPrev': '000000000141a8721757560bb837c6d7be9aa85cf5333bfeaff2602c9d9a1d7a', 'height': Decimal('12441'), ...}
<type 'exceptions.KeyError'>: 'is_proof_of_stake' 
      args = ('is_proof_of_stake',) 
      message = 'is_proof_of_stake'

Never saw this message before, what does it mean?

Thanks

EDIT: from what I understand, Abe test for the block to see if it's proof-of-stake...

EDIT: I used an older version of abe.py and the error went away. Still any clues on why I get this error would be appreciated

I made a VeriCoin block explorer (http://blocks.vericoin.info) which is a NovaCoin deriv. and am getting the same error as you. Could you pastebin your abe.py file? Thanks!

I see you got it fixed... did you find the bug or just used an older abe.py file?

[unick]

Hello,

I have three issue's with ABE.
And I will Pay 0.25 btc for solve each.

First:

Code:

Opened /home/adbeloader/.feathercoin/blocks/blk00000.dat
Exception at 8
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 100000, 'chain_id': 22, 'loader': None, 'conf': u'feathercoin.conf', 'dirname': u'/home/adbeloader/.feathercoin', 'id': 72L}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2507, in catch_up
store.catch_up_dir(dircfg)
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2773, in catch_up_dir
store.import_blkdat(dircfg, ds, blkfile['name'])
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2894, in import_blkdat
if not store.offer_existing_block(hash, chain.id):
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2389, in offer_existing_block
store.offer_block_to_chains(b, frozenset([chain_id]))
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2271, in offer_block_to_chains
b['top'] = store.adopt_orphans(b, 0, chain_ids, chain_ids)
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1291, in adopt_orphans
stack.pop()()
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1296, in doit
store._adopt_orphans_1(stack)
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1407, in _adopt_orphans_1
store._populate_block_txin(int(next_id))
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1240, in _populate_block_txin
(block_id, txin_id, oblock_id))
File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 249, in sql
store._sql.sql(stmt, params)
File "/usr/local/lib/python2.7/dist-packages/Abe/SqlAbstraction.py", line 458, in sql
sql._execute(cached, params)
File "/usr/local/lib/python2.7/dist-packages/Abe/SqlAbstraction.py", line 439, in _execute
sql.cursor().execute(stmt, params)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: (1062, "Duplicate entry '1617197-10493495' for key 'PRIMARY'")

My config:

datadir += [{
"dirname": "/home/adbeloader/.feathercoin",
"chain": "FeatherCoin",
"code3": "FTC",
"loader" : "blkfile",
"conf": "feathercoin.conf" }] 

There is no context here so it's hard to pinpoint, but it looks like a corrupted database. try to load abe with the --rescan switch. if it doesn't work remove the database and load again

Second:

Code:

Exception at 2605896434586747018
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 29, 'loader': None, 'conf': u'blackcoin.conf', 'dirname': u'/home/adbeloader/.blackcoin', 'id': 114L}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2507, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2773, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2895, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 93, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 86, in ds_parse_transaction
    return deserialize.parse_Transaction(ds)
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 90, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 46, in parse_TxIn
    d['sequence'] = vds.read_uint32()
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 71, in read_uint32
    def read_uint32 (self): return self._read_num('<I')
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 4 bytes

Config
#17 ## BlackCoin
datadir += [{
        "dirname": "/home/adbeloader/.blackcoin",
        "chain": "Blackcoin",
        "conf": "blackcoin.conf" }] 

Abe can't deserialize the block chain properly, could be that you have the wrong magic bytes

Third:

Code:

Opened /home/adbeloader/.hawaiicoin/blocks/blk00000.dat
incomplete block of length 1016530668 chain 16

#17 ## BlackCoin
datadir += [{
        "dirname": "/home/adbeloader/.hawaiicoin",
        "chain": "Hawaiicoin",
        "conf": "hawaiicoin.conf" }] 

never encountered this error before, could it be a corrupted blk0001.dat file?  try reloading your coin daemon with the -rescan switch

Thk & B'rgds
Coinwall

[unick]

I have a question, The new Abe database schema added a bunch of binary columns to tx, block and key hashes.  What kind of performance increase does that give us?

[BitCoinDream]

Source code on Github: https://github.com/bitcoin-abe/bitcoin-abe
Demonstration site: http://abe.john-edwin-tobey.org/

I can see the Demonstration site is down. Does the Source code on Github still work ? Can I run it on a semi-dedicated shared hosting where I get 30 GB disk space, 350 GB BW per month and can have 250 000 MySQL queries per hour ?

[coinwallet]

Public demo -> http://bitcoin-abe.info

Thk & B'rgds
Coinwall

[BitCoinDream]

Public demo -> http://bitcoin-abe.info

Thk & B'rgds
Coinwall

Whatever address I'm providing to check balance, it is saying "Address not seen on the network.", though blockchain.info is perfectly showing its balance.

[coinwallet]

Whatever address I'm providing to check balance, it is saying "Address not seen on the network.", though blockchain.info is perfectly showing its balance.

Now in demo are block to 2013-09-06 13:40:54 .

If you want try http://bitcoin-abe.info/address/1PWC7PNHL1SgvZaN7xEtygenKjWobWsCuf it is Bitcoin-Abe donate address.
Soon in demo will be all blocks from bitcoin, litecoin and namecoin.

Thk & B'rgds
Coinwall

[OPEC Coin Dev]

hey! OPEC Coin needs Abe block explorer to be more transparent!

We are announcing bounty of 20,000 OPC for setting up Abe block explorer (we can provide you with VPS if needed)

PM me if interested!

Thanks!

[pnosker]

I'm trying to set a new block explorer... Abe reads the blockchain fine and loaded the database without a itch.

I get an error when I try to access the block details page.  Any other page loads fine. I use FastCGI

here's the error (last call)

Code:

is_stake_block undefined, is_stake_chain = True, b = {'chain_candidates': [{'chain': <Abe.Chain.AsiaCoin object>, 'in_longest': Decimal('1')}], 'chain_satoshi_seconds': 2517292340726300000000L, 'chain_satoshis': 3342705000000000, 'chain_work': 13618916087790469, 'fees': 57000, 'generated': 5000000000, 'hash': '00000000001930548377e3e15c927d8d79278cb8ba53b1d015468609e3085b13', 'hashMerkleRoot': '36f8065b8c6c95b37c426fbf95bb0bcf98dc04817de8257ffd624d8311cb2603', 'hashPrev': '000000000141a8721757560bb837c6d7be9aa85cf5333bfeaff2602c9d9a1d7a', 'height': Decimal('12441'), ...}
<type 'exceptions.KeyError'>: 'is_proof_of_stake' 
      args = ('is_proof_of_stake',) 
      message = 'is_proof_of_stake'

Never saw this message before, what does it mean?

Thanks

EDIT: from what I understand, Abe test for the block to see if it's proof-of-stake...

EDIT: I used an older version of abe.py and the error went away. Still any clues on why I get this error would be appreciated

I made a VeriCoin block explorer (http://blocks.vericoin.info) which is a NovaCoin deriv. and am getting the same error as you. Could you pastebin your abe.py file? Thanks!

I see you got it fixed... did you find the bug or just used an older abe.py file?

I ended up removing some code that was getting called to be displayed on the pages. I forget which honestly, but I can push my version to git and compare.

[ShadesOfMarble]

Is there any support for collecting data about "entities" planned? I.e. knowing which addresses belong - possibly - to the same person/company.

[molecular]

someone found an XSS vulnerability on blockexplorer.auroracoin.eu (bitcoin-abe).

Subject= XSS (Cross Site Scripting) Vulnerability Found In Your Website


Hi!

I am Muhammad Abdullah ,I want to inform you that while testing your

website I found XSS (Cross Site Scripting)

Vulnerability in your website. The Vulnerability is a serious one, the

details of the vulnerability are given below.




Details Of The Vulnerability (XSS):

###################################################################
~Website=http://blockexplorer.auroracoin.eu
~Vulnerability=Cross Site Scripting (XSS)
~Tested on=Firefox,OWASP Mantra Browser
~OS=Windows 7,win XP
~Discovered By=Muhammad Abdullah
~Risk=high
###################################################################



The Vulnerability was found in the following links.


http://blockexplorer.auroracoin.eu/address/</title>1<ScRiPt>prompt("xss/by/M.Abdullah/")</ScRiPt>

http://blockexplorer.auroracoin.eu/b/"1%3CScRiPt%3Eprompt("xss/by/M.Abdullah/")%3C/ScRiPt%3E

http://blockexplorer.auroracoin.eu/chai ... %3Eprompt("xss/by/M.Abdullah/")%3C/ScRiPt%3E



In all the above links URI was set to

</title><ScRiPt>prompt("xss/by/M.Abdullah/")</ScRiPt>

OR

1<ScRiPt>prompt("xss/by/M.Abdullah/")</ScRiPt>

The input is reflected inside a text element






http://blockexplorer.auroracoin.eu/clai ... Pt%3e.html

In the above link

Path Fragment (suffix .html) input / was set to claims'"()&%<ScRiPt
prompt(document.domain)</ScRiPt>

The input is reflected inside a text element.




http://blockexplorer.auroracoin.eu/sear ... dprompt%28"xss/by/M.Abdullah/"%29%20bad%3d%22

In the above link

URL encoded GET input q was set to 1"
onmouseover=prompt("xss/by/M.Abdullah/") bad="
The input is reflected inside a tag parameter between double quotes.

MOVE MOUSE CURSOR TO SEARCH BOX IN ORDER TO EXECUTE THE VECTOR("
onmouseover=prompt("xss/by/M.Abdullah/") bad=")



#######################################################################

###########################




Pictures are attached to verify the vulnerability.
I have only attached pictures for 3 URLs other can also be verified in
the same way....





Patch It as soon as possible......

Hopping to get some bounty from your side..................


Looking forward towards your response.....


Warm Regards

Muhammad Abdullah

mahitman@gmail.com

Images:
https://www.dropbox.com/s/tbp2avf548fqwv0/AUR1.bmp
https://www.dropbox.com/s/2xg61nfwxa520r8/AUR2.bmp
https://www.dropbox.com/s/6n0hz8mdb5krnkb/AUR3.bmp

thanks to joiblumen to let me know.

I'm not sure what to do about this or how serious this is in this case (probably not very).

[foodies123]

Hi, every once in a while my abe gets stuck on a http request and to unstuck it I have to attach to screen and control+c it, any idea how to get around this ?

[boxuser]

+1 have the same issue.

Hi, every once in a while my abe gets stuck on a http request and to unstuck it I have to attach to screen and control+c it, any idea how to get around this ?

[bulldozer2003]

I'm loading Abe with the bitcoin blockchain into Abe using postgres, but it quits at the same point every time with an error.

I've tried running Abe with "--rescan" and it quits at the same point. I also ran "bitcoind -reindex" to reindex my copy of the blockchain, but Abe still quits at the same block.

Short of dropping the database and starting over, does anyone have any other suggestions?

The error message is:

InternalError: right sibling's left-link doesn't match: block 54203 links to 247894 instead of expected 123135 in index "tx_tx_hash_key"

The full error output is:

Code:

$ python -m Abe.abe --config .abe/abe.conf --commit-bytes 100000 --no-serve
no chain_id
catch_up_rpc: abort
Opened /home/user/.bitcoin/blocks/blk00055.dat
Exception at 49123557
Failed to catch up {'blkfile_offset': 49065373, 'blkfile_number': 100055, 'chain_id': None, 'loader': None, 'dirname': '/home/user/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2518, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2778, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2910, in import_blkdat
    store.import_block(b, chain = chain)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1054, in import_block
    tx['tx_id'] = store.import_tx(tx, pos == 0, chain)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1785, in import_tx
    store.intin(tx['lockTime']), tx['size']))
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 250, in sql
    store._sql.sql(stmt, params)
  File "/usr/local/lib/python2.7/dist-packages/Abe/SqlAbstraction.py", line 457, in sql
    sql._execute(cached, params)
  File "/usr/local/lib/python2.7/dist-packages/Abe/SqlAbstraction.py", line 438, in _execute
    sql.cursor().execute(stmt, params)
InternalError: right sibling's left-link doesn't match: block 54203 links to 247894 instead of expected 123135 in index "tx_tx_hash_key"

[cazze]

Can somebody explain me how i can find the string to import-tx? I tried
getrawtransaction 0
but that gives me
error: {"code":-5,"message":"No information available about transaction"}

[roundrobin]

Having some problems getting Abe to work with a Litecoin fork. Nothing odd about the fork, it's a plain PoW Litecoin clone which uses a different "magic" (pchMessage) and different "version_address" which I set with "magic":"x..x..x..x.." and "version_address":"x..", however....

When using blkfile loader, it commits 1234 blocks then complains that the merkleroot hash differs, but the blockchain of this coin is not broken, it has over 65000 blocks, and the daemon can verify the chain 100%.

When using rpc loader instead, without "import-tx" set, I get:

Code:

RPC data not understood: block hash mismatch
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 15, 'loader': u'rpc', 'conf': u'randomcoin.conf', 'dirname': u'/home/randomcoin/.randomcoin/', 'id': 1}

When I set "import-tx = 0100000..." to the 160 character hex string from debug.log (input=........) I get:

Code:

Traceback (most recent call last):
  File "/usr/lib/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/home/randomcoin/bitcoin-abe-master/Abe/abe.py", line 2093, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/randomcoin/bitcoin-abe-master/Abe/abe.py", line 2087, in main
    store = make_store(args)
  File "/home/randomcoin/bitcoin-abe-master/Abe/abe.py", line 156, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3274, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 221, in __init__
    store.maybe_import_binary_tx(chain_name, str(hex_tx).decode('hex'))
  File "/usr/lib/python2.6/encodings/hex_codec.py", line 42, in hex_decode
    output = binascii.a2b_hex(input)
TypeError: Odd-length string

My config I start Abe with:

Code:

datadir += [{
        "chain":"Randomcoin",
        "policy":"NovaCoin",
        "dirname":"/home/randomcoin/.randomcoin/",
        "conf":"randomcoin.conf",
        "code3":"RND",
        "address_version":"x12",
        "magic":"x12x34x56x78",
        "loader":"rpc",
        "rpcpassword":"..."
      }]

import-tx="01000000 ..."     # (160 characters long hexscii string taken from debug.log genesis block entry, "input=")
dbtype = sqlite3
connect-args = abe.sqlite
int-type str
port 8000
host 127.0.0.1

I have installed the ltc_scrypt module, but still nothing works. I suspect that I need to create a new policy/chain file in Abe/Chain/ instead of using NovaCoin which has been suggested, but I tried using the LtcScryptCoin.py with no luck. Going nuts, any help is appreciated.

[xxTeDYxx]

Hi I have a big problem three days I try to put it

Code:

root@vps74571:/home/plnc/bitcoin-abe#     python -m Abe.abe --config abe.conf --commit-bytes 100000 --no-serve
ddl_implicit_commit=true
create_table_epilogue=''
Abe/SqlAbstraction.py:468: Warning: Converting column 'a' from VARCHAR to TEXT
  sql.cursor().execute(stmt)
Abe/SqlAbstraction.py:468: Warning: Converting column 'b' from VARCHAR to TEXT
  sql.cursor().execute(stmt)
max_varchar=4294967295
No native type found for CLOB.
binary_type=binary
int_type=int
sequence_type=mysql
limit_style=native
Abe/SqlAbstraction.py:468: Warning: Converting column 'txout_scriptPubKey' from VARBINARY to BLOB
  sql.cursor().execute(stmt)
Abe/SqlAbstraction.py:468: Warning: Converting column 'txin_scriptSig' from VARBINARY to BLOB
  sql.cursor().execute(stmt)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/plnc/bitcoin-abe/Abe/abe.py", line 2093, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/plnc/bitcoin-abe/Abe/abe.py", line 2087, in main
    store = make_store(args)
  File "/home/plnc/bitcoin-abe/Abe/abe.py", line 156, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3275, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 187, in __init__
    store.initialize()
  File "Abe/DataStore.py", line 841, in initialize
    chain = Chain.create(policy, **conf)
  File "Abe/Chain/__init__.py", line 21, in create
    mod = __import__(__name__ + '.' + policy, fromlist=[policy])
ImportError: No module named PLNcoin

Please HELP :<

[roundrobin]

Hi I have a big problem three days I try to put it

...

Please HELP :<

Same problem as me. The reason it doesn't work for you is that there is no chain module for PLNcoin in Abe/Chain/ directory. You have set "chain":"PLNcoin" (or "policy":"PLNcoin") which wants the file Abe/Chain/PLNcoin.py to know how to handle the blocks, but there is no such file.

I wish the author had written a working litecoin scrypt clone chain file, but, none available...

[th3joker]

trying to set this up for conspiracycoin I get the following error: A server error occurred.  Please contact the administrator.

The front page looks ok and then you can click individual blocks which also work ok but if you try and browse you get that.

http://block.conspiracycoin.org:81/

Anyone had this problem?

[elbandi]

Hi I have a big problem three days I try to put it

...

Please HELP :<

Same problem as me. The reason it doesn't work for you is that there is no chain module for PLNcoin in Abe/Chain/ directory. You have set "chain":"PLNcoin" (or "policy":"PLNcoin") which wants the file Abe/Chain/PLNcoin.py to know how to handle the blocks, but there is no such file.

I wish the author had written a working litecoin scrypt clone chain file, but, none available...

Update you database (sqlite/mysql/etc):
Set the right chain in chain_policy field in chain table. Use google

PLNcoin is a scrypt, so sha256chain (search the right name!!).