[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[exdirrk]

Hi! I have a problem with ppcoin. Quantities seems to require a 100 multiplication to be right. For example, a transaction that ABE stores as 0.100 should be 0.100*100= 10 PPC

With bitcoin, litecoin and testnet ABE works very well

PS: I'm using the ppcoin branch merged to the master from git

How did you get past this ?
Chain not found for magic number e6e8e9e5 in block file /home/dirrk/.ppcoin/blk0001.dat at offset 0.[/u][/u]


Nevermind I was able to fix the issue.  I was not using the peercoin branch.  Now my issue is that the address are in the wrong format.  I changed the config to reference the 0x0037 type but it still displays incorrectly.  I am rebuilding the database now in hopes that it will fix it.

[wing_hk]

I can setup and run Abe successfully. (All blocks are loaded during installation)

However, it just shows 0 block at homepage. Is there anything wrong?

Details can be find here:
https://github.com/bitcoin-abe/bitcoin-abe/issues/49

Using "ppcoin" branch.

[molecular]

Hello,

I really miss my abe database since I threw it away.

I've been trying to rebuild it from scratch, but starting around block 180,000 it gets really really slow (something like 1000 blocks per night). It seems almost undoable.

I'm using postgres and increased the buffers generously.

Is this (1000 block per 8 hours) a speed that can be confirmed by others?

Any suggestions as to what I could do to speed up the process?

[internetbitcoins]

Hi, and happy new year.

Everything was working fine with abe, and I had to stop it for some days. We are talking about an Ubuntu 13.10 Server on Amazon EC2 small with mysql server running on it (not RDS) and apache (think is not important) and a 300Gb Volume.

Now when I try to restart (sudo python -m Abe.abe --config abe-my.conf) I'm getting this error:

Code:

catch_up_rpc: abort
Opened /home/ubuntu/.bitcoin/blocks/blk00105.dat
Exception at 62110992
Failed to catch up {'blkfile_offset': 61209101, 'blkfile_number': 100105, 'chain_id': None, 'loader': None, 'dirname': '/home/ubuntu/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2639, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2897, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3027, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1741, in import_block
    tx['tx_id'] = store.import_tx(tx, pos == 0)
  File "Abe/DataStore.py", line 2195, in import_tx
    pubkey_id = store.script_to_pubkey_id(txout['scriptPubKey'])
  File "Abe/DataStore.py", line 2563, in script_to_pubkey_id
    return store.pubkey_hash_to_id(match.group(1))
  File "Abe/DataStore.py", line 2596, in pubkey_hash_to_id
    return store._pubkey_id(pubkey_hash, None)
  File "Abe/DataStore.py", line 2607, in _pubkey_id
    WHERE pubkey_hash = ?""", (dbhash,))
  File "Abe/DataStore.py", line 638, in selectrow
    store.sql(stmt, params)
  File "Abe/DataStore.py", line 510, in sql
    store._execute(cached, params)
  File "Abe/DataStore.py", line 487, in _execute
    store.cursor.execute(stmt, params)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (2013, 'Lost connection to MySQL server during query')
Reconnecting after rollback error: (2006, 'MySQL server has gone away')
Reconnecting to database.
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/var/www/abe/Abe/abe.py", line 2248, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/var/www/abe/Abe/abe.py", line 2242, in main
    store = make_store(args)
  File "/var/www/abe/Abe/abe.py", line 157, in make_store
    store.catch_up()
  File "Abe/DataStore.py", line 2647, in catch_up
    store.rollback()
  File "Abe/DataStore.py", line 871, in rollback
    store.reconnect()
  File "Abe/DataStore.py", line 271, in reconnect
    store.init_conn()
  File "Abe/DataStore.py", line 213, in init_conn
    store.conn = store.connect()
  File "Abe/DataStore.py", line 224, in connect
    conn = store._connect(cargs)
  File "Abe/DataStore.py", line 256, in _connect
    return store.module.connect(**cargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)")

Mysql server is running, I can access trough mysql -u user -p and show my tables and databases, insert, select, etc. I can do it trough a python script, so I think is not a problem with python or mysql server.

I tried this too:

Code:

sudo find / -name '*.sock'
/run/mysqld/mysqld.sock
sudo ln -s /run/mysqld/mysqld.sock /var/run/mysqld/mysqld.sock
ln: failed to create symbolic link ‘/var/run/mysqld/mysqld.sock’: File exists

So, I'm a little bit lost... Of course, I restarted mysql server sometimes, and entire server, and nothing...

Any idea?? Thanks a lot.

[internetbitcoins]

Hi again.

The problem appears to be in the big size of pubkey table. Aprox 24.270.000 inputs, so mysqld can't handle it and go away.

I have an small amazon ec2 ubuntu server 13.10 64bits with 300Gb HDD. The only difference I can see is that some days ago I was using a 1Tb HDD instead of 300Gb but there are some space remaining yet in 300Gb HDD.

For example:
SELECT COUNT(*) FROM pubkey = MySQL Server gone away
SELECT * FROM pubkey LIMIT 10000000 (aprox half of data amount) =  Output data without problems.

Some idea?? Thanks again.

[crabel]

I love abe and it is giving me some incredible capabilities. Thank you for developing it!

Here is a bug report and a recommendation. 
There is an error in how chain summary generates the table. You can see this at block_id 278440 the block_height 278271 skips to 278160. Also as a comment would be convenient if abe.block had the previous block hash, allowing construction of the chain from one db entry.

[gigamike]

Hi,

how may i know if its working correctly.

Currently

[root@CentOS-63-64-minimal bitcoin-abe]# python -m Abe.abe --config abe.conf
no chain_id
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://localhost:2750
...then stops here

Is this correct i mean its downloading block or something?

Thanks,

Mike

[gigamike]

Hi,

how may i know if its working correctly.

Currently

[root@CentOS-63-64-minimal bitcoin-abe]# python -m Abe.abe --config abe.conf
no chain_id
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://localhost:2750
...then stops here

Is this correct i mean its downloading block or something?

Thanks,

Mike

ok, seems working now, i saw some display

commit
block_tx 3658 3714
commit
block_tx 3659 3715
commit
block_tx 3660 3716
commit
block_tx 3661 3717
commit
block_tx 3662 3718
commit
...

question, possible i can get data of bitcoins days destroyed?

Thanks,

Mike

[lifeboat]

Hi! I have a problem with ppcoin. Quantities seems to require a 100 multiplication to be right. For example, a transaction that ABE stores as 0.100 should be 0.100*100= 10 PPC

With bitcoin, litecoin and testnet ABE works very well

PS: I'm using the ppcoin branch merged to the master from git

This problem still exists. For example, http://ppc.cryptocoinexplorer.com/address/PLifetQ51s7WYHr4WCdHmWaALUzyWNmXdP needs a 100 multiplication on all values.

[gigamike]

Hi,

how may i know if its working correctly.

Currently

[root@CentOS-63-64-minimal bitcoin-abe]# python -m Abe.abe --config abe.conf
no chain_id
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://localhost:2750
...then stops here

Is this correct i mean its downloading block or something?

Thanks,

Mike

ok, seems working now, i saw some display

commit
block_tx 3658 3714
commit
block_tx 3659 3715
commit
block_tx 3660 3716
commit
block_tx 3661 3717
commit
block_tx 3662 3718
commit
...

question, possible i can get data of bitcoins days destroyed?

Thanks,

Mike

Guys,

anyone who can help me if ABE, possible i can get data of bitcoins days destroyed?

Thanks,

Mike

[Bitcoin Doctor]

was this ever completed?

[K1773R]

Hi! I have a problem with ppcoin. Quantities seems to require a 100 multiplication to be right. For example, a transaction that ABE stores as 0.100 should be 0.100*100= 10 PPC

With bitcoin, litecoin and testnet ABE works very well

PS: I'm using the ppcoin branch merged to the master from git

This problem still exists. For example, http://ppc.cryptocoinexplorer.com/address/PLifetQ51s7WYHr4WCdHmWaALUzyWNmXdP needs a 100 multiplication on all values.

still exists -.-
anyone got a solution?

EDIT: nvm. hotfix -> https://github.com/bitcoin-abe/bitcoin-abe/issues/48#issuecomment-31317058

[gigamike]

Guys,

ive been running this for 10 days now. The current `block`.`block_nTime`  is 2013-05-13 23:35:31

Is there anyway i can make it faster?

TIA,

Mike

[gigamike]

Hi,

is the blobk.block_nTime and block.block_ss_destroyed is the data for Bitcoin Days Destroyed?

But strange most of the block.block_ss_destroyed is NULL

TIA,

Mike

[mapleshadow]

Failed to catch up {'blkfile_offset': 1309247667, 'blkfile_number': 1, 'chain_id': 8, 'loader': None, 'dirname': u'/home/mapleshadow/.dogecoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2639, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2804, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2340, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 334, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http:

[wing_hk]

Failed to catch up {'blkfile_offset': 1309247667, 'blkfile_number': 1, 'chain_id': 8, 'loader': None, 'dirname': u'/home/mapleshadow/.dogecoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2639, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2804, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2340, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 334, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http:

try rpc instead of blkfile

[mapleshadow]

Failed to catch up {'blkfile_offset': 1309247667, 'blkfile_number': 1, 'chain_id': 8, 'loader': None, 'dirname': u'/home/mapleshadow/.dogecoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2639, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2804, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2340, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 334, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http:

try rpc instead of blkfile

Will, conf file how to write this code?
And this tip:
ddl_implicit_commit=true
create_table_epilogue=''
max_varchar=4294967295
clob_type=CLOB
binary_type=buffer
int_type=str
Created silly table abe_dual
sequence_type=update
limit_style=native
genesis transaction unavailable via RPC; see import-tx in abe.conf
catch_up_rpc: abort
Opened /home/maple/.digitalcoin/blk0001.dat

[mapleshadow]

Failed to catch up {'blkfile_offset': 1309247667, 'blkfile_number': 1, 'chain_id': 8, 'loader': None, 'dirname': u'/home/mapleshadow/.dogecoin', 'id': 1}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2639, in catch_up
    if not store.catch_up_rpc(dircfg):
  File "Abe/DataStore.py", line 2804, in catch_up_rpc
    format = "binary")
  File "Abe/DataStore.py", line 2340, in export_tx
    'prevout_hash': store.hashout(prevout_hash),
  File "Abe/DataStore.py", line 334, in rev
    return x[::-1]
TypeError: 'NoneType' object has no attribute '__getitem__'
Abe initialized.
Listening on http:

try rpc instead of blkfile

Will, conf file how to write this code?
And this tip:
ddl_implicit_commit=true
create_table_epilogue=''
max_varchar=4294967295
clob_type=CLOB
binary_type=buffer
int_type=str
Created silly table abe_dual
sequence_type=update
limit_style=native
genesis transaction unavailable via RPC; see import-tx in abe.conf
catch_up_rpc: abort
Opened /home/maple/.digitalcoin/blk0001.dat

Should be written so?
default-loader = blkfile

[unreal37]

Before I went to upgrade, I found that Abe had stopped working. My block file just exceeded 1GB in size a couple hours ago which is causing this error:

Failed to catch up {'blkfile_number': 1, 'dirname': 'C:\\Documents and Settings\
\Jon\\Application Data\\Bitcoin', 'chain_id': None, 'id': 1L, 'blkfile_offset':
0}
Traceback (most recent call last):
  File "Abe\DataStore.py", line 2169, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe\DataStore.py", line 2184, in catch_up_dir
    ds = open_blkfile()
  File "Abe\DataStore.py", line 2180, in open_blkfile
    ds.map_file(open(filename, "rb"), 0)
  File "Abe\BCDataStream.py", line 27, in map_file
    self.input = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
WindowsError: [Error 8] Not enough storage is available to process this command

It is because I'm running a 32-bit system. Apparently this issue will eventually affect Linux 32-bit systems as well because of the way mmap works; except that it will occur when the block file hits 2 or 3 GB in size. Is Abe particularly tied to mmap? If it is, it seems like it will soon be (or is now) for 64 bit systems only.

For those who try to use Abe and encounter this error, upgrading from 32-bit Python to 64-bit Python (on a 64-bit system) solves this problem.

[ddirt]

Does Abe work with coins that use transaction messaging?

I'm trying to set this up with SolarCoin and hitting the same "overflowerror: Python int too large to convert to C long" that others seemed to be getting with coins using transaction messaging. Anyone get past that problem yet?