[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[internetbitcoins]

I run

Code:

python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve

but 2 or 3 times I had to restart the server, so now in the block table, I check on block_id and there are some missing blocks.

block_id is not the same as block number (a.k.a. block_height).  The gaps in block_id are not a problem.

Hi, thanks. Finally I can return to the abe staff today. I launched the web server and I can get the data. For example, for this one:
http://localhost:2750/chain/Bitcoin/q/getreceivedbyaddress/19vDdUsaiQrra5CDDKNn8yooai2pF3XU6A
I get 0.43250932

And on http://blockexplorer.com/address/19vDdUsaiQrra5CDDKNn8yooai2pF3XU6A
I get 0.47998327

So I think there is something not updated on my database.

On launching abe web server I get this error:

Code:

python -m Abe.abe --config abe-my.conf
no chain_id
catch_up_rpc: abort
Opened /xx/xx/.bitcoin/blocks/blk00074.dat
Exception at 87267839
Failed to catch up {'blkfile_offset': 87139457, 'blkfile_number': 100074, 'chain_id': None, 'loader': None, 'dirname': '/xx/xx/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2637, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2895, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3025, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1737, in import_block
    tx['tx_id'] = store.import_and_commit_tx(tx, pos == 0)
  File "Abe/DataStore.py", line 2259, in import_and_commit_tx
    tx_id = store.import_tx(tx, is_coinbase)
  File "Abe/DataStore.py", line 2193, in import_tx
    pubkey_id = store.script_to_pubkey_id(txout['scriptPubKey'])
  File "Abe/DataStore.py", line 2573, in script_to_pubkey_id
    for opcode, data, i in deserialize.script_GetOp(script):
  File "Abe/deserialize.py", line 236, in script_GetOp
    opcode |= ord(bytes[i])
IndexError: string index out of range
Abe initialized.
Listening on http://localhost:2750

Any idea? Thanks (again) in advance!

[John Tobey]

Code:

  File "Abe/deserialize.py", line 236, in script_GetOp
    opcode |= ord(bytes[i])
IndexError: string index out of range

Please pull and try the latest code.  This was fixed a few days ago.

[internetbitcoins]

Code:

  File "Abe/deserialize.py", line 236, in script_GetOp
    opcode |= ord(bytes[i])
IndexError: string index out of range

Please pull and try the latest code.  This was fixed a few days ago.

Thanks, I'll do it.

[unick]

This is the output

Code:

<module 'Abe.abe' from '/usr/local/lib/python2.7/dist-packages/Abe/abe.pyc'>

This means you are getting the previously installed version.  You want the one from from 'Abe/abe.py'.  Either install the new version (as root, python setup.py install) or set PYTHONPATH to the git clone directory (bitcoin-abe, not the Abe subdirectory).  Normally, sys.path starts with an empty string, so if you "cd" to the correct directory, Python should find the local modules.

that did it... I'm updating the chain so everything seems to run smooth... I'll keep you posted if I ever find out about what you were talking earlier.

thanks a bunch

[internetbitcoins]

Code:

  File "Abe/deserialize.py", line 236, in script_GetOp
    opcode |= ord(bytes[i])
IndexError: string index out of range

Please pull and try the latest code.  This was fixed a few days ago.

Hi again, just to say thank you. I did it and it worked.

Cheers.

[brishtiteveja]

Hi,
  I just started using Abe. But I am not being able to run the server on my macbook i.g. 127.0.0.1:2750 is showing no connection. I was using sqlite database. And I was reading other users are creating their own abe-my.conf file. Why did you create that and what it contains.. I am sorry if these questions sound silly and pretty basic. Would you please show me a direction? My sqlite database hasn't been complete. But It gathered nearly 200000 blocks.Do I need to wait till it finishes? Or am I missing some fundamentals? Thank you.

My terminal results:

MacBook-Pro:bitcoin-abe zehadyzbdullahkhan$ python -m Abe.abe --dbtype=sqlite3 --connect-args=abe.sqlite --port 2750
no chain_id
catch_up_rpc: abort
Opened /Users/zehadyzbdullahkhan/Library/Application Support/Bitcoin/blocks/blk00018.dat
block_tx 192674 5701191
block_tx 192674 5701192
block_tx 192674 5701193
............................................

[John Tobey]

MacBook-Pro:bitcoin-abe zehadyzbdullahkhan$ python -m Abe.abe --dbtype=sqlite3 --connect-args=abe.sqlite --port 2750
no chain_id
catch_up_rpc: abort
Opened /Users/zehadyzbdullahkhan/Library/Application Support/Bitcoin/blocks/blk00018.dat
block_tx 192674 5701191
block_tx 192674 5701192
block_tx 192674 5701193
............................................

Until it finishes loading, this process will not serve HTTP requests.  I suggest you run two processes, a dedicated loader and a dedicated server.

Window 1:
python abe.py --dbtype=sqlite3 --connect-args=abe.sqlite --port 2750 --datadir '[]'

Window 2:
while true; do python abe.py --dbtype=sqlite3 --connect-args=abe.sqlite --no-serve; sleep 10; done

"--datadir []" makes it skip loading and start serving immediately.  "--no-serve" makes it exit immediately after loading all blocks, and the while-loop makes it check again after a pause.

[brishtiteveja]

Hi Tobey, thank you so much. It worked really great. The database is still updating after days... I wish it gets done quickly. I am watching the ToDo list now. Abe is really great.Hope I can contribute in some way.  

[dashingriddler]

Hello i am facing the issue at block 256522
here is the error i m getting. after getting this error, i deleted my block chain files and loaded the entire block chain again but no use. any help greatly appreciated.

bitcoin@CoinMonk:~/Abe$ python -m Abe.abe --config abe-my.conf --no-serve
no chain_id
catch_up_rpc: abort
Opened /xxx/xxx/.bitcoin/blocks/blk00080.dat
Exception at 30434958
Failed to catch up {'blkfile_offset': 30096859, 'blkfile_number': 100080, 'chain_id': None, 'loader': None, 'dirname': '/xxx/xxx/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2637, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2895, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3025, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1740, in import_block
    if tx['unlinked_count'] > 0:
KeyError: 'unlinked_count'

[John Tobey]

  File "Abe/DataStore.py", line 1740, in import_block
    if tx['unlinked_count'] > 0:
KeyError: 'unlinked_count'

Thanks, I think I have fixed this.  Please pull and try the latest code.

[dashingriddler]

  File "Abe/DataStore.py", line 1740, in import_block
    if tx['unlinked_count'] > 0:
KeyError: 'unlinked_count'

Thanks, I think I have fixed this.  Please pull and try the latest code.

I started from a snapshot about 1 month ago and loaded the rest of block chain and Abe continued and did catch up completely this time. I dont know why i did not face any issue this time though.
I am afraid to touch the scripts now. But i will as soon as i face any other issue and let u know here.

Thank you for your attempt to solve my issue.

[triggermage]

Hey guys, trying to run Abe for the first time on Ubuntu 12.10 the command python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve returns:



Anyone have a clue?

Thank you very much!

[bitbudget]

We are looking for a python developer who would setup the Bitcoin Abe script on our server and modify it for our needs.
Long term cooperation is also an option.

Please email us at hr@bit-b.com with your offers.

[internetbitcoins]

Hi again,

since my last message Abe was working ok, but some days ago I was expecting a very slow response.

For example, for the command wget
wget http://localhost:2750/chain/Bitcoin/q/addressbalance/1EweSrwuG6av7Lsk9fHzRbo91cavBDbNQM

I get this log

Code:

--2013-12-04 09:30:09--  http://localhost:2750/chain/Bitcoin/q/addressbalance/1EweSrwuG6av7Lsk9fHzRbo91cavBDbNQM
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:2750... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

--2013-12-04 09:30:24--  (try: 2)  http://localhost:2750/chain/Bitcoin/q/addressbalance/1EweSrwuG6av7Lsk9fHzRbo91cavBDbNQM
Connecting to localhost (localhost)|127.0.0.1|:2750... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1 [text/plain]
Saving to: '1EweSrwuG6av7Lsk9fHzRbo91cavBDbNQM'

Finally, I got the balance after the second try, but in some cases I have 16 or more...and I need ABE to retrieve about 350 Bitcoin address balances every day, so I have to find a quickly way or find why is so slowly... same operation in blockchain.info api (with wget or curl via php) is very fast, so I think it has to be with my abe installation, but I don't know what happens. I restarted the server and killed old processes, but nothing happens.

With curl is even worst, I never get the value, always a 0 balance, I think because of the waiting time is too much.

Some ideas on what can be happening?
Any help will be appreciated. Thanks in advance and regards.

[internetbitcoins]

My last restart of the server, right now, has this log:

Code:

...block_tx 273388 32443649
block_tx 273388 32443650
block_tx 273388 32443651
commit
Abe initialized.
Listening on http://localhost:2750
no chain_id
catch_up_rpc: abort
Opened /home/ubuntu/.bitcoin/blocks/blk00098.dat
127.0.0.1 - - [05/Dec/2013 11:41:43] "GET /chain/Bitcoin/q/addressbalance/1Q3sRVHTKzik5tB2ZxmvYR2pxk2ZE5PjFW HTTP/1.1" 200 1
no chain_id
catch_up_rpc: abort
Opened /home/ubuntu/.bitcoin/blocks/blk00098.dat
127.0.0.1 - - [05/Dec/2013 11:41:43] "GET /chain/Bitcoin/q/addressbalance/1Ns3UYRqRcwmNLrmm5rwfTD4HF5K2p2U1A HTTP/1.1" 200 1

I don't know if is relevant...Thanks again.

[phelix]

We are considering using Abe for Namecoin. Is there a demo version online somewhere?

[Bale80]

Hi there,
and thanks for making this.

I'm currently trying to set up bitcoin-abe for Netcoin (https://github.com/netcoinproject/netcoin) and I'm having a bit of problems.
I'm running the latest from github and have added the following to Datastore.py:

  {"chain":"Netcoin",
     "code3":"NET", "address_version":"\x70", "magic":"\xfd\xb6\xa5\xdb"}

...
....
block_tx 4071 3398
Skipped 1 bytes at block end
Exception at 7099887448213790924

Failed to catch up {'blkfile_offset': 39107, 'blkfile_number': 1, 'chain_id': 9, 'loader': None, 'dirname': '/home/myuser/.netcoin', 'id': Decimal('32')}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2637, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2901, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3028, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "Abe/DataStore.py", line 3059, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "Abe/deserialize.py", line 92, in parse_Transaction
    d['txOut'].append(parse_TxOut(vds))
  File "Abe/deserialize.py", line 67, in parse_TxOut
    d['value'] = vds.read_int64()
  File "Abe/BCDataStream.py", line 72, in read_int64
    def read_int64  (self): return self._read_num('<q')
  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 8 bytes

Any ideas on what I'm doing wrong ?

[ddirt]

Hi, hoping somebody can help me get Abe up and running for SolarCoin. I'm hitting this error


block 2082 already in chain 9
Exception at 69523
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 9, 'loader': None, 'dirname': '/home/ubuntu/.solarcoin', 'id': Decimal('30')}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2639, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2897, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3024, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3055, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 92, in parse_Transaction
    for i in xrange(n_vout):
OverflowError: Python int too large to convert to C long


I've tried using the PPCoin branch, no help.

[mapleshadow]

datastore.py----------------------
    {"chain":"CinnamonCoin",
     "code3":"CIN", "address_version":"\x1c", "magic":"\xcd\xf2\xc0\xef"},

cin.conf--------------
default-loader = blkfile
dbtype MySQLdb
connect-args {"user":"abe","db":"abe_cin","passwd":"<xxx>"}
datadir = /home/xxx/.CinnamonCoin
upgrade
port 2755
host 12.10.2.222

shell------------
python -m Abe.abe --config config/cin.conf --commit-bytes 100000 --no-serve

error:
Opened /home/xxx/.CinnamonCoin/blk0001.dat
Chain not found for magic number dbc0dbf2 in block file /home/xxx/.CinnamonCoin/blk0001.dat at offset 0.

why？？？？？？？？？
Should not find cdf2c0ef thing?
Why instead to search dbc0dbf2?

[mapleshadow]

ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:526: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:526: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
Abe/DataStore.py:526: Warning: Converting column 'test_varbit' from VARCHAR to TEXT
  store.cursor.execute(stmt)
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
Abe/DataStore.py:526: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:526: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Opened /home/mapleshadow/.chncoin/blk0001.dat
block_tx 1 1
block_tx 2 2
block_tx 3 3