[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[John Tobey]

Another question, is it possible to set the magic address in the config and not DataStore.py? I would like to parse multiple blocks but have separate databases for them. It seems I can only do this by running separate instances of Abe.

Looks like I might be able to specify the magic when doing 'datadir+=' in the .conf but it wont insert it into the database.

You don't need the magic number if you supply the chain name.  Magic number is for datadir=/path so Abe can find the chain name given only the magic number found in the block files.

When Abe sees a datadir with a previously unknown chain name, it copies the magic number and some other fields to the chain table.  On subsequent runs, it does not overwrite these values based on datadir.  You would have to update the row or rebuild the database with the modified datadir configuration.

Hope this helps.

[John Tobey]

Code:

sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: no tty present and no askpass program specified
Sorry, try again.
sudo: 3 incorrect password attempts
[Fri Feb 28 09:37:59 2014] [warn] [client 108.162.221.55] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 09:37:59 2014] [error] [client 108.162.221.55] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

Please verify the line that you added to /etc/sudoers.

[John Tobey]

Somehow the current block height displayed on Abe's main page no longer changes as new blocks are processed by Abe.

Right now the current block displayed on main page is 521941. However I am able to access a newer block 521948 by going to http://localhost:2750/block/a4436cbcf8986ab38e64fc88983c973a76cd938157dbb2f0d3a20748842dc8d5. Abe's title also shows Block a443...48842dc8d5 instead of Block 521948

q/getblockcount also shows the wrong block height of 521941.

Is it possible to do like a rescan to update the Abe's current block height on the front page?

The chain was probably forked.  Once the current chain catches up to the previous highest block it will sort itself out.  I've seen this a few times.

Abe keeps orphan blocks and side chains and reuses them if they join the main chain.

Thanks for the quick reply! The chain I'm using is LTC, is this just a temp fork that affects some people, and also the reason why we wait for several confirmations for a transaction?

Well, this looks to me like Abe failing to find block 521941 by its hash in block 521942.  This can occur because of bugs, especially when two or more processes simultaneously load blocks.  This can also occur because of block format changes, such as changes to the hash algorithm.

Can you browse 521942 by its hash?  You might have to delete 521941 and/or 521942 from the database and rescan the blockfile that contains them.  I don't have a tool to do exactly that, but you may be able to avoid a full reload with this, assuming "Litecoin" is what Abe calls the chain:

• Stop all processes that might load blocks.
• Run: python -m Abe.admin delete-chain-blocks Litecoin.  This should delete all block header information but not the transactions.
• Run Abe with --no-serve --rescan options.  This should reinsert the block information.
• Restart stopped processes.

Alternatively, a full reload using a single process will work, if anything will.  Try to separate loading from serving web pages, if you haven't already.  Only a single process should load blocks until we fix the bugs.

[StakeHunter]

Please verify the line that you added to /etc/sudoers.

Code:

www-data ALL=(nathan) NOPASSWD: /home/nathan/cgi-bin/abe

No longer getting tty, still getting the fastcgi errors...

Code:

[Fri Feb 28 11:53:21 2014] [warn] [client 108.162.237.97] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 11:53:21 2014] [error] [client 108.162.237.97] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

[John Tobey]

Please verify the line that you added to /etc/sudoers.

No longer getting tty, still getting the fastcgi errors...

Code:

[Fri Feb 28 11:53:21 2014] [warn] [client 108.162.237.97] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Fri Feb 28 11:53:21 2014] [error] [client 108.162.237.97] Premature end of script headers: abe.fcgi

Trying to use FastCGI as outlined...having issues.

Sorry, but I can not troubleshoot the FastCGI setup here.  Does Abe work in non-FCGI mode with --port=2750?  Are your scripts executable?  Can you find the output of abe.fcgi?  It may include a useful error message.

[John Tobey]

Tried Everything But all I get is:

A server error occurred.  Please contact the administrator.

Help.

Block Chain Explorer Link: http://107.170.63.233:2750/

What do you see at the command window where you started python?  See: http://www.catb.org/~esr/faqs/smart-questions.html#beprecise

[anush3070]

Tried Everything But all I get is:

A server error occurred.  Please contact the administrator.

Help.

Block Chain Explorer Link: http://107.170.63.233:2750/

What do you see at the command window where you started python?  See: http://www.catb.org/~esr/faqs/smart-questions.html#beprecise

Hello,
Sorry for not Being Precise,
Here is the Output in the Command Line when I visit the Block Chain Explorer Website:

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/root/bce/bitcoin-abe/Abe/abe.py", line 209, in __call__
    handler(page)
  File "/root/bce/bitcoin-abe/Abe/abe.py", line 307, in handle_chains
    100.0 - (100.0 * (ss + more) / (total_ss + more)))
ZeroDivisionError: float division by zero
183.87.156.69 - - [28/Feb/2014 12:25:58] "GET / HTTP/1.1" 500 59

[John Tobey]

Hello,
Sorry for not Being Precise,
Here is the Output in the Command Line when I visit the Block Chain Explorer Website:

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/root/bce/bitcoin-abe/Abe/abe.py", line 209, in __call__
    handler(page)
  File "/root/bce/bitcoin-abe/Abe/abe.py", line 307, in handle_chains
    100.0 - (100.0 * (ss + more) / (total_ss + more)))
ZeroDivisionError: float division by zero
183.87.156.69 - - [28/Feb/2014 12:25:58] "GET / HTTP/1.1" 500 59

This looks like an older version of bitcoin-abe.  Can you please try the latest from https://github.com/bitcoin-abe/bitcoin-abe/ ?

[altcoin]

While running:

Code:

sudo python -m Abe.abe --config arc.conf --commit-bytes 100000 --no-serve

I got the following errors, every block has been skipped, can some one help me?

Code:

ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:459: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:459: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
1389784906
block_tx 1 1
Skipped 1 bytes at block end
1389816604
block_tx 2 2
Skipped 72 bytes at block end
1389816625
block_tx 3 3
Skipped 73 bytes at block end
1389816648
block_tx 4 4
Skipped 72 bytes at block end
1389816671
block_tx 5 5
Skipped 71 bytes at block end
1389816681
block_tx 6 6
Skipped 72 bytes at block end
1389816682
block_tx 7 7
Skipped 71 bytes at block end
1389816697
block_tx 8 8
Skipped 73 bytes at block end
1389816710
block_tx 9 9
Skipped 71 bytes at block end
1389816731
block_tx 10 10
Skipped 73 bytes at block end
1389816740
block_tx 11 11
Skipped 71 bytes at block end
1389816749
block_tx 12 12
Skipped 73 bytes at block end
1389816752
block_tx 13 13
Skipped 72 bytes at block end
1389816758
block_tx 14 14
Skipped 72 bytes at block end
1389816763
block_tx 15 15
Skipped 72 bytes at block end
1389816789
block_tx 16 16
Skipped 72 bytes at block end
1389816793
block_tx 17 17
Skipped 72 bytes at block end
1389816797
block_tx 18 18
Skipped 72 bytes at block end
1389816843
block_tx 19 19
Skipped 73 bytes at block end
1389816847
block_tx 20 20
Skipped 72 bytes at block end
1389816852
block_tx 21 21
Skipped 73 bytes at block end
1389816852
block_tx 22 22
Skipped 72 bytes at block end
1389816859
block_tx 23 23
Skipped 71 bytes at block end
1389816864
block_tx 24 24
Skipped 71 bytes at block end
1389816863
block_tx 25 25
Skipped 72 bytes at block end
1389816863
block_tx 26 26
Skipped 71 bytes at block end
1389816875
block_tx 27 27
Skipped 72 bytes at block end
1389816889
block_tx 28 28
Skipped 73 bytes at block end
1389816893
block_tx 29 29
Skipped 72 bytes at block end
1389816903
block_tx 30 30
Skipped 71 bytes at block end
1389816914
block_tx 31 31
Skipped 73 bytes at block end
1389816918
block_tx 32 32
Skipped 72 bytes at block end
1389816937
block_tx 33 33
Skipped 71 bytes at block end
1389816939
block_tx 34 34
Skipped 73 bytes at block end
1389816942
block_tx 35 35
Skipped 72 bytes at block end
1389816945
block_tx 36 36
Skipped 71 bytes at block end
1389816952
block_tx 37 37
Skipped 72 bytes at block end
1389816969
block_tx 38 38
Skipped 73 bytes at block end
1389816979
block_tx 39 39
Skipped 72 bytes at block end
1389816982
block_tx 40 40
Skipped 71 bytes at block end
1389816986
block_tx 41 41
Skipped 72 bytes at block end
1389817005
block_tx 42 42
Skipped 73 bytes at block end
1389817005
block_tx 43 43
Skipped 72 bytes at block end
1389817025
block_tx 44 44
Skipped 72 bytes at block end
1389817036
block_tx 45 45
Skipped 73 bytes at block end
1389817038
block_tx 46 46
Skipped 73 bytes at block end
1389817044
block_tx 47 47
Skipped 72 bytes at block end
1389817049
block_tx 48 48
Skipped 73 bytes at block end
1389817051
block_tx 49 49
Skipped 71 bytes at block end
1389817060
block_tx 50 50
Skipped 71 bytes at block end
1389817064
block_tx 51 51
Skipped 72 bytes at block end
1389817067
block_tx 52 52
Skipped 73 bytes at block end
1389817090
block_tx 53 53
Skipped 72 bytes at block end
1389817120
block_tx 54 54
Skipped 72 bytes at block end
1389817122
block_tx 55 55
Skipped 73 bytes at block end
1389817137
block_tx 56 56
Skipped 72 bytes at block end
1389817147
block_tx 57 57
Skipped 72 bytes at block end
1389817153
block_tx 58 58
Skipped 72 bytes at block end
1389817157
block_tx 59 59
Skipped 71 bytes at block end
1389817166
block_tx 60 60
Skipped 73 bytes at block end
1389817178
block_tx 61 61
Skipped 73 bytes at block end
1389817182
block_tx 62 62
Skipped 72 bytes at block end
1389817186
block_tx 63 63
Skipped 71 bytes at block end
1389817196
block_tx 64 64
Skipped 73 bytes at block end
1389817203
block_tx 65 65
Skipped 72 bytes at block end
1389817196
block_tx 66 66
Skipped 73 bytes at block end
1389817218
block_tx 67 67
Skipped 71 bytes at block end
1389817217
block_tx 68 68
Skipped 72 bytes at block end
1389817226
block_tx 69 69
Skipped 73 bytes at block end
1389817237
block_tx 70 70
Skipped 71 bytes at block end
1389817248
block_tx 71 71
Skipped 73 bytes at block end
1389817262
block_tx 72 72
Skipped 72 bytes at block end
1389817262
block_tx 73 73
Skipped 71 bytes at block end
1389817288
block_tx 74 74
Skipped 71 bytes at block end
1389817288
block_tx 75 75
Skipped 72 bytes at block end
1389817294
block_tx 76 76
Skipped 71 bytes at block end
1389817305
block_tx 77 77
Skipped 73 bytes at block end
1389817307
block_tx 78 78
Skipped 72 bytes at block end
1389817310
block_tx 79 79
Skipped 73 bytes at block end
1389817314
block_tx 80 80
Skipped 71 bytes at block end
1389817317
block_tx 81 81
Skipped 71 bytes at block end
1389817336
block_tx 82 82
Skipped 73 bytes at block end
1389817352
block_tx 83 83
Skipped 72 bytes at block end
1389817367
block_tx 84 84
Skipped 72 bytes at block end
1389817381
block_tx 85 85
Skipped 72 bytes at block end
1389817384
block_tx 86 86
Skipped 72 bytes at block end
1389817399
block_tx 87 87
Skipped 73 bytes at block end
1389817401
block_tx 88 88
Skipped 73 bytes at block end
1389817423
block_tx 89 89
Skipped 73 bytes at block end
1389817428
block_tx 90 90
Skipped 72 bytes at block end
1389817433
block_tx 91 91
Skipped 71 bytes at block end
1389817441
block_tx 92 92
Skipped 72 bytes at block end
1389817463
block_tx 93 93
Skipped 73 bytes at block end
1389817465
block_tx 94 94
Skipped 72 bytes at block end
1389817465
block_tx 95 95
Skipped 72 bytes at block end
1389817469
block_tx 96 96
Skipped 72 bytes at block end
1389817466
block_tx 97 97
Skipped 72 bytes at block end
1389817478
block_tx 98 98
Skipped 71 bytes at block end
1389817493
block_tx 99 99
Skipped 73 bytes at block end
1389817505
block_tx 100 100
Skipped 72 bytes at block end
1389817513
block_tx 101 101
Skipped 72 bytes at block end
1389817536
block_tx 102 102
Skipped 71 bytes at block end
1389817531
block_tx 103 103
Skipped 71 bytes at block end
1389817547
block_tx 104 104
Skipped 73 bytes at block end
1389817568
block_tx 105 105
Skipped 71 bytes at block end
1389817583
block_tx 106 106
Skipped 72 bytes at block end
1389817597

[altcoin]

While running:

Code:

sudo python -m Abe.abe --config arc.conf --commit-bytes 100000 --no-serve

I got the following errors, every block has been skipped, can some one help me?

Code:

ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:459: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:459: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
1389784906
block_tx 1 1
Skipped 1 bytes at block end
1389816604
block_tx 2 2
Skipped 72 bytes at block end
1389816625
block_tx 3 3
Skipped 73 bytes at block end
1389816648
block_tx 4 4
Skipped 72 bytes at block end
1389816671
block_tx 5 5
Skipped 71 bytes at block end
1389816681
block_tx 6 6
Skipped 72 bytes at block end
1389816682
block_tx 7 7
Skipped 71 bytes at block end
1389816697
block_tx 8 8
Skipped 73 bytes at block end
1389816710
block_tx 9 9
Skipped 71 bytes at block end
1389816731
block_tx 10 10
Skipped 73 bytes at block end
1389816740
block_tx 11 11
Skipped 71 bytes at block end
1389816749
block_tx 12 12
Skipped 73 bytes at block end
1389816752
block_tx 13 13
Skipped 72 bytes at block end
1389816758
block_tx 14 14
Skipped 72 bytes at block end
1389816763
block_tx 15 15
Skipped 72 bytes at block end
1389816789
block_tx 16 16
Skipped 72 bytes at block end
1389816793
block_tx 17 17
Skipped 72 bytes at block end
1389816797
block_tx 18 18
Skipped 72 bytes at block end
1389816843
block_tx 19 19
Skipped 73 bytes at block end
1389816847
block_tx 20 20
Skipped 72 bytes at block end
1389816852
block_tx 21 21
Skipped 73 bytes at block end
1389816852
block_tx 22 22
Skipped 72 bytes at block end
1389816859
block_tx 23 23
Skipped 71 bytes at block end
1389816864
block_tx 24 24
Skipped 71 bytes at block end
1389816863
block_tx 25 25
Skipped 72 bytes at block end
1389816863
block_tx 26 26
Skipped 71 bytes at block end
1389816875
block_tx 27 27
Skipped 72 bytes at block end
1389816889
block_tx 28 28
Skipped 73 bytes at block end
1389816893
block_tx 29 29
Skipped 72 bytes at block end
1389816903
block_tx 30 30
Skipped 71 bytes at block end
1389816914
block_tx 31 31
Skipped 73 bytes at block end
1389816918
block_tx 32 32
Skipped 72 bytes at block end
1389816937
block_tx 33 33
Skipped 71 bytes at block end
1389816939
block_tx 34 34
Skipped 73 bytes at block end
1389816942
block_tx 35 35
Skipped 72 bytes at block end
1389816945
block_tx 36 36
Skipped 71 bytes at block end
1389816952
block_tx 37 37
Skipped 72 bytes at block end
1389816969
block_tx 38 38
Skipped 73 bytes at block end
1389816979
block_tx 39 39
Skipped 72 bytes at block end
1389816982
block_tx 40 40
Skipped 71 bytes at block end
1389816986
block_tx 41 41
Skipped 72 bytes at block end
1389817005
block_tx 42 42
Skipped 73 bytes at block end
1389817005
block_tx 43 43
Skipped 72 bytes at block end
1389817025
block_tx 44 44
Skipped 72 bytes at block end
1389817036
block_tx 45 45
Skipped 73 bytes at block end
1389817038
block_tx 46 46
Skipped 73 bytes at block end
1389817044
block_tx 47 47
Skipped 72 bytes at block end
1389817049
block_tx 48 48
Skipped 73 bytes at block end
1389817051
block_tx 49 49
Skipped 71 bytes at block end
1389817060
block_tx 50 50
Skipped 71 bytes at block end
1389817064
block_tx 51 51
Skipped 72 bytes at block end
1389817067
block_tx 52 52
Skipped 73 bytes at block end
1389817090
block_tx 53 53
Skipped 72 bytes at block end
1389817120
block_tx 54 54
Skipped 72 bytes at block end
1389817122
block_tx 55 55
Skipped 73 bytes at block end
1389817137
block_tx 56 56
Skipped 72 bytes at block end
1389817147
block_tx 57 57
Skipped 72 bytes at block end
1389817153
block_tx 58 58
Skipped 72 bytes at block end
1389817157
block_tx 59 59
Skipped 71 bytes at block end
1389817166
block_tx 60 60
Skipped 73 bytes at block end
1389817178
block_tx 61 61
Skipped 73 bytes at block end
1389817182
block_tx 62 62
Skipped 72 bytes at block end
1389817186
block_tx 63 63
Skipped 71 bytes at block end
1389817196
block_tx 64 64
Skipped 73 bytes at block end
1389817203
block_tx 65 65
Skipped 72 bytes at block end
1389817196
block_tx 66 66
Skipped 73 bytes at block end
1389817218
block_tx 67 67
Skipped 71 bytes at block end
1389817217
block_tx 68 68
Skipped 72 bytes at block end
1389817226
block_tx 69 69
Skipped 73 bytes at block end
1389817237
block_tx 70 70
Skipped 71 bytes at block end
1389817248
block_tx 71 71
Skipped 73 bytes at block end
1389817262
block_tx 72 72
Skipped 72 bytes at block end
1389817262
block_tx 73 73
Skipped 71 bytes at block end
1389817288
block_tx 74 74
Skipped 71 bytes at block end
1389817288
block_tx 75 75
Skipped 72 bytes at block end
1389817294
block_tx 76 76
Skipped 71 bytes at block end
1389817305
block_tx 77 77
Skipped 73 bytes at block end
1389817307
block_tx 78 78
Skipped 72 bytes at block end
1389817310
block_tx 79 79
Skipped 73 bytes at block end
1389817314
block_tx 80 80
Skipped 71 bytes at block end
1389817317
block_tx 81 81
Skipped 71 bytes at block end
1389817336
block_tx 82 82
Skipped 73 bytes at block end
1389817352
block_tx 83 83
Skipped 72 bytes at block end
1389817367
block_tx 84 84
Skipped 72 bytes at block end
1389817381
block_tx 85 85
Skipped 72 bytes at block end
1389817384
block_tx 86 86
Skipped 72 bytes at block end
1389817399
block_tx 87 87
Skipped 73 bytes at block end
1389817401
block_tx 88 88
Skipped 73 bytes at block end
1389817423
block_tx 89 89
Skipped 73 bytes at block end
1389817428
block_tx 90 90
Skipped 72 bytes at block end
1389817433
block_tx 91 91
Skipped 71 bytes at block end
1389817441
block_tx 92 92
Skipped 72 bytes at block end
1389817463
block_tx 93 93
Skipped 73 bytes at block end
1389817465
block_tx 94 94
Skipped 72 bytes at block end
1389817465
block_tx 95 95
Skipped 72 bytes at block end
1389817469
block_tx 96 96
Skipped 72 bytes at block end
1389817466
block_tx 97 97
Skipped 72 bytes at block end
1389817478
block_tx 98 98
Skipped 71 bytes at block end
1389817493
block_tx 99 99
Skipped 73 bytes at block end
1389817505
block_tx 100 100
Skipped 72 bytes at block end
1389817513
block_tx 101 101
Skipped 72 bytes at block end
1389817536
block_tx 102 102
Skipped 71 bytes at block end
1389817531
block_tx 103 103
Skipped 71 bytes at block end
1389817547
block_tx 104 104
Skipped 73 bytes at block end
1389817568
block_tx 105 105
Skipped 71 bytes at block end
1389817583
block_tx 106 106
Skipped 72 bytes at block end
1389817597

PS: this is for yacoin.

[lunalauncher]

When no-statistics Abe starts working on Bitcoin blocks in late Oct 2013, it is showing a IndexError: string index out of range. Retrying does not help the problem. Any ideas how to solve this?

Code:

# python -m Abe.abe --config abe-my.conf --commit-bytes 1000000 --no-serve;

no chain_id
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00088.dat
Exception at 38408358
Failed to catch up {'blkfile_offset': 38175818, 'blkfile_number': 100088, 'chain_id': None, 'loader': None, 'dirname': '/root/.bitcoin', 'id': Decimal('1')}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1786, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2044, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2174, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1009, in import_block
    tx['tx_id'] = store.import_tx(tx, pos == 0)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1351, in import_tx
    pubkey_id = store.script_to_pubkey_id(txout['scriptPubKey'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 1723, in script_to_pubkey_id
    for opcode, data, i in deserialize.script_GetOp(script):
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 247, in script_GetOp
    nSize = ord(bytes[i])
IndexError: string index out of range

[John Tobey]

When no-statistics Abe starts working on Bitcoin blocks in late Oct 2013, it is showing a IndexError: string index out of range. Retrying does not help the problem. Any ideas how to solve this?

I've just copied a fix from the master branch that I think handles this.  Please pull and retry.  Thanks.

[lunalauncher]

I've just copied a fix from the master branch that I think handles this.  Please pull and retry.  Thanks.

Thank you for the quick update! I pulled the fix and reran Abe but the same error still appears.

Code:

remote: Counting objects: 3, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From https://github.com/bitcoin-abe/bitcoin-abe
   6049d56..3eb0032  no-statistics -> origin/no-statistics
Updating 6049d56..3eb0032
Fast-forward
 Abe/deserialize.py |   24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

[John Tobey]

Thank you for the quick update! I pulled the fix and reran Abe but the same error still appears.

If the error is still at

Code:

File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 247, in script_GetOp

then Python is using the old, installed version.  Either reinstall Abe or adjust PYTHONPATH to use the new code.

[lunalauncher]

Thank you for the quick update! I pulled the fix and reran Abe but the same error still appears.

If the error is still at

Code:

File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 247, in script_GetOp

then Python is using the old, installed version.  Either reinstall Abe or adjust PYTHONPATH to use the new code.

It works, thank you so much.

[jeffmthomas]

I have been having a heck of a time getting this to work with dogecoin.  I have downloaded the blockchain and rescanned.  Here is my abe.conf:

Code:

dbtype sqlite3
connect-args abe.sqlite
upgrade
port 7000

datadir = [{
        "dirname": "/home/ubuntu/.dogecoin",
        "chain":   "Dogecoin",
        "code3":   "DOGE",
        "address_version": "\u001e" }]

I have applied the fix located at: https://bitcointalk.org/index.php?topic=22785.msg750417#msg750417

This is the error I am getting when I run:

Code:

python -m Abe.abe --config abe.sqlite.conf --commit-bytes 100000 --no-serve

Code:

Skipped 95 bytes at block end
block_tx 1170 1
Skipped 95 bytes at block end
block_tx 1171 1
Skipped 95 bytes at block end
block_tx 1172 1
Skipped 95 bytes at block end
block_tx 1173 1
Skipped 96 bytes at block end
block_tx 1174 1
Skipped 95 bytes at block end
block_tx 1175 1
Skipped 95 bytes at block end
block_tx 1176 1
Skipped 95 bytes at block end
block_tx 1177 1
Skipped 96 bytes at block end
block_tx 1178 1
Skipped 95 bytes at block end
block_tx 1179 1
Skipped 96 bytes at block end
block_tx 1180 1
Skipped 96 bytes at block end
block_tx 1181 1
Skipped 95 bytes at block end
Exception at 235523
Failed to catch up {'blkfile_number': 100000, 'dirname': u'/home/ubuntu/.dogecoin', 'chain_id': 8, 'id': 1, 'blkfile_offset': 208572}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2422, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2475, in catch_up_dir
    store.import_blkdat(dircfg, ds, filename[0])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2596, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2630, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 90, in parse_Transaction
    for i in xrange(n_vout):
OverflowError: Python int too large to convert to C long

[John Tobey]

Code:

block_tx 1181 1
Skipped 95 bytes at block end
Exception at 235523
Failed to catch up {'blkfile_number': 100000, 'dirname': u'/home/ubuntu/.dogecoin', 'chain_id': 8, 'id': 1, 'blkfile_offset': 208572}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2422, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2475, in catch_up_dir
    store.import_blkdat(dircfg, ds, filename[0])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2596, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2630, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 90, in parse_Transaction
    for i in xrange(n_vout):
OverflowError: Python int too large to convert to C long

Most likely the transaction format differs from Bitcoin's.  NovaCoin and CryptoCash also changed their transaction format, and Abe/Chain.py accommodates them (by passing has_nTime in PpcPosChain.ds_parse_transaction).  If Doge shared their format, it could do the same, but it seems different, since the default parser got to Block 1181.  It is a matter of translating the C++ definition of CTransaction into Python.

[luckytiger]

Oke so i have managed to setup, my Abe blockexplorer.
will release it soon and its newly design after i have completed my work,
but i have this problem that i think its extremly slow.

I didnt want to host it on some server for 5 to 50$ a month
but hosted it on my pc trough a VM running ubuntu 12.04.

now my machine specs are
CPU: 6 core 3.4 ghz
RAM(dedicated to VM): 12GB 1800mhz
HDD: 200GB

now i think my problem lays with the internet connected since i am not directly connected to the router,
what means i use wifi of about 20mbs instead of the 100mb i should get from my fiber wire.

any tips or idea's on this, on how to make the website run faster.
or is it just abe that is slow?

also another question,
What would be faster, using one DB for all the blocks
or each chain one DB
LT
 

[clinraus]

are u accessing the database with your own queries?

[luckytiger]

are u accessing the database with your own queries?

no not my own queries, i dont know if your replying on this topic as a person that uses Abe, or someone that is intrested.

i apriciate both ,but to explain abe uses sql queries to recall data (blocks, hashes etc)
Now the blocks of all chains get put in one table of blocks and then the database identifies those blocks by their chain number(or something else not sure, but this is the most logic in my eyes)

now if you have 20 chains running next too each other and giving every block a different id, so their can be: block id=1 from doge, block=2 from bitcoin and block=5 from LiteCoin. and now i have just 2 chains one of 700k and one of 150k blocks, 2 of the biggest chains out there. but still the database has to check if not one of those 150k blocks is lost in those 700k blocks since i mybe wouldnt always start that wallet.

so thats my problem i might be wrong but if your going to run 20 coins or more(what i am planning) you can get yourself in deep shit since the queries has to look trough 1-5mil blocks or more if you have 20 big chains running.

so thats why i think of a db for every chains, since your mostly searching for 1 block within a chain than looking within 200k is easyer than looking within 5 mil. since it has only one critiria(block_id) instead of 2 (block_id+ chain_id)

ahh well i have just started to learn that stuff at uni about DB, atleast in the sence of going deeper, so anything i might just say is just complete BS.

thxn upfront, for any advice your going to give.

LT