[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[dotvibe]

need some help please,

1st, my address_version:

        PUBKEY_ADDRESS = 23, 
        SCRIPT_ADDRESS = 83,
        PUBKEY_ADDRESS_TEST = 111,
        SCRIPT_ADDRESS_TEST = 196,

and 2nd, my "magic number"

unsigned char pchMessageStart[4] = { 0x70, 0x35, 0x22, 0x05 };

Sorry, but i just seem to be too stoopid to figure this out! 

[danbi]

Trying to figure out how to create 'rich list' from the Abe database. Should be easy doable with SQL. Any ideas?

[roundrobin]

Trying to figure out how to create 'rich list' from the Abe database. Should be easy doable with SQL. Any ideas?

You don't need to store in a database if you don't want to, it doesn't make it any easier or harder. You can build an associative array with each unique address as key for quick counting. One entry for every unique address you find, count both in and outgoing and keep coinbase addresses separate to avoid confusion.

[dotvibe]

Hii,

Trying to set up ABE for an X13 Coin,

I've imported the hash module and installed it's python libraries, edited the Datastore.py and added the chain details with the address_version & magic number. The abe.conf is pointed to the correct datadir.

got as far as:

Code:

 
python -m Abe.abe --config abe.conf --commit-bytes 100000 --no-serve

this kicks out the following:

Code:

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/ubuntu/bitcoin-abe/Abe/abe.py", line 32, in <module>
    import DataStore
  File "Abe/DataStore.py", line 1076
    hashPrev = b['hashPrev']
           ^
IndentationError: expected an indented block

Help will be much appreciated. 

[gjhiggins]

... edited the Datastore.py

Code:

IndentationError: expected an indented block

I'll hazard a guess that it's a whitespace issue, mixed spaces and tabs or 8-space tabs, or something.

Cheers,

Graham

[dotvibe]

thanks gjhiggens,

the error occurs when this:

Code:

      
        # Get a new block ID.
        block_id = int(store.new_id("block"))
        b['block_id'] = block_id

        if chain is not None:
            # Verify Merkle root.
            if b['hashMerkleRoot'] != chain.merkle_root(tx_hash_array):
                raise MerkleRootMismatch(b['hash'], tx_hash_array)

        # Look for the parent block.
        hashPrev = b['hashPrev']
        if chain is None:
            # XXX No longer used.
            is_genesis = hashPrev == util.GENESIS_HASH_PREV
        else:
            is_genesis = hashPrev == chain.genesis_hash_prev

is changed to this:

Code:

     
        # Get a new block ID.
        block_id = int(store.new_id("block"))
        b['block_id'] = block_id

        if chain is not None:
            # Verify Merkle root.
            #if b['hashMerkleRoot'] != chain.merkle_root(tx_hash_array):
                #raise MerkleRootMismatch(b['hash'], tx_hash_array)

        # Look for the parent block.
        hashPrev = b['hashPrev']
        if chain is None:
            # XXX No longer used.
            is_genesis = hashPrev == util.GENESIS_HASH_PREV
        else:
            is_genesis = hashPrev == chain.genesis_hash_prev

in the DataStore.py (commenting out the Merkle root verification). strange.

[gjhiggins]

Comment out the entire check, as below.

Code:

     
        # Get a new block ID.
        block_id = int(store.new_id("block"))
        b['block_id'] = block_id

        # if chain is not None:
        #    # Verify Merkle root.
        #    if b['hashMerkleRoot'] != chain.merkle_root(tx_hash_array):
        #        raise MerkleRootMismatch(b['hash'], tx_hash_array)

        # Look for the parent block.
        hashPrev = b['hashPrev']
        if chain is None:
            # XXX No longer used.
            is_genesis = hashPrev == util.GENESIS_HASH_PREV
        else:
            is_genesis = hashPrev == chain.genesis_hash_prev

[dotvibe]

Ok Thanks,

I'm now at the point where it's trying to import the chain module:

Code:

  File "Abe/DataStore.py", line 840, in initialize
    chain = Chain.create(policy, **conf)
  File "Abe/Chain/__init__.py", line 21, in create
    mod = __import__(__name__ + '.' + policy, fromlist=[policy])
ImportError: No module named myX13coin

It's looking in the bitcoin-abe\Abe\Chain\ directory for the X13 chain python files, which are obviously not there. someone has graciously pointed me to an x13 hash module on github: https://github.com/mindfox/x13-hash. I ran the setup.py file, and it installed the python packages.

I still need the x13 hash .py and .pyc files to copy to ABE's \Chain\ directory. Then i can set a new policy fpr the coin that points to the correct hash, and hopefully the chain will import. Any directions on how to achieve this?

much appreciating the help so far!  

[gjhiggins]

Ok Thanks,

I'm now at the point where it's trying to import the chain module:

someone has graciously pointed me to an x13 hash module ... I ran the setup.py file, and it installed the python packages.

I still need the x13 hash .py and .pyc files to copy to ABE's \Chain\ directory.

1. General advice: you do not want to copy .pyc or .pyo files, they are machine-specific (byte-)compiled code, Python's equivalent of C's .o files. Your machine will generate its own from the Python source.

2. Save the code below as Abe/Chain/X13Chain.py

Code:

 Copyright(C) 2014 by Abe developers.

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Affero General Public License for more details.
# 
# You should have received a copy of the GNU Affero General Public
# License along with this program.  If not, see
# <http://www.gnu.org/licenses/agpl.html>.

from . import BaseChain

class X13Chain(BaseChain):
    """
    A blockchain that hashes block headers using the X13 algorithm.
    The current implementation requires the x13_hash module:
    https://github.com/mindfox/x13-hash
    """

    def block_header_hash(chain, header):
        import x13_hash
        return x13_hash.getPoWHash(header)

A quick comparison of the above code with the existing X11Chain.py code will be repaid by a better understanding of the programming skills used here 

Code:

ImportError: No module named myX13coin

Given the above, you can now move to importing X13Chain instead of myx13chain.

If your coin has PoS, you'll probably need to import and inherit from PpcPosChain. If so, save this code as Abe/Chain/X13PosChain.py and use “X13PosChain” as the policy.

Code:

# Copyright(C) 2014 by Abe developers.

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Affero General Public License for more details.
# 
# You should have received a copy of the GNU Affero General Public
# License along with this program.  If not, see
# <http://www.gnu.org/licenses/agpl.html>.

from .X13Chain import X13Chain
from .PpcPosChain import PpcPosChain

class X13PosChain(X13Chain, PpcPosChain):
    pass

HTH

Cheers

Graham

[dotvibe]

Thanks Graham, i'm making progress here,

everything seems to be loading now, except that when point my browser at the ip address, i get the Apache2 ubuntu default page.

maybe this has something to do with it:

after running python -m Abe.abe --config abe-my.conf, i get:

Abe initialized.
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
fixed path_info
Status: 301 Moved Permanently
Location: http://localhost/
Content-Type: text/html

<html><head><title>Moved</title></head>
<body><h1>Moved</h1><p>This page has moved to <a href="http://localhost/">http://localhost/</a></body>
<html><head><title>Moved</title></head>

anyway, i'm assuming abe is creating an html file somewhere, and i need to get Apache2 to point to it by default. From what i can see, this file should be in the /bitcoin-abe/Abe/htdocs directory, but there is no html file there.
So somewhere something has gone awry.

[gjhiggins]

anyway, i'm assuming abe is creating an html file somewhere, and i need to get Apache2 to point to it by default.

No, Abe generates the html on the fly, the htdocs directory holds common styling and graphics.

You'll probably find it easier to uncomment the section in the config file:

Code:

# Specify port and/or host to serve HTTP instead of FastCGI:
#port 2750
#host localhost

and make sure it works in vanilla mode first before settling down to enjoy the delights of FCGI param config.

Cheers

Graham

[dotvibe]

ok thanks,

i've done that, and now it seems be working from the abe end:

Code:

Abe initialized.
Listening on http://localhost:2750

but i'm getting a "webpage unavailable" when connecting to server's IP with my browser.

Though is probably an issue on the AWS instance side. Strange, i've pretty much opened all the necessary ports in the security groups settings.

[dotvibe]

nvm, got it going, thanks.

i'll try to figure out why the fastcgi isn't working a bit later.

[roundrobin]

nvm, got it going, thanks.

i'll try to figure out why the fastcgi isn't working a bit later.

you should give RPC ACE a try, it's not an advanced block explorer but it will let you explore the chain to see it's working and you can get it running in 30 second.

[CryptoBlox]

Hey John, great work.  Love the project!

I'm looking for some guidance on creating a SQL query to display a top 100 richlist for each coin that I have been using Abe for.  Looking at the code I'm having a difficult time wrapping my head around obtaining the information and also converting hashes to addresses all at the same time.  Does anyone have some examples they might share with me to grab the top 100 addresses descending by total balance?

[GrimDemon]

Hi! I want to load my chain form altocin. But all I've got is this:

Code:

ImportError: No module named Polcoin

What am I doing wrong?

My config:

Code:

dbtype MySQLdb
connect-args {"user":"abe","db":"abe","passwd":"pass"}
port 2750
host localhost
upgrade
datadir += [{
        "dirname":"/root/.polcoin",
        "chain":"Polcoin",
        "conf":"polcoin.conf",
        "policy":"Sha256Chain",
        "code3":"PLC",
        "address_version":"u0000",
        "magic":"u00a5u00072u0059u0082"
        }]
no-serve
commit-bytes = 10000

[roundrobin]

Hi! I want to load my chain form altocin.

What am I doing wrong?

My config:

Code:

dbtype MySQLdb
connect-args {"user":"abe","db":"abe","passwd":"pass"}
port 2750
host localhost
upgrade
datadir += [{
        "dirname":"/root/.polcoin",
        "chain":"Polcoin",
        "conf":"polcoin.conf",
        "policy":"Sha256Chain",
        "code3":"PLC",
        "address_version":"u0000",
        "magic":"u00a5u00072u0059u0082"
        }]
no-serve
commit-bytes = 10000

It's a bug in Abe, it doesn't use "policy" to load chain file like documentation say. you can clone a chain file in abe/chain and rename, or you can make a change in the database generated for you.

[roundrobin]

Hey John, great work.  Love the project!

I'm looking for some guidance on creating a SQL query to display a top 100 richlist for each coin that I have been using Abe for.  Looking at the code I'm having a difficult time wrapping my head around obtaining the information and also converting hashes to addresses all at the same time.  Does anyone have some examples they might share with me to grab the top 100 addresses descending by total balance?

The guy who makes RPC ACE has a PHP file for making a richlist without Abe, i've used it and it works but it can take a long while if the blockchain is huge with lots of transactions, it only talks over RPC no database so you have to do it from block #1 each time.

I tried this on an altcoin

Code:

87711 blocks ... #1000 (tx: 1018/60)   #2000 (tx:1244/260)   etc etc etc etc   #87000 (tx: 2067/1245)    done! 188731 transactions through 99781 unique addresses parsed

It took 9 minutes, Core 2 Xeon 3.2ghz.

[dotvibe]

Still trying to get fastcgi version of Abe going. MySQL Webserver version working ok.

/var/log/apache2/error.log output:

Code:

FastCGI: can't start server "/usr/lib/cgi-bin/abe.fcgi" (pid 12272), execle() failed: Exec format error
[Sat Jul 26 17:06:30.639430 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" (pid 12272) terminated by calling exit with status '255'
[Sat Jul 26 17:06:30.639439 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:32.637924 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:35.637950 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:38.637980 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:41.637940 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:44.637955 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:47.637932 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to$
[Sat Jul 26 17:06:50.638025 2014] [:warn] [pid 12164:tid 140647052720000] FastCGI: (dyna
FastCGI: can't start server "/usr/lib/cgi-bin/abe.fcgi" (pid 12272), execle() failed: Exec format error
mic) server "/usr/lib/cgi-bin/abe.fcgi" has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to 600 seconds

Abe_error.log output:

Code:

[Sat Jul 26 16:33:29.324356 2014] [core:info] [pid 4147:tid 139699054118656] [client 105.236.208.23:50415] AH00130: File does not exist: /usr/lib/cgi-bin/abe.fcgi/
[Sat Jul 26 16:34:34.916076 2014] [core:info] [pid 11986:tid 140280435025664] [client 105.236.208.23:50431] AH00130: File does not exist: /usr/lib/cgi-bin/abe.fcgi/
[Sat Jul 26 16:36:48.961275 2014] [fastcgi:error] [pid 12165:tid 140646891386624] [client 105.236.208.23:50478] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" aborted: (first read) idle timeout (30 sec)
[Sat Jul 26 16:36:48.961471 2014] [fastcgi:error] [pid 12165:tid 140646891386624] [client 105.236.208.23:50478] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/abe.fcgi"
[Sat Jul 26 16:37:04.785932 2014] [fastcgi:error] [pid 12166:tid 140646782281472] [client 105.236.208.23:50496] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" aborted: (first read) idle timeout (30 sec)
[Sat Jul 26 16:37:04.786144 2014] [fastcgi:error] [pid 12166:tid 140646782281472] [client 105.236.208.23:50496] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/abe.fcgi"
[Sat Jul 26 16:41:34.861125 2014] [fastcgi:error] [pid 12165:tid 140646882993920] [client 105.236.208.23:50577] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" aborted: (first read) idle timeout (30 sec)
[Sat Jul 26 16:41:34.861268 2014] [fastcgi:error] [pid 12165:tid 140646882993920] [client 105.236.208.23:50577] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/abe.fcgi"
[Sat Jul 26 17:05:37.867536 2014] [fastcgi:error] [pid 12165:tid 140646866208512] [client 105.236.208.23:51200] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" aborted: (first read) idle timeout (30 sec)
[Sat Jul 26 17:05:37.867692 2014] [fastcgi:error] [pid 12165:tid 140646866208512] [client 105.236.208.23:51200] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/abe.fcgi"
[Sat Jul 26 17:06:50.637914 2014] [fastcgi:error] [pid 12165:tid 140646857815808] [client 105.236.208.23:51228] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/abe.fcgi" aborted: (first read) idle timeout (30 sec)
[Sat Jul 26 17:06:50.638050 2014] [fastcgi:error] [pid 12165:tid 140646857815808] [client 105.236.208.23:51228] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/abe.fcgi"

clues?

[gjhiggins]

clues?

Yes, some pastebinned clues would be useful, the Apache fcgi config for one and the contents of abe.fcgi for another.

You could also try out some of the suggestions mooted by the Perl Monks in the top results of this search.

Cheers,

Graham


Edit: credit Perl Monks
Edit: Edit