[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[molecular]

What happens if there is a catch-up triggered by request A, then request B comes in?

B tries to "help" A catch up.  Which would be okay if the loader code were free of bugs.  Probably the easiest fix (when I--or someone--has time) is to enforce single-threaded loading with a database lock.

Your suggested workaround (one instance with --no-load, one with --no-serve) works like a charm and solves the problem.

Haven't had a problem since, thank you 1000!

I agree using some db-lock to only ever have one "db updater" would likely solve it, too.

[1713256934]

1713256934

[1713256934]

1713256934

[unick]

I modified the default template and I want ABE to load variables on page refresh (current price of coin for instance).

I doesn't refresh, I have to do a sevice apache2 reload or service apache2 restart so ABE can fetch the correct value for the variable.

Any clue on where I should define my variables so ABE can look them up on page load rather than on service load?  I'm on FCGI config obviously.

thanks

[John Tobey]

I modified the default template and I want ABE to load variables on page refresh (current price of coin for instance).

I doesn't refresh, I have to do a sevice apache2 reload or service apache2 restart so ABE can fetch the correct value for the variable.

Any clue on where I should define my variables so ABE can look them up on page load rather than on service load?  I'm on FCGI config obviously.

thanks

With the current template support, I suggest inserting a <script> that fetches the dynamic content with Ajax.  But if you are working on the front end, please note the big facelift in the works: https://github.com/bitcoin-abe/bitcoin-abe/issues/81

[unick]

I modified the default template and I want ABE to load variables on page refresh (current price of coin for instance).

I doesn't refresh, I have to do a sevice apache2 reload or service apache2 restart so ABE can fetch the correct value for the variable.

Any clue on where I should define my variables so ABE can look them up on page load rather than on service load?  I'm on FCGI config obviously.

thanks

With the current template support, I suggest inserting a <script> that fetches the dynamic content with Ajax.  But if you are working on the front end, please note the big facelift in the works: https://github.com/bitcoin-abe/bitcoin-abe/issues/81

thanks,

Indeed I am working on the front end design too  I really like what geekz has done... good job!

Here's what I've done so far http://hbn.blockx.info/

Next step would be to implement AJAX for faster loading and dynamic content refresh... I'll have a closer look at what geekz has done since it seems he already did that part.

again great work

[unick]

I saw that Geekz showed on the first page of the block explorer the latest transactions. However, it's the latest CONFIRMED transaction (AKA included in blocks). But I believe blockchain.info shows the latest transactions NOT YET included in the block chain.

Since ABE reads the blk0001.dat file, it can only read the confirmed transactions.  Does anyone know how blockchain.info achieves showing the latest posted transactions on the network?

[ShadesOfMarble]

If you have bitcoind running, you can read pending tx from the mempool.

[Jouke]

New Abe feature: Standard Bitcoin multisig and pay-to-script-hash (P2SH) support is in the master branch, thanks to Jouke's generous sponsorship.

It was my comany (Bitonic) that sponsored these.

Since ABE reads the blk0001.dat file, it can only read the confirmed transactions.  Does anyone know how blockchain.info achieves showing the latest posted transactions on the network?

Abe has a option to read the mempool from RPC, so it would show unconfirmed transactions as well.

[n00bnoxious]

For anybody who's interested I've fixed the problems dasource was having with porting ABE to X11. It's pretty transparent what needs doing - you mainly need the xcoin_hash python module and to add a chain type that does the X11 hashing. Check out my Github repo for the changes... I'll issue a pull to officially support it by making a less cut down branch.

EDIT: Pull request issued

[unick]

Since ABE reads the blk0001.dat file, it can only read the confirmed transactions.  Does anyone know how blockchain.info achieves showing the latest posted transactions on the network?

Abe has a option to read the mempool from RPC, so it would show unconfirmed transactions as well.

How would that be setup?

[John Tobey]

Since ABE reads the blk0001.dat file, it can only read the confirmed transactions.  Does anyone know how blockchain.info achieves showing the latest posted transactions on the network?

Abe has a option to read the mempool from RPC, so it would show unconfirmed transactions as well.

How would that be setup?

See the comments above "default-loader" in abe.conf.

[publicjud]

Any help with the following error when trying to run abe from apache?

Code:

<class 'sqlite3.ProgrammingError'>: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140731607066368 and this is thread id 140731448555264
      args = ('SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264',)
      message = 'SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264'

Screen shot of the error when I try to visit abe via apache web browser fastcgi.

[unick]

Any help with the following error when trying to run abe from apache?

Code:

<class 'sqlite3.ProgrammingError'>: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140731607066368 and this is thread id 140731448555264
      args = ('SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264',)
      message = 'SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264'

Screen shot of the error when I try to visit abe via apache web browser fastcgi.

can you paste your config file for your fast cgi setup?

I've setup the default loader to

Code:

default-loader=no-serve

I make the fastcgi script only read the database and I use another instance of abe to update the database.

[publicjud]

Any help with the following error when trying to run abe from apache?

Code:

<class 'sqlite3.ProgrammingError'>: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140731607066368 and this is thread id 140731448555264
      args = ('SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264',)
      message = 'SQLite objects created in a thread can only be u...31607066368 and this is thread id 140731448555264'

Screen shot of the error when I try to visit abe via apache web browser fastcgi.

can you paste your config file for your fast cgi setup?

I've setup the default loader to

Code:

default-loader=no-serve

I make the fastcgi script only read the database and I use another instance of abe to update the database.

Code:

# Config file for Abe.

# dbtype and connect-args are required.
# If not configured to run under FastCGI, host OR port is required.

# "NAME=VALUE" and "NAME VALUE" are equivalent.  Just "NAME" sets
# value to True.  Values of the form "..." or [...]  or {...} are
# parsed as extended JSON, as are the values true, false, and null.
# Abe extends JSON syntax to allow strings to contain control
# characters (tab, LF, and CR) and to allow comments (from # to end of
# line) between tokens.

# "NAME += VALUE" appends VALUE to an array or updates an object
# defined by NAME.  Hyphen in NAME is read as underscore, so
# "connect_args" and "connect-args" are synonymous.  Later values
# override earlier ones (unless using "+=").

# All values may be passed as command-line arguments by preceding NAME
# with "--".  Example: "python -m Abe.abe --upgrade --dbtype=sqlite3
# --connect-args=abe.sqlite --port 2750".

# dbtype is the name of a Python module.  connect-args are arguments
# to pass to the module's "connect" function.  Consult the module's
# documentation for values.  The user needs permission to create
# tables.  Consult your database documentation on how to configure a
# database.  See also README.txt.

# SQLite3 example:
dbtype = sqlite3
#connect-args = /tmp/abe/abe.sqlite
#    dbtype sqlite3
    connect-args /usr/local/bin/abe/abe.sqlite
    connect-args check_same_thread=False
    int-type str
    upgrade
#    port 2750
# Oracle example:
#dbtype=cx_Oracle
#connect-args=user/pass@SID

# ODBC examples.  Don't use the same schema via ODBC and non-ODBC!
#dbtype = pyodbc
#connect-args = DSN=Abe;PWD=B!tc0iN;
#connect-args = DRIVER={ORACLE};DB=ORCL;UID=abe;PWD=Bitcoin;
#connect-args = DRIVER={PostgreSQL ANSI};Database=abe;

# Workaround for ODBC drivers that begin in autocommit mode:
#connect-args {"":"DRIVER={PostgreSQL ANSI};Database=abe;","autocommit":false}

# IBM DB2 example:
#dbtype = ibm_db_dbi
#connect-args {"dsn":"DATABASE=abe;UID=db2inst1;PWD=B!tCo1N","conn_options":{"102":0}}

# Specify port and/or host to serve HTTP instead of FastCGI:
#port 8080
#host 107.170.94.10

# Specify no-serve to exit immediately after importing block files:
#no-serve

datadir += [{
        "dirname": "/root/.blackcoin",
        "chain": "BlackCoin",
        "code3":"BC",
        "address_version":"\u0019",
        "policy": "NovaCoin"  # ... but use the "NovaCoin" loader.
      }]


#
# The binary-type setting affects only the first run, where Abe
# creates its tables.
#binary-type hex

# Some databases have trouble with the large integers that Abe uses
# for statistics.  Setting int-type=str causes Abe to pass certain
# integers to the database as strings and cast them to floating point
# in SQL expressions that calculate statistics.  Try this if SQLite
# complains "OverflowError: long too big to convert".
#int-type str

# URL component for static content: logos, style sheets.
#static-path =

# Filesystem location of static content, if served by Abe.
#document-root = Abe/htdocs

# Uncomment "auto-agpl" to add a "Source" link to each page pointing
# to a "/download" URL that streams the directory containing abe.py
# and all subdirectories as a compressed TAR archive.  This exposes
# files outside of the htdocs directory to the client, so use it with
# caution.
#auto-agpl

# Directory name and tarfile name prefix for auto-agpl source
# download.
#download-name = abe


# Save blocks to the database after reading this many bytes from a
# file.  Abe may run faster with a higher value, for example 10000,
# while loading lots of data with a single process.  Non-zero values
# can lead to errors when two processes load transaction data
# simultaneously.
#commit-bytes = 0

# "rescan" causes Abe to search all block files for new blocks.  This
# can take several minutes on a large chain, longer if many of the
# blocks are not already in Abe's database.  You might want to do this
# if Abe becomes stuck at an old block number: new blocks are loaded
# but not attached to the chain.  You must run once with "rescan"
# after switching to another copy of the block files.
#rescan

# For displaying short links, base-url overrides the first part of the
# URL.  This does not affect the link target, only the visible text.
# The value must include the trailing slash (/) if applicable.
#base-url = http://abe.example.org/

# History pages of addresses with more than this many receipts will be
# not be shown.  This protects against denial of service.  Use -1 for
# no limit.  This also limits the total inputs shown by
# /unspent/ADDR|ADDR|...
#address-history-rows-max 1000

# Argument to logging.config.dictConfig.  Requires Python 2.7 or later.
# http://docs.python.org/library/logging.config.html#logging-config-dictschema
#logging = {
#    "version":1,
#    "handlers": {
#        "console": {
#            "class": "logging.StreamHandler",
#            "formatter": "full",
#            "level": "DEBUG"}},
#    "formatters": {
#        "full": {
#            "format": "%(asctime)s [%(process)d:%(threadName)s] %(name)s %(levelname)s - %(message)s"}},
#    "root": {
#        "handlers": ["console"],
#        "level": "DEBUG"}}

# --log-sql logs all queries.  This is quite verbose, so it is disabled
# by default.
#log-sql

# Create and use the abe_firstbits table.  This affects only the first
# run, where Abe creates its tables, or the first run after an upgrade
# to firstbits-enabled Abe.  This is disabled by default.
#use-firstbits

# shortlink-type=firstbits constructs address short links using
# firstbits.  This is the default when use-firstbits is true.
# Otherwise, shortlink-type is a number of address characters to use.
# Fewer characters result in more collisions, where the page shows a
# list of matches instead of redirecting to the address history.
#shortlink-type 10

# keep-scriptsig=false prevents storage and display of transaction
# input signature scripts (scriptSig) and sequence numbers.  This
# reduces the database size by about 20%.
#keep-scriptsig

# Add transactions to the database.  The genesis transaction is
# unavailable via RPC and must be specified to enable full loading
# over RPC.  See Abe/genesis_tx.py.
#import-tx += 01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4d04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73ffffffff0100f2052a01000000434104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac00000000

# Use default-loader=blkfile to force Abe to load data by scanning
# blockfiles.  Use default-loader=rpc to force the use of RPC
# with a running bitcoind (or *coin daemon).  This requires:
#
#     * A readable bitcoin.conf with the RPC username and password
#
#     * HTTP access to bitcoind (SSL is not yet supported.)
#
#     * Bitcoin 0.8 or newer with the -txindex option.  (Run bitcoind
#       once with -reindex if you previously ran it without -txindex.)
#
#     * A "datadir" option with "chain":"Bitcoin" (or other chain
#       name)
#
#     * The "import-tx" option specifying the genesis transaction, if
#       "bitcoid getrawtransaction" does not return it and it is not
#       in Abe/genesis_tx.py.  (Bitcoin's genesis transaction is in
#       Abe/genesis_tx.py.  Otherwise, it requires manually parsing
#       the first blockfile.)
#
# By default, Abe tries RPC and falls back to blkfile if that fails.
# Use "rpc,blkfile" to specify this behaviour across upgrades, or
# "default" to let Abe choose.  The "loader" specified in a "datadir"
# entry takes precedence over "default-loader".
#
default-loader=no-serve

[unick]

Can you connect with the built-in webserver?

did you follow this guide ? https://github.com/bitcoin-abe/bitcoin-abe/blob/master/README-FASTCGI.txt

you could try to make a new config file with only these parameters

Code:

default-loader=no-serve
dbtype = sqlite3
connect-args {"user":"YOURDBUSER","db":"YOURDBNAME","passwd":"YOURDBPASSWORD"}
datadir=[]

and make sure you follow the guide in the previous link.  and that you point your FastCGI script to this newly created config file

check line 92 on that link, where it says

Code:

-config /home/USER/abe.conf --static-path static/ --watch-pid="$1"

replace /home/USER/abe.conf with the path of your new config file

essentially, you want that your fastcgi script reads the database and prints out it's content.  you will need to run another instance of ABE to update the database.

[geekz]

Hi Everyone,
Check out the new design of Abe at http://ancblockchain.com
You can see the new branch in github at https://github.com/bitcoin-abe/bitcoin-abe/tree/geekz-patch-1

currently,  @jtobey is reviewing the code and soon it will be part of main ABE explorer

Feel free to provide any suggestions for improvements in the design And of course, You can also make donation if you liked the work and would like to see continued improvement. 

[hawaiicoin]

Anyone know how to have Abe show top wallet balances?  If not already built in, how could I query the mysql DB for this info?

[unick]

Anyone know how to have Abe show top wallet balances?  If not already built in, how could I query the mysql DB for this info?

No, Abe does not have this feature built in, and the addresses are not store in the database.

What you have is the public keys.  So the way to go would be to query the database for all the public keys, transform the public keys to the network address, then ask Abe to give you the balance for that specific address.

You do that in a loop and then show the results with descending by top address

[John Tobey]

Anyone know how to have Abe show top wallet balances?  If not already built in, how could I query the mysql DB for this info?

As unick says, it is not built in.  You can search this thread for "txout_value" to find a query that gives the amount received by an address ("pubkey").  A similar query (joining txin with txout on txout_id) can give the amount received.  Subtract to get the "balance" for that particular public key (which, you must know, is not the same as a wallet.dat file).  Use GROUP BY to put them all into a table and select from it ordering by balance.  Remember to limit the results to where chain_candidate.in_longest=1 in case your database contains side-chains.

[unick]

Abe is stuck on a block height... when I search for the block after, it gives me 3 choices and if I click on either of them... it crashes with this

Code:

<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'has_feature'
      args = ("'NoneType' object has no attribute 'has_feature'",)
      message = "'NoneType' object has no attribute 'has_feature'"

So I guess those are orphans and Abe doesn't know what to do with those (or witch one to chose)

My question would be, am I correct? how can this be fixed?

[Jimmy2011]

As my disk is not so big and I don't want to download further block chain data, so I changed the loader of Abe to "blkfile' from 'rpc', and I expected that the database won't increase. However, it seems Abe reloaded the blkfile into the database from the scratch and the block_tx number continued to increase though the block data was not updated. It did re-index the data into database according to the Abe browser that the block number just several thousands.

What's the problem? How can I recover to the right database?

Thanks.