[ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff

[MORA]

Got this after MySQL was restarted, maybe while Abe was using it.

Code:

Failed to catch up {'blkfile_number': 1, 'dirname': '/home/bitcoin/.bitcoin', 'chain_id': None, 'id': Decimal('1'), 'blkfile_offset': 945735848}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2141, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2162, in catch_up_dir
    store.import_blkdat(dircfg, ds)
  File "Abe/DataStore.py", line 2277, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1479, in import_block
    tx['tx_id'] = store.import_and_commit_tx(tx, pos == 0)
  File "Abe/DataStore.py", line 1866, in import_and_commit_tx
    tx_id = store.import_tx(tx, is_coinbase)
  File "Abe/DataStore.py", line 1798, in import_tx
    store.intin(tx['lockTime']), len(tx['tx'])))
  File "Abe/DataStore.py", line 403, in sql
    store.cursor.execute(cached, params)
  File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 166, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35, in defaulterrorhandler
    raise errorclass, errorvalue
IntegrityError: (1062, "Duplicate entry '1474' for key 'PRIMARY'")
ROLLBACK

1474 seems to refer to the tx table, which is ofcause at a much higher id than that (2.2m+)

Any suggestion as how to reset the id Abe will use ?
I will look more into it, hopefully getting it working again



[UPDATE] Ran the SQL to reset seq numbers from last page, and its running again.
Seems to be happy, at least no errors

[1714232249]

1714232249

[1714232249]

1714232249

[1714232249]

1714232249

[1714232249]

1714232249

[tiker]

Removed my previous post.. figured it out...  now I need to find a reliable way to determine what the address_version is...

[tiker]

Also, is there any way to setup Abe so that the web process doesn't trigger an update of data from the blk* files?  My current testing environment isn't exactly the latest hardware and if there new blocks to import for multiple chains at once it times out and spits back a 500 internal error page.  I've got a cron job setup that updates on a regular basis.

[tiker]

Is there an easy way to remove a block chain from Abe?

I tried to bring in Coiledcoin to play with but it didn't seem to work. Addresses show as "UNKNOWN" and transaction hashes are messed up.. then I noticed the bold text from the original announcement https://bitcointalk.org/index.php?topic=56675.0 so I'm looking for an uninstall option for that chain.  

That or a way to make it work with Abe.

[John Tobey]

@tiker

Regarding address_version in datadir configuration, I've added the following documentation:

For address_version, if you have a valid address for the new chain, append it to http://abe.john-edwin-tobey.org/q/decode_address/. For example, Testnet address mgnQ32RSjvmTLB3jVZ9L2xUTT512cCX9b8 gives
http://abe.john-edwin-tobey.org/q/decode_address/mgnQ32RSjvmTLB3jVZ9L2xUTT512cCX9b8, which shows 6f:0de3da453bfd284cd1c94902dbb9bc28bbed139f. Take the part to the left of the colon ( (6f for Testnet) and replace "XX" with it in "\u00XX" ("\u006f" for Testnet) That is the value for address_version in the config file's JSON format.

address_version comes from the first byte of the input to SHA256 used in address computation. In Bitcoin as of this writing, this information is in src/base58.h:

    enum
    {
        PUBKEY_ADDRESS = 0,
        SCRIPT_ADDRESS = 5,
        PUBKEY_ADDRESS_TEST = 111,
        SCRIPT_ADDRESS_TEST = 196,
    };

The byte is 111 for Testnet and 0 for regular Bitcoin. You would translate byte 111 to a JSON string as follows: 111 = '6f' (hexadecimal). In JSON, a 1-byte string is encoded as "\u00XX" where XX are the hex digits. So Testnet would be "\u006f".

If you get the wrong address_version value, everything will work except for address display. You could look up addresses, but they would appear different on web pages.

To prevent the web process from reading blk*, I give it an empty directory in datadir.

To remove a chain would be a nice feature.  If I remember right, the steps are:

1. Find chain_id in the chain table.  (SELECT chain_id FROM chain WHERE chain_code3='CLC')
2. Delete from chain_candidate where chain_id = ?(value found in #1)
3. Delete from datadir where chain_id = ?(value found in #1)
4. Delete from chain where chain_id = ?(value found in #1)

This does not delete the chain's block data, which is harmless apart from taking space.  You could delete all rows from block, block_tx, tx, txin, block_txin, txout, and pubkey that relate only to the deleted chain.  That would be a nice function to have.

[tiker]

Sweet, thanks!

That'll help my 3 box setup - 1 for clients to download chains and import to db, 1 for the db, and 1 for the web interface. 

For now I think I'm going to leave CLC and try not to remove it.  Once everything is up and running decently I'm going to see if I can figure out the addressing and update Abe to support CoiledCoin.  I'll post the code for you to add if you want but don't expect it to happen any time soon as I'm still learning things.

[andrehorta]

Dears

Why when i use bitcoin-abe (http://abe.john-edwin-tobey.org/), is showing the message:

Address 14K9E8xNBgn1ZomQYPcUVXbmnW2CjHkQCk

Address not seen on the network.

After i install the bitcoin-abe on my server, the same message is show: Address not seen on the network.

This address is of my wallet on bitcoin client official. At bitcoin explorer this addess is showing corret.

Please, help me to understand!

[John Tobey]

Why when i use bitcoin-abe (http://abe.john-edwin-tobey.org/), is showing the message:

Address 14K9E8xNBgn1ZomQYPcUVXbmnW2CjHkQCk

Address not seen on the network.

That site is now six days behind, and your address appeared on the network only after the site stopped updating.  I have messaged the site maintainer but had no response yet.  If you (or anyone) would like to host the site with an up-to-date block chain and get free monitoring from me via PM I will happily switch the DNS record to your server.

After i install the bitcoin-abe on my server, the same message is show: Address not seen on the network.

This address is of my wallet on bitcoin client official. At bitcoin explorer this addess is showing corret.

Please, help me to understand!

On your server's home page, what block number does it show for BTC?  Are you running the latest bitcoind?  Can you post the last few lines of Abe's output?

[tiker]

Checked it against my install and you're showing with 0.006 BTC.

http://blockexplorer.funkymonkey.org/address/14K9E8xNBgn1ZomQYPcUVXbmnW2CjHkQCk

(It's an older box so it's quite slow on pages with lots of addresses to hash out.  Give it time. )

[andrehorta]

Why when i use bitcoin-abe (http://abe.john-edwin-tobey.org/), is showing the message:

Address 14K9E8xNBgn1ZomQYPcUVXbmnW2CjHkQCk

Address not seen on the network.

That site is now six days behind, and your address appeared on the network only after the site stopped updating.  I have messaged the site maintainer but had no response yet.  If you (or anyone) would like to host the site with an up-to-date block chain and get free monitoring from me via PM I will happily switch the DNS record to your server.

After i install the bitcoin-abe on my server, the same message is show: Address not seen on the network.

This address is of my wallet on bitcoin client official. At bitcoin explorer this addess is showing corret.

Please, help me to understand!

On your server's home page, what block number does it show for BTC?  Are you running the latest bitcoind?  Can you post the last few lines of Abe's output?

OK, i will prepare my server to make visible for everyone.

[andrehorta]

How can I call a function like: getreceivedbyaddress???

[notawake]

How can I call a function like: getreceivedbyaddress???

For some API calls such as getreceivedbyaddress, you have to specify which block chain (Bitcoin, Namecoin, etc.) to query in the URL. Example usage for Bitcoin:

Code:

https://hostname/abe/chain/Bitcoin/q/getreceivedbyaddress/1HZRowLTkmF6M8V11vj2dmVf2r9VK7dVfo

Namecoin example:

Code:

https://hostname/abe/chain/Namecoin/q/getreceivedbyaddress/MxT6tjJZcYZeWUGNgsAMxJ7wpDgKMdub4y

Other API calls can be used for addresses on any block chain. Example:

Code:

https://hostname/abe/q/addresstohash/1HZRowLTkmF6M8V11vj2dmVf2r9VK7dVfo

The following URL shows the API calls currently supported by the Abe installation and provides links to documentation for each API call.

Code:

https://hostname/abe/q

[andrehorta]

OK, thank you.

For my application, i need to know how can i make selects on the database to get some information, like:

How much some address send to another addres?

Hom many transactions was done between address? Filter by date and value?

Thank you a lot!

I´m making a opensource and free website to use bitcoin abe (in english, spanish, portugues and chinese)

[John Tobey]

andrehorta,

I appreciate your goal!

To extend Abe with new page types and HTTP-API functions, I suggest you follow the model of ecdsa.org.  See contrib/ecdsa.py for sample code.  It subclasses Abe and defines methods named with a "handle_" prefix.  Abe calls handle_* when it gets a request for /*/... .  Unfortunately, the structure of the "page" argument is not currently documented and may change, so you will have to figure it out from the existing handlers in abe.py and ecdsa.py.  Likewise, Abe maps /q/* and /chain/.../q/* to the q_* method with "page" and "chain" arguments; chain is None when /chain/ does not appear in the URL.

Pasted from doc/FAQ.html:

Where in the database are bitcoin addresses?

A bitcoin address is an encoding of the hash of the public part of a keypair in someone's wallet. Abe stores only the public key hash, in pubkey.pubkey_hash. Abe converts hash values to bitcoin addresses as needed using the hash_to_address function in abe.py.

To find an address history in the database, you must first convert the address to a public key hash and look up its pubkey_id in the pubkey table.  As an example, the SQL for /address/ pages is in Abe/abe.py: handle_address.  pubkey_id links to the txout table, where txout_value is the amount received.  Abe internally uses units of 1 satoshi (.00000001 BTC), so it would store 1 BTC as txout_value=100000000 (1e8).

When money leaves an address, there will be a row in txin referencing an older txout row (via txin.txout_id) where txout.pubkey_id refers to the address.

To find when transactions entered the block chain, you must follow the joins from txin/txout to tx, block_tx, block, and chain_candidate.  Abe stores blocks that are on side-chains, and you should normally ignore those by restricting results to where chain_candidate.in_longest=1.  Abe is designed to support multiple currencies in one database, so you should also restrict by chain_id, where the values are listed in the chain table, and BTC is chain_id=1 by default.  block.block_nTime is the time recorded in the block header in units of seconds since 1 Jan 1970 UTC.

I hope this helps you get started.

[andrehorta]

OK, thank you! I understand.

How can i convert the address to a public key hash?

[John Tobey]

OK, thank you! I understand.

How can i convert the address to a public key hash?

abe.py: decode_check_address or decode_address.  They return two values: address_version and pubkey_hash.  The "version" should be "\0" for BTC.

Also there is base58.py: bc_address_to_hash_160 returning only the hash.

Here's the algorithm in pseudocode: https://en.bitcoin.it/wiki/Protocol_specification#Addresses

[andrehorta]

Why it´s not working?

I´m whating for (e5ec4ef0b857699ca315ea0a05645691a92369ff) but i´m seeing 5eFYXDVERjk3BfotTnR1SA5BD4sTzWTwMB7tTTbcDYU7

string sSourceData;
            byte[] tmpSource;
            byte[] tmpHash;
            string final;
            sSourceData = "1MxipvUHfKaZzWVrCGhrLKVPtNxwESRoq2";

            tmpSource = BitCoinSharp.Base58.Decode(sSourceData);
           
            System.Security.Cryptography.HMACSHA256 h = new HMACSHA256();
            tmpHash = h.ComputeHash(tmpSource);
            final = BitCoinSharp.Base58.Encode(tmpHash);

            System.Security.Cryptography.RIPEMD160 r160 = new RIPEMD160Managed();
            byte[] tmpHash2 = r160.ComputeHash(tmpHash);           
            final = BitCoinSharp.Base58.Encode(tmpHash);

[andrehorta]

Dears

How can i get the Date and Time of transaction?

Thank you

[John Tobey]

Sorry, I think I confused you by linking to the algorithm going from hash to address.

To go from address to hash, you would reverse it: decode base58, discard last 4 bytes (checksum), discard first byte (version), and hex-encode the result.

If you want the date/time a transaction appeared in the chain, it is block.block_nTime (seconds since 0:00 1 Jan 1970 UTC).  You find the block row by joining tx to block_tx to block to chain_candidate, restricting in_longest=1 and chain_id=1 (for BTC).  Note that date is not a property of transactions.  (tx has a lock_time but it is unused.)  A transaction doesn't say when it was signed, and it doesn't know when it will become part of the chain.  The joins could produce different results at different times because of reorganizations, which change the value of chain_candidate.in_longest.

[someguy123]

Scanning the block chain... but not starting the web server til it thinks its done...
There's a huge problem with that, it seems on a lot of systems, ABE takes a LOOONG time to scan blockchains of various whateverCoins, and during that time, it refuses to start the web server til it's done =_=
It'd be better if it would start the web server FIRST so that the users of an ABE block explorer don't have to wait HOURS for ANYTHING AT ALL to work, if the admin decides to add a new block chain...