Bitcoin Forum
December 08, 2016, 06:20:22 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 [34] 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 175765 times)
luckytiger
Member
**
Offline Offline

Activity: 96


View Profile
March 08, 2014, 01:47:24 AM
 #661

are u accessing the database with your own queries?

no not my own queries, i dont know if your replying on this topic as a person that uses Abe, or someone that is intrested.

i apriciate both ,but to explain abe uses sql queries to recall data (blocks, hashes etc)
Now the blocks of all chains get put in one table of blocks and then the database identifies those blocks by their chain number(or something else not sure, but this is the most logic in my eyes)

now if you have 20 chains running next too each other and giving every block a different id, so their can be: block id=1 from doge, block=2 from bitcoin and block=5 from LiteCoin. and now i have just 2 chains one of 700k and one of 150k blocks, 2 of the biggest chains out there. but still the database has to check if not one of those 150k blocks is lost in those 700k blocks since i mybe wouldnt always start that wallet.

so thats my problem i might be wrong but if your going to run 20 coins or more(what i am planning) you can get yourself in deep shit since the queries has to look trough 1-5mil blocks or more if you have 20 big chains running.

so thats why i think of a db for every chains, since your mostly searching for 1 block within a chain than looking within 200k is easyer than looking within 5 mil. since it has only one critiria(block_id) instead of 2 (block_id+ chain_id)

ahh well i have just started to learn that stuff at uni about DB, atleast in the sence of going deeper, so anything i might just say is just complete BS.

thxn upfront, for any advice your going to give.

LT
1481221222
Hero Member
*
Offline Offline

Posts: 1481221222

View Profile Personal Message (Offline)

Ignore
1481221222
Reply with quote  #2

1481221222
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481221222
Hero Member
*
Offline Offline

Posts: 1481221222

View Profile Personal Message (Offline)

Ignore
1481221222
Reply with quote  #2

1481221222
Report to moderator
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 10, 2014, 08:21:36 PM
 #662

What would be faster, using one DB for all the blocks
or each chain one DB

I guess you should use a separate database per chain.  It complicates the HTTP side but simplifies deleting a corrupt (or abandoned) chain and spreading an increasing load.  I don't know if it would be much faster.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 10, 2014, 10:54:33 PM
 #663

Hi John,

I have a quick question (don't know if the answer would be as quick and easy) so here's the deal in a nutshell.

I am trying to setup a block explorer for FairCoin.

it reads the blockchain just fine but on the HTTP side, I can't get it to read the FairCoin chain name. It defaults to NovaCoin no matter what I do.

It seems that the magic number for NovaCoin is the same as the one for FairCoin. could that be it?

how can this be fixed/where should I look?

thanks in advance

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 10, 2014, 11:15:56 PM
 #664

ok I found where NovaCoin took it's config from in Chain.py... and like I said they both have the same magic number so I hardcoded the FairCoin names into that file, witch I find is the unelegant way to go but it works for now... If you have a better solution, I would love to know Smiley

edit: well after putting more thoughts to it... I just added a conditional statement for the FairCoin name, it's a bit more clean this way... still if there is a better solution (as an update to the code will wipe this setting) let me know (or maybe update the github with this conditional statement)

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
elcoco
Member
**
Offline Offline

Activity: 109


View Profile
March 13, 2014, 01:13:13 AM
 #665

I'm trying to run a blockchain explorer for HeavyCoin, by using the newest bitcoin-abe, but i keep getting this error
Opened /home/azureuser/.heavycoin/blocks/blk00000.dat
Exception at 433754408174527358
Failed to catch up {'blkfile_offset': 40726, 'blkfile_number': 100000, 'chain_id': 19, 'loader': u'blkfile', 'conf': None, 'dirname': '/home/azureuser/.heavycoin', 'id': Decimal('84')}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2686, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2950, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3072, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 59, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 52, in ds_parse_transaction
    return deserialize.parse_Transaction(ds)
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 90, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 46, in parse_TxIn
    d['sequence'] = vds.read_uint32()
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 71, in read_uint32
    def read_uint32 (self): return self._read_num('<I')
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 4 bytes


I installed the ltc_scrypt module, by doing python setup.py install, but i'm not sure what to do next.
at first I was getting
raise MerkleRootMismatch(b['hash'], tx_hash_array)
but I changed lines in     
# Verify Merkle root.
    if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        raise MerkleRootMismatch(b['hash'], tx_hash_array)
from the Abe/DataStore.py

but now i'm getting the error posted

CoolCoin -Forever Free IPO. CMdtHbZCWo7d9C2oXU6HYPmUnFVSwaFhGk
PM if in need of Mac Wallet, Block Explorer,Rich List, or Node.
www.coinexplorers.com
jeffmthomas
Newbie
*
Offline Offline

Activity: 4


View Profile
March 13, 2014, 02:13:01 AM
 #666

So I was finally able to get Dogecoin working.  I was using the wrong magic.  I have everything up on a live server now and it doesn't seem to be polling the database for the queries, instead it is opening up the block file every single time.

I am on the no-statistics branch using this config file:

Code:
dbtype sqlite3
connect-args doge.sqlite
port 2750
host 0.0.0.0
#upgrade
datadir = [{
        "dirname": "/home/ubuntu/.dogecoin",
        "chain": "Dogecoin",
        "code3": "DOGE",
        "address_version": "\u001e",
        "magic": "\u00c0\u00c0\u00c0\u00c0",
        "conf": "dogecoin.conf",
        "trim_depth": 40
        }]
commit-bytes = 100000
keep-scriptsig = false
int-type str

Here is what shows up on the server:

Code:
[13/Mar/2014 02:06:12] "GET /q/addressbalance/D9f7RRygjA7N91TLDoshLwNVoFxqCjvv7i HTTP/1.1" 200 83
failed to load /home/ubuntu/.dogecoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/ubuntu/.dogecoin/bitcoin.conf'
catch_up_rpc: abort
Opened /home/ubuntu/.dogecoin/blocks/blk00022.dat

John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 13, 2014, 02:37:03 PM
 #667

So I was finally able to get Dogecoin working.  I was using the wrong magic.  I have everything up on a live server now and it doesn't seem to be polling the database for the queries, instead it is opening up the block file every single time.

Please search for "no-serve" in this thread.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
lunalauncher
Newbie
*
Offline Offline

Activity: 15


View Profile
March 13, 2014, 07:57:16 PM
 #668

I'm transferring Abe from one server to the next. After restoring the MySQL dump from the previous server and starting Abe, the block number no longer increases and it is stuck at block 289302 for Bitcoin. The MySQL dump was made for at block height 289302. The previous server's copy of Abe is able to go past block 289302.

On another pair of servers, the same transfer was made and Ltc had no problems updating the new blocks.

Any idea what has happened?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 13, 2014, 10:45:07 PM
 #669

Hi, I'm trying to setup GrowthCoin block explorer.

This is what I get

Code:
root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Opened /root/.GrowthCoin/blk0001.dat
Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0.

But we can clearly see the firt byte from

Code:
hexdump -n 300 -C /root/.GrowthCoin/blk0001.dat


are correct!


Code:
00000000  a5 ef db fd 69 01 00 00  01 00 00 00 00 00 00 00  |....i...........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 bf f2 79 9b  |..............y.|
00000030  bf ec 52 41 32 ea 94 54  5f 53 83 7c 5c eb 46 26  |..RA2..T_S.|\.F&|
00000040  b6 3d 18 70 76 01 0c cb  96 10 53 14 31 0b fa 51  |.=.pv.....S.1..Q|
00000050  ff ff 0f 1e ba 6e 15 01  01 02 00 00 00 1d 0b fa  |.....n..........|
00000060  51 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 ff ff ff ff b9 04  ff ff 00 1d 02 0f 27 4c  |..............'L|
00000090  af 4a 75 6c 79 20 33 31  2c 20 32 30 31 33 20 31  |.July 31, 2013 1|
000000a0  32 70 6d 20 45 44 54 3a  20 54 68 65 20 55 2e 53  |2pm EDT: The U.S|
000000b0  2e 20 65 63 6f 6e 6f 6d  79 20 67 72 65 77 20 31  |. economy grew 1|
000000c0  2e 37 25 20 69 6e 20 74  68 65 20 73 65 63 6f 6e  |.7% in the secon|
000000d0  64 20 71 75 61 72 74 65  72 2c 20 61 69 64 65 64  |d quarter, aided|
000000e0  20 62 79 20 73 6f 6c 69  64 20 63 6f 6e 73 75 6d  | by solid consum|
000000f0  65 72 20 73 70 65 6e 64  69 6e 67 20 61 6e 64 20  |er spending and |
00000100  61 20 72 61 6d 70 2d 75  70 20 69 6e 20 62 75 73  |a ramp-up in bus|
00000110  69 6e 65 73 73 20 69 6e  76 65 73 74 6d 65 6e 74  |iness investment|
00000120  2c 20 74 68 65 20 67 6f  76 65 72 6e              |, the govern|
0000012c


What did I miss to get this message?

thanks

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
Remember remember the 5th of November
Legendary
*
Offline Offline

Activity: 1526

Reverse engineer from time to time


View Profile
March 13, 2014, 10:46:31 PM
 #670

Hi, I'm trying to setup GrowthCoin block explorer.

This is what I get

Code:
root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Opened /root/.GrowthCoin/blk0001.dat
Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0.

But we can clearly see the firt byte from

Code:
hexdump -n 300 -C /root/.GrowthCoin/blk0001.dat


are correct!


Code:
00000000  a5 ef db fd 69 01 00 00  01 00 00 00 00 00 00 00  |....i...........|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  00 00 00 00 bf f2 79 9b  |..............y.|
00000030  bf ec 52 41 32 ea 94 54  5f 53 83 7c 5c eb 46 26  |..RA2..T_S.|\.F&|
00000040  b6 3d 18 70 76 01 0c cb  96 10 53 14 31 0b fa 51  |.=.pv.....S.1..Q|
00000050  ff ff 0f 1e ba 6e 15 01  01 02 00 00 00 1d 0b fa  |.....n..........|
00000060  51 01 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |Q...............|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 ff ff ff ff b9 04  ff ff 00 1d 02 0f 27 4c  |..............'L|
00000090  af 4a 75 6c 79 20 33 31  2c 20 32 30 31 33 20 31  |.July 31, 2013 1|
000000a0  32 70 6d 20 45 44 54 3a  20 54 68 65 20 55 2e 53  |2pm EDT: The U.S|
000000b0  2e 20 65 63 6f 6e 6f 6d  79 20 67 72 65 77 20 31  |. economy grew 1|
000000c0  2e 37 25 20 69 6e 20 74  68 65 20 73 65 63 6f 6e  |.7% in the secon|
000000d0  64 20 71 75 61 72 74 65  72 2c 20 61 69 64 65 64  |d quarter, aided|
000000e0  20 62 79 20 73 6f 6c 69  64 20 63 6f 6e 73 75 6d  | by solid consum|
000000f0  65 72 20 73 70 65 6e 64  69 6e 67 20 61 6e 64 20  |er spending and |
00000100  61 20 72 61 6d 70 2d 75  70 20 69 6e 20 62 75 73  |a ramp-up in bus|
00000110  69 6e 65 73 73 20 69 6e  76 65 73 74 6d 65 6e 74  |iness investment|
00000120  2c 20 74 68 65 20 67 6f  76 65 72 6e              |, the govern|
0000012c


What did I miss to get this message?

thanks

Maybe it's an endiannes thing?

BTC:1AiCRMxgf1ptVQwx6hDuKMu4f7F27QmJC2
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 14, 2014, 01:49:37 AM
 #671

Hi, I'm trying to setup GrowthCoin block explorer.

This is what I get

Code:
root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Opened /root/.GrowthCoin/blk0001.dat
Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0.

This coin is new to Abe.  Please follow the example "datadir" syntax in abe.conf, and add datadir = [{....}] to your abe-mysql.conf file.  If the currency is scrypt-based, it may need more changes discussed recently in this thread.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 14, 2014, 02:02:41 AM
 #672

ok I found where NovaCoin took it's config from in Chain.py... and like I said they both have the same magic number so I hardcoded the FairCoin names into that file, witch I find is the unelegant way to go but it works for now... If you have a better solution, I would love to know Smiley

edit: well after putting more thoughts to it... I just added a conditional statement for the FairCoin name, it's a bit more clean this way... still if there is a better solution (as an update to the code will wipe this setting) let me know (or maybe update the github with this conditional statement)

I'd put the FairCoin parameters in abe.conf datadir=[{...}] and not in Python.  That should survive upgrades.  See the datadir examples in abe.conf.

Given a simple directory name (datadir="...") Abe relies on the magic number.  We could support two chains with the same magic number, but we would need some way to distinguish them.  That's why I introduced the expanded syntax for datadir, a JSON array of objects.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
jeffmthomas
Newbie
*
Offline Offline

Activity: 4


View Profile
March 14, 2014, 06:53:28 PM
 #673

I am attempting to get this running on a postgresql database.  I have followed the instructions to the letter and this is the error that I get:

Code:
admin@ip-10-196-49-5:~$ python -m Abe.abe --config doge.conf --commit-bytes 100000 --no-serve
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2276, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2270, in main
    store = make_store(args)
  File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 153, in make_store
    store = DataStore.new(args)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3445, in new
    return DataStore(args)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 199, in __init__
    store._init_datadirs()
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 753, in _init_datadirs
    dircfg.get('policy', chain_name), decimals))
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 522, in sql
    store._execute(cached, params)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 499, in _execute
    store.cursor.execute(stmt, params)
psycopg2.DataError: value too long for type character(3)

I am on Debain Wheezy running python 2.7 and postgresql 9.1
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 14, 2014, 07:08:01 PM
 #674

Code:
psycopg2.DataError: value too long for type character(3)

Either shorten your chain's "code3" to 3 characters or widen the database column:

Code:
ALTER TABLE chain ALTER COLUMN chain_code3 TYPE VARCHAR(4);

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 14, 2014, 10:10:16 PM
 #675

Maybe it's an endiannes thing?

I'm not familiar with the concepet of endiannes. After relooking at the code, I had made a typo on the class name


This coin is new to Abe.  Please follow the example "datadir" syntax in abe.conf, and add datadir = [{....}] to your abe-mysql.conf file.  If the currency is scrypt-based, it may need more changes discussed recently in this thread.



Well I modified the Chain.py file as I did for FairCoin (it worked for FairCoin) so the abe-mysql.conf file would be a better option on the perspective of future upgrades, but the specs of the GrowthCoin blockchain were indeed entered. I will revert this to the config file when I sorted out the issue.

Meanwhile I triple checked the code and found that I made a typo on the class name.

That being said, I have another issue now. This time with the address version

Code:
root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT
  store.cursor.execute(stmt)
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/bitcoin-abe/Abe/abe.py", line 2276, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main
    store = make_store(args)
  File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3446, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 181, in __init__
    store.initialize()
  File "Abe/DataStore.py", line 1262, in initialize
    store.insert_chain(chain)
  File "Abe/DataStore.py", line 1301, in insert_chain
    chain.name, chain.decimals))
  File "Abe/DataStore.py", line 523, in sql
    store._execute(cached, params)
  File "Abe/DataStore.py", line 500, in _execute
    store.cursor.execute(stmt, params)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
_mysql_exceptions.OperationalError: (1048, "Column 'chain_address_version' cannot be null")

if I decode the address, this is what I get

Code:
root@grw:~/bitcoin-abe# python -m Abe.abe --query /q/decode_address/GTjYKganLhDtMTcWCinzbQZpsrSjXfB29m
26:6c7c51acb76a0b62c10b6cb17b39cc0982f5d289

and this is my GrowthCoin class

Code:
class GrowthCoin(NvcChain):
    def __init__(chain, **kwargs):
        chain.name = 'GrowthCoin'
        chain.code3 = 'GRW'
        chain.address_version = "\x26"
        chain.magic = "\xa5\xef\xdb\xfd"
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = "GrowthCoin.conf"
    datadir_rpcport = 17178

I also added CHAIN_CONFIG = [{"chain":"GrowthCoin"}] in the DataStore.py file...

Everything looks ok, what could be the issue?

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
crabel
Newbie
*
Offline Offline

Activity: 18


View Profile
March 14, 2014, 11:06:06 PM
 #676

I developed a method for analyzing the bitcoin blockchain using very simple analytical tools. While my initial work was with bitcoin it works with altcoins in general. I wrote a post describing the method and theory. In it I posted the MatLab files I used to perform the analysis.

I am not familiar with programing in general. I do not see any reason why these functions and methods can't be incorporated into abe's source code. They provide powerful description of these currencies and are very easily implemented.

I used abe to parse the bitcoin blockchain and greatly appreciate the effort that you put into developing the source code. Please take my work as a thank you for the work that you did.

http://statisticaleconomics.org/2014/03/13/quantifying-the-value-of-bitcoin-2/
lunalauncher
Newbie
*
Offline Offline

Activity: 15


View Profile
March 15, 2014, 09:42:40 AM
 #677

Abe is giving me the error

Code:
block_tx 290732 34916394
block_tx 290732 34916395
block_tx 290732 34916396
block_tx 290732 34916397
block_tx 290732 34916398
block_tx 290732 34916399
commit
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat
RPC service lacks full txindex
catch_up_rpc: abort
Opened /root/.bitcoin/blocks/blk00123.dat

Made bitcoind do a reindex but it's still the same. Any ideas?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 15, 2014, 10:29:10 AM
 #678



Ok I'm still trying to figure this thing out... I've manage to get passed the invalid address error (not sure why but I tried putting the chain specifcation into the config file instead of the python file).  But now moving forward on this I get yet another error!

Code:
root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/bitcoin-abe/Abe/abe.py", line 2276, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main
    store = make_store(args)
  File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3446, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 165, in __init__
    store.init_conn()
  File "Abe/DataStore.py", line 226, in init_conn
    store.conn = store.connect()
  File "Abe/DataStore.py", line 237, in connect
    conn = store._connect(cargs)
  File "Abe/DataStore.py", line 269, in _connect
    return store.module.connect(**cargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'abe'@'localhost' (using password: YES)")
root@tmp:~/bitcoin-abe# nano /home/abe/abe-mysql.conf
root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT
  store.cursor.execute(stmt)
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Assigned chain_id 1 to GrowthCoin
Opened /root/.GrowthCoin/blk0001.dat
Chain 1 genesis tx: 020000001d0bfa51010000000000000000000000000000000000000000000000000000000000000000ffffffffb904ffff001d020f274caf4a756c792033312c2032303133203132706d204544543a2054686520552e532e2065636f6e6f6d79206772657720312e372520696e20746865207365636f6e6420717561727465722c20616964656420627920736f6c696420636f6e73756d6572207370656e64696e6720616e6420612072616d702d757020696e20627573696e65737320696e766573746d656e742c2074686520676f7665726e6d656e742073616964205765646e65736461792effffffff0100000000000000000000000000
Exception at 368
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1L, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 1L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2687, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2951, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3083, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1805, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=d5e4fc54c53a9a20bbe01e70d8b47ce66f15c1bd3504f7e80390bf5601150ded

I've read to comment out in DataStore.py
Code:
# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)

But as you can image if I post this... it didn't work!

It gave me a few of these

Code:
Skipped 73 bytes at block end
block_tx 970 970
Skipped 71 bytes at block end
block_tx 971 971
Skipped 72 bytes at block end
block_tx 972 972
Skipped 72 bytes at block end
Exception at 1677721600398062
Failed to catch up {'blkfile_offset': 206488, 'blkfile_number': 1, 'chain_id': 1, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 2L}
Traceback (most recent call last):

Then
Code:
error: unpack_from requires a buffer of at least 4 bytes

Witch I had earlier in my attempts to get Abe read the blockchain.  It looks to me as the magic bytes are causing troubles again.

Any inputs/help/solutions on this would be appreciated  Wink

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
adversor
Member
**
Offline Offline

Activity: 112


View Profile
March 16, 2014, 03:09:26 PM
 #679

John Tobey, great job with Abe. Thanks for sharing this work with the community!

I'm trying to set up a website with several altcoin block explorers. The first one I tried was Klondikecoin, this works already like a charm.

Now I'm trying Noblecoin, but get same error, same as unick, just one post above.

Code:
$ python -m Abe.abe --config NobleCoin.conf --commit-bytes 100000 --no-serve
Opened /home/adversor/.noblecoin/blk0001.dat
Chain 2 genesis tx: 02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4604ffff001d01043e36204a616e20323031353a20434e4e3a204d696c6c696f6e7320696e20552e532e20627261636520666f72207265636f7264206465657020667265657a65ffffffff010000000000000000434104dcba12349012341234900abcd12223abcd455abcd77788abcd000000aaaaabbbbbcccccdddddeeeeeff00ff00ff00ff001234567890abcdef0022446688abc11ac00000000
Exception at 286
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 3L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2686, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2950, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3082, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1804, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=dbf8ee8b91b5f03d7786cb1d58f2588538b2d48d8c23a7913db5aac7bb58c481

After I have commented out the Merkle root verification in DataStore.py, it reads 214 blocks from chain and this is the output:

Code:
.....
Skipped 1 bytes at block end
block_tx 212 212
Skipped 1 bytes at block end
block_tx 213 213
Skipped 1 bytes at block end
block_tx 214 214
Skipped 1 bytes at block end
Exception at 60825391604920033
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 4L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2686, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2950, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3072, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "Abe/Chain.py", line 59, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "Abe/Chain.py", line 52, in ds_parse_transaction
    return deserialize.parse_Transaction(ds)
  File "Abe/deserialize.py", line 90, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "Abe/deserialize.py", line 46, in parse_TxIn
    d['sequence'] = vds.read_uint32()
  File "Abe/BCDataStream.py", line 71, in read_uint32
    def read_uint32 (self): return self._read_num('<I')
  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 4 bytes

I'm using

Code:
default-loader = blkfile

Someone has a solution?


LISK Develop Decentralized Applications & Sidechains in JavaScript with Lisk!
wing_hk
Full Member
***
Offline Offline

Activity: 182


View Profile
March 17, 2014, 04:16:14 PM
 #680

Well I have a ABE working well for a particular Altcoin now.

Is there any existing data mining tool for ABE? (i.e. largest coin holder distribution etc. )  Huh
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 [34] 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!