John Tobey (OP)
|
|
March 10, 2014, 08:21:36 PM |
|
What would be faster, using one DB for all the blocks or each chain one DB
I guess you should use a separate database per chain. It complicates the HTTP side but simplifies deleting a corrupt (or abandoned) chain and spreading an increasing load. I don't know if it would be much faster.
|
|
|
|
unick
|
|
March 10, 2014, 10:54:33 PM |
|
Hi John,
I have a quick question (don't know if the answer would be as quick and easy) so here's the deal in a nutshell.
I am trying to setup a block explorer for FairCoin.
it reads the blockchain just fine but on the HTTP side, I can't get it to read the FairCoin chain name. It defaults to NovaCoin no matter what I do.
It seems that the magic number for NovaCoin is the same as the one for FairCoin. could that be it?
how can this be fixed/where should I look?
thanks in advance
|
|
|
|
unick
|
|
March 10, 2014, 11:15:56 PM Last edit: March 10, 2014, 11:41:38 PM by unick |
|
ok I found where NovaCoin took it's config from in Chain.py... and like I said they both have the same magic number so I hardcoded the FairCoin names into that file, witch I find is the unelegant way to go but it works for now... If you have a better solution, I would love to know edit: well after putting more thoughts to it... I just added a conditional statement for the FairCoin name, it's a bit more clean this way... still if there is a better solution (as an update to the code will wipe this setting) let me know (or maybe update the github with this conditional statement)
|
|
|
|
elcoco
Member
Offline
Activity: 111
Merit: 10
|
|
March 13, 2014, 01:13:13 AM |
|
I'm trying to run a blockchain explorer for HeavyCoin, by using the newest bitcoin-abe, but i keep getting this error Opened /home/azureuser/.heavycoin/blocks/blk00000.dat Exception at 433754408174527358 Failed to catch up {'blkfile_offset': 40726, 'blkfile_number': 100000, 'chain_id': 19, 'loader': u'blkfile', 'conf': None, 'dirname': '/home/azureuser/.heavycoin', 'id': Decimal('84')} Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2686, in catch_up store.catch_up_dir(dircfg) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2950, in catch_up_dir store.import_blkdat(dircfg, ds, blkfile['name']) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3072, in import_blkdat b = chain.ds_parse_block(ds) File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 59, in ds_parse_block d['transactions'].append(chain.ds_parse_transaction(ds)) File "/usr/local/lib/python2.7/dist-packages/Abe/Chain.py", line 52, in ds_parse_transaction return deserialize.parse_Transaction(ds) File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 90, in parse_Transaction d['txIn'].append(parse_TxIn(vds)) File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 46, in parse_TxIn d['sequence'] = vds.read_uint32() File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 71, in read_uint32 def read_uint32 (self): return self._read_num('<I') File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 110, in _read_num (i,) = struct.unpack_from(format, self.input, self.read_cursor) error: unpack_from requires a buffer of at least 4 bytes
I installed the ltc_scrypt module, by doing python setup.py install, but i'm not sure what to do next. at first I was getting raise MerkleRootMismatch(b['hash'], tx_hash_array) but I changed lines in # Verify Merkle root. if b['hashMerkleRoot'] != util.merkle(tx_hash_array): raise MerkleRootMismatch(b['hash'], tx_hash_array) from the Abe/DataStore.py
but now i'm getting the error posted
|
|
|
|
jeffmthomas
Newbie
Offline
Activity: 4
Merit: 0
|
|
March 13, 2014, 02:13:01 AM |
|
So I was finally able to get Dogecoin working. I was using the wrong magic. I have everything up on a live server now and it doesn't seem to be polling the database for the queries, instead it is opening up the block file every single time. I am on the no-statistics branch using this config file: dbtype sqlite3 connect-args doge.sqlite port 2750 host 0.0.0.0 #upgrade datadir = [{ "dirname": "/home/ubuntu/.dogecoin", "chain": "Dogecoin", "code3": "DOGE", "address_version": "\u001e", "magic": "\u00c0\u00c0\u00c0\u00c0", "conf": "dogecoin.conf", "trim_depth": 40 }] commit-bytes = 100000 keep-scriptsig = false int-type str Here is what shows up on the server: [13/Mar/2014 02:06:12] "GET /q/addressbalance/D9f7RRygjA7N91TLDoshLwNVoFxqCjvv7i HTTP/1.1" 200 83 failed to load /home/ubuntu/.dogecoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/ubuntu/.dogecoin/bitcoin.conf' catch_up_rpc: abort Opened /home/ubuntu/.dogecoin/blocks/blk00022.dat
|
|
|
|
John Tobey (OP)
|
|
March 13, 2014, 02:37:03 PM Last edit: March 25, 2014, 03:49:20 PM by John Tobey |
|
So I was finally able to get Dogecoin working. I was using the wrong magic. I have everything up on a live server now and it doesn't seem to be polling the database for the queries, instead it is opening up the block file every single time.
Please search for "no-serve" in this thread.
|
|
|
|
lunalauncher
Newbie
Offline
Activity: 15
Merit: 0
|
|
March 13, 2014, 07:57:16 PM |
|
I'm transferring Abe from one server to the next. After restoring the MySQL dump from the previous server and starting Abe, the block number no longer increases and it is stuck at block 289302 for Bitcoin. The MySQL dump was made for at block height 289302. The previous server's copy of Abe is able to go past block 289302.
On another pair of servers, the same transfer was made and Ltc had no problems updating the new blocks.
Any idea what has happened?
|
|
|
|
unick
|
|
March 13, 2014, 10:45:07 PM |
|
Hi, I'm trying to setup GrowthCoin block explorer. This is what I get root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve Opened /root/.GrowthCoin/blk0001.dat Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0. But we can clearly see the firt byte from hexdump -n 300 -C /root/.GrowthCoin/blk0001.dat are correct! 00000000 a5 ef db fd 69 01 00 00 01 00 00 00 00 00 00 00 |....i...........| 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 bf f2 79 9b |..............y.| 00000030 bf ec 52 41 32 ea 94 54 5f 53 83 7c 5c eb 46 26 |..RA2..T_S.|\.F&| 00000040 b6 3d 18 70 76 01 0c cb 96 10 53 14 31 0b fa 51 |.=.pv.....S.1..Q| 00000050 ff ff 0f 1e ba 6e 15 01 01 02 00 00 00 1d 0b fa |.....n..........| 00000060 51 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |Q...............| 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000080 00 00 ff ff ff ff b9 04 ff ff 00 1d 02 0f 27 4c |..............'L| 00000090 af 4a 75 6c 79 20 33 31 2c 20 32 30 31 33 20 31 |.July 31, 2013 1| 000000a0 32 70 6d 20 45 44 54 3a 20 54 68 65 20 55 2e 53 |2pm EDT: The U.S| 000000b0 2e 20 65 63 6f 6e 6f 6d 79 20 67 72 65 77 20 31 |. economy grew 1| 000000c0 2e 37 25 20 69 6e 20 74 68 65 20 73 65 63 6f 6e |.7% in the secon| 000000d0 64 20 71 75 61 72 74 65 72 2c 20 61 69 64 65 64 |d quarter, aided| 000000e0 20 62 79 20 73 6f 6c 69 64 20 63 6f 6e 73 75 6d | by solid consum| 000000f0 65 72 20 73 70 65 6e 64 69 6e 67 20 61 6e 64 20 |er spending and | 00000100 61 20 72 61 6d 70 2d 75 70 20 69 6e 20 62 75 73 |a ramp-up in bus| 00000110 69 6e 65 73 73 20 69 6e 76 65 73 74 6d 65 6e 74 |iness investment| 00000120 2c 20 74 68 65 20 67 6f 76 65 72 6e |, the govern| 0000012c
What did I miss to get this message? thanks
|
|
|
|
Remember remember the 5th of November
Legendary
Offline
Activity: 1862
Merit: 1011
Reverse engineer from time to time
|
|
March 13, 2014, 10:46:31 PM |
|
Hi, I'm trying to setup GrowthCoin block explorer. This is what I get root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve Opened /root/.GrowthCoin/blk0001.dat Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0. But we can clearly see the firt byte from hexdump -n 300 -C /root/.GrowthCoin/blk0001.dat are correct! 00000000 a5 ef db fd 69 01 00 00 01 00 00 00 00 00 00 00 |....i...........| 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 bf f2 79 9b |..............y.| 00000030 bf ec 52 41 32 ea 94 54 5f 53 83 7c 5c eb 46 26 |..RA2..T_S.|\.F&| 00000040 b6 3d 18 70 76 01 0c cb 96 10 53 14 31 0b fa 51 |.=.pv.....S.1..Q| 00000050 ff ff 0f 1e ba 6e 15 01 01 02 00 00 00 1d 0b fa |.....n..........| 00000060 51 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |Q...............| 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000080 00 00 ff ff ff ff b9 04 ff ff 00 1d 02 0f 27 4c |..............'L| 00000090 af 4a 75 6c 79 20 33 31 2c 20 32 30 31 33 20 31 |.July 31, 2013 1| 000000a0 32 70 6d 20 45 44 54 3a 20 54 68 65 20 55 2e 53 |2pm EDT: The U.S| 000000b0 2e 20 65 63 6f 6e 6f 6d 79 20 67 72 65 77 20 31 |. economy grew 1| 000000c0 2e 37 25 20 69 6e 20 74 68 65 20 73 65 63 6f 6e |.7% in the secon| 000000d0 64 20 71 75 61 72 74 65 72 2c 20 61 69 64 65 64 |d quarter, aided| 000000e0 20 62 79 20 73 6f 6c 69 64 20 63 6f 6e 73 75 6d | by solid consum| 000000f0 65 72 20 73 70 65 6e 64 69 6e 67 20 61 6e 64 20 |er spending and | 00000100 61 20 72 61 6d 70 2d 75 70 20 69 6e 20 62 75 73 |a ramp-up in bus| 00000110 69 6e 65 73 73 20 69 6e 76 65 73 74 6d 65 6e 74 |iness investment| 00000120 2c 20 74 68 65 20 67 6f 76 65 72 6e |, the govern| 0000012c
What did I miss to get this message? thanks Maybe it's an endiannes thing?
|
BTC:1AiCRMxgf1ptVQwx6hDuKMu4f7F27QmJC2
|
|
|
John Tobey (OP)
|
|
March 14, 2014, 01:49:37 AM |
|
Hi, I'm trying to setup GrowthCoin block explorer. This is what I get root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve Opened /root/.GrowthCoin/blk0001.dat Chain not found for magic number a5efdbfd in block file /root/.GrowthCoin/blk0001.dat at offset 0. This coin is new to Abe. Please follow the example "datadir" syntax in abe.conf, and add datadir = [{....}] to your abe-mysql.conf file. If the currency is scrypt-based, it may need more changes discussed recently in this thread.
|
|
|
|
John Tobey (OP)
|
|
March 14, 2014, 02:02:41 AM |
|
ok I found where NovaCoin took it's config from in Chain.py... and like I said they both have the same magic number so I hardcoded the FairCoin names into that file, witch I find is the unelegant way to go but it works for now... If you have a better solution, I would love to know edit: well after putting more thoughts to it... I just added a conditional statement for the FairCoin name, it's a bit more clean this way... still if there is a better solution (as an update to the code will wipe this setting) let me know (or maybe update the github with this conditional statement) I'd put the FairCoin parameters in abe.conf datadir=[{...}] and not in Python. That should survive upgrades. See the datadir examples in abe.conf. Given a simple directory name (datadir="...") Abe relies on the magic number. We could support two chains with the same magic number, but we would need some way to distinguish them. That's why I introduced the expanded syntax for datadir, a JSON array of objects.
|
|
|
|
jeffmthomas
Newbie
Offline
Activity: 4
Merit: 0
|
|
March 14, 2014, 06:53:28 PM |
|
I am attempting to get this running on a postgresql database. I have followed the instructions to the letter and this is the error that I get: admin@ip-10-196-49-5:~$ python -m Abe.abe --config doge.conf --commit-bytes 100000 --no-serve Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2276, in <module> sys.exit(main(sys.argv[1:])) File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 2270, in main store = make_store(args) File "/usr/local/lib/python2.7/dist-packages/Abe/abe.py", line 153, in make_store store = DataStore.new(args) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3445, in new return DataStore(args) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 199, in __init__ store._init_datadirs() File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 753, in _init_datadirs dircfg.get('policy', chain_name), decimals)) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 522, in sql store._execute(cached, params) File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 499, in _execute store.cursor.execute(stmt, params) psycopg2.DataError: value too long for type character(3) I am on Debain Wheezy running python 2.7 and postgresql 9.1
|
|
|
|
John Tobey (OP)
|
|
March 14, 2014, 07:08:01 PM |
|
psycopg2.DataError: value too long for type character(3) Either shorten your chain's "code3" to 3 characters or widen the database column: ALTER TABLE chain ALTER COLUMN chain_code3 TYPE VARCHAR(4);
|
|
|
|
unick
|
|
March 14, 2014, 10:10:16 PM |
|
Maybe it's an endiannes thing?
I'm not familiar with the concepet of endiannes. After relooking at the code, I had made a typo on the class name This coin is new to Abe. Please follow the example "datadir" syntax in abe.conf, and add datadir = [{....}] to your abe-mysql.conf file. If the currency is scrypt-based, it may need more changes discussed recently in this thread.
Well I modified the Chain.py file as I did for FairCoin (it worked for FairCoin) so the abe-mysql.conf file would be a better option on the perspective of future upgrades, but the specs of the GrowthCoin blockchain were indeed entered. I will revert this to the config file when I sorted out the issue. Meanwhile I triple checked the code and found that I made a typo on the class name. That being said, I have another issue now. This time with the address version root@grw:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve ddl_implicit_commit=true create_table_epilogue='' Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT store.cursor.execute(stmt) Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT store.cursor.execute(stmt) max_varchar=4294967295 clob_type=LONGTEXT Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT store.cursor.execute(stmt) binary_type=hex int_type=int Created silly table abe_dual sequence_type=mysql limit_style=native Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT store.cursor.execute(stmt) Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT store.cursor.execute(stmt) Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/bitcoin-abe/Abe/abe.py", line 2276, in <module> sys.exit(main(sys.argv[1:])) File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main store = make_store(args) File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store store = DataStore.new(args) File "Abe/DataStore.py", line 3446, in new return DataStore(args) File "Abe/DataStore.py", line 181, in __init__ store.initialize() File "Abe/DataStore.py", line 1262, in initialize store.insert_chain(chain) File "Abe/DataStore.py", line 1301, in insert_chain chain.name, chain.decimals)) File "Abe/DataStore.py", line 523, in sql store._execute(cached, params) File "Abe/DataStore.py", line 500, in _execute store.cursor.execute(stmt, params) File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute self.errorhandler(self, exc, value) File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue _mysql_exceptions.OperationalError: (1048, "Column 'chain_address_version' cannot be null") if I decode the address, this is what I get root@grw:~/bitcoin-abe# python -m Abe.abe --query /q/decode_address/GTjYKganLhDtMTcWCinzbQZpsrSjXfB29m 26:6c7c51acb76a0b62c10b6cb17b39cc0982f5d289 and this is my GrowthCoin class class GrowthCoin(NvcChain): def __init__(chain, **kwargs): chain.name = 'GrowthCoin' chain.code3 = 'GRW' chain.address_version = "\x26" chain.magic = "\xa5\xef\xdb\xfd" Chain.__init__(chain, **kwargs)
datadir_conf_file_name = "GrowthCoin.conf" datadir_rpcport = 17178 I also added CHAIN_CONFIG = [{"chain":"GrowthCoin"}] in the DataStore.py file... Everything looks ok, what could be the issue?
|
|
|
|
crabel
Newbie
Offline
Activity: 18
Merit: 0
|
|
March 14, 2014, 11:06:06 PM |
|
I developed a method for analyzing the bitcoin blockchain using very simple analytical tools. While my initial work was with bitcoin it works with altcoins in general. I wrote a post describing the method and theory. In it I posted the MatLab files I used to perform the analysis. I am not familiar with programing in general. I do not see any reason why these functions and methods can't be incorporated into abe's source code. They provide powerful description of these currencies and are very easily implemented. I used abe to parse the bitcoin blockchain and greatly appreciate the effort that you put into developing the source code. Please take my work as a thank you for the work that you did. http://statisticaleconomics.org/2014/03/13/quantifying-the-value-of-bitcoin-2/
|
|
|
|
lunalauncher
Newbie
Offline
Activity: 15
Merit: 0
|
|
March 15, 2014, 09:42:40 AM |
|
Abe is giving me the error block_tx 290732 34916394 block_tx 290732 34916395 block_tx 290732 34916396 block_tx 290732 34916397 block_tx 290732 34916398 block_tx 290732 34916399 commit RPC service lacks full txindex catch_up_rpc: abort Opened /root/.bitcoin/blocks/blk00123.dat RPC service lacks full txindex catch_up_rpc: abort Opened /root/.bitcoin/blocks/blk00123.dat RPC service lacks full txindex catch_up_rpc: abort Opened /root/.bitcoin/blocks/blk00123.dat RPC service lacks full txindex catch_up_rpc: abort Opened /root/.bitcoin/blocks/blk00123.dat
Made bitcoind do a reindex but it's still the same. Any ideas?
|
|
|
|
unick
|
|
March 15, 2014, 10:29:10 AM Last edit: March 15, 2014, 10:41:44 AM by unick |
|
Ok I'm still trying to figure this thing out... I've manage to get passed the invalid address error (not sure why but I tried putting the chain specifcation into the config file instead of the python file). But now moving forward on this I get yet another error! root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/bitcoin-abe/Abe/abe.py", line 2276, in <module> sys.exit(main(sys.argv[1:])) File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main store = make_store(args) File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store store = DataStore.new(args) File "Abe/DataStore.py", line 3446, in new return DataStore(args) File "Abe/DataStore.py", line 165, in __init__ store.init_conn() File "Abe/DataStore.py", line 226, in init_conn store.conn = store.connect() File "Abe/DataStore.py", line 237, in connect conn = store._connect(cargs) File "Abe/DataStore.py", line 269, in _connect return store.module.connect(**cargs) File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect return Connection(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__ super(Connection, self).__init__(*args, **kwargs2) _mysql_exceptions.OperationalError: (1045, "Access denied for user 'abe'@'localhost' (using password: YES)") root@tmp:~/bitcoin-abe# nano /home/abe/abe-mysql.conf root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve ddl_implicit_commit=true create_table_epilogue='' Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT store.cursor.execute(stmt) Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT store.cursor.execute(stmt) max_varchar=4294967295 clob_type=LONGTEXT Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT store.cursor.execute(stmt) binary_type=hex int_type=int Created silly table abe_dual sequence_type=mysql limit_style=native Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT store.cursor.execute(stmt) Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT store.cursor.execute(stmt) Assigned chain_id 1 to GrowthCoin Opened /root/.GrowthCoin/blk0001.dat Chain 1 genesis tx: 020000001d0bfa51010000000000000000000000000000000000000000000000000000000000000000ffffffffb904ffff001d020f274caf4a756c792033312c2032303133203132706d204544543a2054686520552e532e2065636f6e6f6d79206772657720312e372520696e20746865207365636f6e6420717561727465722c20616964656420627920736f6c696420636f6e73756d6572207370656e64696e6720616e6420612072616d702d757020696e20627573696e65737320696e766573746d656e742c2074686520676f7665726e6d656e742073616964205765646e65736461792effffffff0100000000000000000000000000 Exception at 368 Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1L, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 1L} Traceback (most recent call last): File "Abe/DataStore.py", line 2687, in catch_up store.catch_up_dir(dircfg) File "Abe/DataStore.py", line 2951, in catch_up_dir store.import_blkdat(dircfg, ds, blkfile['name']) File "Abe/DataStore.py", line 3083, in import_blkdat store.import_block(b, chain = chain) File "Abe/DataStore.py", line 1805, in import_block raise MerkleRootMismatch(b['hash'], tx_hash_array) MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=d5e4fc54c53a9a20bbe01e70d8b47ce66f15c1bd3504f7e80390bf5601150ded I've read to comment out in DataStore.py # Verify Merkle root. #if b['hashMerkleRoot'] != util.merkle(tx_hash_array): #raise MerkleRootMismatch(b['hash'], tx_hash_array) But as you can image if I post this... it didn't work! It gave me a few of these Skipped 73 bytes at block end block_tx 970 970 Skipped 71 bytes at block end block_tx 971 971 Skipped 72 bytes at block end block_tx 972 972 Skipped 72 bytes at block end Exception at 1677721600398062 Failed to catch up {'blkfile_offset': 206488, 'blkfile_number': 1, 'chain_id': 1, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 2L} Traceback (most recent call last): Then error: unpack_from requires a buffer of at least 4 bytes Witch I had earlier in my attempts to get Abe read the blockchain. It looks to me as the magic bytes are causing troubles again. Any inputs/help/solutions on this would be appreciated
|
|
|
|
adversor
Member
Offline
Activity: 112
Merit: 10
|
|
March 16, 2014, 03:09:26 PM |
|
John Tobey, great job with Abe. Thanks for sharing this work with the community! I'm trying to set up a website with several altcoin block explorers. The first one I tried was Klondikecoin, this works already like a charm. Now I'm trying Noblecoin, but get same error, same as unick, just one post above. $ python -m Abe.abe --config NobleCoin.conf --commit-bytes 100000 --no-serve Opened /home/adversor/.noblecoin/blk0001.dat Chain 2 genesis tx: 02000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4604ffff001d01043e36204a616e20323031353a20434e4e3a204d696c6c696f6e7320696e20552e532e20627261636520666f72207265636f7264206465657020667265657a65ffffffff010000000000000000434104dcba12349012341234900abcd12223abcd455abcd77788abcd000000aaaaabbbbbcccccdddddeeeeeff00ff00ff00ff001234567890abcdef0022446688abc11ac00000000 Exception at 286 Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 3L} Traceback (most recent call last): File "Abe/DataStore.py", line 2686, in catch_up store.catch_up_dir(dircfg) File "Abe/DataStore.py", line 2950, in catch_up_dir store.import_blkdat(dircfg, ds, blkfile['name']) File "Abe/DataStore.py", line 3082, in import_blkdat store.import_block(b, chain = chain) File "Abe/DataStore.py", line 1804, in import_block raise MerkleRootMismatch(b['hash'], tx_hash_array) MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=dbf8ee8b91b5f03d7786cb1d58f2588538b2d48d8c23a7913db5aac7bb58c481
After I have commented out the Merkle root verification in DataStore.py, it reads 214 blocks from chain and this is the output: ..... Skipped 1 bytes at block end block_tx 212 212 Skipped 1 bytes at block end block_tx 213 213 Skipped 1 bytes at block end block_tx 214 214 Skipped 1 bytes at block end Exception at 60825391604920033 Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 2, 'loader': None, 'conf': u'noblecoin.conf', 'dirname': u'/home/adversor/.noblecoin', 'id': 4L} Traceback (most recent call last): File "Abe/DataStore.py", line 2686, in catch_up store.catch_up_dir(dircfg) File "Abe/DataStore.py", line 2950, in catch_up_dir store.import_blkdat(dircfg, ds, blkfile['name']) File "Abe/DataStore.py", line 3072, in import_blkdat b = chain.ds_parse_block(ds) File "Abe/Chain.py", line 59, in ds_parse_block d['transactions'].append(chain.ds_parse_transaction(ds)) File "Abe/Chain.py", line 52, in ds_parse_transaction return deserialize.parse_Transaction(ds) File "Abe/deserialize.py", line 90, in parse_Transaction d['txIn'].append(parse_TxIn(vds)) File "Abe/deserialize.py", line 46, in parse_TxIn d['sequence'] = vds.read_uint32() File "Abe/BCDataStream.py", line 71, in read_uint32 def read_uint32 (self): return self._read_num('<I') File "Abe/BCDataStream.py", line 110, in _read_num (i,) = struct.unpack_from(format, self.input, self.read_cursor) error: unpack_from requires a buffer of at least 4 bytes
I'm using Someone has a solution?
|
LISK Develop Decentralized Applications & Sidechains in JavaScript with Lisk!
|
|
|
wing_hk
|
|
March 17, 2014, 04:16:14 PM |
|
Well I have a ABE working well for a particular Altcoin now. Is there any existing data mining tool for ABE? (i.e. largest coin holder distribution etc. )
|
|
|
|
gdassori
|
|
March 18, 2014, 10:27:05 AM Last edit: March 18, 2014, 02:04:52 PM by gdassori |
|
Hi commit block_tx 54506 88356 block_tx 54507 88357 Exception at 22941369 Failed to catch up {'blkfile_offset': 21942607, 'blkfile_number': 100000, 'chain_id': 2, 'loader': None, 'dirname': u'/home/guido/testnet/testnet3', 'id': 1L} Traceback (most recent call last): File "Abe/DataStore.py", line 2689, in catch_up store.catch_up_dir(dircfg) File "Abe/DataStore.py", line 2948, in catch_up_dir store.import_blkdat(dircfg, ds, blkfile['name']) File "Abe/DataStore.py", line 3072, in import_blkdat store.import_block(b, chain = chain) File "Abe/DataStore.py", line 1784, in import_block tx['tx_id'] = store.import_tx(tx, pos == 0) File "Abe/DataStore.py", line 2248, in import_tx store.binin(txout['scriptPubKey']), pubkey_id)) File "Abe/DataStore.py", line 521, in sql store._execute(cached, params) File "Abe/DataStore.py", line 498, in _execute store.cursor.execute(stmt, params) File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute self.errorhandler(self, exc, value) File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue OperationalError: (2006, 'MySQL server has gone away') Reconnecting after rollback error: (2006, 'MySQL server has gone away') Reconnecting to database.
happened twice, block_tx 54507 88357 blockchain is bitcoin testnet. mysql server is still there. ideas? Enabled log-sql: EXEC: INSERT INTO tx (tx_id, tx_hash, tx_version, tx_lockTime, tx_size) VALUES (%s, %s, %s, %s, %s) (88379L, 'c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c', 1, 0, 998171) EXEC: INSERT INTO txout_seq () VALUES () () EXEC: SELECT LAST_INSERT_ID() () FETCH: (115923L,) EXEC: INSERT INTO txout ( txout_id, tx_id, txout_pos, txout_value, txout_scriptPubKey, pubkey_id ) VALUES (%s, %s, %s, %s, %s, %s) (115923L, 88379L, 0, 0, 'fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff ffff fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff fffff [...cut... many, many, MANY, 'f']
This is the query that screws everything up Edit: WELL, found the problem: http://blockexplorer.com/testnet/tx/c333a53f0174166236e341af9cad795d21578fb87ad7a1b6d2cf8aa9c722083c (warning, it could crash or hang the browser) a crazy block in testnet blockchain. Now I have to figure out how to skip this block. Edit2: Asked myself "why skipping the block if you can include it, also if its crazy?". Answer was setting max_allowed_packet in my.cnf from 1MB to 8MB. Now it works.
|
|
|
|
|