Bitcoin Forum
March 26, 2017, 09:21:01 AM *
News: Latest stable version of Bitcoin Core: 0.14.0  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 [36] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 183866 times)
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 22, 2014, 07:25:37 AM
 #701

Is there a reason why my questions don't get answered ?

Two possible reasons: bad timing (I am busy) and no obvious answer.

You have several questions, and it is not clear what you are currently using in terms of configuration and modifications to the code.  You mentioned commenting out the Merkle root check, but your latest error shows it still occurring.  Are you able to load blocks without the check?

I have converted your hex dump back to binary, commented out the Merkle root check, installed the ltc_scrypt module, and loaded the block with this command:

Code:
python -m Abe.abe --dbtype sqlite3 --connect-args :memory: --port 2750 --datadir '[{"dirname":".", "chain":"GrowthCoin", "policy":"NovaCoin"}]'

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
1490520061
Hero Member
*
Offline Offline

Posts: 1490520061

View Profile Personal Message (Offline)

Ignore
1490520061
Reply with quote  #2

1490520061
Report to moderator
1490520061
Hero Member
*
Offline Offline

Posts: 1490520061

View Profile Personal Message (Offline)

Ignore
1490520061
Reply with quote  #2

1490520061
Report to moderator
1490520061
Hero Member
*
Offline Offline

Posts: 1490520061

View Profile Personal Message (Offline)

Ignore
1490520061
Reply with quote  #2

1490520061
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1490520061
Hero Member
*
Offline Offline

Posts: 1490520061

View Profile Personal Message (Offline)

Ignore
1490520061
Reply with quote  #2

1490520061
Report to moderator
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 22, 2014, 02:19:31 PM
 #702

Is there a reason why my questions don't get answered ?

Two possible reasons: bad timing (I am busy) and no obvious answer.



ok thanks, no offense, it seemed as my questions were overlooked, but I understand  Wink

Is there a reason why my questions don't get answered ?

You have several questions, and it is not clear what you are currently using in terms of configuration and modifications to the code.  You mentioned commenting out the Merkle root check, but your latest error shows it still occurring.  Are you able to load blocks without the check?

I have converted your hex dump back to binary, commented out the Merkle root check, installed the ltc_scrypt module, and loaded the block with this command:

Code:
python -m Abe.abe --dbtype sqlite3 --connect-args :memory: --port 2750 --datadir '[{"dirname":".", "chain":"GrowthCoin", "policy":"NovaCoin"}]'


I wasn't able to load after commenting out the merkle root check, I'll give it a shot like you did, maybe I passed too many parameters on datadir option.

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 23, 2014, 08:34:17 PM
 #703



Ok I'm still trying to figure this thing out... I've manage to get passed the invalid address error (not sure why but I tried putting the chain specifcation into the config file instead of the python file).  But now moving forward on this I get yet another error!

Code:
root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/bitcoin-abe/Abe/abe.py", line 2276, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/root/bitcoin-abe/Abe/abe.py", line 2270, in main
    store = make_store(args)
  File "/root/bitcoin-abe/Abe/abe.py", line 153, in make_store
    store = DataStore.new(args)
  File "Abe/DataStore.py", line 3446, in new
    return DataStore(args)
  File "Abe/DataStore.py", line 165, in __init__
    store.init_conn()
  File "Abe/DataStore.py", line 226, in init_conn
    store.conn = store.connect()
  File "Abe/DataStore.py", line 237, in connect
    conn = store._connect(cargs)
  File "Abe/DataStore.py", line 269, in _connect
    return store.module.connect(**cargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'abe'@'localhost' (using password: YES)")
root@tmp:~/bitcoin-abe# nano /home/abe/abe-mysql.conf
root@tmp:~/bitcoin-abe# python -m Abe.abe --config /home/abe/abe-mysql.conf --commit-bytes 100000 --no-serve
ddl_implicit_commit=true
create_table_epilogue=''
Abe/DataStore.py:536: Warning: Converting column 'a' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'b' from VARCHAR to TEXT
  store.cursor.execute(stmt)
max_varchar=4294967295
clob_type=LONGTEXT
Abe/DataStore.py:536: Warning: Converting column 'test_varbit' from VARCHAR to TEXT
  store.cursor.execute(stmt)
binary_type=hex
int_type=int
Created silly table abe_dual
sequence_type=mysql
limit_style=native
Abe/DataStore.py:536: Warning: Converting column 'txout_scriptPubKey' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Abe/DataStore.py:536: Warning: Converting column 'txin_scriptSig' from VARCHAR to TEXT
  store.cursor.execute(stmt)
Assigned chain_id 1 to GrowthCoin
Opened /root/.GrowthCoin/blk0001.dat
Chain 1 genesis tx: 020000001d0bfa51010000000000000000000000000000000000000000000000000000000000000000ffffffffb904ffff001d020f274caf4a756c792033312c2032303133203132706d204544543a2054686520552e532e2065636f6e6f6d79206772657720312e372520696e20746865207365636f6e6420717561727465722c20616964656420627920736f6c696420636f6e73756d6572207370656e64696e6720616e6420612072616d702d757020696e20627573696e65737320696e766573746d656e742c2074686520676f7665726e6d656e742073616964205765646e65736461792effffffff0100000000000000000000000000
Exception at 368
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': 1L, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 1L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2687, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2951, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3083, in import_blkdat
    store.import_block(b, chain = chain)
  File "Abe/DataStore.py", line 1805, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=d5e4fc54c53a9a20bbe01e70d8b47ce66f15c1bd3504f7e80390bf5601150ded

I've read to comment out in DataStore.py
Code:
# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)

But as you can image if I post this... it didn't work!

It gave me a few of these

Code:
Skipped 73 bytes at block end
block_tx 970 970
Skipped 71 bytes at block end
block_tx 971 971
Skipped 72 bytes at block end
block_tx 972 972
Skipped 72 bytes at block end
Exception at 1677721600398062
Failed to catch up {'blkfile_offset': 206488, 'blkfile_number': 1, 'chain_id': 1, 'loader': None, 'conf': u'GrowthCoin.conf', 'dirname': u'/root/.GrowthCoin', 'id': 2L}
Traceback (most recent call last):

Then
Code:
error: unpack_from requires a buffer of at least 4 bytes

Witch I had earlier in my attempts to get Abe read the blockchain.  It looks to me as the magic bytes are causing troubles again.

Any inputs/help/solutions on this would be appreciated  Wink

Ok tried what you said John, but I reposted that quote since I get the same message as previously when I tryed to comment out the MerkleRoot check.

It goes pass the 370th byte but fails at byte 1677721600398062

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 24, 2014, 02:42:34 PM
 #704

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field.  By default, Abe uses double-SHA256, the same as Bitcoin.  Many coins use scrypt.  I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code.  Look for "NovaCoin" in the sources for an example.


Hi John,
Could you elaborate abit further on this? much appreciated. What few lines of code are needed?

^ I am with STUPID!
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 24, 2014, 03:31:49 PM
 #705

Ok tried what you said John, but I reposted that quote since I get the same message as previously when I tryed to comment out the MerkleRoot check.

It goes pass the 370th byte but fails at byte 1677721600398062

"Exception at <impossibly huge file offset>" means an unexpected block format.  Where Abe expects a block size or script length, there is some other data.  Abe interprets it as a huge number, uses it as the blockfile pointer, and reports the error as coming from there.  See: https://github.com/bitcoin-abe/bitcoin-abe/issues/4.  Try either the ppcoin patch or RPC mode.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 24, 2014, 03:39:32 PM
 #706

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field.  By default, Abe uses double-SHA256, the same as Bitcoin.  Many coins use scrypt.  I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code.  Look for "NovaCoin" in the sources for an example.


Hi John,
Could you elaborate abit further on this? much appreciated. What few lines of code are needed?

Come to think of it, if the format is exactly NovaCoin's, you might not need a code change.  In datadir, add "policy":"NovaCoin" along with "chain" etc.

The changes are mostly in Abe/Chain.py: add a class defined like NovaCoin (or CryptoCash) and a corresponding conditional in the create function.  In Abe/DataStore.py, where it says
Code:
   {"chain":"NovaCoin"},
    {"chain":"CryptoCash"},

add a similar line.

Edit: and if it works, post your changes, or better yet, create a GitHub pull request so I can add them.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 24, 2014, 03:42:47 PM
 #707

Shows only the first block. The thing I do not understand.

When Abe loads many blocks but shows only the genesis block, it means that Abe is using the wrong algorithm to calculate the block header hash as it appears in the next block's hashPrev field.  By default, Abe uses double-SHA256, the same as Bitcoin.  Many coins use scrypt.  I would like to make this selectable by datadir=[{...,"hashPrev":"scrypt"}], but for now it requires a few lines of code.  Look for "NovaCoin" in the sources for an example.


Hi John,
Could you elaborate abit further on this? much appreciated. What few lines of code are needed?

Come to think of it, if the format is exactly NovaCoin's, you might not need a code change.  In datadir, add "policy":"NovaCoin" along with "chain" etc.

The changes are mostly in Abe/Chain.py: add a class defined like NovaCoin (or CryptoCash) and a corresponding conditional in the create function.  In Abe/DataStore.py, where it says
Code:
   {"chain":"NovaCoin"},
    {"chain":"CryptoCash"},

add a similar line.

Edit: and if it works, post your changes, or better yet, create a GitHub pull request so I can add them.

Thanks John, will give this a shot. Actually trying this on HiroCoin which is a fork of litecoin.

EDIT : Am I being dumb or have I missed something ?
Code:
ImportError: No module named ltc_scrypt

Here is where I think the issue is (block_id 2) :
Code:
*************************** 1. row ***************************
             block_id: 1
           block_hash: ae94c51e61c87f32a2a6c755df00b1ffa130c73b1bd165aaccaa3f90f8d243a9
        block_version: 1
 block_hashMerkleRoot: b0019d92bc054f7418960c91e252e7d24c77719c7a30128c5f6a827c73095d2a
          block_nTime: 1394723131
          block_nBits: 504365040
         block_nNonce: 1234746574
         block_height: 0
        prev_block_id: NULL
      search_block_id: NULL
     block_chain_work: 0000000000000000000000000000000000000000000000000000000000000000000000100010
       block_value_in: 0
      block_value_out: 40000000000
 block_total_satoshis: 40000000000
  block_total_seconds: 0
block_satoshi_seconds: 0
       block_total_ss: 0
         block_num_tx: 1
   block_ss_destroyed: 0
*************************** 2. row ***************************
             block_id: 2
           block_hash: 54a642a1aabb71a5b9d57a6455ba4ab7f69641e2ebe24ca18dfdc4bd9d98e198
        block_version: 2
 block_hashMerkleRoot: cf42adf0dbda83114f23f3c771700b1daf107aaf57f935f1d6bfcc568f38aa4f
          block_nTime: 1394867933
          block_nBits: 504365040
         block_nNonce: 264378
         block_height: NULL
        prev_block_id: NULL
      search_block_id: NULL
     block_chain_work: NULL
       block_value_in: 0
      block_value_out: 40000000000
 block_total_satoshis: -1
  block_total_seconds: NULL
block_satoshi_seconds: NULL
       block_total_ss: NULL
         block_num_tx: 1
   block_ss_destroyed: NULL

^ I am with STUPID!
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 24, 2014, 04:11:01 PM
 #708

Thanks John, will give this a shot. Actually trying this on HiroCoin which is a fork of litecoin.

EDIT : Am I being dumb or have I missed something ?
Code:
ImportError: No module named ltc_scrypt

Install ltc_scrypt as per README.txt:

Quote
NovaCoin and CryptoCash support depends on the ltc_scrypt module
available from https://github.com/CryptoManiac/bitcoin-abe (see
README-SCRYPT.txt).

Here is where I think the issue is (block_id 2) :
Code:
*************************** 1. row ***************************
             block_id: 1
           block_hash: ae94c51e61c87f32a2a6c755df00b1ffa130c73b1bd165aaccaa3f90f8d243a9
        block_version: 1
 block_hashMerkleRoot: b0019d92bc054f7418960c91e252e7d24c77719c7a30128c5f6a827c73095d2a
          block_nTime: 1394723131
          block_nBits: 504365040
         block_nNonce: 1234746574
         block_height: 0
        prev_block_id: NULL
      search_block_id: NULL
     block_chain_work: 0000000000000000000000000000000000000000000000000000000000000000000000100010
       block_value_in: 0
      block_value_out: 40000000000
 block_total_satoshis: 40000000000
  block_total_seconds: 0
block_satoshi_seconds: 0
       block_total_ss: 0
         block_num_tx: 1
   block_ss_destroyed: 0
*************************** 2. row ***************************
             block_id: 2
           block_hash: 54a642a1aabb71a5b9d57a6455ba4ab7f69641e2ebe24ca18dfdc4bd9d98e198
        block_version: 2
 block_hashMerkleRoot: cf42adf0dbda83114f23f3c771700b1daf107aaf57f935f1d6bfcc568f38aa4f
          block_nTime: 1394867933
          block_nBits: 504365040
         block_nNonce: 264378
         block_height: NULL
        prev_block_id: NULL
      search_block_id: NULL
     block_chain_work: NULL
       block_value_in: 0
      block_value_out: 40000000000
 block_total_satoshis: -1
  block_total_seconds: NULL
block_satoshi_seconds: NULL
       block_total_ss: NULL
         block_num_tx: 1
   block_ss_destroyed: NULL

If you mean all those NULLs, they result from the incorrect block hash algorithm.  Abe does not see block_id 1 as the parent of block_id 2, so it can not compute chain statistics or even height (block number).  See if ltc_scrypt helps.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 24, 2014, 04:21:26 PM
 #709

<snip>
If you mean all those NULLs, they result from the incorrect block hash algorithm.  Abe does not see block_id 1 as the parent of block_id 2, so it can not compute chain statistics or even height (block number).  See if ltc_scrypt helps.


Ok tried this with ltc_scrypt and policy:novacoin and now get this..
Dies at block 9590 rather than before it was getting to full 26000 .... Same output on the web interface, only shows block 0.

Code:
block_tx 9590 1
Skipped 92 bytes at block end
commit
Exception at 5777995176649507762
Failed to catch up {'blkfile_offset': 1917723, 'blkfile_number': 100000, 'chain_id': 11, 'loader': None, 'conf': None, 'dirname': u'/home/hiro/.hirocoin', 'id': 4L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2789, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 3053, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3175, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "Abe/Chain.py", line 121, in ds_parse_block
    d = Chain.ds_parse_block(chain, ds)
  File "Abe/Chain.py", line 59, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "Abe/Chain.py", line 118, in ds_parse_transaction
    return deserialize.parse_Transaction(ds, has_nTime=True)
  File "Abe/deserialize.py", line 94, in parse_Transaction
    d['txOut'].append(parse_TxOut(vds))
  File "Abe/deserialize.py", line 67, in parse_TxOut
    d['value'] = vds.read_int64()
  File "Abe/BCDataStream.py", line 72, in read_int64
    def read_int64  (self): return self._read_num('<q')
  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 8 bytes
Abe initialized.

^ I am with STUPID!
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 24, 2014, 04:35:42 PM
 #710

<snip>
If you mean all those NULLs, they result from the incorrect block hash algorithm.  Abe does not see block_id 1 as the parent of block_id 2, so it can not compute chain statistics or even height (block number).  See if ltc_scrypt helps.


Ok tried this with ltc_scrypt and policy:novacoin and now get this..
Dies at block 9590 rather than before it was getting to full 26000 .... Same output on the web interface, only shows block 0.

Code:
block_tx 9590 1
Skipped 92 bytes at block end
commit
Exception at 5777995176649507762
Failed to catch up {'blkfile_offset': 1917723, 'blkfile_number': 100000, 'chain_id': 11, 'loader': None, 'conf': None, 'dirname': u'/home/hiro/.hirocoin', 'id': 4L}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2789, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 3053, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "Abe/DataStore.py", line 3175, in import_blkdat
    b = chain.ds_parse_block(ds)
  File "Abe/Chain.py", line 121, in ds_parse_block
    d = Chain.ds_parse_block(chain, ds)
  File "Abe/Chain.py", line 59, in ds_parse_block
    d['transactions'].append(chain.ds_parse_transaction(ds))
  File "Abe/Chain.py", line 118, in ds_parse_transaction
    return deserialize.parse_Transaction(ds, has_nTime=True)
  File "Abe/deserialize.py", line 94, in parse_Transaction
    d['txOut'].append(parse_TxOut(vds))
  File "Abe/deserialize.py", line 67, in parse_TxOut
    d['value'] = vds.read_int64()
  File "Abe/BCDataStream.py", line 72, in read_int64
    def read_int64  (self): return self._read_num('<q')
  File "Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
error: unpack_from requires a buffer of at least 8 bytes
Abe initialized.

Not sure if that was going to work anyway since its using NovaCoin which HiroCoin is not a fork of.
Trying the method on defined class with :

Code:
class HiroCoin(NvcChain):                
    def __init__(chain, **kwargs):
        chain.name = 'HiroCoin'
        chain.code3 = 'HIC'      
        chain.address_version = "\x28"        
        chain.magic = "\xfe\xc3\xb9\xde"
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = "hirocoin.conf"      
    datadir_rpcport = 9347

And adding "policy: HiroCoin" to the datadir made no difference .... it runs through the chain perfectly but again lots of NULLS in the DB.

P.S. HiroCoin is not POS like NovaCoin so not sure if I am going down the wrong path here. Its based around DRK Coin and X11.

^ I am with STUPID!
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
March 25, 2014, 05:41:18 AM
 #711

Update:

So I tried to use ppcoin branch to get Abe to read the GrowthCoin database without success.

I also tried to manually load the genesis transaction without success (I had to hexdump the blk0001.dat file since getblock 0 returned and error in the client, similar to the bitcoin issue).  ABE complained of an odd lenght transaction with import-tx parameter in abe.conf.

So if I load with the blkfile loader and merkleroot check commented out, I get up to 971 blocks then it fails with

Code:
block_tx 969 969
Skipped 72 bytes at block end
block_tx 970 970
Skipped 73 bytes at block end
block_tx 971 971
Skipped 73 bytes at block end
Exception at 266991
Failed to catch up {'blkfile_offset': 264462, 'blkfile_number': 1, 'chain_id': Decimal('8'), 'loader': None, 'dirname': u'/root/.GrowthCoin', 'id': 2L}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2631, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2895, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3022, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3053, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 89, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 45, in parse_TxIn
    d['scriptSig'] = vds.read_bytes(vds.read_compact_size())
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 59, in read_bytes
    result = self.input[self.read_cursor:self.read_cursor+length]
MemoryError


With current master version of ABE, If I try to load with rpc, it will work but the frontend crashes as int(ss) is of the NoneType.  I checked the database and I have NULL values after ~3k blocks mark.... So it will load the blocks but it misses some info.

With PPCOIN branch, it was prior to ABE's NovaCoin upgrade so I assumen datadir wasn't implemented yet at that point and it doesn't accept my growthcoin.conf file (it says bitcoin.conf)  still trying to find were can I input the correct conf file but I am not sure it will work since I guess the block format is a lot different from NovaCoin yet it is a descendent of it (and Diamond).

Trying to contact the dev to have more info on that block format

EDIT: does this decription give you any clue if ABE can parse the blockchain data correctly
GrowthCoin - a fork of Diamond, with several algorithms fixed and improved. It is decendent of Novacoin/PPCoin with Pow/PoS. It combines the great features from Luckycoin (random blocks) and Florincoin (transaction message).

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 25, 2014, 07:14:15 AM
 #712

Folks,

The block format is defined in C++ in a call to the IMPLEMENT_SERIALIZE macro in classes CBlockHeader, CTransaction, CTxIn, and CTxOut.  The relevant sections of Bitcoin source code are:

https://github.com/bitcoin/bitcoin/blob/v0.9.0/src/core.h#L363-369
https://github.com/bitcoin/bitcoin/blob/v0.9.0/src/core.h#L201-205
https://github.com/bitcoin/bitcoin/blob/v0.9.0/src/core.h#L87-89
https://github.com/bitcoin/bitcoin/blob/v0.9.0/src/core.h#L134-135

That is what we translate into Python here:

https://github.com/bitcoin-abe/bitcoin-abe/blob/v0.7.2/Abe/deserialize.py#L163-168
https://github.com/bitcoin-abe/bitcoin-abe/blob/v0.7.2/Abe/deserialize.py#L82-91
https://github.com/bitcoin-abe/bitcoin-abe/blob/v0.7.2/Abe/deserialize.py#L42-45
https://github.com/bitcoin-abe/bitcoin-abe/blob/v0.7.2/Abe/deserialize.py#L65-66

If you can figure out the correspondence between those two sets of code snippets, you can read any chain that uses Bitcoin's serialization facility.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 25, 2014, 07:22:35 AM
 #713

With current master version of ABE, If I try to load with rpc, it will work but the frontend crashes as int(ss) is of the NoneType.

My knee-jerk fix for this type of bug is to replace "int(ss)" with "None if ss is None else int(ss)".

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
ShadesOfMarble
Donator
Hero Member
*
Offline Offline

Activity: 543



View Profile
March 25, 2014, 10:29:25 AM
 #714

I haven't kept up with the development of Abe, but is P2SH/Multisig supported now?

Review of the Spondoolies-Tech SP10 „Dawson“ Bitcoin miner (1.4 TH/s)

[22:35] <Vinnie_win> Did anyone get paid yet? | [22:36] <Isokivi> pirate did!
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 25, 2014, 10:48:15 AM
 #715


Hi John, double checked all that and the DarkCoin/HiroCoin code has had no alterations on any of those classes on IMPLEMENT_SERIALIZE macro.
Any other ideas on what could be the issue?

^ I am with STUPID!
dasource
Hero Member
*****
Offline Offline

Activity: 815



View Profile
March 25, 2014, 11:26:41 AM
 #716

This is how I setup the class...

Code:
class HiroCoin(LtcScryptChain):          
    def __init__(chain, **kwargs):
        chain.name = 'HiroCoin'
        chain.code3 = 'HIC'      
        chain.address_version = "\x28"        
        chain.magic = "\xfe\xc3\xb9\xde"
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = "hirocoin.conf"      
    datadir_rpcport = 9347        

EDIT: This may or may not help, but all the block past block 0 are all in orphan_block table.
Checking the code concludes this is because it cannot see the previous block hash correctly.

^ I am with STUPID!
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 25, 2014, 03:05:19 PM
 #717

I haven't kept up with the development of Abe, but is P2SH/Multisig supported now?

Apart from an issue affecting Namecoin and the question whether to include "escrow" outputs in address balances, yes, on the multisig branch, thanks to Jouke's sponsorship.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 25, 2014, 03:14:32 PM
 #718

This is how I setup the class...

Code:
class HiroCoin(LtcScryptChain):          
    def __init__(chain, **kwargs):
        chain.name = 'HiroCoin'
        chain.code3 = 'HIC'      
        chain.address_version = "\x28"        
        chain.magic = "\xfe\xc3\xb9\xde"
        Chain.__init__(chain, **kwargs)

    datadir_conf_file_name = "hirocoin.conf"      
    datadir_rpcport = 9347        

EDIT: This may or may not help, but all the block past block 0 are all in orphan_block table.
Checking the code concludes this is because it cannot see the previous block hash correctly.


Please see: https://bitcointalk.org/index.php?topic=22785.msg5785774#msg5785774 (and note that "policy":"NovaCoin" has an effect similar to the proposed "hashPrev":"scrypt").

If the algorithm is neither Bitcoin's nor NovaCoin's, we have to override block_header_hash in the new class:
Code:
class Sha256Chain(Chain):
    def block_header_hash(chain, header):
        return util.double_sha256(header)
Code:
class LtcScryptChain(Chain):
    def block_header_hash(chain, header):
        import ltc_scrypt
        return ltc_scrypt.getPoWHash(header)

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
molecular
Donator
Legendary
*
Online Online

Activity: 2240



View Profile
March 25, 2014, 06:26:42 PM
 #719

I have a question about mempool transactions regarding performance:

So I'm running http://blockexplorer.auroracoin.eu and because I have allocated quite a machine to the task everything has been zappy and fine this morning.

However when I checked back from work an hour later I saw loads of exceptions saying "error: [Errno 32] Broken pipe", the nginx I have in front reporting gateway timeouts. I'm hypothesizing the db queries are the bottleneck.

I tried rebuilding the database (drop it completely and rebuild)... that didn't help, it started again right away.

There are loads of mempool transactions in AURoracoin because we're being pool-hopping-attacked to the point where there hasn't been a block for 6 hours or so.

Another instance I run with same setup, but on quite weak machine (vm) was able to cope with the load quite well and didn't suffer broken pipes.

What fixed it was to re-initialize BOTH the blockchain of the AuroraCoind AND the db (just db didn't help).

Now the question: how are mempool transactions handled and could the existance of many mempool transactions have considerable impact on db (or abe.py) performance?

I'm a bit confused... does anyone have an idea what could've been causing this?

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
jeffmthomas
Newbie
*
Offline Offline

Activity: 4


View Profile
March 25, 2014, 06:31:19 PM
 #720

Just posted a bounty for Memorycoin

https://bountify.co/get-memorycoin-working-with-bitcoin-abe
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 [36] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!