Bitcoin Forum
May 27, 2019, 12:44:22 PM *
News: Latest Bitcoin Core release: 0.18.0 [Torrent] (New!)
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: [help request] Setting up ABE and Electrum for Terracoin  (Read 3301 times)
crazy_rabbit
Legendary
*
Offline Offline

Activity: 1176
Merit: 1001


RUM AND CARROTS: A PIRATE LIFE FOR ME


View Profile
December 20, 2012, 04:15:54 PM
 #1

So I'm trying to setup ABE for Terracoin with the final goal of setting up an Electrum Service for Terracoin, but I'm a bit stuck on the ABE setup.

I think I've gotten everything setup for ABE but I keep getting the following message when I try to run abe so it can scan the blockfile:

:~/bitcoin-abe$ python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
Chain not found for magic number 'B\xba\xbeV' in block file /home/ubuntu/.terracoin/blocks/blk00000.dat at offset 0.  If file contents have changed, consider forcing a rescan: UPDATE datadir SET blkfile_number=1, blkfile_offset=0 WHERE dirname='/home/ubuntu/.terracoin/'

I really don't understand what the "Magic Number" part is talking about, but I tried forcing a rescan and that doesn't work. MySQL says there is an error in my syntax near the "dirname='......' part.

So I'm stumped at the moment. It seems to otherwise work, but this part is tricky.


more or less retired.
PLAY OVER 3000 GAMES
LIGHTNING FAST WITHDRAWALS
PLAY NOW
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1558961062
Hero Member
*
Offline Offline

Posts: 1558961062

View Profile Personal Message (Offline)

Ignore
1558961062
Reply with quote  #2

1558961062
Report to moderator
celkaris
Full Member
***
Offline Offline

Activity: 148
Merit: 100


View Profile
December 20, 2012, 04:33:20 PM
 #2

in network applications, a "magic byte" (or in our case, "magic bytes") is the first byte contained in a packet, part of a custom binary protocol, so application may immediately identify its own protocol by just inspecting this magic byte.

Terracoin "live net" uses the following :
( https://github.com/terracoin/terracoin/blob/master/src/main.cpp#L2709 )
unsigned char pchMessageStart[4] = { 0x42, 0xba, 0xbe, 0x56 };
crazy_rabbit
Legendary
*
Offline Offline

Activity: 1176
Merit: 1001


RUM AND CARROTS: A PIRATE LIFE FOR ME


View Profile
December 20, 2012, 04:36:17 PM
 #3

in network applications, a "magic byte" (or in our case, "magic bytes") is the first byte contained in a packet, part of a custom binary protocol, so application may immediately identify its own protocol by just inspecting this magic byte.

Terracoin "live net" uses the following :
( https://github.com/terracoin/terracoin/blob/master/src/main.cpp#L2709 )
unsigned char pchMessageStart[4] = { 0x42, 0xba, 0xbe, 0x56 };


Thanks! I didn't know that. A starting point in trying to get this working. :-)

more or less retired.
celkaris
Full Member
***
Offline Offline

Activity: 148
Merit: 100


View Profile
December 20, 2012, 04:37:37 PM
 #4

good luck, i earlier wanted to setup a home-made blockchain parser, with http/web rendering, but finally gave up when http://cryptocoinexplorer.com:3750/ went live Smiley
dreamwatcher
Legendary
*
Offline Offline

Activity: 1064
Merit: 1000


View Profile WWW
December 20, 2012, 04:59:44 PM
 #5

I can help.... Grin

You will need to edit the Datastore.py file both for simplicity (No need to have it checking for all the coins it is preset for) and I have found the IP address transaction REGEX structure needs to be changed (For both PPC and TRC)

In the Datastore.py file change:


Code:
CHAIN_CONFIG = [
    {"chain":"Terracoin",
     "code3":"TRC", "address_version":"\x00", "magic":"\x42\xba\xbe\x56"},
    ]

Just delete the other coin definitions.

The Regex for IP address transactions needs to be changed:

Code:
# Regex to match a pubkey ("IP address transaction") in txout_scriptPubKey.
SCRIPT_PUBKEY_RE = re.compile("\x21(.{33})\xac\\Z", re.DOTALL)


The Cryptocoin Explorer site uses this as its config file:

Code:
dbtype MySQLdb
connect-args {"user":"<your database user name for abe>","db":"<database name>","passwd":"<user password>"}
upgrade
port 3750
host 96.126.122.8
datadir = [{
        "dirname": "<your .terracoin directory (expressed as a full path) >",
        "chain":   "Terracoin",
        "code3":   "TRC",
        "address_version": "\u0000" }]

The host will default to localhost. If you need to have ABE available outside a local network, you will need to host set to the outside facing IP address (Like Cryptocoin explorer is at 96.126.122.8 )

Port defaults to 2750, but I have it set at 3750 on the server so it does not conflict with the PPC explorer.


If you have any more questions/issues I will try to help the best I can.
crazy_rabbit
Legendary
*
Offline Offline

Activity: 1176
Merit: 1001


RUM AND CARROTS: A PIRATE LIFE FOR ME


View Profile
December 20, 2012, 06:06:55 PM
 #6

I can help.... Grin
If you have any more questions/issues I will try to help the best I can.

Looks like it's working so far....Thanks! I'll let everyone know if I get Electrum Working. :-)

Now I have to wait some time for ABE to index everything.

more or less retired.
crazy_rabbit
Legendary
*
Offline Offline

Activity: 1176
Merit: 1001


RUM AND CARROTS: A PIRATE LIFE FOR ME


View Profile
December 20, 2012, 06:37:21 PM
 #7

Got to the end of the Terracoin chain and got this error:

Exception at 12175175
Failed to catch up {'blkfile_number': 100000, 'dirname': u'/home/ubuntu/.terracoin/', 'chain_id': Decimal('1'), 'id': 1L, 'blkfile_offset': 12125505}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2389, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2442, in catch_up_dir
    store.import_blkdat(dircfg, ds, filename[0])
  File "Abe/DataStore.py", line 2566, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1642, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=4be7570e8f70eb093640c8468274ba759745a7aa2b7d25ab1e0421b259845014

 Huh
I was running Terracoind at the same time as Abe, would that be a problem? Although at the time it got this exception, it was at: block_tx 33750 42496

And Terracoind was saying there were this many blocks: 33747

hmmm....

more or less retired.
dreamwatcher
Legendary
*
Offline Offline

Activity: 1064
Merit: 1000


View Profile WWW
December 20, 2012, 07:21:58 PM
 #8

Got to the end of the Terracoin chain and got this error:

Exception at 12175175
Failed to catch up {'blkfile_number': 100000, 'dirname': u'/home/ubuntu/.terracoin/', 'chain_id': Decimal('1'), 'id': 1L, 'blkfile_offset': 12125505}
Traceback (most recent call last):
  File "Abe/DataStore.py", line 2389, in catch_up
    store.catch_up_dir(dircfg)
  File "Abe/DataStore.py", line 2442, in catch_up_dir
    store.import_blkdat(dircfg, ds, filename[0])
  File "Abe/DataStore.py", line 2566, in import_blkdat
    store.import_block(b, chain_ids = chain_ids)
  File "Abe/DataStore.py", line 1642, in import_block
    raise MerkleRootMismatch(b['hash'], tx_hash_array)
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=4be7570e8f70eb093640c8468274ba759745a7aa2b7d25ab1e0421b259845014

 Huh
I was running Terracoind at the same time as Abe, would that be a problem? Although at the time it got this exception, it was at: block_tx 33750 42496

And Terracoind was saying there were this many blocks: 33747

hmmm....


Nope, it is a problem with the way TRC and PPC handle transactions. Not really a problem with the coins, but ABE does not understand some of the tweaks that were put into them. The ABE creator even gave this advice at one time, and this is what I do now:

In Datastore.py: Comment out ( With the # like below)

Code:
# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)

IF somebody wants to spend the time reprogramming the Merkle root structure  that ABE expects, more power to them. But for now, this is a 'dirty' way around it.
If anybody has found a better way around this issue, please tell because I would like a real solution myself.

Also With Terracoin (not PPC), after this fix, there is a second issue with ABE and the way difficulty is calculated. This bug does not effect anything because the issue is with predicting difficulty and corrects itself once the block is ready for hashing. The warning will also not terminate ABE, you just end up with alot of screen spam if you are monitoring it. I have tried to fix this one, but it has fallen down on my list of things to do because the difficulty ABE produces matches what the  daemon says (It differs slightly a few decimal places down, but it is an insignificant amount).

Code:
File "Abe/util.py", line 57, in calculate_target
    return (nBits & 0xffffff) << (8 * ((nBits >> 24) - 3))
ValueError: negative shift count

Anyway, good luck with your project, and if you come up with better ideas about these bugs, please let me know. I am always looking to improve my projects.
Sunny King
Legendary
*
Offline Offline

Activity: 1205
Merit: 1001



View Profile WWW
December 20, 2012, 10:16:35 PM
 #9


Nope, it is a problem with the way TRC and PPC handle transactions. Not really a problem with the coins, but ABE does not understand some of the tweaks that were put into them. The ABE creator even gave this advice at one time, and this is what I do now:

In Datastore.py: Comment out ( With the # like below)

Code:
# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)


Sounds strange. PPC did not change the way bitcoin calculates and verifies the merkle root hash, I don't think terracoin did either. So it's a mystery to me why abe would complain about merkle root mismatch here.

Any further info?
dreamwatcher
Legendary
*
Offline Offline

Activity: 1064
Merit: 1000


View Profile WWW
December 20, 2012, 11:20:07 PM
 #10


Nope, it is a problem with the way TRC and PPC handle transactions. Not really a problem with the coins, but ABE does not understand some of the tweaks that were put into them. The ABE creator even gave this advice at one time, and this is what I do now:

In Datastore.py: Comment out ( With the # like below)

Code:
# Verify Merkle root.
        #if b['hashMerkleRoot'] != util.merkle(tx_hash_array):
        #raise MerkleRootMismatch(b['hash'], tx_hash_array)


Sounds strange. PPC did not change the way bitcoin calculates and verifies the merkle root hash, I don't think terracoin did either. So it's a mystery to me why abe would complain about merkle root mismatch here.

Any further info?


I think it is mostly because Abe does not use Berkleydb (now leveldb in Bitcoin 0.7+) to acquire the block information. Instead it reads the BLK001 ...etc file and deserializes it looking for certain markers (Like the "magic key" to see the start of blocks). This makes it sensitive to any changes in structure, although John does continue to improve it.

For example, in PPC the extra transaction time stamp really threw it for a loop because it was not expecting it and thought it was part of the transaction hash. Software that uses the database libraries would have just skipped over that field because it wasn't looking for that field. However Abe reads everything directly from the file byte for byte looking for markers and assuming some information is always a certain length.

Anyway, John has released a new version and I will try to have a look at it when i have time. However the explorer site is running smoothly..<crossed fingers> and I find myself drawn to other projects.

roy7
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


View Profile
February 17, 2014, 02:28:48 PM
 #11

With an extra year of improvements to ABE, do you know if an Electrum wallet for TRC/PPC is feasible yet?
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!