Bitcoin Forum
December 08, 2016, 10:14:20 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2] 3 4 5 6 »  All
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 23890 times)
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
July 24, 2013, 04:24:50 AM
 #21

Is there any more information you need? Is there someway I can tell abe to connect to bitcoind with RPC over a specific port?

Sorry for the delayed reply.  Please post the last few lines of output before "catch_up_rpc: abort" or "Exception: RPC load failed."  Have you run Bitcoin with -reindex since adding txindex to bitcoin.conf?

RPC is slower for the initial load, I think over twice as slow as blockfiles.  Once caught up, it usually inserts blocks faster than the blockfile method, since RPC has already loaded most transactions by the time the block comes (assuming that you arrange for it to catch up frequently).

Abe normally does not "read the blockfiles over and over again": it stores the current position in the database.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
1481192060
Hero Member
*
Offline Offline

Posts: 1481192060

View Profile Personal Message (Offline)

Ignore
1481192060
Reply with quote  #2

1481192060
Report to moderator
1481192060
Hero Member
*
Offline Offline

Posts: 1481192060

View Profile Personal Message (Offline)

Ignore
1481192060
Reply with quote  #2

1481192060
Report to moderator
1481192060
Hero Member
*
Offline Offline

Posts: 1481192060

View Profile Personal Message (Offline)

Ignore
1481192060
Reply with quote  #2

1481192060
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481192060
Hero Member
*
Offline Offline

Posts: 1481192060

View Profile Personal Message (Offline)

Ignore
1481192060
Reply with quote  #2

1481192060
Report to moderator
katriel
Newbie
*
Offline Offline

Activity: 3


View Profile
July 24, 2013, 03:59:33 PM
 #22

Abe is really terrific! I used it to create a transaction network and play around with that.

However, on my machine I left it running (writing to a SQLite database) for I think about two weeks and only got about 18 months into the blockchain (2.4 million transactions). The readme says it typically takes several days to get through the chain--is there something I'm probably doing wrong?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
July 24, 2013, 04:21:50 PM
 #23

Abe is really terrific! I used it to create a transaction network and play around with that.

Good to hear!

However, on my machine I left it running (writing to a SQLite database) for I think about two weeks and only got about 18 months into the blockchain (2.4 million transactions). The readme says it typically takes several days to get through the chain--is there something I'm probably doing wrong?

I don't use SQLite for the full block chain, so I don't really know.  One user, K1773R, reported a fast load with the database file in tmpfs (RAM).  (https://bitcointalk.org/index.php?topic=22785.msg2639709#msg2639709)  I think K1773R uses SQLite.  If your system has enough memory, you could try that.  Abe takes a lot of space, though.  I estimate 4-5 times the size of all block files combined.  (~/.bitcoin/blocks/blk00???.dat in Unix)  You can reduce this somewhat by setting keep-scriptsig=false or by using the no-statistics branch and default-trim-depth.  See the comments in abe.conf for what those options entail.

K1773R mentioned possibly distributing the SQLite file via torrent: https://github.com/jtobey/bitcoin-abe/issues/15

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
johntawaki
Member
**
Offline Offline

Activity: 63



View Profile
July 24, 2013, 05:07:04 PM
 #24

nice one, but I heard it's not working for some new coins, such as Diamond (DMD).
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
July 24, 2013, 05:43:46 PM
 #25

nice one, but I heard it's not working for some new coins, such as Diamond (DMD).
Yes, several new coins trace their ancestry to PPCoin and need the "ppcoin" branch in Git.  See: https://github.com/jtobey/bitcoin-abe/issues/19

Another one, CopperLark, has its own format, which the "copperlark" branch handles.

Ideally, Abe would automatically detect among known variants and handle them without code changes.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
internetbitcoins
Newbie
*
Offline Offline

Activity: 12


View Profile
August 13, 2013, 11:13:58 PM
 #26

Hi,

I run
Code:
python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
but 2 or 3 times I had to restart the server, so now in the block table, I check on block_id and there are some missing blocks.

Is there a way to recover them without restart all the process of reading all the blocks on the blockchain??
For example, any way to launch the
Code:
python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
only for missing ids?

Thanks in advances and for Abe!
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 14, 2013, 01:47:19 PM
 #27

I run
Code:
python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
but 2 or 3 times I had to restart the server, so now in the block table, I check on block_id and there are some missing blocks.

block_id is not the same as block number (a.k.a. block_height).  The gaps in block_id are not a problem.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
August 22, 2013, 11:40:07 PM
 #28

Hey John Tobey, I am trying to set up the explorer for the DMD coin, I've read the issue about PPcoin branches, could you give any help on setting this up for DMD?

thanks

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 23, 2013, 12:54:23 AM
 #29

Hey John Tobey, I am trying to set up the explorer for the DMD coin, I've read the issue about PPcoin branches, could you give any help on setting this up for DMD?

thanks
Sure, just post how far you got and any detailed error message.  If the transactions have a new format, we may need to find how the coin's source code serializes CTransaction.  Bitcoin example: src/core.h line 194.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
August 23, 2013, 03:23:01 AM
 #30

Hey John Tobey, I am trying to set up the explorer for the DMD coin, I've read the issue about PPcoin branches, could you give any help on setting this up for DMD?

thanks
Sure, just post how far you got and any detailed error message.  If the transactions have a new format, we may need to find how the coin's source code serializes CTransaction.  Bitcoin example: src/core.h line 194.



This is the error message I get when trying to read the block chain

Code:

root@blockchains:/var/www/blockchain/bitcoin-abe/Abe# python -m Abe.abe --config abe-my.conf --commit-bytes 100000 --no-serve
failed to load /root/.Diamond/bitcoin.conf: [Errno 2] No such file or directory: u'/root/.Diamond/bitcoin.conf'
catch_up_rpc: abort
Opened /root/.Diamond/blk0001.dat
Exception at 72056513853653130
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': Decimal('9'), 'loader': None, 'dirname': u'/root/.Diamond', 'id': 24L}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2637, in catch_up
    store.catch_up_dir(dircfg)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2895, in catch_up_dir
    store.import_blkdat(dircfg, ds, blkfile['name'])
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3022, in import_blkdat
    b = store.parse_block(ds, chain_id, magic, length)
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 3053, in parse_block
    d['transactions'].append(deserialize.parse_Transaction(ds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 88, in parse_Transaction
    d['txIn'].append(parse_TxIn(vds))
  File "/usr/local/lib/python2.7/dist-packages/Abe/deserialize.py", line 46, in parse_TxIn
    d['sequence'] = vds.read_uint32()
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 71, in read_uint32
    def read_uint32 (self): return self._read_num('<I')
  File "/usr/local/lib/python2.7/dist-packages/Abe/BCDataStream.py", line 110, in _read_num
    (i,) = struct.unpack_from(format, self.input, self.read_cursor)
OverflowError: Python int too large to convert to C long


I am not sure if this is what you are looking for

https://github.com/diamondcoinproject/diamondcoin/blob/dd388c28bf89abeaaad761982bb4d04f18995c27/src/serialize.h#L59

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 23, 2013, 09:22:56 PM
 #31


Close enough, it's here, and the new field is strTxComment: https://github.com/diamondcoinproject/diamondcoin/blob/dd388c28bf89abeaaad761982bb4d04f18995c27/src/main.h#L452-462

I've downloaded some of the DMD chain and got Abe to load it, but only over RPC, since this coin uses strange block hashing rules.  Loading from the blockfile resulted in only Block 0 being displayed, since Abe could not link any block to its parent.  Also, I had to disable verification and make Abe trust the RPC server's block hashes.  (If you have info about the differences between DMD and Bitcoin block hashing, I may be able to fix this.)

Please pull branch "diamond", copy Diamond.conf as bitcoin.conf, add "rpcport=17772" to that file if not already present, and use a config such as this:

Code:
datadir [{"dirname":"/root/.Diamond", "chain":"Diamond", "code3":"DMD", "address_version":"\u005a"}]
default-loader rpc

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
tacoman71
Sr. Member
****
Offline Offline

Activity: 323



View Profile
August 23, 2013, 10:48:59 PM
 #32

You should make a newbie friendly .exe version of this that they can just double click to install.

Feeling generous? Like my post? Leave a tip at BTC: 1NZJ8cceqEiKDZGAJged2vNGCyfFMUEYPt
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 23, 2013, 11:07:22 PM
 #33

You should make a newbie friendly .exe version of this that they can just double click to install.
I have too little time.  I would happily show another developer around, should one wish to package it nicely.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
b!z
Legendary
*
Offline Offline

Activity: 1330



View Profile
August 24, 2013, 08:01:42 AM
 #34

Very cool. This looks like a good way to view the blockchain without relying on an external site.
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
August 26, 2013, 07:46:22 PM
 #35


Close enough, it's here, and the new field is strTxComment: https://github.com/diamondcoinproject/diamondcoin/blob/dd388c28bf89abeaaad761982bb4d04f18995c27/src/main.h#L452-462

I've downloaded some of the DMD chain and got Abe to load it, but only over RPC, since this coin uses strange block hashing rules.  Loading from the blockfile resulted in only Block 0 being displayed, since Abe could not link any block to its parent.  Also, I had to disable verification and make Abe trust the RPC server's block hashes.  (If you have info about the differences between DMD and Bitcoin block hashing, I may be able to fix this.)

Please pull branch "diamond", copy Diamond.conf as bitcoin.conf, add "rpcport=17772" to that file if not already present, and use a config such as this:

Code:
datadir [{"dirname":"/root/.Diamond", "chain":"Diamond", "code3":"DMD", "address_version":"\u005a"}]
default-loader rpc


I am having the following error

Code:
RPC data not understood: block hash mismatch
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': Decimal('9'), 'loader': None, 'dirname': u'/root/.Diamond', 'id': 36L}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/Abe/DataStore.py", line 2635, in catch_up
    raise Exception("RPC load failed")
Exception: RPC load failed
Abe initialized.

I have pulled the diamond branch from the repository
I have cp Diamond.conf. to bitcoin.conf (making sure the rpc port was on 17772)
change abe-my.conf accordingly

while the server listens... I can't seem to have the RPC loaded... did I miss something ?

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 26, 2013, 08:28:06 PM
 #36

Code:
RPC data not understood: block hash mismatch

Are you sure you are using the latest from the diamond branch?  These changes should be reflected in your Python files:

https://github.com/jtobey/bitcoin-abe/commit/829890b0c3bd2599e66108c24d90d16a5a1b0ea8

In particular, the part about "block hash mismatch" should appear commented out.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
August 27, 2013, 06:58:25 AM
 #37

yep, Git tells me I'm up to date and I have manually checked for those changes and everything is there.

any thing else?

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 27, 2013, 03:19:31 PM
 #38

yep, Git tells me I'm up to date and I have manually checked for those changes and everything is there.

any thing else?

Perhaps you have an older version installed on your system?  Otherwise, I don't see how Abe could produce that message.  What does this show:
Code:
python -c 'import Abe.abe; print Abe.abe;'

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
unick
Sr. Member
****
Offline Offline

Activity: 378


View Profile
August 27, 2013, 03:48:42 PM
 #39

yep, Git tells me I'm up to date and I have manually checked for those changes and everything is there.

any thing else?

Perhaps you have an older version installed on your system?  Otherwise, I don't see how Abe could produce that message.  What does this show:
Code:
python -c 'import Abe.abe; print Abe.abe;'


This is the output

Code:
<module 'Abe.abe' from '/usr/local/lib/python2.7/dist-packages/Abe/abe.pyc'>

Awesome Explorers for Awesome Coins | Show some BTC love here: 1AAYAZgaz2me7hyumexUZzcyGRZEYtCx5C
HoboNickels: hbn.blockx.info | BottleCaps: cap.blockx.info | GrowthCoin: grw.blockx.info
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
August 27, 2013, 04:12:26 PM
 #40

This is the output

Code:
<module 'Abe.abe' from '/usr/local/lib/python2.7/dist-packages/Abe/abe.pyc'>

This means you are getting the previously installed version.  You want the one from from 'Abe/abe.py'.  Either install the new version (as root, python setup.py install) or set PYTHONPATH to the git clone directory (bitcoin-abe, not the Abe subdirectory).  Normally, sys.path starts with an empty string, so if you "cd" to the correct directory, Python should find the local modules.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Pages: « 1 [2] 3 4 5 6 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!