Hi, I've been away the last few days and will take a while to catch up. datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks
Leave /blocks off. This should be: datadir = /Users/thisuser/Library/Application Support/Bitcoin
If I leave it off, the error message is "../Application Support/Bitcoin/blck0001.dat not found". In case I use /blocks, the error message points correctly to "../Application Support/Bitcoin/blocks/blck0001.dat". Now I am even more confused. While ABE searches after blck0001.dat, blck000 01.dat is existent, and would be perhaps the correct file? Is this a misconfiguration? Old versions of Bitcoin start with blk0001.dat, and new (0.8.x) ones start with blocks/blk00000.dat. To accommodate all known versions, including users who upgrade, Abe first searches in datadir for the blk*.dat series, and then in the blocks subdirectory for blk00000.dat. Since you use a new bitcoin-qt, it should not find blk0001.dat but should continue and load blocks/blk00000.dat. The blockchain loading is still at the very beginning, I just ran bitcoin-qt once, as listed in the read me.
How many blk*.dat files are in "../Application Support/Bitcoin/blocks" ? Does Abe exit after the error message about blck0001.dat? Is that a typo, and did you mean blk0001.dat? Please paste the last several lines of output if Abe exits before it finishes loading. |
|
|
|
Now after about 14 days of importing it's finally done  Anyway everything seems to be working just fine I only got one small problem. It's only importing new blocks when I load a new webpage in the Abe interface. Should it really be like that? So if I for example stop using the web interface for about 1 hour then when I request one page load it will star importing the 6 missing blocks. Why doesn't Abe just import the block as soon my bitcoin-qt instance accepts the new block? Yup, this is an underdocumented quirk. There are many ways to keep Abe up to date, an no one fits every need. You can use a script like abe_loader. Actually, abe_loader is more complex than needed. It watches bitcoind's debug.log for hints that a block has arrived. Now that bitcoind supports blocknotify, we could just add something like this to bitcoin.conf: blocknotify=python -m Abe.abe --config /etc/abe/abe.conf --no-serve
Another strategy is to run in a loop, like this: while true; do python -m Abe.abe --config abe.conf --no-serve; sleep 10; done
I used to have a cron job that just loaded the homepage, serving as both a system monitor and a blockchain updater: * * * * * /usr/bin/wget -q -O /dev/null http://john-edwin-tobey.org:2750/
One more question is there a way to check so the information in the mysql database is intact? (That does not take 14 days)
A complete check of all invariants does not exist. It would be a nice feature. Abe/verify.py does a small fraction of the work, calculating Merkle roots from stored transaction hashes. I haven't tested it lately. |
|
|
|
datadir = /Users/thisuser/Library/Application Support/Bitcoin/blocks
Leave /blocks off. This should be: datadir = /Users/thisuser/Library/Application Support/Bitcoin
|
|
|
|
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found. Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture Sorry, this is an issue with Python modules on Mac, and I have no relevant experience. Put this in a file and run it with "python FILE.py". It will have to complete without error before you can use Abe: |
|
|
|
That I can build wdc of abe services?
I don't know. Can you get Abe working with BTC or NMC or LTC? Those are known to work. I need a little more information to understand your problem. What is your python command line? How big is /home/mapleshadow/.worldcoin/blocks/blk00000.dat ? What exact version of Abe do you have? (Post the output of "git rev-parse HEAD" and "git --no-pager diff") |
|
|
|
my conf is wdc wdc.conf: dbtype sqlite3 connect-args abe.sqlite datadir = [{ "dirname": "/home/mapleshadow/.worldcoin", "chain": "Worldcoin", "code3": "WDC", "address_version": "\u0049" }] int-type str upgrade port 2750 host 12.16.2.24 DataStore.py: {"chain":"Worldcoin", "code3":"WDC", "address_version":"\x49", "magic":""}, ERROR-1: failed to load /home/maple/.worldcoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/maple/.worldcoin/bitcoin.conf' catch_up_rpc: abort Opened /home/mapleshadow/.worldcoin/blocks/blk00000.dat Abe initialized. Listening on http://12.16.2.24:2750WHY? Abe tries to load data over RPC, and when that fails, it reads block files. (See default-loader in abe.conf to choose one or the other.) Abe does not know about Worldcoin, so it looks for RPC credentials in bitcoin.conf. There is no bitcoin.conf, so it logs an error and tries to load blocks from files, starting with blocks/blk00000.dat. It did not find any blocks in that file, so perhaps the file is empty or the block format is not the same as Bitcoin's. I am surprised that the configured dirname is in /home/mapleshadow but Abe looked for bitcoin.conf in /home/maple. |
|
|
|
Thanks for your awesome contribution to the open source community. I really appreciate the work you've done!
Happy to hear it. Actually, this ought to be very simple. I'd do it right now, but I am not at my secure workstation. Where q_getreceivedbyaddress calls abe.store.get_received, it could pass the optional block_height parameter to ignore blocks after a certain number. That number would be something like: abe.store.get_block_number(chain['id']) + 1 - N |
|
|
|
Ah, I can look up mempool transactions, but it does not add the addresses in that transaction to the database? I was hoping I could do a /unspent/ on mempool transactions as well.
This is a little tricky to do. Abe does not delete transactions that can never get into the main chain because they contain inputs that are spent elsewhere. /unspent/ excludes these from its output by filtering on block_tx and chain_candidate.in_longest = 1. This also excludes mempool transactions. Can you see them on the address page? |
|
|
|
I'll probably just pick one of those chains. I didn't suspect trying to get 30 chains loaded in Abe would require you to fix a bunch of stuff  Your reports are helping me improve the code. Thank you!! |
|
|
|
In trying to find out the cause of my issue, I found that LTC, FTC, CNC & GME all have the same genesis block hash. Would that cause my issue?
Yes, probably, but it is a bug in Abe, and I'll try to fix it. Thanks. |
|
|
|
Run this SQL and see if it is fixed: UPDATE datadir SET chain_id = 1, loader = 'rpc' WHERE datadir_id = 1;
In the datadir table I have the following columns: datadir_id dirname blkfile_number blkfile_number blkfile_offset chain_id datadir_loader Whoops, I guess I meant: UPDATE datadir SET chain_id = 1, datadir_loader = 'rpc' WHERE datadir_id = 1;How does Abe keep track of blockfiles and blockfile offsets when it is updating from RPC?
Putting chain_id and datadir_loader in the table was bad design. I think they get their values from the "chain" and "loader" (or "default-loader") config elements when the datadir is first used, and thereafter, the config values are ignored. I plan to make it use the config values only, but meanwhile, I think you can copy the correct values into the table row, and it will work. |
|
|
|
where you able to find out what was wrong with testnet3?
Not yet, thanks for the reminder. I think this is fixed in the latest commit (ecfe350). Thanks for the report. |
|
|
|
@diatonic Please try the latest code and report back. The errors I posted above happen when I try to load Litecoin, CHNCoin or GameCoin. I've got about 27 chains loaded now I think at http://altcha.in . I am working on a fix for this old bug. |
|
|
|
The errors I posted above happen when I try to load Litecoin, CHNCoin or GameCoin. I've got about 27 chains loaded now I think at http://altcha.in . I am working on a fix for this old bug. |
|
|
|
I am trying to get Abe to connect through RPC, but I am getting the following error. The initial database loading was trough blkfile.
2013-06-27 23:39:43,338 [5344:MainThread] DataStore DEBUG - no chain_id
2013-06-27 23:39:43,339 [5344:MainThread] DataStore ERROR - Failed to catch up {'blkfile_offset': 40774615, 'blkfile_number': 100068, 'chain_id': None, 'loader': None, 'dirname': '/home/node/.bitcoin-fullindex', 'id': Decimal('1')}
Traceback (most recent call last):
File "/home/node/bitcoin-abe/bitcoin-abe-master/Abe/DataStore.py", line 2594, in catch_up
raise Exception("RPC load failed")
Exception: RPC load failed
Run this SQL and see if it is fixed: UPDATE datadir SET chain_id = 1, loader = 'rpc' WHERE datadir_id = 1;
It's a bug, Abe should take the chain and loader from the config, not the database. |
|
|
|
I really appreciate your help with this. I updated, but now I get a different error: Opened /home/diatonic/.chncoin/blk0001.dat
Exception at 8
Failed to catch up {'blkfile_offset': 0, 'blkfile_number': 1, 'chain_id': Decimal('30'), 'loader': None, 'dirname': u'/home/diatonic/.chncoin', 'id': 92L}
Traceback (most recent call last):
File "Abe/DataStore.py", line 2599, in catch_up
store.catch_up_dir(dircfg)
File "Abe/DataStore.py", line 2857, in catch_up_dir
store.import_blkdat(dircfg, ds, blkfile['name'])
File "Abe/DataStore.py", line 2981, in import_blkdat
if not store.offer_existing_block(hash, chain_id):
File "Abe/DataStore.py", line 2468, in offer_existing_block
store.offer_block_to_chains(b, frozenset([chain_id]))
File "Abe/DataStore.py", line 2350, in offer_block_to_chains
b['top'] = store.adopt_orphans(b, 0, chain_ids, chain_ids)
File "Abe/DataStore.py", line 2097, in adopt_orphans
next_ret = store.adopt_orphans(nb, new_work, None, chain_mask)
File "Abe/DataStore.py", line 2029, in adopt_orphans
util.get_search_height(height), int(block_id))
File "Abe/util.py", line 91, in get_search_height
if n & 1:
TypeError: unsupported operand type(s) for &: 'Decimal' and 'int' Fixed, please try again. Sorry if there are more errors, this code path is untested. The lack of a test suite is what keeps me from calling it 1.0. |
|
|
|
File "Abe/DataStore.py", line 2465, in offer_existing_block
store.offer_block_to_chains(b, frozenset([chain_id]))
File "Abe/DataStore.py", line 2349, in offer_block_to_chains
b['top'] = store.adopt_orphans(b, 0, chain_ids, chain_ids)
File "Abe/DataStore.py", line 2015, in adopt_orphans
if b['total_ss'] is None or b['satoshis'] is None:
KeyError: 'total_ss' Fixed, please retry with the latest. Thanks. |
|
|
|
2013-06-27 23:39:43,338 [5344:MainThread] DataStore DEBUG - no chain_id
Does datadir have "chain":"Bitcoin"? |
|
|
|
Thanks for ABE. I've been trying it out with a few different alt-coins. I could do with a hand understanding address_version.
For most coins it seems to be obvious, just the unicode of the first character of an address. But some coins have addresses like this (feathercoin in this example)
6nh62XDjhbqjjit78jfStECNcHz89xbNeq
72HzQ77Xbpg67MadosyWDjd5YZsVHz4Npt
etc...
Plugging it into decode_address, for example: http://explorer.doubloons.net/chain/Doubloons/q/decode_address/6nh62XDjhbqjjit78jfStECNcHz89xbNeqgives "0e" as the hex version byte for both those addresses, so I would think it is "\u000e". |
|
|
|
I was able to get it to run from Python with Bitgems & BottleCaps (NovaCoin/PPcoin forks) this way. Thought I would post it here in hopes that it helps someone else.
Great, thanks! |
|
|
|
|
Show Posts
