|
|
abn123
Newbie
Offline
Activity: 33
Merit: 0
|
|
August 27, 2014, 04:27:20 PM |
|
I give up....I have searched Google and this forum at length. Attempted to implement the changes I found for items that were kinda close to my issue, but I cannot find this specific error anywhere else. Trying to populate the db for the first time with my alt.... I just can't seem to figure out what it is trying to store, and how to make it smaller Much thanks in advance.... Keep getting this error: [INDENT][INDENT]Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 2206, in <module> sys.exit(main(sys.argv[1:])) File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 2200, in main store = make_store(args) File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 119, in make_store store = DataStore.new(args) File "Abe/DataStore.py", line 2872, in new return DataStore(args) File "Abe/DataStore.py", line 148, in __init__ store.initialize() File "Abe/DataStore.py", line 1139, in initialize conf["magic_name"])) File "Abe/DataStore.py", line 450, in sql store.cursor.execute(cached, params) psycopg2.DataError: value too long for type character( [/INDENT][/INDENT] Posted from Bitcointa.lk - #dDu0XS4nINCsuzHl
|
|
|
|
abn123
Newbie
Offline
Activity: 33
Merit: 0
|
|
August 28, 2014, 07:55:34 PM |
|
nevermind.....figured it out on my own for any others with this issue -- this specific error appears to be directly related to a malformed magic number (in mine, i left out an 'x' from one of the hex bytes) I give up....I have searched Google and this forum at length. Attempted to implement the changes I found for items that were kinda close to my issue, but I cannot find this specific error anywhere else. Trying to populate the db for the first time with my alt.... I just can't seem to figure out what it is trying to store, and how to make it smaller Much thanks in advance.... Keep getting this error: [INDENT][INDENT]Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 2206, in <module> sys.exit(main(sys.argv[1:])) File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 2200, in main store = make_store(args) File "/home/david/Projects/bitcoin-abe/Abe/abe.py", line 119, in make_store store = DataStore.new(args) File "Abe/DataStore.py", line 2872, in new return DataStore(args) File "Abe/DataStore.py", line 148, in __init__ store.initialize() File "Abe/DataStore.py", line 1139, in initialize conf["magic_name"])) File "Abe/DataStore.py", line 450, in sql store.cursor.execute(cached, params) psycopg2.DataError: value too long for type character( [/INDENT][/INDENT] Posted from Bitcointa.lk - #GPMw8JdVXjFIcJs4
|
|
|
|
Hoshimaru
|
|
September 09, 2014, 07:50:17 PM Last edit: September 09, 2014, 08:19:35 PM by Hoshimaru |
|
Good evening I've prepared a VM with Ubuntu Server 14.04/1GB RAM/8GB VDI and had some fun setting it up to host ABE. It's very slow, but I that's due to the hardware it runs on (old 6 year old laptop with a 1.6ghz athlon neox2 and 4GB ram. Anyway, since my tests are successful, I was thinking to host it somewhere, but I have no clue about the hardware requirements needed. When I look at VPS solutions like Digital Ocean's 5$ and 10$ packages, they look similar to my VM, memory-wise, but I don't know about the other virtualized hardware. I'd like to avoid getting Error 500's or time-outs as the daemon will be running in the background to keep it updating at any time. On my VM, the wallet daemon takes up to 15% of the cpu load, python around 35% and postgresql is sometime at 80% while loading blocks in the database (in huge chunks) during the initial loading. When no blocks are added as the come, it's around 10% for postgresql and python, daemon runs at <2% load. If I query the block explorer for an address, it pyhon gets spikes as high as 45%, postgresql 50% and apache2 45%... So, I'm wondering what the minimum hardware requirements (or VPS equivalent) are to host this kind of service. Also, is there a way to filter out the staking transactions? I also think there's a problem. When looking at transaction details, it show always the same kind of addresses for any sender and receiver, for any transaction... for the first one to the last one in the whole blockchain. But I can query a specific transaction and it shows all the transactions that I see in my wallet as well. Did I miss something during the setup. Is there an (d)e(n)coding error? The address receiving those 5000 coins should be the one in my sig. M.....eXe iso 6KZ...PW9 Thanks for insights
|
|
|
|
xxTeDYxx
Member
Offline
Activity: 69
Merit: 10
|
|
September 10, 2014, 08:19:52 AM |
|
Hi my abe work: http://explorer.zaplnc.pl/Abe work in fast cgi mode and adresses is unhashed My server: http://www.kimsufi.com/KS-1 I have on this serwer: pool.zaplnc.pl explorer.zaplnc.pl beta.zaplnc.pl - demo MPOS
|
|
|
|
Hoshimaru
|
|
September 10, 2014, 09:12:28 PM Last edit: September 10, 2014, 09:34:51 PM by Hoshimaru |
|
Thank you for your reaction. I think something more performing is needed for a block explorer. I get and error when trying to query http://explorer.zaplnc.pl/address/A1PxdQEhJTAy5tVit6kKrPWv3Ggn8qABcm for example. My VM does it correctly, but slowly, once blocks are synched as they trickle in by the wallet daemon and there are not too many transactions for the address. Some with over 20000 transactions ran for more than an hour with the 1000 rows limit removed and FcgidIOTimeout at 900 seconds iso 40 seconds in the apache vhost's config file.
|
|
|
|
roundrobin
|
|
September 13, 2014, 06:51:22 PM |
|
Good evening I've prepared a VM with Ubuntu Server 14.04/1GB RAM/8GB VDI and had some fun setting it up to host ABE. It's very slow, but I that's due to the hardware it runs on (old 6 year old laptop with a 1.6ghz athlon neox2 and 4GB ram. Anyway, since my tests are successful, I was thinking to host it somewhere, but I have no clue about the hardware requirements needed. When I look at VPS solutions like Digital Ocean's 5$ and 10$ packages, they look similar to my VM, memory-wise, but I don't know about the other virtualized hardware. I'd like to avoid getting Error 500's or time-outs as the daemon will be running in the background to keep it updating at any time. On my VM, the wallet daemon takes up to 15% of the cpu load, python around 35% and postgresql is sometime at 80% while loading blocks in the database (in huge chunks) during the initial loading. When no blocks are added as the come, it's around 10% for postgresql and python, daemon runs at <2% load. If I query the block explorer for an address, it pyhon gets spikes as high as 45%, postgresql 50% and apache2 45%... So, I'm wondering what the minimum hardware requirements (or VPS equivalent) are to host this kind of service. Also, is there a way to filter out the staking transactions? I also think there's a problem. When looking at transaction details, it show always the same kind of addresses for any sender and receiver, for any transaction... for the first one to the last one in the whole blockchain. But I can query a specific transaction and it shows all the transactions that I see in my wallet as well. Did I miss something during the setup. Is there an (d)e(n)coding error? The address receiving those 5000 coins should be the one in my sig. M.....eXe iso 6KZ...PW9 Thanks for insights Abe will run ok on Digital Ocean's 1GB RAM VPS ($10/month). It is possible to get it to work ok also on 512MB RAM if you set up swap memory, pick the right distro and disable some unnecessary things to conserve memory use. Don't worry about CPU usage because even if Abe is written in Python which is a shit slow performing language that is not where problem or bottleneck with Abe is, it will run good on some old Pentium 3 If you want smaller footprint blockexplorer (but with some limited feature) check out RPC Ace https://bitcointalk.org/index.php?topic=686177.0
|
LTC: Lbha3tRmE75oHfF4SjSKpxVK2fY9PxrPUX VTC: VguAuTdGRcQzihEgrJDYPYUuugGMMGFCNn FTC: 6fVWQ3eHhhgH1haqThQbxTFV8XjrqyuKY2 SOL: 8X6dLCY8MeZ6RNdBxzYQkd5kxWj8VVPJmL DOGE: DMBQta9ME9cWnRPVXtEbi57CDk1uNpwzSh
|
|
|
dagi
|
|
September 15, 2014, 08:11:10 AM Last edit: September 15, 2014, 12:58:52 PM by dagi |
|
Hi, i had problem with the last bitcoin-abe release v0.7.2 and last bitcoin client. The problem was with blkXXXX.dat files: blk0001.dat (old btc client) VS blk00000.dat (new btc client) my fix in 3 steps :-) : edit file DataStore.py1. line nr 651 store.datadirs.append({ "id": store.new_id("datadir"), "dirname": dirname, "blkfile_number": 1, "blkfile_offset": 0, "chain_id": chain_id, }) to store.datadirs.append({ "id": store.new_id("datadir"), "dirname": dirname, "blkfile_number": 0, "blkfile_offset": 0, "chain_id": chain_id, }) 2. line nr 918 """CREATE TABLE datadir ( datadir_id NUMERIC(10) NOT NULL PRIMARY KEY, dirname VARCHAR(2000) NOT NULL, blkfile_number NUMERIC(4) NULL, blkfile_offset NUMERIC(20) NULL, chain_id NUMERIC(10) NULL )""", to """CREATE TABLE datadir ( datadir_id NUMERIC(10) NOT NULL PRIMARY KEY, dirname VARCHAR(2000) NOT NULL, blkfile_number NUMERIC(5) NULL, blkfile_offset NUMERIC(20) NULL, chain_id NUMERIC(10) NULL )""", 3. line nr 2597 def blkfile_name(store, dircfg): return os.path.join(dircfg['dirname'], "blk%04d.dat" % (dircfg['blkfile_number'],)) to def blkfile_name(store, dircfg): return os.path.join(dircfg['dirname'], "blk%05d.dat" % (dircfg['blkfile_number'],))
|
|
|
|
Hoshimaru
|
|
September 17, 2014, 01:23:08 PM |
|
Abe will run ok on Digital Ocean's 1GB RAM VPS ($10/month). It is possible to get it to work ok also on 512MB RAM if you set up swap memory, pick the right distro and disable some unnecessary things to conserve memory use. Don't worry about CPU usage because even if Abe is written in Python which is a shit slow performing language that is not where problem or bottleneck with Abe is, it will run good on some old Pentium 3 If you want smaller footprint blockexplorer (but with some limited feature) check out RPC Ace https://bitcointalk.org/index.php?topic=686177.0I managed to load the whole blockchain, but made a mistake with the hex value for processing addresses... So they all look weird. Currently I'm also experimenting with Bitcore's Insight. I noticed it loads it DB extremely fast compared to ABE, but also ran into some little bumps along the road, but will hopefully flatten them soon
|
|
|
|
coinflow
Legendary
Offline
Activity: 840
Merit: 1000
|
|
September 20, 2014, 03:47:29 PM Last edit: September 20, 2014, 04:55:32 PM by coinflow |
|
Thank you for your reaction. I think something more performing is needed for a block explorer. I get and error when trying to query http://explorer.zaplnc.pl/address/A1PxdQEhJTAy5tVit6kKrPWv3Ggn8qABcm for example. My VM does it correctly, but slowly, once blocks are synched as they trickle in by the wallet daemon and there are not too many transactions for the address. Some with over 20000 transactions ran for more than an hour with the 1000 rows limit removed and FcgidIOTimeout at 900 seconds iso 40 seconds in the apache vhost's config file. To add to the discussion: If the explorer is not used for a few hours/days and then has to fetch all the latest block-data from the blockchain to store it into the database with the fcgi-call by the visitor, you'll most probably get an timeout-error. I'd recommend the following, to keep the database up-to-date automatically in the background through a cronjob: In /etc/crontab add: */2 * * * * USER cd /home/USER/bitcoin-abe; python -m Abe.abe --config abe.conf --no-serve >/dev/null 2>&1Replace USER with the actual username, which runs Abe on your machine. Maybe this helps some people on a slower machine. It makes Abe check the blockchain every 2 minutes by the cronjob and add new transactions to the database, even if no website-visitor runs an Abe-call by opening the page. So normally by this the database should be mostly current and only the latest few transactions are added by the call of the real user on the website, hence he does not get an error-message. Just make sure, that the paths and names of the files in the above command are set correctly, according to your installation. Edit: Regarding the 1000 rows-limit, just remove the outcommenting hash mark in the config-file and set to -1 to check, whether your system can cope with unlimited tx-numbers for an address: https://github.com/bitcoin-abe/bitcoin-abe/blob/54ee67eafff81ec9cd1f5ef5cebea3514c267ae5/abe.conf#L267
|
|
|
|
Hoshimaru
|
|
September 30, 2014, 09:44:24 AM |
|
To add to the discussion: If the explorer is not used for a few hours/days and then has to fetch all the latest block-data from the blockchain to store it into the database with the fcgi-call by the visitor, you'll most probably get an timeout-error. I'd recommend the following, to keep the database up-to-date automatically in the background through a cronjob: In /etc/crontab add: */2 * * * * USER cd /home/USER/bitcoin-abe; python -m Abe.abe --config abe.conf --no-serve >/dev/null 2>&1Replace USER with the actual username, which runs Abe on your machine. Maybe this helps some people on a slower machine. It makes Abe check the blockchain every 2 minutes by the cronjob and add new transactions to the database, even if no website-visitor runs an Abe-call by opening the page. So normally by this the database should be mostly current and only the latest few transactions are added by the call of the real user on the website, hence he does not get an error-message. Just make sure, that the paths and names of the files in the above command are set correctly, according to your installation. Edit: Regarding the 1000 rows-limit, just remove the outcommenting hash mark in the config-file and set to -1 to check, whether your system can cope with unlimited tx-numbers for an address: https://github.com/bitcoin-abe/bitcoin-abe/blob/54ee67eafff81ec9cd1f5ef5cebea3514c267ae5/abe.conf#L267Thank you for your tips. I've forgot to chekc this thread lately as I've been trying to get Insight up & running with X11 POW/POS coins, but this requires more changes to the different components than I can handle within my limited time. For now, I'll be putting my attention back to ABE
|
|
|
|
Hoshimaru
|
|
October 02, 2014, 01:19:11 PM |
|
Call me stupid, but I managed to get it up & running not so long ago and now I can't even remember if I did something special except RTFM to get it up & running. Bitcoin-abe works perfectly with its built-in web server and the block chain got loaded: akira@NGC4826:~/minerals-abe$ python -m Abe.abe --config abe.conf --host 192.168.241.128 --port 2750 RPC failed: [Errno socket error] [Errno 111] Connection refused catch_up_rpc: abort Opened /home/akira/.Minerals/blk0001.dat Abe initialized. Listening on http://192.168.241.128:2750 RPC failed: [Errno socket error] [Errno 111] Connection refused catch_up_rpc: abort Opened /home/akira/.Minerals/blk0001.dat 192.168.241.1 - - [02/Oct/2014 15:14:46] "GET / HTTP/1.1" 200 1545 192.168.241.1 - - [02/Oct/2014 15:14:47] "GET /abe.css HTTP/1.1" 200 1064 192.168.241.1 - - [02/Oct/2014 15:14:47] "GET /logo32.png HTTP/1.1" 200 568 However, when I launch it for use with apache2/fastcgi I get into trouble with it... Or at least launching bitcoin-abe doesn't want to launch. akira@NGC4826:~/minerals-abe$ python -m Abe.abe --config abe.conf RPC failed: [Errno socket error] [Errno 111] Connection refused catch_up_rpc: abort Opened /home/akira/.Minerals/blk0001.dat Abe initialized. WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI! WSGIServer: missing FastCGI param SERVER_NAME required by WSGI! WSGIServer: missing FastCGI param SERVER_PORT required by WSGI! WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI! fixed path_info Status: 301 Moved Permanently Location: http://localhost/ Content-Type: text/html
<html><head><title>Moved</title></head> <body><h1>Moved</h1><p>This page has moved to <a href="http://localhost/">http://localhost/</a></body></html> akira@NGC4826:~/minerals-abe$ What kind of directory or file or symlink could be 'permantently moved'? I don't remember what I did to get it up & running at two weeks ago. Unfortunately, while reverting to another snapshot of my Virtualbox VM, that software killed the whole tree and there's no way to recover the working bitcoin-abe snapshot I saved back then Currently using other virtualization software, but yeah... I must be doing something wrong this time :s PS: the wallet daemon is not running atm, but that shouldn't be a problem? The Virtual Host is configured like this: <VirtualHost *:80> ServerName nightminerals.com #ServerAdmin webmaster@localhost #ServerAlias www.nightminerals.com #DocumentRoot /var/www/nightminerals.com/public_html Alias /static/ /home/akira/minerals-abe/Abe/htdocs/ Alias /robots.txt /home/akira/minerals-abe/Abe/htdocs/robots.txt Alias /favicon.ico /home/akira/minerals-abe/Abe/htdocs/favicon.ico Alias / /usr/lib/cgi-bin/abe.fcgi/ ErrorLog ${APACHE_LOG_DIR}/abe-error.log CustomLog ${APACHE_LOG_DIR}/abe-access.log combined #FcgidIOTimeout 40 </VirtualHost> And abe.conf like this: datadir += [{ "dirname":"/home/akira/.Minerals", "chain":"Minerals" }]
dbtype MySQLdb connect-args {"user":"***","db":"abe","passwd":"***"}
Basically, according to the instructions for fastcgi/mysql.
|
|
|
|
|
Hoshimaru
|
|
October 02, 2014, 02:08:19 PM |
|
I'm running this on Ubuntu Server 14.04 LTS and vaguely remember something about +ExecCGI with /usr/lib/cgi-bin, but can't say if that was the solution or not. I don't remember bumping into this 301 status before. It's so annoying that all the snapshot tree got destroyed I just could have doublecheck the configuration with it ;_; Anyhow, I've been checking and double checking what I did with the instructions and there's no difference at all. Except last time, with Virtualbox, I ran postgresql iso mysql, but that doesn't matter. I might just revert this to the previous "clean" snapshot and try again.
|
|
|
|
coinflow
Legendary
Offline
Activity: 840
Merit: 1000
|
|
October 02, 2014, 02:12:33 PM |
|
I'm running this on Ubuntu Server 14.04 LTS and vaguely remember something about +ExecCGI with /usr/lib/cgi-bin, but can't say if that was the solution or not. I don't remember bumping into this 301 status before. It's so annoying that all the snapshot tree got destroyed I just could have doublecheck the configuration with it ;_; Anyhow, I've been checking and double checking what I did with the instructions and there's no difference at all. Except last time, with Virtualbox, I ran postgresql iso mysql, but that doesn't matter. I might just revert this to the previous "clean" snapshot and try again. Check this one out: https://github.com/bitcoin-abe/bitcoin-abe/issues/140Might contain helpful information for you. Only at the very end of this lengthy thread it turned out, that he copied code that contained spaces, which led to the system not working ... What does your frontend show, if you try to access it? If you're using Ubuntu, be sure to check this one out: https://www.digitalocean.com/community/tutorials/installing-mod_wsgi-on-ubuntu-12-04
|
|
|
|
Hoshimaru
|
|
October 03, 2014, 09:04:03 PM |
|
It works \(^_^)/ Looks like the problem was not my abe configuration, but rather Apache's virtualhost (the domain is a fictive one I added to the OS' hosts file for testing). <VirtualHost *:80> ServerName explorer.minerals.com DocumentRoot /home/akira/minerals-abe/Abe/htdocs Alias /static/ /home/akira/minerals-abe/Abe/htdocs/ Alias /robots.txt /home/akira/minerals-abe/Abe/htdocs/robots.txt Alias /favicon.ico /home/akira/minerals-abe/Abe/htdocs/favicon.ico Alias / /usr/lib/cgi-bin/min-abe.fcgi/
# Raise this if you get server errors mentioning "mod_fcgid: # read data timeout in 40 seconds" #FcgidIOTimeout 40
# Uncomment to log Abe requests. ErrorLog /var/log/apache2/abe_error.log LogLevel info CustomLog /var/log/apache2/abe_access.log combined <Directory /usr/lib/cgi-bin> Options ExecCGI Indexes FollowSymLinks MultiViews AllowOverride all Allow from all Require all granted </Directory> <Directory /home/akira/minerals-abe/Abe/htdocs> Options Indexes FollowSymLinks MultiViews AllowOverride all Allow from all Require all granted </Directory> </Virtualhost>
When you execute "python -m Abe.abe", it supposes bitcoin-abe is installed in /usr/lib/python/dist-files/ ... or something like that. Is it possible to run it too when not installing it and instead do "python /home/akira/minerals/abe/Abe/abe.py" in that /home/USER/cgi-bin/abe file? If so, how do you cleanly uninstall such python module? $ python setup.py uninstall won't do it
|
|
|
|
Vizakenjack
Newbie
Offline
Activity: 38
Merit: 0
|
|
October 05, 2014, 04:42:07 PM Last edit: November 15, 2014, 08:48:59 AM by Vizakenjack |
|
I'm trying to run abe-blockexplorer. I have found "magic number" and "address_version", modified DataStore.py and abe.conf. If I use this version - https://github.com/bitcoin-abe/bitcoin-abeit can't find litecoin module: Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/bitcoin-abe/Abe/abe.py", line 2096, in <module> sys.exit(main(sys.argv[1:])) File "/root/bitcoin-abe/Abe/abe.py", line 2090, in main store = make_store(args) File "/root/bitcoin-abe/Abe/abe.py", line 156, in make_store store = DataStore.new(args) File "Abe/DataStore.py", line 3275, in new return DataStore(args) File "Abe/DataStore.py", line 185, in __init__ store.initialize() File "Abe/DataStore.py", line 839, in initialize chain = Chain.create(policy, **conf) File "Abe/Chain/__init__.py", line 21, in create mod = __import__(__name__ + '.' + policy, fromlist=[policy]) ImportError: No module named Litecoin If I use version with litecoin module: https://github.com/CryptoManiac/bitcoin-abeIt starts to scan but then it crashes: Exception at 762842 Failed to catch up {'blkfile_number': 100000, 'dirname': '/root/.litecoin', 'chain_id': 8, 'id': Decimal('4'), 'blkfile_offset': 762707} Traceback (most recent call last): File "Abe/DataStore.py", line 2422, in catch_up store.catch_up_dir(dircfg) File "Abe/DataStore.py", line 2475, in catch_up_dir store.import_blkdat(dircfg, ds, filename[0]) File "Abe/DataStore.py", line 2596, in import_blkdat b = store.parse_block(ds, chain_id, magic, length) File "Abe/DataStore.py", line 2630, in parse_block d['transactions'].append(deserialize.parse_Transaction(ds)) File "Abe/deserialize.py", line 90, in parse_Transaction for i in xrange(n_vout): OverflowError: Python int too large to convert to C long What's the problem? Which version should I use?
|
|
|
|
xxTeDYxx
Member
Offline
Activity: 69
Merit: 10
|
|
October 05, 2014, 08:15:34 PM |
|
"/root/.litecoin"
Add new user !!!! DON'T USE ROOT !
adduser abe
/home/abe/.litecoin !
|
|
|
|
fronti
Legendary
Offline
Activity: 2912
Merit: 1309
|
|
October 06, 2014, 11:05:55 AM |
|
Hallo
running ABE over the WebIF is very nice and fast.
But just want to know, before starting to write my own, id there is an API to access ABE, or some SQL statements to query on console?
I guess this will be helpfull for some services?
|
If you like to give me a tip: bc1q8ht32j5hj42us5qfptvu08ug9zeqgvxuhwznzk
"Bankraub ist eine Unternehmung von Dilettanten. Wahre Profis gründen eine Bank." Bertolt Brecht
|
|
|
|