Bitcoin Forum
December 09, 2016, 11:44:04 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 [8] 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 175848 times)
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 04, 2011, 04:07:56 PM
 #141

No luck?
When you have finished following the instructions, the file /home/user/cgi-bin/abe should exist and be executable.  Issue "ls -l /home/user/cgi-bin/abe" and post its output if unsure.

Then test the site and post any error message from your server log or browser.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
1481327044
Hero Member
*
Offline Offline

Posts: 1481327044

View Profile Personal Message (Offline)

Ignore
1481327044
Reply with quote  #2

1481327044
Report to moderator
1481327044
Hero Member
*
Offline Offline

Posts: 1481327044

View Profile Personal Message (Offline)

Ignore
1481327044
Reply with quote  #2

1481327044
Report to moderator
"Your bitcoin is secured in a way that is physically impossible for others to access, no matter for what reason, no matter how good the excuse, no matter a majority of miners, no matter what." -- gmaxwell
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 04, 2011, 04:33:54 PM
 #142

No luck?
When you have finished following the instructions, the file /home/user/cgi-bin/abe should exist and be executable.  Issue "ls -l /home/user/cgi-bin/abe" and post its output if unsure.

Then test the site and post any error message from your server log or browser.

Yeah, I got it up and running. What I've been doing is reverse-proxying the requests from nginx to the built in server Abe uses. That isn't feasible for a production environment, though.

My next big challenge is getting Abe to run in a spawned FastCGI process using nginx.
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 04, 2011, 04:34:27 PM
 #143

I might also add that getting it run over Apache using FastCGI was easy as pie.
btc_artist
Full Member
***
Offline Offline

Activity: 154


Bitcoin!


View Profile WWW
November 04, 2011, 04:53:09 PM
 #144

Following this discussion.  Looks like a very nice tool.

BTC: 1CDCLDBHbAzHyYUkk1wYHPYmrtDZNhk8zf
LTC: LMS7SqZJnqzxo76iDSEua33WCyYZdjaQoE
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 04, 2011, 08:33:03 PM
 #145

Yeah, I got it up and running. What I've been doing is reverse-proxying the requests from nginx to the built in server Abe uses. That isn't feasible for a production environment, though.

My next big challenge is getting Abe to run in a spawned FastCGI process using nginx.
I have no experience with nginx, but FastCGI should work the same on any server.  I might add that the "sudo" overhead is not essential, it's just how I set things up to keep any Abe exploits out of Apache and vice versa.  If your security policy doesn't need that separation, I suggest naming the script /usr/lib/cgi-bin/abe and removing --watch-pid="$1":

Code:
#! /bin/sh
PYTHONUNBUFFERED=1 exec python -m Abe.abe \
--config /home/USER/abe.conf --static-path static/

And set "Alias / /usr/lib/cgi-bin/abe/" or the nginx equivalent in the server configuration.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
notawake
Jr. Member
*
Offline Offline

Activity: 58


View Profile
November 07, 2011, 08:26:29 AM
 #146

I used Abe to output transactions for a given address to CSV format: https://bitcointalk.org/index.php?topic=51121

Bitcoin: 1HZRowLTkmF6M8V11vj2dmVf2r9VK7dVfo | Namecoin: MxT6tjJZcYZeWUGNgsAMxJ7wpDgKMdub4y

Output Transactions for a Given Address to CSV Format
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 07, 2011, 03:38:45 PM
 #147

I used Abe to output transactions for a given address to CSV format: https://bitcointalk.org/index.php?topic=51121
Thanks for sharing!

I wonder if your GetTransactions function would work as a VIEW.  That might translate easily to MySQL etc.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
notawake
Jr. Member
*
Offline Offline

Activity: 58


View Profile
November 07, 2011, 07:41:32 PM
 #148

I used Abe to output transactions for a given address to CSV format: https://bitcointalk.org/index.php?topic=51121
Thanks for sharing!

I wonder if your GetTransactions function would work as a VIEW.  That might translate easily to MySQL etc.


I don't think views can take a dynamic parameter, which is why I went with a function.

However, one could make a view with all the transactions and then just use a select statement from the view for records with the given address. I don't know which method (function vs. view) would be faster for selecting transactions and updating the database with new transactions. I guess the view would need to either be persisted, meaning it gets updated on every update, or dynamically generated on every query, which I would guess does not have a significant performance difference from the current function method.

As for compatibility, I know contemporary versions of MySQL have function capabilities, but I'm not sure about some of the older versions. Actually, I think older versions of PostgreSQL may not allow for the type of function I'm using, i.e. returning a table.

Bitcoin: 1HZRowLTkmF6M8V11vj2dmVf2r9VK7dVfo | Namecoin: MxT6tjJZcYZeWUGNgsAMxJ7wpDgKMdub4y

Output Transactions for a Given Address to CSV Format
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 12, 2011, 10:52:04 PM
 #149

And set "Alias / /usr/lib/cgi-bin/abe/" or the nginx equivalent in the server configuration.
It's not quite that simple. nginx's lightweight nature does not automagically spawn FastCGI threads.

If anyone cares, you can use spawn-fcgi, 'fastcgi_pass'ing the requests from nginx to loopback or UNIX socket or whatever you decide to use. Remember to have nginx serve your static files, or you'll lose the entire benefit of using it in the first place.
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 13, 2011, 12:24:35 AM
 #150

If I'd like to modify Abe without modifying the original code, what's the go-to way of hooking or wrapping into it so as to maintain compatibility with the original codebase and its updates?
molecular
Donator
Legendary
*
Offline Offline

Activity: 2142



View Profile
November 13, 2011, 06:55:22 PM
 #151

If I'd like to modify Abe without modifying the original code, what's the go-to way of hooking or wrapping into it so as to maintain compatibility with the original codebase and its updates?

+1. I've been "hacking up" bitcoin-abe by adding functions "q_whatever(...)" to abe.py. Now I can't easily update.

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 14, 2011, 03:46:04 PM
 #152

If I'd like to modify Abe without modifying the original code, what's the go-to way of hooking or wrapping into it so as to maintain compatibility with the original codebase and its updates?

+1. I've been "hacking up" bitcoin-abe by adding functions "q_whatever(...)" to abe.py. Now I can't easily update.
Just uploaded an example (untested by me) of subclassing Abe.abe to add handle_* functions.  I suspect q_* would work too.

https://github.com/jtobey/bitcoin-abe/blob/master/contrib/ecdsa.py

If you come to rely on an undocumented interface, the best way to keep it around is to document it and ask me to commit the doc. Smiley

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 16, 2011, 08:01:10 PM
 #153

I'm getting "Commands out of sync; you can't run this command now" MySQL errors. Could this tie in to Litecoin experiencing spamming in the form of hundreds if not thousands of 0.00000001 outputs per block?

Case in point: http://explorer.liteco.in/
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 17, 2011, 04:24:13 PM
 #154

I'm getting "Commands out of sync; you can't run this command now" MySQL errors.
Hmm, that's a new one to me.  Could you post a way to reliably produce the error in my own environment?  Or the next best thing would be to run with --log-sql and post a section of log including, say, 5 SQL commands leading up to the error.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
terrytibbs
Hero Member
*****
Offline Offline

Activity: 560



View Profile
November 17, 2011, 05:40:58 PM
 #155

I'm getting "Commands out of sync; you can't run this command now" MySQL errors.
Hmm, that's a new one to me.  Could you post a way to reliably produce the error in my own environment?  Or the next best thing would be to run with --log-sql and post a section of log including, say, 5 SQL commands leading up to the error.

I've switched to Postgres, and it seems to be holding up fairly well so far, other than the FastCGI process dying every once in a while.

I will set a test environment up this weekend and work on reproducing it myself.
molecular
Donator
Legendary
*
Offline Offline

Activity: 2142



View Profile
November 18, 2011, 01:16:54 AM
 #156

Hey John,

I ran into some weird and massive performance issues (litecoin data, older version of abe, mysql). Couldn't find out what exactly is wrong. mysqld just seems to be really slow. It's inserting the blocks almost as slowly as they are mined and mysqld cripples my system hogging i/o, I guess.

So I just upgraded to newest version from git (actually made a fork) and ran into a typo, I guess:

Quote from: update.py:341
        if count % 1000 == 0:
            store.commit()
            stor.log.info("Updated %d blocks", count)

NameError: global name 'stor' is not defined


PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 18, 2011, 02:51:12 PM
 #157

Hey John,

I ran into some weird and massive performance issues (litecoin data, older version of abe, mysql). Couldn't find out what exactly is wrong. mysqld just seems to be really slow. It's inserting the blocks almost as slowly as they are mined and mysqld cripples my system hogging i/o, I guess.

If it persists, I find an easy way to "profile" abe is to interrupt it (ctrl-C) and note the stack trace, then restart, let it get going, and repeat a few times.  If the interrupt usually happens in the same query or two, I know what to optimize.

NameError: global name 'stor' is not defined
Fixed, thanks.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 18, 2011, 02:54:21 PM
 #158

I'm getting "Commands out of sync; you can't run this command now" MySQL errors.
Hmm, that's a new one to me.  Could you post a way to reliably produce the error in my own environment?  Or the next best thing would be to run with --log-sql and post a section of log including, say, 5 SQL commands leading up to the error.
Come to think of it, a plain old stack trace would be better than nothing if you have one around...

I've switched to Postgres, and it seems to be holding up fairly well so far, other than the FastCGI process dying every once in a while.
Any error message in the log or browser?

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
molecular
Donator
Legendary
*
Offline Offline

Activity: 2142



View Profile
November 20, 2011, 01:40:05 AM
 #159

I ran into some weird and massive performance issues (litecoin data, older version of abe, mysql). Couldn't find out what exactly is wrong. mysqld just seems to be really slow. It's inserting the blocks almost as slowly as they are mined and mysqld cripples my system hogging i/o, I guess.

If it persists, I find an easy way to "profile" abe is to interrupt it (ctrl-C) and note the stack trace, then restart, let it get going, and repeat a few times.  If the interrupt usually happens in the same query or two, I know what to optimize.

It persists. Even accross db venors, meaning that postgres is - while faster than mysql - also having a hard time doing these inserts (manly tx, I guess)

You don't happen to have a part with O

I think

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 20, 2011, 09:41:23 PM
 #160

It persists. Even accross db venors, meaning that postgres is - while faster than mysql - also having a hard time doing these inserts (manly tx, I guess)
Please try with --commit-bytes=100000 and let me know.  This will prevent a commit after every tx insertion but may lead to errors when concurrent processes insert.  I recently changed this setting from being the default.

How is system load?  Is the database server busy with CPU?  IO?  Is there free memory?

When I have time, I will try some load testing.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Pages: « 1 2 3 4 5 6 7 [8] 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!