PoolServerJ - Tech Support

[shads]

Support thread for PoolServerJ.  Ask you questions here.

In case you're wondering what PoolServerJ is http://forum.bitcoin.org/index.php?topic=33142.0

If you are using a bitcoind v0.3.24 or below ensure you have the 4diff patch. There is a known bug that will bring poolserverj to a grinding halt if it occurs.  It is fixed in this patch and it also contains some massive performance improvements that will benefit poolserverj.  See this post for an explanation of how it affects poolserverj and why the fact that it doesn't slow down pushpool is not necessarily a good thing.

Support for performance issues

If you're having performance issues (high CPU load, slow response to workers etc) your problem is probably fixable.  Poolserverj has been proven to handle very high loads in production environments with minimal cpu usage.  There are a number of things that can cause these symptoms though.

If you want me to help you troubleshoot these sort of issues please help me out by providing the following info:

• Poolserverj version
• A summary of your hardware specs.
• What patches are applied to your bitcoind (and if it's an alt chain daemon make sure you tell me that
• How you've got yr pool components splits among servers.  i.e. are poolserverj, bitcoind, frontend, database on the same server?  Multiple bitcoinds?  On which servers.  What type of database.
• A copy of your poolserverj properties file (Please remember to remove your passwords).
• A sample of your log output (error and stdout) with debug=true.  About 10 mins worth is good but try to include a block change in that.
  If your load is not particularly high then also set trace=true and traceTargets=all.  Note: the only data this should contain that might be considered sensitive is worker login names and possibly ip addresses (if trace=true).  If you're concerned about this then scrub the log file first but please use some sort of regex find/replace with a marker so I can see where these events actually happened.
• The output from running the script found in this post.  This will dump two 2sec snapshots of server stats to log directory. This should be from the same time window as the log sample.

If you can't find me on IRC then PM for an email address to send this to.

[de4l]

this is probably an error on my part, but I was trying to setup the mysql database and went to import the example sql file provided but am getting this error:

Code:

#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE `poolserverj_native`.`shares` ( `id` bigint(30) NOT NULL AUTO_IN' at line 10
CREATE TABLE  `poolserverj_native`.`pool_worker` (
 `id` INT( 255 ) NOT NULL AUTO_INCREMENT ,
 `associatedUserId` INT( 255 ) NOT NULL ,
 `username` VARCHAR( 50 ) DEFAULT NULL ,
 `password` VARCHAR( 255 ) DEFAULT NULL ,
 `allowed_hosts` TEXT,
PRIMARY KEY (  `id` )
) ENGINE = MYISAM AUTO_INCREMENT =1 DEFAULT CHARSET = latin1 CREATE TABLE  `poolserverj_native`.`shares` (
 `id` BIGINT( 30 ) NOT NULL AUTO_INCREMENT ,

just needed 2 semicolons, 1 after each statement

[de4l]

I have noticed very poor performance when trying to run poolserverj with a solidcoin daemon, could I get you opinion if this is a) poor config on my part b) poolserverj does not like solidcoin c) solidcoin needs multi-threaded keepalive connections like this https://bitcointalk.org/index.php?topic=22585.0 d) other

[shads]

I have noticed very poor performance when trying to run poolserverj with a solidcoin daemon, could I get you opinion if this is a) poor config on my part b) poolserverj does not like solidcoin c) solidcoin needs multi-threaded keepalive connections like this https://bitcointalk.org/index.php?topic=22585.0 d) other

It could be several of those things...

I haven't tried psj with solidcoin but I can't see any reason it wouldn't work ok with so I wouldn't jump straight to b).

With regards to config yes there are some easy ways to turn psj into a pig and in fact some of the default settings in the sample config are probably not ideal as defaults.  With the right tuning it should fly though.  I've written a bit of a guide to performance tuning here: http://poolserverj.org/documentation/performance-memory-tuning/

But definately the first step is to ensure you've got a patched bitcoind.  I saw 10-20x performance increase in the rate of incoming works when I patched my daemon.  I don't know how heavily solidcoin has been modified so not sure how easy/hard it will be to apply that patch but it really is essential.  If patching isn't an option (or if it is and you just want more)... Then you could always setup a whole bunch of bitcoind's and feed them all into the poolserver. 

[shads]

just needed 2 semicolons, 1 after each statement

thanks, I've fixed the sample sql files in 0.2.9.  BTW 0.2.9 fixes a critical memory leak so please update.

[de4l]

I updated but now I'm getting this error, tried re-installing the service too

Code:

[2011-08-27 09:56:56] [info]  Running 'PoolServerJServicec' Service...
[2011-08-27 09:56:56] [info]  Starting service...
[2011-08-27 09:56:56] [error] %1 is not a valid Win32 application.
[2011-08-27 09:56:56] [error] Failed creating java C:\Program Files\Java\jdk1.7.0\jre\bin\server\jvm.dll
[2011-08-27 09:56:56] [error] %1 is not a valid Win32 application.
[2011-08-27 09:56:56] [error] ServiceStart returned 1
[2011-08-27 09:56:56] [error] %1 is not a valid Win32 application.
[2011-08-27 09:56:56] [info]  Run service finished.
[2011-08-27 09:56:56] [info]  Commons Daemon procrun finished

[shads]

Not sure what that one's all about I just did a diff between the 0.2.8 and 0.2.9 win-service.bat and they haven't changed... Are you running from the same directory?  If not you may need to remove the service before reinstalling.  Remember you *must* run win-service.bat from the bin directory or none of the paths will match up properly.

You can try running PoolServerJServicew.exe which is a gui interface to show all the service settings.  Check that all the paths in there are actually valid.  I'm just about to head to bed so if you don't have any luck let me know and I'll try a win-service install from the 0.2.9 binary in the morning and see what I can find.

[shads]

oops... that's a little embarrassing.  I put the 0.2.8 binary in the 0.2.9 distribution...
I've updated the archive on the download page and also put the single jar file there if you want to just download that and copy it over the old one.

Sorry about that...

[de4l]

Not sure what that one's all about I just did a diff between the 0.2.8 and 0.2.9 win-service.bat and they haven't changed... Are you running from the same directory?  If not you may need to remove the service before reinstalling.  Remember you *must* run win-service.bat from the bin directory or none of the paths will match up properly.

You can try running PoolServerJServicew.exe which is a gui interface to show all the service settings.  Check that all the paths in there are actually valid.  I'm just about to head to bed so if you don't have any luck let me know and I'll try a win-service install from the 0.2.9 binary in the morning and see what I can find.

I removed the older service and am starting over in a new folder, I re-downloaded the archive since you have updated it, still getting the same error, not sure what it is, could having 64bit java be messing with it? Even though I did have it running previously...

[shads]

did you try running the service gui and verifying that all paths etc it's recorded for the service are valid?

[shads]

ok just tried a service install using 0.2.9a.tar.gz (the one on the website).  It work both for a 1.6_u25 Sun JDK.  I tested both the 32bit and 64bit versions ok.  I haven't tried openJDK1.7 but I can't see any reason why it would fail.  Unless "C:\Program Files\Java\jdk1.7.0\jre\bin\server\jvm.dll" doesn't happen to exist in openJDK?

You might try editing win-service.bat and changing the options : --StartMode=jvm and --StopMode=jvm to either 'java' or 'exe' (no quotes).

Detail doc on service options available here: http://commons.apache.org/daemon/procrun.html

procrun can be a bitch to get right so I'm scared to touch it once I've found a working config.

[de4l]

ok just tried a service install using 0.2.9a.tar.gz (the one on the website).  It work both for a 1.6_u25 Sun JDK.  I tested both the 32bit and 64bit versions ok.  I haven't tried openJDK1.7 but I can't see any reason why it would fail.  Unless "C:\Program Files\Java\jdk1.7.0\jre\bin\server\jvm.dll" doesn't happen to exist in openJDK?

You might try editing win-service.bat and changing the options : --StartMode=jvm and --StopMode=jvm to either 'java' or 'exe' (no quotes).

Detail doc on service options available here: http://commons.apache.org/daemon/procrun.html

procrun can be a bitch to get right so I'm scared to touch it once I've found a working config.

I got it running and it seems much more stable than the last version, just need to do some performance tuning.  I fixed it by finally getting the gui to run and then selecting 'use default' instead of C:\Program Files\Java\jdk1.7.0\jre\bin\server\jvm.dll even though this file did exist, something was not working with it, no idea why.  I think the default might be using my 32bit jre instead of the 64bit jdk that I installed just for this, but after reading your tips that 32bit java was better in this instance.... I'll just let it go since it's running now.

[Leandro César]

Hi,

I am using the last poolserverj on Fedora 15 64 bits, jdk 1.6 update 26, bitcoind 0.3.24 and get the follow error:

org.json.JSONException: A JSONObject text must begin with '{' at character 1

Can you help me?

L.

[shads]

Hi,

I am using the last poolserverj on Fedora 15 64 bits, jdk 1.6 update 26, bitcoind 0.3.24 and get the follow error:

org.json.JSONException: A JSONObject text must begin with '{' at character 1

Can you help me?

L.

I'll try but I need a bit more information.  Can you make sure debug=true and logStacktrace=true then post the full stack strace.  Also can you describe what's actually happening?  Is it straight after start up?  Is it when you send a getwork request from a miner?  Is bitcoind running etc?

[wtfman]

Hey, I am trying to test PoolServerJ but I have a problem with the db inserts.

Using v0.29a

Error:

Code:

work submit success, result: false
Submit Throttling on: false
Doing database flush for Shares: 1
Failed to commit to database.
java.sql.BatchUpdateException: Field 'block_num' doesn't have a default value

Query being executed when exception was thrown:
INSERT INTO shares (rem_host, username, our_result, upstream_result, reason, solution, time, source) VALUES ('xx.xx.xx.xx', 'test', 1, 0, null, '0001000131dcbc30ad5ae6021834e79139879666f1e628de6c78e9e600007d8a000000003c7fef66abefaba3bd824e32ab44351b8f0186196944b909c198b96ee01fc77a4e6de5471b00b269cce138bf000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', '2011-09-12 11:09:38', 'namecoind-localhost')


        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2024)
        at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1449)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at com.shadworld.poolserver.db.shares.DefaultPreparedStatementSharesDBFlushEngine.flushToDatabase(DefaultPreparedStatementSharesDBFlushEngine.java:111)
        at com.shadworld.poolserver.logging.ShareLoggingThread.run(ShareLoggingThread.java:156)
Caused by: java.sql.SQLException: Field 'block_num' doesn't have a default value

Query being executed when exception was thrown:
INSERT INTO shares (rem_host, username, our_result, upstream_result, reason, solution, time, source) VALUES ('xx.xx.xx.xx', 'test', 1, 0, null, '0001000131dcbc30ad5ae6021834e79139879666f1e628de6c78e9e600007d8a000000003c7fef66abefaba3bd824e32ab44351b8f0186196944b909c198b96ee01fc77a4e6de5471b00b269cce138bf000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', '2011-09-12 11:09:38', 'namecoind-localhost')


        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415)
        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1976)
        ... 5 more
Flushed 1 shares to DB in 35.0ms (28/sec)

Prepare Statement from Conf

Code:

### native - ensure usePushPoolCompatibleFormat=false
db.stmt.insertShare=INSERT INTO shares (rem_host, username, our_result, upstream_result, reason, solution, time, source) VALUES (?, ?, ?, ?, ?, ?, ?, ?)

Now, there does seem the insert for block_num to be missing, but no idea where I should put it in. I have looked a bit at the source code but didnt get an exact info. Pls help

[shads]

Prepare Statement from Conf

Code:

### native - ensure usePushPoolCompatibleFormat=false
db.stmt.insertShare=INSERT INTO shares (rem_host, username, our_result, upstream_result, reason, solution, time, source) VALUES (?, ?, ?, ?, ?, ?, ?, ?)

Now, there does seem the insert for block_num to be missing, but no idea where I should put it in. I have looked a bit at the source code but didnt get an exact info. Pls help

It's trying to write a null to a NOT NULL column... try

INSERT INTO shares (rem_host, username, our_result, upstream_result, reason, solution, time, source, block_num, prev_block_hash) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ? ,?)

or alternately remove the NOT NULL restriction from the block_num and prev_block_hash columns.

The supplied sql scripts in 0.2.9 set those columns to NOT NULL which they probably shouldn't have.  Will be changed in the next release.

BTW Unless you've got a good reason for it I'd leave out the prev_block_hash column.

[wtfman]

thanks!

[wtfman]

hey, I tried to use latest poolserverj in pushpool compatible mode but it didnt work for me. I have a share table that isnt called 'shares' and I also set it up in the properties file, but still it tried to insert data into 'shares'

I adjusted the db to your design, it works so far, but just now I found this in the stdout:

Code:

Flushed 61 shares to DB in 35.0ms (1742/sec)
Pausing Share logging due to block change.
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)
java.lang.IllegalStateException: IDLE,initial
        at org.eclipse.jetty.server.AsyncContinuation.complete(AsyncContinuation.java:449)
        at com.shadworld.poolserver.BlockChainTracker$NotifyLongpollClientsThread.run(BlockChainTracker.java:178)

[shads]

Hi wtfman,

I assume you're running 0.2.9 then?  If so then those errors aren't a problem.  It's an expected exception and probably be logged.  I'm pretty sure it isn't unless you've got debug enabled.

However I really would recommend you get 0.3.0 working.  The longpolling has been rewritten from scratch which has given it huge performance boosts as well as fixing a of obscure errors.

Post some detail about the sql errors and I'll see if can help.  There should be no problem using a different share table name since you can specify the query in the properties file.  Post the query yr using, the table CREATE statement and any exceptions you're seeing in the log.

[Remember remember the 5th of November]

I am currently 'testing' the latest 0.3.0rc1. The first thing i notice is the CPU usage. I am running in pushpoold compatability. So why this CPU usage? Pushpoold rarely used any CPU but PJS uses one full core.

The second thing i notice is that PSJ inserts shares quite a bit slower than pushpoold. And what it affects is the speed counted on my frontend. I am currently getting 40mh/s less detected in the frontend thereby people WILL lose coins.