PoolServerJ - Tech Support

[shads]

hey again.

there was even another issue with running mm-0.5. It worked great until first Long Poll, then CPU usage skyrocketed and didnt drop any more. After a couple of mins the work queue was emptied out so miners only received No Work available message and pool hash rate dropped to 0.0 Hash/s.

I have tried it before with only 2 miners in testnet and there was no problem. Problem occured with approximately 15Ghash/s and 60 workers.

I have exactly the same issue, poolserverj mm is running smooth at low cpu usage and finding both nmc and btc blocks until network block is found and lp happens.
Then bitcoind is under heavy cpu usage until i restart the poolserverj. We currently use a 4diff patched version of vinced bitcoind.

When you see this happen can you try checking http://localhost:8997/?method=getsourcestats and report back is Duplicate rate or Reject rate are above zero?

If not can you set debug=true trace=true traceTargets=merged and throw the output onto pastebin?

[1714808749]

1714808749

[DavinciJ15]

hey again.

there was even another issue with running mm-0.5. It worked great until first Long Poll, then CPU usage skyrocketed and didnt drop any more. After a couple of mins the work queue was emptied out so miners only received No Work available message and pool hash rate dropped to 0.0 Hash/s.

I have tried it before with only 2 miners in testnet and there was no problem. Problem occured with approximately 15Ghash/s and 60 workers.

I have exactly the same issue, poolserverj mm is running smooth at low cpu usage and finding both nmc and btc blocks until network block is found and lp happens.
Then bitcoind is under heavy cpu usage until i restart the poolserverj. We currently use a 4diff patched version of vinced bitcoind.

This also happens to me with mm-psj  one suggestion from shadders was to set forceAllSubmitsUpstream to false.

I have not tried it.  Also you should lower your work cache sizes big time.

I know you where upset with me in the chat room for not using/testing it but I had a working part of my pool and many other parts that where not working I had to prioritize them as people wanted me to expand into other countries.

Also I was hoping you would find the bug and issue a new version before I got a chance to retry your suggestions.

[urstroyer]

hey again.

there was even another issue with running mm-0.5. It worked great until first Long Poll, then CPU usage skyrocketed and didnt drop any more. After a couple of mins the work queue was emptied out so miners only received No Work available message and pool hash rate dropped to 0.0 Hash/s.

I have tried it before with only 2 miners in testnet and there was no problem. Problem occured with approximately 15Ghash/s and 60 workers.

I have exactly the same issue, poolserverj mm is running smooth at low cpu usage and finding both nmc and btc blocks until network block is found and lp happens.
Then bitcoind is under heavy cpu usage until i restart the poolserverj. We currently use a 4diff patched version of vinced bitcoind.

When you see this happen can you try checking http://localhost:8997/?method=getsourcestats and report back is Duplicate rate or Reject rate are above zero?

If not can you set debug=true trace=true traceTargets=merged and throw the output onto pastebin?

Running psj mm for an hour now with 35gh/s load an got some new test results.

I seems like the heavy cpu load (70-80%) on bitcoind happens CAN happen on every lp which was trigged from a new network block.

When this happens, i noticed that the field block_num in shares table gets the current NMC block number!

It also happend that on another network block and lp the cpu usage got back to normal 3-5% load and guess what! The field block_num in shares table got the BTC block number again on every share.

The situation can actually switch every on every lp.

Here are two snapshots of getsourcestats:
#### 1 minute after starting psj (heavy cpu load, nmc block_num in shares table) ####
http://pastebin.com/BMAx0vgG

#### 30 minute after starting psj (heavy cpu load, nmc block_num in shares table) ####
http://pastebin.com/zViA7K5b

#### Console on lp (normal cpu load to heavy cpu load) ####
http://pastebin.com/zhfQG1ES

I couldn't figure out what circumstances cause this effect, it seems pretty random to me. Maybe can help.

[shads]

Thanks for the info Urstroyer...  I think this is the indicator I needed:

Code:

#
Incoming Rate: 3,387.53/sec
Incoming Fullfillment Rate: 100%
Outgoing Requested Rate: 29.5/sec

Somewhere in the chain it's discarding valid work and getting more from the daemon.  Not because of duplicates in this case.  One of the validation checks it does is to ensure the work is from the current block before sending out.  It seems having different sets of block numbers is causing a bit confusion.

I'm actually nearly ready to release the fix that should make longpolling work for both chains and I've replaced the blocknum field (which was just a long int) which a new BlockNumbers class that tracks blocks for all chains so that should be the end of it with a bit of luck.

[shads]

MM edition .06 is now on the downloads page.  This is first version of MM that I'm reasonably happy with.

It should now be sending longpolls if either block changes and the server is now actively monitoring the state of all block chains..  Issues with high CPU load caused by PSJ rejecting valid work should also be resolved.

If you are going to try it I'd appreciate if you could set the following:
debug=true
trace=true
traceTargets=blockmon

If anything odd happens this should give me some useful info to play with.

[wtfman]

MM edition .06 is now on the downloads page.  This is first version of MM that I'm reasonably happy with.

It should now be sending longpolls if either block changes and the server is now actively monitoring the state of all block chains..  Issues with high CPU load caused by PSJ rejecting valid work should also be resolved.

If you are going to try it I'd appreciate if you could set the following:
debug=true
trace=true
traceTargets=blockmon

If anything odd happens this should give me some useful info to play with.

great, testing it right now

[urstroyer]

Running very smooth now! Great work! Even lp is happening quite often, seems to work out.

I'll report back after some more hours on load.

[DavinciJ15]

MM edition .06 is now on the downloads page.  This is first version of MM that I'm reasonably happy with.

It should now be sending longpolls if either block changes and the server is now actively monitoring the state of all block chains..  Issues with high CPU load caused by PSJ rejecting valid work should also be resolved.

If you are going to try it I'd appreciate if you could set the following:
debug=true
trace=true
traceTargets=blockmon

If anything odd happens this should give me some useful info to play with.

100 GHs and looking good!
I have everything set except:
traceTargets=blockmon


I have a good feeling about this one.

[Jine]

If the connection between MySQL and poolserverj goes down (timeout and/or other reasons), the server having issues with doing reconnects, please read the following error message.

java.sql.BatchUpdateException: The last packet successfully received from the server was 4,664,997 milliseconds ago.  The last packet sent successfully to the server was 1,950,645 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.

How do i set autoReconnect=true?

I can increase the timeout on the serverside, but feels kinda non-optional.

/ Jim


EDIT:
This is for the non-MM version.

[Jine]

Hmm, strange.. noticed the JDBC-url which contains autoReconnect=true.
I'm having issues with writing shares to the db so... Get the errormessage above in the log.

[shads]

I could have sworn I'd set that because I've seen this problem before.  I've just changed it as a default setting which will require a full release since it's buried in once of the support libs... Since yr obviously building from source you can set it in the Conf class.

Add the line:
workerSql.getJdbcOptionMap().put("autoReconnect", "true");

just after:
workerSql = new MySql(whost, wport == null ? "3306" : String.valueOf(wport), wschema, wuser, wpassword);

[shads]

Hmm, strange.. noticed the JDBC-url which contains autoReconnect=true.
I'm having issues with writing shares to the db so... Get the errormessage above in the log.

aaahh when I had that issue previously it was with the workerSql connection not the shares one so whatever I changed may have only affected that.  Add that same line but where I said 'workerSql' use 'sharesSql'.

[Jine]

I'm going to throw that in soon, i restarted PSJ and now I'm getting this error...

Failed to commit to database.
java.sql.BatchUpdateException: Field 'block_num' doesn't have a default value

Query being executed when exception was thrown:
INSERT INTO pool_shares (rem_host, username, our_result, upstream_result, reason, solution, time) VALUES ('213.112.59.222', 'Ly5pv6', 1, 0, null, '00000001763915294e20 .... 000000000080020000', UNIX_TIMESTAMP('2011-11-01 16:26:22'))



        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2024)
        at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1449)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at com.shadworld.poolserver.db.shares.DefaultPreparedStatementSharesDBFlushEngine. flushToDatabase(DefaultPreparedStatementSharesDBFlushEngine.java:127)
        at com.shadworld.poolserver.logging.ShareLoggingThread.run(ShareLoggingThread.java:156)

Any ideas?

My config (snippet):

usePushPoolCompatibleFormat=false

...

db.stmt.insertShare=INSERT INTO pool_shares (rem_host, username, our_result, upstream_result, reason, solution, time) VALUES (?, ?, ?, ?, ?, ?, UNIX_TIMESTAMP(?))

[shads]

block_num column in sample db script is set to 'not null'... change it and it should be fine..

[Jine]

Dooooooh! Stupid me, thanks

[Jine]

Hmm, strange.. noticed the JDBC-url which contains autoReconnect=true.
I'm having issues with writing shares to the db so... Get the errormessage above in the log.

aaahh when I had that issue previously it was with the workerSql connection not the shares one so whatever I changed may have only affected that.  Add that same line but where I said 'workerSql' use 'sharesSql'.

               if (workerSql != null)
                        workerSql.getJdbcOptionMap().put("autoReconnect", "true");
                if (sharesSql != null)
                        sharesSql.getJdbcOptionMap().put("autoReconnect", "true");

Seems like it's already there.
Does the 0.3.0-FINAL version contain that to? I'm using that for testing now, to avoid any potential issues with my own patches/fixes

A 0.3.1 version based upon the latest hg would be awesome, as i having a bit of issues with compiling the code from bitbucket. (Have to setup a better development environment)

/ Jim

[Jine]

There is something seriously wrong with my setup (MySQL+PSJ), getting all kinds of mysql-issues, this is the latest:

Failed to commit to database.
java.sql.BatchUpdateException: Communications link failure

The last packet successfully received from the server was 703,780 milliseconds ago.  The last packet sent successfully to the server was 9 milliseconds ago.

Query being executed when exception was thrown:
INSERT INTO pool_shares (rem_host, username, our_result, upstream_result, reason, solution, time) VALUES ('213.112.59.222', 'Ly5pv6', 1, 0, null, '000000019e4eb299110ef922bf6f3cfe7dbb2c69157066e567b36266000008d200000000340305a 3500db5479f929646294649725a231962027d44a503d5f67af87fbdd34eb021d51a0df0ca3c9ee9 0200000080000000000000000000000000000000000000000000000000000000000000000000000 0000000000080020000', UNIX_TIMESTAMP('2011-11-01 17:45:09'))


        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2024)
        at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1449)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
        at com.shadworld.poolserver.db.shares.DefaultPreparedStatementSharesDBFlushEngine. flushToDatabase(DefaultPreparedStatementSharesDBFlushEngine.java:127)
        at com.shadworld.poolserver.logging.ShareLoggingThread.run(ShareLoggingThread.java:156)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure

The last packet successfully received from the server was 703,780 milliseconds ago.  The last packet sent successfully to the server was 9 milliseconds ago.

Any ideas?

[Jine]

A restart of PSJ solved it, but the issue is kinda strange. The connection between mysql and PSJ seems to break from time to time with 0.3.0-FINAL.

[Remember remember the 5th of November]

Does poolserverj have rollntime support?

[Jine]

Yes, it does.