[ATTN: POOL OPERATORS] PoolServerJ - scalable java mining pool backend

[shads]

Pushpoold Vs PoolServerJ performance test results now published

As a necessary component to a larger project I've had to write a complete pool server in java.  The rest of the project is still in progress but the pool server is ready to release into beta.  The key problem I needed to solve and part of the reason I rolled my own rather than just using pushpoold is scalability.  Pushpoold is great at what it does but the difference I needed was to get around the bitcoind bottleneck.  PoolServerJ solves that problem rather neatly by caching work and also allowing the pool operator to run as many bitcoin daemons as they want to and feed them all into the poolserver.

A few of the features of PoolServerJ:

    * Native merged mining support.  Will work with standard bitcoind 0.5
    * SCrypt blockchain support (e.g litecoin)
    * Customisable coinbase transaction (pay block rewards to any wallet perhaps an offline secure wallet)
    * Internal work generation (more than 10x faster than rpc getwork)
    * Work caching - caches work from bitcoind or internal WorkMaker to handle short term spikes in requests
    * Database resilience.  Server can keep running if database goes down and shares will be sent to database when connection is reestablished
    * DoS resilience with QoS support to ensure workers who've submitted valid work are serviced with priority
    * Capable of pulling work from multiple bitcoin daemons to get around the getwork bottleneck and also to provide some redundancy
    * Notify of block change via HTTP to a user configurable URL to support event triggered share processing
    * Cached database handling (optional) to reduce round trips.
    * Supports Mysql, Postgresql, sqlite3 (JDBC based so others can be added easily - currently only tested on mysql)
    * Runs as a Windows service
    * Longpolling support
    * Integrated block monitoring using all available bitcoin daemon work sources
    * Dump shares to Database, log file or stdout
    * Safe restart - shares issued to client are dumped to file on shutdown and reloaded on startup so your miners won't get stale shares if you restart a live server
    * Only dependency is a Java 6+ JDK.

Performance

Extensive performance testing against pushpoold has shown PoolServerJ capable of more than 4 times the capacity for share submits and 3 times the capacity for getwork requests.  This is the limit of what could tested on limited available hardware however it's architecture allows for significant scaling and theoretically these limits could be significantly exceeded.

update: Since the release of 0.4.0 WorkMaker Edition getwork capacity has improved by 4 times and share submit capacity by 2 times over the original performance test results.

Please give it a go, feel free to send me any feedback, rip into it if you like.

You can find it here

The webserver may be a bit slow, I'm testing out EC2 micros to see how they handle basic web server duties so sorry about that.  Will have a test pool up and running soon (as soon as I finish the front end). <-- update: this never happened and probably won't... I'm too busy working on the software and there's plenty of pools using it now so I've got enough guinea pigs

[1714977517]

1714977517

[1714977517]

1714977517

[1714977517]

1714977517

[Sukrim]

No Derivative Works — You may not alter, transform, or build upon this work.

Might be a bit hindering... a lot of pools have rewritten parts of pushpoold for example.

Some ideas I had for pool backends:

• Shares evaluation (stale/valid/"winning") done by backend, not bitcoind and only shares that are >=current_difficutly actually get relayed to bitcoind
• "metapool"/"inceptpool" mode that requests getworks from other pools like a miner and relays them
• certain share counts of users as event trigger

[talpan]

Wow,

i definitely will test it.
Although I don't care much about java, this seems to have some very unique features .

What about the stales, do you have some data about that?


regards, talpan

[shads]

No Derivative Works — You may not alter, transform, or build upon this work.

Might be a bit hindering... a lot of pools have rewritten parts of pushpoold for example.

As it says on the download page I plan to fully open source it in the near future, as soon as the larger project is released.

Some ideas I had for pool backends:

• Shares evaluation (stale/valid/"winning") done by backend, not bitcoind and only shares that are >=current_difficutly actually get relayed to bitcoind
• "metapool"/"inceptpool" mode that requests getworks from other pools like a miner and relays them
• certain share counts of users as event trigger

backend currently does evaluate share status, until a real live instance has found a winning block I've got it set to force all shares upstream just in case but that is just a switch and when it's proved it's evaluating a winning solution I will turn it off.

Event triggers are easy to implement, the question is the notification mechanism.  Currently it's just a http request to a URL of your choice but there's got to be a less ugly way.  Open to suggestions.

Re: metapool mode, actually it can already do that.  I just haven't released the doco for it yet.  Coming soon...

[shads]

Although I don't care much about java, this seems to have some very unique features .

What about the stales, do you have some data about that?

Java's not so much of a feature, just a fact.  Although it does give the benefit of a single binary for all platforms.

I haven't done any solid testing on stales but it delivers work back to longpoll requests as fast as it can get them in from the work sources after a block change.  Using multiple daemons should speed this up.  The only instances of stales I saw were in testing when I was bringing the server up and down constantly so the server was losing it's work-worker map and the miner kept humming along with old work.  After I implemented the work-worker map dump to disk on shutdown I haven't seen one.

[Sukrim]

Wow, sounds great so far!

What about Namecoin support? Then you could even quite fast test if your "winning solution relay" works (15 times faster than Bitcoin...).

[shads]

What about Namecoin support?

Haven't tried it yet, just trying to work out if it is now: http://dot-bit.org/forum/viewtopic.php?f=6&t=263&p=1499#p1499

[shads]

Shares evaluation (stale/valid/"winning") done by backend, not bitcoind and only shares that are >=current_difficutly actually get relayed to bitcoind[/li][/list]

BTW I'm pretty sure pushpool does that.  I probably got the idea when I was peering at pushpool source (and getting a headache because I've never learned C).

[phorensic]

Bitcoind (with pushpoold) has been patched to allow 1,000GH/s (on a nice server) and solve the too many open connections problem by Jeff Garzik - http://forum.bitcoin.org/index.php?topic=22585.0 .  It has other nice added features as well.  You say you released this mostly due to scalability.  What is your theoretical limit with your setup?

[shads]

Bitcoind (with pushpoold) has been patched to allow 1,000GH/s (on a nice server) and solve the too many open connections problem by Jeff Garzik - http://forum.bitcoin.org/index.php?topic=22585.0 .  It has other nice added features as well.  You say you released this mostly due to scalability.  What is your theoretical limit with your setup?

By the time the patched bitcoind was announced I was already well into the development process... However I still think there's some significant scalability benefits.  I've just a run a test to try and get some metrics.  The difficult part is eliminating the bottleneck (I don't have a patched bitcoind atm).

Here's the test setup:

Ubuntu 10.4 64 bit running inside vmware on winXP 64 host - Dual core 3Ghz - 8gb ram (3 gb available to ubuntu).
2 * bitcoind, one running on host and one on ubuntu vm.
mysql running in ubuntu vm

To avoid the bottleneck I set the cache size to 20000 getworks per source and waited until the caches were 1/2 full so all works were served from memory.  But the server was still pulling work from the bitcoind's in the background while the tests were running.

Set the difficult to below easy, average 2 hashes required to solve a block.

Client test also run inside ubuntu vm.  
50 concurrent clients continuously issuing getworks.  
Each second 10 of those solve a block (using CPU only) and submit.
Clients shared 10 different sets of worker credentials.

Result:

Avg rate of work delivery was about 1200/sec until the cache ran out then it dropped dramatically due to not being able to get work from daemon fast enough.
about 200 works were submitted and flushed to database.

With 30 block solve attempts/sec:
Avg work delivery 1000/sec
580 works flushed to database

At this point I remembered starcraft 2 was running in the background and killed it...

With 67 block solve attempts per second
Avg work delivery 1200/sec
1000 works flushed to database

To increase the number of block solves/sec anymore I'd need to rewrite some of the test code to synchronize it, due to the way each iteration decides whether to solve a block it becomes unreliable and have no real way of seeing how many block solves it's attempting per sec.

There's some obvious weakness in the test

Unrealistic advantages:
all network communication was local

Unrealistic disadvantages:
Everything (including clients and block hashing) running on same machine.

Hopefully someone who's got more real world experience with pushpool can give those numbers some context.

[phorensic]

I'm sorry, Joel Katz did the patching, not Jeff Garzik, oops!

Unrealistic advantages:
all network communication was local

I'm glad you mentioned the above.  While latency and bandwidth is usually not the bottleneck of a pool server, virtual networking is almost unlimited compared to a typical WAN link.  Not to downplay your development and testing or anything!  Very good info here!

I'm a little bit confused as to the new bottleneck you got to.  Are you saying bitcoind is your bottleneck?  If so, that's why Joel Katz hacked on it.  A month ago the large pool operators were screaming at pushpoold as the bottleneck, including Eleuthria with his awesome experiments and communication with his members.  Then some people started to discover it was actually bitcoind, hence the patches.  With all due respect, I hope you aren't trying to scale the wrong side of the puzzle.  It may be faster than pushpoold, but all for naught if bitcoind can't feed the monster.

[shads]

[0.2.1]

- fix: maxCacheSize was being ignored and hardcoded to 50
- fix: thread pool size for work fetchers not being set correctly.
- enabled allowedManagementAddresses property
- added source.local.<n>.disabled=true option so you don’t have to comment out every line of a source to disable it.

[Sukrim]

2 more features off the top of my head:

* Google app engine compatibility
* Hash offloading to a GPU (I have no idea if that is a major bottleneck of your program though) - I recently read something about doing SSL calculations on a GPU, getwork verifications etc. should also be an easy task.

[shads]

2 more features off the top of my head:

* Google app engine compatibility
* Hash offloading to a GPU (I have no idea if that is a major bottleneck of your program though) - I recently read something about doing SSL calculations on a GPU, getwork verifications etc. should also be an easy task.

I looked at google app engine but it has a couple of gotcha's. 

- Time per request: max 30 sec

Which effectively rules out longpolling.

- Memory cap 128MB.

Depends on how hard you are running it but if you run a large pool you may find this restrictive.  A smallish pool will comfortably run in about 50mb but once you start scaling you are consuming memory two ways, first is caching work.  Second is mapping work delivered to workers for current block and last block.  I can't actually remember why I did this, it was partly to ensure no duplicate work but there was one other reason. 

Another option for scalable hosting would be Amazon elastic beanstalk.  It would require some refactoring to strip out the embedded server and turn it into a fully compliant webapp, I think it's backend is Jetty (have to double check), if it that's good because poolserverj uses some of jettys advanced features (which are outside the servlet spec) to avoid thread saturation and handle long polling.  If there's a reasonable demand then I'll look into doing it.

Interesting idea about the GPU calc but I wonder if in this scenario it would really yield much benefit?  Under a mining scenario the overhead of translating the text to a byte array, sending to GPU and starting calculation happens once then the gpu does many billions of calcs.  In this scenario that overhead would be incurred and the GPU would do a single calculation.  The text-byte array conversion happens anyway I suppose but I still wonder whether the net gain would be comparable to overheard?  I'm not a GPU programmer so it's hard to say.

[shads]

I'm a little bit confused as to the new bottleneck you got to.  Are you saying bitcoind is your bottleneck?  If so, that's why Joel Katz hacked on it.  A month ago the large pool operators were screaming at pushpoold as the bottleneck, including Eleuthria with his awesome experiments and communication with his members.  Then some people started to discover it was actually bitcoind, hence the patches. 

Yes it was bitcoind that was the bottleneck, I built the patched version yesterday and it was a vast improvement. 

With all due respect, I hope you aren't trying to scale the wrong side of the puzzle.  It may be faster than pushpoold, but all for naught if bitcoind can't feed the monster.

I don't know if it's faster than pushpoold.  It quite different in a lot ways.  It just handles some things differently, the key difference being it can use multiple bitcoinds if they are proving to be a bottleneck.  So under some scenarios it will probably be faster and under others it might be slower.

[shads]

Due to the issues noted in the changelog I’ve decided it was probably premature to call PoolServerJ a beta release.  Until those issues are resolved it is reverted to alpha status.

[0.2.2]

- fix: safe restart throwing concurrent modification exceptions due to requests still being serviced during shutdown
- added property enableSafeRestart to allow disabling
- trimmed unnecessary data out of work-worker map entries, reduced overall memory usage by 80%
- enable sharesToStdout and requestsToStdout options
- added useEasiestDifficulty property to enabled stress testing of work submits

Known issues
- Under extreme load database flush thread can become overwhelmed particularly if the server
is started under high load conditions and doesn’t have time for JIT compiler to kick in.
This is currently managed by throttling work submit connections if cache entries exceeds
maxCacheSize.  Need to implement concurrent database writes.
- Block change notification currently doesn’t take into account the state of the database
cache.  This should wait until all shares from the previous block have been flushed to
DB before notifying of block change.
- Cache flushing should be temporarily suspended after a block change to allow resources
to focusing on completing longpoll requests.
- Memory usage.  work-worker map entries size has been reduced by 80% but it still presents
a theoretical limit.  With 768mb max heap size the server can handle about 1.2 million
cached works requests before OOM error occurs.  At about 1 million performance seriously
degrades due to frequent garbage collection.  The map contains shares issue in current
block and previous block.  So this will only be an issue for very busy servers.  Current
workaround is to assign more memory to JVM.  Fix will require a revisit of the work map
strategy to either trim the map entries even more dramatically or find a safe method of
pruning the map.

[shads]

It has other nice added features as well.  You say you released this mostly due to scalability.  What is your theoretical limit with your setup?

I can answer your question properly now.  On the test setup I used it achieved a max long term avg of 2154GH/s.  

Bear in mind that the clients were running on the same machine and performing CPU hashes (approx 2 per submit).  The test was pretty clearly CPU bound from all the observations I made so I'm quite sure if the client wasn't running on the same machine it could have been close to double that.  The server performs 1 hash/submit.  So with additional overhead it's probably a reasonable guess that they were using about 50% of the cpu each (which ps indicated as well).  The next hurdle is database write speed.  Which seemed to on the borderline of becoming the next bottleneck.  I'm going to test concurrent writes to see if that yields an improvement.

[phorensic]

Oh wow, that's impressive.  You might want to contact the big pool owners to see if they are interested.  Currently the new cap on pushpoold seems to be 1,000GH/s on a fully tweaked setup.

[shads]

Oh wow, that's impressive.  You might want to contact the big pool owners to see if they are interested.  Currently the new cap on pushpoold seems to be 1,000GH/s on a fully tweaked setup.

So the bottleneck seem to be submissions rather that getworks?  Can you point me to where this is discussed?  Particularly the 1000gh/s limit?

[phorensic]

Check it out - https://bitcointalk.org/index.php?topic=22585.0

[eleuthria]

Right now BTC Guild is forced to do some very awkward things to get pushpoold to scale to 1 TH/s per server.  I'm very interested in this project, but until the source is out I won't be able to give it any real-world testing since my DB Schema is very customized for faster reads/multi-server support.

[Furyan]

Right now BTC Guild is forced to do some very awkward things to get pushpoold to scale to 1 TH/s per server.  I'm very interested in this project, but until the source is out I won't be able to give it any real-world testing since my DB Schema is very customized for faster reads/multi-server support.

+1.  My DB is heavily customized for much the same purposes.  The addition of bits of information can reduce future lookups by millions.

[phorensic]

+1.  My DB is heavily customized for much the same purposes.  The addition of bits of information can reduce future lookups by millions.

Adding one index or changing one line of code on our webpages can cause our mysql server to go from 15 million rows scanned per minute to 5 million.  It's nuts!

[Furyan]

+1.  My DB is heavily customized for much the same purposes.  The addition of bits of information can reduce future lookups by millions.

Adding one index or changing one line of code on our webpages can cause our mysql server to go from 15 million rows scanned per minute to 5 million.  It's nuts!

The joys of database administration and design

[shads]

Well I just needed a shove I guess.  I was going to wait a few weeks while I finished some other thing but since there's demand for it I'm refactoring the code to split out the parts that I'm not ready to release yet (which you won't need anyway).  So I should be able to publish the source very soon.  I'd really like to see the results of other people's tests so I'm prioritising this.

As part of the refactoring I'm splitting out some parts to engine classes that can be easily overridden for the things that are most likely to need customisation.  Namely: Worker db lookups, authentication, share database writes.  That should hopefully minimize the need to fork the code so you won't get clobbered by every update.

I'll do this all as part of release 0.2.3 which is a pretty major improvement.

- fixed memory hog issues so memory usage has stabilised.
- switched duplicate maps and work-worker maps to trove THashMap implementation and added customised hashing strategy which has yielded about a 50% boost to getwork speed
- implemented several bulkloader engines for share db writes which has boosted db write speed by 5-20 times depending on which engine is used.
- partial implementation of mining extensions (reject-reason complete, rollntime and noncerange still work in progress).

[shads]

0.2.3 released now with source code available. 

This is the first release I'd consider usable in a production environment.  Memory leak issues have been resolved and some hefty speed increases have been gained on both the getwork side and the share submit side.


[0.2.3]

- fix: Memory usage issues for work-worker map and duplicate map
- switched duplicate maps and work-worker maps to trove THashMap implementation and added customised hashing strategy which has yielded about a 50% boost to getwork speed
- share database flushing now paused delayed for 4 seconds after block change to free up resources for fullfilling longpoll requests
- implemented several bulkloader engines for share db writes which has boosted db write speed by 5-20 times depending on which engine is used.
- partial implementation of mining extensions (reject-reason complete, rollntime and noncerange still work in progress).
- implemented rudimentary plugin interface to allow customized engine for Authentication, worker fetching, share logging (to database only, file logging is still hardcoded)
- major refactoring and reorganization of packages in preparation for source release.

Known issues
 - Under heavy load shutdown may fail to trigger on the first try.  Sending a 2nd shutdown signal
 seems to reliably complete shutdown.
 - All new bulkloaders are not considered stable.  Use at your own risk.

[BurningToad]

I think I will try testing this soon to see if it will work for me.  My pool is running up against the 450-500 GH/s pushpool barrier.

I would also be very interested to hear what you are doing about it eleuthria, if you have a single pushpool instance scaling up to 1,100 GH/s per server, or, it sounds like you might have 3 instances running on the same server, I guess with routing/port stuff that you have to set up to do that?

[shads]

Source now published to a repo:

https://bitbucket.org/shadders/bitcoin-poolserverj/

[shads]

Architecture diagram is now on the site.  Also for those of you who would need to customise the code to match you database structure I've written a guide to building DB and authentication plugin modules.

http://poolserverj.org/documentation/

[jamesg]

Definitely interested in this.

[shads]

0.2.4 source now pushed to repo.  This primarily improved the plugin mechanism which is only of interest to people who will build from source so no binary has been published on the main site.

[eyeoh]

I'm testing it out now. Maybe I have a setting in the config wrong but I keep getting the good old "Problems communicating with bitcoin RPC" error and I've verified bitcoind is up and happy and all the IP/login/pass/port settings are correct. Hitting it with Firefox gives me "Problem accessing /. Reason HTTP method GET is not supported by this URL" and I'm seeing "33022 [main-con-qtp-22] WARN org.eclipse.jetty.util.log - / java.lang.NullPointerException" in the error log with debugging turned on. Any hints?

[shads]

Can you pm me the full stacktrace. (need to set logStacktraces=true in properties file).

The message you're getting from the brower is correct.  JSON-RPC should not accept GET requests.  The request has to have content so it has to be a POST.  Although some mining clients use it for LP so the longpoll servlet will allow it.

If you've enable the management interface which is enabled in the sample properties file try hitting
http://localhost:8997/?method=getsourcestats from your browser.  That should at least tell if it's up and if it's communicating with the daemon

if you want to test an actual request then try in one browser tab hit the longpoll url, you'll need to include the user:pass in the url:
http://user:pass@mylongpollurl

Then in another browser tab hit:
http://localhost:8997/?method=fireblockchange

This should force your longpoll request to finish and at least give you some kind of status message.

[shads]

BTW if the nullpointer stack trace looks like this one:

Code:

36142 [main-con-qtp-18] WARN org.eclipse.jetty.util.log - /
java.lang.NullPointerException
        at com.shadworld.poolserver.WorkProxy.handleRequest(WorkProxy.java:412)
        at com.shadworld.poolserver.servlet.PoolServerJServlet.getResponse(PoolServerJServlet.java:26)
        at com.shadworld.poolserver.servlet.AbstractJsonRpcServlet.doPost(AbstractJsonRpcServlet.java:176)

Someone else found this issue last night... The problem is caused by client not setting 'id' field in json-rpc request.  I've updated json-rpc lib to fallback to a default '-1' instead of throwing a nullpointer and commited to repo.  Will try an put up a new binary dist today.

if you need a workaround, use a miner that does json-rpc according to spec.  Phoenix seems to do it right...

[shads]

I'm testing it out now. Maybe I have a setting in the config wrong but I keep getting the good old "Problems communicating with bitcoin RPC" error and I've verified bitcoind is up and happy and all the IP/login/pass/port settings are correct. Hitting it with Firefox gives me "Problem accessing /. Reason HTTP method GET is not supported by this URL" and I'm seeing "33022 [main-con-qtp-22] WARN org.eclipse.jetty.util.log - / java.lang.NullPointerException" in the error log with debugging turned on. Any hints?

0.2.5 is available on downloads page now.  If you're was the problem I was describing it should be fixed now.

[0.2.5]

 - update to jsonrpc lib - force JsonRpcRequest.getId() to return a default value (-1)
 if not set.  Some clients don't implement spec properly and this was causing a nullpointerException.
 - Cleaned up directory layout for distribution.
 - Fix: classpath problem for windows service so no longer need to build into one monolithic jar
 - added lib/plugins directory which is in the default classpath so plugins no longer require classpath changes.
 - update procrunsrv to 1.07 to support wildcard classpaths

[0.2.4] - unreleased

 - synchronised share flush to database to prevent concurrent attempts.
 - set server header to PoolServerJ(v<version>)
 - implemented resource pools for recycling objects used in WorkProxy.chooseSourceForWork()
 - Added reset(args) methods to a number of heavily used classes for recycling
 - added example AnyPasswordWorkerAuthenticator class as used in plugin build guide: http://poolserverj.org/documentation/plugin-guide/

[Dusty]

@shads:
Do you thing it would be possible to have a pure java implementation of getWork() without the need of calling the bitcoin daemon, using for example the bitcoinj library?
Could this speed up things?

[shads]

@shads:
Do you thing it would be possible to have a pure java implementation of getWork() without the need of calling the bitcoin daemon, using for example the bitcoinj library?
Could this speed up things?

Dusty I looked at doing this but Mike Hearn pretty much flat out told me that bitcoinj was miles off being ready to serve getworks.  It's not really a problem at the moment though, patched bitcoind's are fast and if you do hit the wall you can always hook up several separate bitcoinds to feed poolserverj and get a big speed boost that way.  The biggest bottleneck I can see is writing to databases.  The bulkloaders I added in 0.2.3 speed that up by somewhere between 500%-2000% depending on which engine you use.

When bitcoinj is ready though I definately will incorporate it.

[eyeoh]

Just tried 0.2.5 and ran into this:

Code:

:/usr/src/poolserverj-0.2.5# /usr/bin/java -jar bin/poolserverj.jar start config.cfg      
Args - [2]: start config.cfg 
PoolServerJ Service Starting Sat Aug 13 16:49:33 MST 2011
java.lang.NoClassDefFoundError: javax/servlet/http/HttpServlet
        at com.shadworld.poolserver.servicelauncher.PoolServerService.start(PoolServerService.java:88)
        at com.shadworld.poolserver.servicelauncher.PoolServerService.windowsService(PoolServerService.java:48)
        at com.shadworld.poolserver.servicelauncher.PoolServerService.main(PoolServerService.java:18)
Caused by: java.lang.ClassNotFoundException: javax.servlet.http.HttpServlet
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        ... 3 more

Anything I can do?

[shads]

Ah sorry the startup command has changed a bit, I've updated http://poolserverj.org/quick-start/

The command is uglier but makes things a lot more flexible.  Probably best to put it in a script...

Code:

java -classpath poolserverj.jar:../lib/*:../lib/lib_non-maven/*:../lib/plugins com.shadworld.poolserver.servicelauncher.PoolServerService start config.cfg

if you create poolserverj.sh

Code:

#!/bin/bash

java -classpath poolserverj.jar:../lib/*:../lib/lib_non-maven/*:../lib/plugins com.shadworld.poolserver.servicelauncher.PoolServerService $1 $2 $3

then it's a bit less ugly:

Code:

chmod 777 poolserverj.sh
./poolserverj.sh start config.cfg

Currently has to be started from the bin dir...

[eyeoh]

Just want to let you know that worked! Up and running with 0.2.5 and Guiminer, i'm about to start testing Diablo and Phx and all the others and stress test it.

[shads]

Just want to let you know that worked! Up and running with 0.2.5 and Guiminer, i'm about to start testing Diablo and Phx and all the others and stress test it.

great!  please let me know how you go.  I've only tested with phoenix.  I'm lurking in #bitcoin-dev and #bitcoin-poolowners btw if you need any real time help.

[relm9]

Maybe I screwed up my config somewhere but I can't seem to get this to work.

-bash-3.2$ ./minerd --url http://ip:8999 --userpass test:test
[2011-08-13 20:51:18] JSON key 'midstate' not found
[2011-08-13 20:51:18] JSON inval midstate
[2011-08-13 20:51:18] json_rpc_call failed, retry after 30 seconds
[2011-08-13 20:51:18] Long-polling activated for http://ip:8999/LP/
[2011-08-13 20:55:04] longpoll failed, sleeping for 30s

I can connect to my Bitcoin RPC directly fine. I've also temp set rpcallowip to * to rule out any possible conflicts, no luck. It happens if I disable longpolling too. Pushpoold on the same box is running fine.

Paste of config: http://pastebin.com/JFnhMrgY

Oh and I'm running the latest JDK 1.7

[shads]

The only I can think of is that you've got: useEasiestDifficulty=true

This should have been false in the sample-config.  That difficulty is actually less than 1 so any validation by the miner will probably fail.  Though it's odd it claims 'midstate' is not found...

I've made changes necessary to allow GET requests via browser so I'll upload those in a minute then you should be able to see what it's returning at least.

[shads]

correction:  midstate is actually missing... That can't be good... Will get it fixed asap...

[shads]

This is includes a fix for a fairly serious bug introduced in 0.2.4 where getwork responses were missing some fields if difficulty was rewritten.

relm9 this should fix the problem you were having... it's uploading atm so give it 10 minutes for my slow uplink to squeeze it through

[0.2.6]

 - enable different DB connection parameters (and engines) for shares and workers
 - basic bash startup script
 - shutdown called from command line with properties file will attempt to shutdown the server described in props file via managements interface
 - GET requests to main listener will now assume it's a getwork request rather than refusing to respond.
 - Resource pool implemented for request content StringBuilders
 - fix: getwork missing some fields if target is rewritten

[relm9]

Thanks shads   That fixed it, works perfect now. Will let you know if I come across any other bugs.

By the way, useEasiestDifficulty=true is actually default in the sample config.

[shads]

By the way, useEasiestDifficulty=true is actually default in the sample config.

Yeah that's what I meant, it shouldn't be.  I was supposed to change it before I put it in the distribution but forgot to.  It's only there for stress testing work submisssions.  I don't think it had anything to with your issue though but make sure you change it or you'll be accepting work of pretty much any difficulty.

[shads]

Thanks heaps to the people that have been testing it... We've uncovered a few major bugs that have been rectified and will be in next release 0.2.7

If you're using 0.2.6 or below then please be aware one of those bugs prevents valid shares from being accepted.

I have noticed during all the debug/troubleshooting processes I've gone though with various people the size of the config file is pretty cumbersome.... What's peoples thoughts on making it optional to split it... e.g. use a master properties file with an 'include' that allows you seperate for e.g. database, sources into different files.  That way you could have a main config, then plug in a different database config for different scenarios...  What do people think?  Is it worthwhile?
Its not really difficult to do timewise... But the benefit would be mainly for debug/test scenarios, allowing you switch over to test setups but with more confidence that you other configuration is consistent.

[shads]

0.2.7 released

[0.2.7]

- fix: accidentally hardcoding our_result = false
- enabled reporting winning share to blockChainTracker which enables notifyBlockChangeMethod to report if the block was won or not.
- fix: UniqueDataPortiton.equals not comparing properly.  preventing hashmap lookups making valid work submits report as unknown.
- added method “flushWorkers” to mgmt interface which works just like “flushWorker” except it take a “names” param with a comma delimited list

[eyeoh]

I've been having exceptional results while testing this. Very low stales compared to pushpoold, like ~0.1% or less average. Very stable compared to pushpoold. Fast restarts and it picks up the shares it had in the air when you do have to restart. And it runs fast and doesn't eat endless memory, the GC process seems work great. Really fantastic so far.

Wanted to clarify - The only reason I've restarted poolj so far was because I wanted to, not had to.

[gentakin]

This is interesting. I'm wondering, since your license says "no derivative works", what exactly am I allowed to do?

If I wanted to start my own pool and needed to change some parts of PoolServerJ (and let's assume I can't do this with a plugin, so I actually need to change YOUR code), this seems to be a "derivative work". Can I do this? I don't think so. Or maybe it's just forbidden to share my derivate work with others?

[shads]

This is interesting. I'm wondering, since your license says "no derivative works", what exactly am I allowed to do?

If I wanted to start my own pool and needed to change some parts of PoolServerJ (and let's assume I can't do this with a plugin, so I actually need to change YOUR code), this seems to be a "derivative work". Can I do this? I don't think so. Or maybe it's just forbidden to share my derivate work with others?

Good catch, that's an oversight on my part... as of v0.2.3 the code was published to https://bitbucket.org/shadders/bitcoin-poolserverj

As far as I'm concerned what's published is OSS.  I haven't updated the licenses yet, will do so in the next release which should be in the next couple of days.  If you want to be covered in the meantime pm me an email address and I'll send you a 'permission to create derivatives' email.  I would just write it here but I don't think a forum post carrys much legal standing.

The only parts that aren't fully OSS with published source are a few utility libraries of my own that I use for numerous unrelated projects.  The reason I haven't published those is because I don't want the hassle of maintaining a public repo for them all.  You are welcome to decompile and inspect code, if you really want the source contact me and I'll send you a copy of the current snapshots.  The one's I'm referring to are all the libs in the /lib/lib_non-maven folder with the exception of the trove library (which is OSS and not written by me).

[gentakin]

Yes, I've seen those libraries. It's very nice of you to open source PoolServerJ! It seems to be a nice piece of software. No need to send me a permission, I simply asked because I was wondering what "no derivative works" means (and why the source code is public then). Now it's all clear, the license has not been updated and it used to be closed source.

I've tested PoolServerJ and so far I'm impressed. The possibility to write your own sharelogger and simply plug it into PoolServerJ with a config file directive is nice. I implemented one that logs invalid shares to their own table (BTW: if "entry.reason == null && entry.ourResult" evaluates to true, is it safe to assume that this share is valid and should be counted for rewards?).

However I'd like to report a bug, or at least I think it is a bug.
I changed phoenix to do this:

Code:

        if self.checkTarget(hash, nr.unit.target):
            for bad in range(nonce, nonce+20):
	            formattedResult = pack('<76sI', nr.unit.data[:76], bad)
	            d = self.miner.connection.sendResult(formattedResult)

So when phoenix finds a share, it takes the nonce and submits it, but then increases the nonce by 1 and submits that "share" as well, and then continues until it has reached nonce+20.

I actually find 20 shares in my table when only one should be valid, and they all have "our_result == 1".
Maybe the problem is with Res.getEasyDifficultyTargetAsInteger()?

[shads]

(BTW: if "entry.reason == null && entry.ourResult" evaluates to true, is it safe to assume that this share is valid and should be counted for rewards?).

Yes that should be a safe assumption except for the bug you pointed out ... Actually the check should just be entry.ourResult.  Currently reason is left null for a valid share and I can see no reason why it would change but it's possible in future the reason field may be used for some sort of meta data.  entry.ourResult is intended to be guaranteed to reflect share valid/invalid from the pool's perspective.  

So when phoenix finds a share, it takes the nonce and submits it, but then increases the nonce by 1 and submits that "share" as well, and then continues until it has reached nonce+20.

I actually find 20 shares in my table when only one should be valid, and they all have "our_result == 1".
Maybe the problem is with Res.getEasyDifficultyTargetAsInteger()?

Thanks for the report.  I'm a bit befuddled to be honest since the validation code hashes the solution before checking.  Any chance you could send me the modded phoenix file?  And let me know what version?  Or if you can tell me the version jand which file it's in if the code you quoted is verbatim.

You didn't happen to have useEasiestDifficulty=true in your properties file did you?  If so could you set it false and test again? useEasiestDifficulty sets difficult so only 2 hashes are required on avg to get a solution.  It's intended for load testing only and should have been set false in the sample properties file but as with the license I forgot to update

I think for next version I might change the property to useRidiculouslyEasyTargetButDontIfThisIsARealPool so it's more obvious that's it's not the same thing as rewritedifficulty in pushpool.

[gentakin]

I'm a little confused as well, because I see the hashing code in WorkProxy.java. But since I'm not so experienced with bitcoin hash targets, BigInter.compareTo and your .getEasyDifficultyTargetAsInteger(), I'm not sure what's wrong!

This is from my properties file, and it's the only line that sets useEasiestDifficulty: (And it is actually a hg clone from yesterday, so I think it was set to false all the time):

useEasiestDifficulty=false

Maybe some more details on what I do/get:
 * Start PoolServerJ with empty shares table
 * Start cheating Phoenix (see below)
 * Wait until Phoenix reports "[18/08/2011 15:15:21] Result: b464f013 accepted" (this will appear only once, but phoenix will send 20 getwork-answers)
 * See PoolServerJ log something like this 20 times: "Forced work submission upstream: 00000001a55604a05d5f30c4d1784f38bd1e1389f3[..]" with slightly changed nonce each time
 * Now PoolServerJ logs 20x "work submit success, result: false"
 * A few seconds later, PoolServerJ flushes the shares to MySQL and indeed, 20 valid shares are now in the table

(I'm not sure if this indicates there's another problem somewhere, but a lot of "RETRY" is sometimes mixed into the debug log output)


I'm using a pretty old SVN checkout of phoenix (I prefer poclbm, but I had a phoenix start script for localhost sitting around, so I used it), it's r101 I believe. However the newest git should be fine as well:
https://github.com/jedi95/Phoenix-Miner/
In KernelInterface.py, change foundNonce.

Original code from git:

Code:

        if self.checkTarget(hash, nr.unit.target):
            formattedResult = pack('<76sI', nr.unit.data[:76], nonce)
            d = self.miner.connection.sendResult(formattedResult)

Cheating code (as always with python, tabs/indents are important, not sure if this was copied correctly):

Code:

        if self.checkTarget(hash, nr.unit.target):
            for bad in range(nonce, nonce+20):
	            formattedResult = pack('<76sI', nr.unit.data[:76], bad)
	            d = self.miner.connection.sendResult(formattedResult)

Edit:
Are you sure Res.getEasyDifficultyTargetAsInteger()=115792089237316195423570985008687907853269984665640564039457584007908834672640 is correct?
I thought it would be 0xffff0000000000000000000000000000000000000000000000000000 for a difficulty-1-share, or 26959535291011309493156476344723991336010898738574164086137773096960 according to wolfram alpha.

[shads]

hmmm... if useEasiestDifficulty != true then

easyDifficultyTargetString = "ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"
easyDifficultyTargetAsInteger = new BigInteger(easyDifficultyTargetString, 16);

Which is basically just parsing the hex string...

There's no other reference to either variable through any codepath.... I'll have a more detailed look in the morning but I' think that string is wrong?  Not sure where I got that from...

[shads]

Edit:
Are you sure Res.getEasyDifficultyTargetAsInteger()=115792089237316195423570985008687907853269984665640564039457584007908834672640 is correct?
I thought it would be 0xffff0000000000000000000000000000000000000000000000000000 for a difficulty-1-share, or 26959535291011309493156476344723991336010898738574164086137773096960 according to wolfram alpha.

I worked it out (thanks to luke-jr for the handholding).. I had the endianess around the wrong way when setting the easy target string and converting to a BigInteger.  I've made a fix now and will create a new release shortly.

[gentakin]

That's great! And thanks, Luke.



Edit to prevent double post:
There seems to be a (or even two more) different bug(s) related to counting invalid shares as valids. This time, I changed Phoenix to submit every share 10 times with a delay of 1 sec between sends (I'm not sure if the delay works, it might send them instantly..). The first share was counted as "valid" two times, and "invalid" two times as well, so a total of 4 shares were recognized. I'm not sure if phoenix stopped sending the share after the 4th time, or otherwise 6 shares were sent but not registered as valid/invalid.

I assume phoenix then found a second nonce in the same getwork, this time producing 10 valid shares. That bug is probably in WorkProxy.java, line ~186.

Code:

					if (entry.solutions == null) {
						entry.solutions = new ArrayList(10);
						entry.solutions.add(data);
					} else if (entry.solutions.contains(data)) {
						return buildGetWorkSubmitResponse((Integer) request.getId(), false, "duplicate");
					}

When the second share of the same getwork is received, "entry.solutions != null" and "entry.solutions.contains(data) == false", so the second share is accepted (this is correct behaviour). However, unless this happens elsewhere in the code, the second nonce is *not* added to entry.solutions, so it can be sent infinitely often and will award a valid share every time.

Moving "entry.solutions.add(data);" below that if/else if should fix this. There's still the problem with the first nonce in a getwork accepted multiple times, but not infinitely often.


Update:
python does not respect my time.sleep(1), so  all the nonces are sent without delay. The problem with identical shares being accepted multiple times could be a race condition. This is from the phoenix log:

[19/08/2011 18:19:31] Result: e36b1881 accepted        
[19/08/2011 18:19:31] Result: e36b1881 accepted        // and indeed, this share was logged twice by PoolServerJ as "valid"
[19/08/2011 18:19:31] Result: e36b1881 rejected        
[19/08/2011 18:19:31] Result: e36b1881 rejected        
[19/08/2011 18:19:31] Result: e36b1881 rejected        
[.. 5 more rejects]

The rejects are not logged in PoolServerJ, so I guess they fail because of connection issues (sending 10 shares at the same moment might cause problems in phoenix).

Yet another update:
My suggestion to move "entry.solutions.add(data);" below the if/else if is wrong, it should probably go pretty far down the validation method, maybe after checking the hash. That should fix the double-accepts.
Are "duplicate" rejects not getting logged on purpose? (Omg, I'm spamming this thread, sorry!)

[shads]

spam away.. It's very helpful.  The faster we can find all the bugs the sooner it's ready for production use.  I'll have a detailed look at all this now.  I would have last night but got hit by the flu from hell and had to go to bed...

[shads]

This release contains some fix's and updates essential for anyone running a live pool.  Please read the changelog for details.

[0.2.8]
- implement 'include' in properties to allow seperation of config blocks into different files for easy changeover
- add check for duplicate solution on submit
- changed 'stale-work' to 'stale' for pushpool compatibility
- change property name 'useEasiestDifficulty' to 'useRidiculouslyEasyTargetForTesingButDONTIfThisIsARealPool' to make is clear this isn't the same as pushpools 'rpc.target.rewrite'
- crude support for share counter table updates rather than full submits.
- fix: cache size set to 1/2 maxCacheSize due to partially implement dynamic cache sizing
- fix: shares accepted below difficulty due to endian issues setting difficulty target. (thanks luke-jr for the help)
- fix: duplicate work checks not working properly due to race conditions.  Moved atomic duplicate check/update operations into synchronized block.
- fix: duplicate work not being logged
- updated json-rpc and utils libs
- update sample properties file to reflect recent changes.

[shads]

- There seems to be a (or even two more) different bug(s) related to counting invalid shares as valids.

- Are "duplicate" rejects not getting logged on purpose?

Hi Gentakin,

The 0.2.8 release should address all these issue.  I've moved all the duplicate handling inside the same synchronized block so this should alleviate the race condition.  There was a reason for duplicate not being logged... Developer forgot... Fixed now though.  I've rewritten the whole duplicate check portion.  I found a couple of other possible problems in addition to the 3 you found so thanks for bringing my focus to this part of the code, it was very much needed.  It was originally added as a bit of an afterthought and not really tested thoroughly.

The tests you've been doing would make great unit tests... If only I could work out how to integrate phoenix into a unit test.  DiabloMiner might be an option for integration as a test platform since it's java based.

[gentakin]

Thanks for fixing the bugs really fast! I appreciate it. And get well soon.

I also thought about creating a test suite to check for these kind of problems (could be useful for other custom pools, like AFAIK bitp.it). I have been digging through the Diablominer source some time ago, and I guess it's possible to use it (if the license allows to do so). It will probably get "a little" cluttered..

[eyeoh]

Still having great results with poolj

[Sukrim]

On the other hand you could also try to use jython and run phoenix too!

...or include a fixed "test" getwork with known solution(s)?

It might however be interesting to have something like a "benchmark"/test of the whole system that consists of several miners (where you know the solutions already beforehand), getwork sources (emulated bitcoinds that always return the same sequence of getworks as well as emulated pools for the proxy mode) where you can then also do regression tests for the performance of the whole software.

[shads]

It appears the last critical update introduced a new bug which if triggered causes a memory leak due the cache flushing thread crashing.  If you're using 0.2.8 upgrade is urgently recommended.

[0.2.9]
- fix: FastEqualsSolution not serializable causing exception dumping workmap during safe restart
- fix: nullpointer exception crashing cache cleaner thread leading to eventual OOM error.
- added generic try catch to all threads to catch unknown exceptions and prevent them stopping. - need to add 'shutdownOnCriticalError' option then these errors can be handled by shutting down the server and a wrapper script can restart.

[urstroyer]

The PoolServerJ works great! Keep development going on, i see really huge potential! Definitely worth some donations!

[eleuthria]

Giving PSJ a bit of a bump here.

BTC Guild's new PPS pool has elected to use PoolServerJ during its beta run.  So far the performance has been outstanding (looking at stale/reject rates of < 0.2%).  It eats a decent chunk of RAM at boot, but the increase in RAM usage is very small as the pool scales.

Excellent work shadders, and hopefully soon we can post that a PSJ pool has found its first block .

[eleuthria]

3 blocks have been found by BTC Guild using PoolServerJ so far.  All 3 had the latest patch where it logs shares detected as matching current difficulty before submission and tracks them.

All 3 blocks were identified by PSJ as full difficulty shares before submission, and all 3 were accepted.  No false positives/false negatives so far.

[DavinciJ15]

What to be a hero with your software?

Help me set it up on an Amazon servers and save Namecoins PoolServerJ will be the hero.

[shads]

What's happening to namecoins?

[DavinciJ15]

What's happening to namecoins?

Someone planning an attack, My pool can't handle the hash rates I can give you an amazon server or servers with pushpool compatible schema tonight (Im at work) and you can set it up PoolServerj.  Just tell me what you need.

read about it here...
Namecoin is Prime for a 51% attack
http://dot-bit.org/forum/viewtopic.php?f=2&t=292

[DavinciJ15]

What's happening to namecoins?

I'm sending you 10 btc at 1LezqRatQz7MeNoCVziYwcdwtqeEbvrdAq for your work even if you don't help.

[wtfman]

hey, nice work so far, keep it up

what I missed at my first test was, that I was not able to connect my miners without supplying a password. Would be awesome if that could be implemented too. Maybe I did not find out how it works, but its possible?

[shads]

hey, nice work so far, keep it up

what I missed at my first test was, that I was not able to connect my miners without supplying a password. Would be awesome if that could be implemented too. Maybe I did not find out how it works, but its possible?

take a look at the guide to building plugins:
http://poolserverj.org/documentation/plugin-guide/

It just so happens the example I used in the tutorial was an AnyPasswordWorkerAuthenticator plugin.  That example plugin is included in the build so you just need to do step 2 to enable it.

[wtfman]

hey, nice work so far, keep it up

what I missed at my first test was, that I was not able to connect my miners without supplying a password. Would be awesome if that could be implemented too. Maybe I did not find out how it works, but its possible?

take a look at the guide to building plugins:
http://poolserverj.org/documentation/plugin-guide/

It just so happens the example I used in the tutorial was an AnyPasswordWorkerAuthenticator plugin.  That example plugin is included in the build so you just need to do step 2 to enable it.

great! thanks. also sorry for not noticing it

[DavinciJ15]

Thanks for the fix for my pool it helped a lot, there is no need for me to switch from pushpool right now I must focus adding more power to my pool so that it can handle more hashes.

Once that's done I can upgrade to PoolServerJ.

[shads]

This is a major milestone release for PoolServerJ.  Many of fixes and improvements were as a result of an extensive stress testing process by BTC Guild while they were migrating the pool over.  To celebrate I've changed to license to fully open source.

Notable changes include:

    * Now licensed under GPL v3
    * complete rewrite of longpolling code which has improved longpolling performance markedly
    * alpha implementation of native longpolling listener with the aid of an intermediate daemon built by Caesium from rfcpool
    * numerous stability fixes.

Complete changelog (including the changes from the unreleased 0.2.10):

[0.3.0rc1]

- license change to GPL v3.0
- added a temporary logger to record all stages of submission where a real solution is found.
- fix from <Eleuthria>: convert submitted data to lowercase, some miners return in uppercase resulting in failed work lookup from map.
- new mgmt interface methods:
?method=setCacheSize&source=<mySourceName>&value=<value>
?method=setMaxConcurrentDl&source=<mySourceName>&value=<value>
?method=setMaxWorkAgeToFlush&source=<mySourceName>&value=<value>
?method=setAllCacheSize&value=<value>
?method=setAllMaxConcurrentDl&value=<value>
?method=setAllMaxWorkAgeToFlush&value=<value>
?method=listWorkerCache

- force cache to be trimmed if shrinking cache size.  If left untrimmed this can result in a lag when all the work expires.  Normally as work is request it will open up a slot for fresh work to be fetched.  However due to cache being oversized this doesn't happen.  It's possible for the entire to end up getting purged so the server has to catch up filling the cache from daemon while servicing requests.
- fix: WorkerProxy was case sensitive.  In cases where database case was different to user supplied case of worker name this would cause a cache miss and force a db query every time.  Thanks to <Eleuthria> for finding.
- complete rewrite of longpolling code
- added async LP dispatch.
- restructured repo to include dependencies as source projects instead binaries
- fix: clean up longpoll shutdown.  Missed a threadpool executor which was preventing JVM exit.
- fix: longpoll connection are now all explicity closed on shutdown.  Some miners were not registering the closed socket so don't attempt to reconnect to LP when the server restarts.
- fix: prevent work being served from a source until it's confirmed on the new block
- pause work fetching for source until it's confirm on the new block
- add prev_block_hash checks to each incoming work as a new block indicator.  This reduces the amount of polling needed when not in native longpoll mode.
- fix: set autoReconnect=true on JDBC connections.  worker caching can leave long intervals between uses of the connection causing sql server to time it out.
- add fixed time worker cache eviction strategy.  resolves issue #6
- implementation of longpoll connection counting and enforcement of optional limits
- trace logging with target groups for granular tracing.
- clean up of sample properties file add new config options
- fix: nullpointer if share output file not specified.
- fix: nullpointer if request output file not specified.
- addresses issue #7.  When setting worker IP first check X-Forwarded-For header then falls back to remoteAddr.  This covers situations where the server is behind a load balancing proxy.
- fix: use username.intern() to gain a per user canonical sync lock object. Prevents an obscure bug where two near simultaneous initial connections from one worker can result in multiple db lookups where one hasn't been put into the cache before the other is looked up.
- add keep-alive header to json-rpc client requests.

improve worksource syncing on block change
- prevent entries entering cache during out of sync period
- synchronize the change of sync status process
- remove redundant sourcesOnCurrentBlockArray
- change NotifyLongpollClients thread to a Runnable task to avoid having to start up a new thread.
- acceptNotifyBlockChange now had double check inside sync block to prevent double accepts.
- prestart LP executor threads
- fix mismatched sync objects for block change syncing
- workSource resync moved inside sync block

native longpolling
- add registration of native-enabled sources to native lp listener
- improved debug logging
- address handling using canonical host name
- enable verification request to report success or failure

[0.2.10] - unreleased

- fix: WorkSource request throttling was only activating for HTTP level failures.  TCP failures (e.g. connection refused) would not activate throttling resulting in thousands of requests/sec and high CPU usage.
- convert blocknum from string to int
- refactor common elements to bitcoin-poolserverj-core
- fix missing semi-colons in sample sql  scripts

 

[DavinciJ15]

Hi Shadders

My testing of the merged-mine-proxy application created for nmc and btc merged mining was interesting.  I was able to get poolserverj running it was not hard but it sure is a CPU pig with just 1 miner, I think that's over the top for the default settings.  Anyhow that's not what I wanted to talk about, as I was able to merge mine on the test net with just merged-mine-proxy app and find block then when I switched poolserverj I found none!   

Then I tried pushpoold and same thing... NO blocks after 10 times the shares need based on the difficulty.  Mind you I did the test over and over again and some cases I did find a block but not within the normal variance of the difficulty. Only directly mining the merged-mine-proxy gave me blocks within normal variances.

So I changed pushpoold's server.json file to disable rewrite of the difficulty since it seemed that merged=mine-proxy already did that.  BAM! Pushpool worked as it should and is now finding blocks within normal variance, and it posted the shares to the database.

Switching back to poolserverj and I tried disabling the rewrite but I am not sure if I got the right setting. I'm not on the computer with setting so I will edit this post with the setting I changed but it did not work.  Poolserverj did not find blocks. 

So is there setting you know off hand that will stop poolserverj from rewriting the difficulty and post the shares to the DB?

Thanks

Davinci
BTW I will download the new version and try that one as well.

[shads]

Hi Shadders

My testing of the merged-mine-proxy application created for nmc and btc merged mining was interesting.  I was able to get poolserverj running it was not hard but it sure is a CPU pig with just 1 miner, I think that's over the top for the default settings.  Anyhow that's not what I wanted to talk about, as I was able to merge mine on the test net with just merged-mine-proxy app and find block then when I switched poolserverj I found none!   

Then I tried pushpoold and same thing... NO blocks after 10 times the shares need based on the difficulty.  Mind you I did the test over and over again and some cases I did find a block but not within the normal variance of the difficulty. Only directly mining the merged-mine-proxy gave me blocks within normal variances.

So I changed pushpoold's server.json file to disable rewrite of the difficulty since it seemed that merged=mine-proxy already did that.  BAM! Pushpool worked as it should and is now finding blocks within normal variance, and it posted the shares to the database.

Switching back to poolserverj and I tried disabling the rewrite but I am not sure if I got the right setting. I'm not on the computer with setting so I will edit this post with the setting I changed but it did not work.  Poolserverj did not find blocks. 

So is there setting you know off hand that will stop poolserverj from rewriting the difficulty and post the shares to the DB?

Thanks

Davinci
BTW I will download the new version and try that one as well.

Hi Davinci,

To disable rewrite difficulty you will a find the following line in the sample properties file:
#source.local.1.rewriteDifficulty=false

just remove the comment marker ('#')

also ensure you've got:
useEasiestDifficulty=false

This is a testing/debugging setting specifically for load testing with the stress test client.

I was under the impression that pool servers needed some modifications to work with merged mining.  I could be wrong though.

Regarding poolserverj being a cpu pig, the default settings in sample config are probably more suited to larger pool.  So yes by default it will use a lot of CPU and memory... This is advantageous to a larger pool but a little wasteful for a smaller one. 

To rectify this please look at the articles in the documentation section: http://poolserverj.org/documentation/

Specifically: Performance & Memory tuning and Troubleshooting Tips.

cheers

[shads]

Apologies to anyone who's downloaded 0.3.0rc1 and tried to use it.  Due a series on unamusing screw ups the distribution contained an earlier version of poolserverj.jar.  A new distro is uploading right now and should be done in a couple of minutes.

[urstroyer]

Hi shads,

just downloaded the release. Is this the updated realase?

poolserverj.jar file size is 246kb

[shads]

Hi shads,

just downloaded the release. Is this the updated realase?

poolserverj.jar file size is 246kb

The one I'm looking at is 245.6kb so yes.  The wrong was was about 220kb I think.

[eleuthria]

BTC Guild's testing has found 15 blocks with PoolServerJ so far, all of them were detected as proper difficulty by PSJ's internal test prior to sending to bitcoind.  No false positives, and no false negatives.

[DavinciJ15]

still on 2.9 and It works WAY WAY better than pushpool,  I could kiss you shadders 

HOWEVER I have only got it to go with half my load before it falls down because it sucks up 100% of the amazon micro CPU this is not good as amazon only gives you burst of 100% usage then throttles the VM down.

Pushpool will take 100% of my load but I get these errors...

Code:

2011-09-19 14:05:09: Listener for "coinserver3 test": 19/09/2011 14:05:09, Problems communicating with bitcoin RPC 0 2
2011-09-19 14:07:01: Listener for "coinserver3 test": 19/09/2011 14:07:01, Problems communicating with bitcoin RPC 0 2

Still trying to fine tune it so that I get the best performance need upgrade and code in a control panel for my web app.  Man I knew when I read your web site that my 10 BTC donation was worth every BTC cent.

If I can get the poolserverj handling 25 GHashs on on micro server with (bitcoind and namecoind running) I will kick you some more BTC or one better higher you on as a part time consultant but we will see.

Davinci

[DavinciJ15]

I would like to clarify some settings, instead of assuming I will ask even if I think I know I am right just can you just confirm or correct my assertion and move on.


.maxConcurrentDownloadRequests
This setting is the number of miners that can make a get work request or your thread making get work request against the bitcoin daemon. I will assume you create threads here and they do the get work to fill a cache or a request.  20 is a safe maximum but if your system is large and on fiber optics You could do more.

.maxConcurrentUpstreamSubmits
This is same as above but its submit to the bitcoin daemon and not really cashed but uses a thread for each thread.

.maxWorkAgeToFlush
When a thread makes a get work request to store in a cache this is how long in seconds that request is valid should no worker grab the cached work it will be flushed and a new get work request is called.  The bitcoin network updates the work every 60 seconds or more if there is no new transactions (people sending coins) that will be placed in the work load so this value should not be too high or the pool will have a lot of stales? (<- I'm guessing here need confirmation ->)  To small and the bitcoin daemon recieves more requests than necessary reducing performance?


.minIntervalBetweenHttpRequests
A pool receives work requests from miners this settings is the number of milliseconds between ANY request can be made to poolserverj?  This means that poolserverj ignores or places in queue any request made under this time threshold?  If too many requests are made to the poolserverj they may timeout waiting for poolserverj to accept their request?  For example a value of 100 means only 10 requests can be made per second to poolserverj?
NOTE: If the above and bottom settings are what I think it is and you coded this right you are one BAD ASS DEVELOPER.

.minIntervalBetweenHttpRequestsWhenFrantic
When a new block is found on the network all previous work becomes invalid and workers need new work thus minIntervalBetweenHttpRequests limit should be lifted to allow a burst of requests.  This setting should be lower than minIntervalBetweenHttpRequests if not zero if your system can handle it.

 
.maxCacheSize
This number is meaningless by it's self you need to figure out how much load you server can handle then this value is a calculation of that number plus head room for Frantic requests.  The data in this cache is removed by workers or timed out by the value of maxWorkAgeToFlush setting however it's all meaningless without knowledge of you hardware and network limits.  SEE optimal settings example below

.cacheWaitTimeout
Your docs say...
### maximum time in milliseconds to wait for the cache to return work before giving up and returning an error result.
Huh?  The cache is a separate application or thread that poolserverj waits for when miners make a work request?  Thus this the length of time it waits to get that data?  I would like to know is this memcache or something else internal to java like .NET cached objects?


Optimal Settings Example
Lets say your system has bitcoind installed on it and can only handle 200 requests per second before falling down. [NOTE: this is an example, your results may vary, please call your government official who knows best about everything to regulate what you do.]   So to be safe you say 100 request per second is your limit.  The first thing you would set is minIntervalBetweenHttpRequests and its easy to figure out since 100 reqests pers second and there is 1000 milliseconds in a second so the value should be 10.   The next setting is (you guessed it) minIntervalBetweenHttpRequestsWhenFrantic and we know our max is 200 so you could put 5 but 6 would be safe.  Next settings are maxWorkAgeToFlush and maxCacheSize that should (maxCacheSize / (maxWorkAgeToFlush / 1000))  to equal 200, your systems maximum requests.   Keeping in mind the maxWorkAgeToFlush should not be to low or too high.


Shadders if any of my assertions are incorrect please correct them.

Looked at my poolserverj worker connection and it has collapsed for some reason. so maybe I'm wrong about the settings.

[shads]

still on 2.9 and It works WAY WAY better than pushpool,  I could kiss you shadders 

HOWEVER I have only got it to go with half my load before it falls down because it sucks up 100% of the amazon micro CPU this is not good as amazon only gives you burst of 100% usage then throttles the VM down.

Pushpool will take 100% of my load but I get these errors...

Code:

2011-09-19 14:05:09: Listener for "coinserver3 test": 19/09/2011 14:05:09, Problems communicating with bitcoin RPC 0 2
2011-09-19 14:07:01: Listener for "coinserver3 test": 19/09/2011 14:07:01, Problems communicating with bitcoin RPC 0 2

Still trying to fine tune it so that I get the best performance need upgrade and code in a control panel for my web app.  Man I knew when I read your web site that my 10 BTC donation was worth every BTC cent.

If I can get the poolserverj handling 25 GHashs on on micro server with (bitcoind and namecoind running) I will kick you some more BTC or one better higher you on as a part time consultant but we will see.

Davinci

The CPU burst profile of an EC2-micro instance really isn't suitable for PSJ.  PSJ will always have a baseline level of load due to keeping the cache full this is an advantage when load increases but an EC2-micro is designed for scenario's where most of the time the load is near zero and burst occasionally.  If you have a constant baseline load it won't allow the CPU to burst.  So I think from memory you end up with about 0.2 of an EC2 compute unit as your constant capacity.  In pushpool's case it basically doesn't do anything until it gets a request so it's probably using the full 2 EC2 compute unit burst capacity..  For low load pools pushpool will probably run better on a micro that psj.  You would be better off running it on a small instance to get a constant 1 EC2 compute unit then raising it a large as your capacity needs increase.   

Amazon have a very detailed article on this but I can't find it... This talks about the same sort of thing though : http://huanliu.wordpress.com/2010/09/10/amazon-ec2-micro-instances-deeper-dive/

[shads]

.maxConcurrentDownloadRequests
This setting is the number of miners that can make a get work request or your thread making get work request against the bitcoin daemon. I will assume you create threads here and they do the get work to fill a cache or a request.  20 is a safe maximum but if your system is large and on fiber optics You could do more.

This refers to psj -> bitcoind requests.  This side is completely asynchronous with miner -> psj requests. You should increase this value if your network latency between psj and bitcoind is higher than normal.  This ensures there's less idle gaps for each end while the packets are traversing the network.  

.maxConcurrentUpstreamSubmits
This is same as above but its submit to the bitcoin daemon and not really cashed but uses a thread for each thread.

yes basically the same as above but going to other direction.  Unless you've forceAllSubmitsUpstream=true this can be low as you'd only expect to be sending submits when you win a block.  Thanks to Eleuthria's testing I'm fairly confident now that it's safe to set forceAllSubmitsUpstream=false

.maxWorkAgeToFlush
When a thread makes a get work request to store in a cache this is how long in seconds that request is valid should no worker grab the cached work it will be flushed and a new get work request is called.  The bitcoin network updates the work every 60 seconds or more if there is no new transactions (people sending coins) that will be placed in the work load so this value should not be too high or the pool will have a lot of stales? (<- I'm guessing here need confirmation ->)  To small and the bitcoin daemon recieves more requests than necessary reducing performance?

Yes this is the cache 'expiry' time.  When a new block is detected the entire cache is dumped however so it's not really an issue with stales.  Just ensuring that work is relatively fresh so new transactions are included.  As long as the work is from the correct block you could in theory dish out work that is 10 mins old and it wouldn't break anything.

.minIntervalBetweenHttpRequests
A pool receives work requests from miners this settings is the number of milliseconds between ANY request can be made to poolserverj?  This means that poolserverj ignores or places in queue any request made under this time threshold?  If too many requests are made to the poolserverj they may timeout waiting for poolserverj to accept their request?  For example a value of 100 means only 10 requests can be made per second to poolserverj?
NOTE: If the above and bottom settings are what I think it is and you coded this right you are one BAD ASS DEVELOPER.

No this is relevent to the psj -> bitcoind side.  It should be set to 0 unless you have an unpatched bitcoind.  It just spaces out the requests a little so as not to overwhelm the bitcoind.  This does improve a little with an unpatched daemon but will only slow you down if you have the 4diff patch.

The client side throttling you're talking about is actually handled by the QoS filter.  And the number of requests it will service concurrently before the filtering kicks in is set by 'QoSMaxRequestsToServiceConcurrently=55'
If the server is under extreme load it will begin prioritizing requests.  Priority is determined from lowest to highest:

worker not found
worker found but password is bad
worker found and authenticated
worker found and has submitted at least 1 valid work
worker found and has submitted at least 10 valid works.

low priority requests will probably never get serviced until the server load drops to a more bearable level.

.minIntervalBetweenHttpRequestsWhenFrantic
When a new block is found on the network all previous work becomes invalid and workers need new work thus minIntervalBetweenHttpRequests limit should be lifted to allow a burst of requests.  This setting should be lower than minIntervalBetweenHttpRequests if not zero if your system can handle it.

Pretty much... This should be set to 0 in 99.9% of cases.

.maxCacheSize
This number is meaningless by it's self you need to figure out how much load you server can handle then this value is a calculation of that number plus head room for Frantic requests.  The data in this cache is removed by workers or timed out by the value of maxWorkAgeToFlush setting however it's all meaningless without knowledge of you hardware and network limits.  SEE optimal settings example below

Yes it's a key tuning parameter.  If it's too high you will have a lot of work wasted and burn CPU cycles uneccessarilly.  Too low and you will not have burst capacity.  The right number is essentially dependant on your getwork load and how much burst capacity you want to have.

.cacheWaitTimeout
Your docs say...
### maximum time in milliseconds to wait for the cache to return work before giving up and returning an error result.
Huh?  The cache is a separate application or thread that poolserverj waits for when miners make a work request?  Thus this the length of time it waits to get that data?  I would like to know is this memcache or something else internal to java like .NET cached objects?

It basically looks like this: bitcoind <-> work fetcher -> [cache: work queue] -> work server <-> miner.

However the the fetch and serve sides are async and barely interact.  The cache is basically just a queue of works with some metadata attached.

when a getwork request is received the server thread polls the queue.  If no work is available it sends a wake up call to the fetcher controller in case it's sleeping (very unlikely) then goes to sleep for cachewaitTimeout.  Whenever the fetcher puts new work in the queue it notifies the sleeping server threads. They wake up and try to poll again.  They may get beaten by another server thread so they go back to sleep again.  If the fetcher can't get a work from the queue for cacheWaitTimeout milliseconds it gives up and returns a JSON-RPC error message.

Optimal Settings Example
Lets say your system has bitcoind installed on it and can only handle 200 requests per second before falling down. [NOTE: this is an example, your results may vary, please call your government official who knows best about everything to regulate what you do.]  So to be safe you say 100 request per second is your limit.  The first thing you would set is minIntervalBetweenHttpRequests and its easy to figure out since 100 reqests pers second and there is 1000 milliseconds in a second so the value should be 10.   The next setting is (you guessed it) minIntervalBetweenHttpRequestsWhenFrantic and we know our max is 200 so you could put 5 but 6 would be safe.  Next settings are maxWorkAgeToFlush and maxCacheSize that should (maxCacheSize / (maxWorkAgeToFlush / 1000))  to equal 200, your systems maximum requests.   Keeping in mind the maxWorkAgeToFlush should not be to low or too high.

Firstly I would set minIntervalBetweenHttpRequests and minIntervalBetweenHttpRequestsWhenFrantic to 0.  If you bitcoind does not have 4diff patch then apply it.

(maxCacheSize / (maxWorkAgeToFlush / 1000))  <- this will ensure on average you waste very little work, you will still waste some due to variance in the rate of requests.  However, you have no headroom for burst capacity.  I would recommend you at least double this number.  In a high load environment increase it quite a bit more.  I hope I'm not giving away trade secrets here but I think BTC guild have their's set to about 10 times this number.

[shads]

I would like to know is this memcache or something else internal to java like .NET cached objects?

Poolserverj does not use memcached.  All of it's caches, queues, maps etc are internal which is part of the reason psj eats a lot more memory than the pushpoold process.  It's an awful lot faster this way because we can avoid LRU overhead, don't have to traverse the network stack and in many cases can avoid using map keys altogether. 

The maps are using the obscenely efficient trove library which boosted getwork performance by about 50% when it was implemented as well as some intelligent hashing/comparison strategies which would be impossible with memcached. 

e.g. string comparisons for duplicate checks don't just check char by char and abort when they find one that isn't equal.  They check the chars in a different order, starting with the ones that are most likely to differ first.  Hash codes only use about 5 chars of a getwork data string and from memory I think they pick those from the merkle root which should be unique to every work.  This provides more than enough uniqueness for the hashmap to work efficiently but saves about 80-90% of the work involved in a hashmap put or get.

[DavinciJ15]

still on 2.9 and It works WAY WAY better than pushpool,  I could kiss you shadders 

HOWEVER I have only got it to go with half my load before it falls down because it sucks up 100% of the amazon micro CPU this is not good as amazon only gives you burst of 100% usage then throttles the VM down.

Pushpool will take 100% of my load but I get these errors...

Code:

2011-09-19 14:05:09: Listener for "coinserver3 test": 19/09/2011 14:05:09, Problems communicating with bitcoin RPC 0 2
2011-09-19 14:07:01: Listener for "coinserver3 test": 19/09/2011 14:07:01, Problems communicating with bitcoin RPC 0 2

Still trying to fine tune it so that I get the best performance need upgrade and code in a control panel for my web app.  Man I knew when I read your web site that my 10 BTC donation was worth every BTC cent.

If I can get the poolserverj handling 25 GHashs on on micro server with (bitcoind and namecoind running) I will kick you some more BTC or one better higher you on as a part time consultant but we will see.

Davinci

The CPU burst profile of an EC2-micro instance really isn't suitable for PSJ.  PSJ will always have a baseline level of load due to keeping the cache full this is an advantage when load increases but an EC2-micro is designed for scenario's where most of the time the load is near zero and burst occasionally.  If you have a constant baseline load it won't allow the CPU to burst.  So I think from memory you end up with about 0.2 of an EC2 compute unit as your constant capacity.  In pushpool's case it basically doesn't do anything until it gets a request so it's probably using the full 2 EC2 compute unit burst capacity..  For low load pools pushpool will probably run better on a micro that psj.  You would be better off running it on a small instance to get a constant 1 EC2 compute unit then raising it a large as your capacity needs increase.   

Amazon have a very detailed article on this but I can't find it... This talks about the same sort of thing though : http://huanliu.wordpress.com/2010/09/10/amazon-ec2-micro-instances-deeper-dive/

Right now I have PSJ running handling 2GH nicely for the miner compared to pushpool my miners are cleanly getting LP new blocks and thus low stales and running a lot faster than before.
3 of the miners go a Timeout error from PSJ once but I have not seen it again after 2 other blocks where found.

Pushpool always gives me timeout errors for LP and the above error.

Not once have I gotten this from Pushpool...

Code:

2011-09-19 22:52:09: Listener for "coinserver5 test": 19/09/2011 22:52:09, long poll: new block 0000589b63ba445d
2011-09-19 22:52:15: Listener for "coinserver5 test": 19/09/2011 22:52:15, Using new LP URL /LP/
2011-09-19 22:52:15: Listener for "coinserver5 test": 19/09/2011 22:52:15, LP connected to test1.nmcbit.com:8332

It's always been a timeout Exception.

So anyways it's looking good with 2GH and my currently incorrect settings.  I will adjust see what happens.


BTW you can't go from small to large as a small is a 32bit computer... unless linux has some magic code I did not know about that allows it to reboot into 64bit.

[shads]

BTW you can't go from small to large as a small is a 32bit computer... unless linux has some magic code I did not know about that allows it to reboot into 64bit.

Sorry the one I was thinking of was High-CPU Medium Instance which is 32bit with 5 EC2 compute units.  2x price/hour but 5x CPU capacity.

[DavinciJ15]

So I was running some test put all 6GH I have on PSJ using poclmb 12 hrs NO problem PSJ on a mirco server does not even flinch as it posts 30+ shares ever 15 seconds to the database.

AWESOME!

I take 1 GPU using CGIMiner default settings... NO problem.  Change scans to 1 and PSJ collapses in a ball of flames.

THAT WAS JUST 1 miner!

I'm going to re-test that and post again if I have more info.
EDIT:
Its confirmed taking the scantime setting down to low number hits the server very hard I'm going to make some adjustments on the settings see if I can prevent this hit.  Also going to try CGIMiner by it's self with scan time high.

[shads]

you are not to be able to control other people's miner settings so you will have to deal with miners who have high scan rates.  I repeat and micro instance is not suitable for running a pool of any scale. 

In recent tests on a large production pool using a dedicated server it took over 1500GH/s before psj had it's first hiccup.  At that point it was servicing over 4000 getworks/sec.  The problem you're having is the CPU throttling on the micro instance.  Once the throttling kicks in you're CPU capacity is squeezed from 2 ECU down 0.35ECU so if your baseline load is 0.5ECU your server is effectively squashed.

[DavinciJ15]

you are not to be able to control other people's miner settings so you will have to deal with miners who have high scan rates.  I repeat and micro instance is not suitable for running a pool of any scale. 

In recent tests on a large production pool using a dedicated server it took over 1500GH/s before psj had it's first hiccup.  At that point it was servicing over 4000 getworks/sec.  The problem you're having is the CPU throttling on the micro instance.  Once the throttling kicks in you're CPU capacity is squeezed from 2 ECU down 0.35ECU so if your baseline load is 0.5ECU your server is effectively squashed.

I know that I just want to learn about clustering and it's issues before having large number of users.  I want to see how the server reacts and how to fail over to a server that can handle all load.  Basiclly I am educating myself on all this stuff and learning linux at the same time.

[shads]

I know that I just want to learn about clustering and it's issues before having large number of users.  I want to see how the server reacts and how to fail over to a server that can handle all load.  Basiclly I am educating myself on all this stuff and learning linux at the same time.

A micro instance is probably a very good tool for learning with.  I'm just pointing out that psj on a micro is pushing a square peg into a round hole... If it's just for experimentation that fine, but you will likely see some behaviours on a micro that you wouldn't on standard ec2 instance.  Having said that I'm running the psj website on a micro... I've run the odd bitcoind on it and even a test psj instance but I didn't expect too much of it when it...

[DavinciJ15]

I know that I just want to learn about clustering and it's issues before having large number of users.  I want to see how the server reacts and how to fail over to a server that can handle all load.  Basiclly I am educating myself on all this stuff and learning linux at the same time.

A micro instance is probably a very good tool for learning with.  I'm just pointing out that psj on a micro is pushing a square peg into a round hole... If it's just for experimentation that fine, but you will likely see some behaviours on a micro that you wouldn't on standard ec2 instance.  Having said that I'm running the psj website on a micro... I've run the odd bitcoind on it and even a test psj instance but I didn't expect too much of it when it...

So I am running PSJ on a micro and I have thrown virtually all of my hashing power on it and I am getting better speeds from my miners than pushpool.  I have to say PSJ is kick ass solid if someone wanted to create a small pool using micro ec2.

[DavinciJ15]

When a new block is found by my pool stales go up big time bu not on PSJ!  Average guy has 3 to 6% stales I have 0.63 and only 3 miners have stales the rest are at ZERO after a block found!  Amazing!

I'm going to upgrade my server then put the full load of users.  Then I will fool around with clustering PSJ on micro servers see if I can get that working.

EDIT: Spoke to soon, PSJ server was throttled.

[DavinciJ15]

So I upgraded to a large EC2 Amazon server and tried 100% of the load and PSJ could not take it.  

The users of CGIMiner can not use nmcbit.com as I use Round Robin and that miner does not stick to a server until it fails like poclmb does.
So I have all my CGIMiners accessing one pushpool server.  When I change the domain's IP address to point to PSJ server they take the PSJ DOWN even on a large EC2 Amazon server!

Here is a look at the linux server values...

Code:

top - 18:13:22 up  1:52,  2 users,  load average: 0.72, 0.65, 0.43
Tasks: 116 total,   1 running, 115 sleeping,   0 stopped,   0 zombie
Cpu(s): 34.4%us, 14.1%sy,  0.0%ni, 38.3%id,  0.0%wa,  0.0%hi,  2.4%si, 10.9%st
Mem:   7645956k total,  1158644k used,  6487312k free,    38172k buffers
Swap:        0k total,        0k used,        0k free,   648712k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
23621 ubuntu    20   0 2292m 140m  11m S   83  1.9  16:03.16 java
  611 root      20   0  236m  15m 8860 S   60  0.2  24:05.11 namecoind
  616 ubuntu    20   0  225m  63m 8924 S    9  0.9   3:18.40 bitcoind

If i get rid of the CGI miner guys PSJ acts like no one is touching the server mean while PSJ is inserting 15 to 20 shares ever 15 seconds.

Code:

top - 18:27:19 up  2:06,  2 users,  load average: 0.12, 0.18, 0.29
Tasks: 116 total,   1 running, 115 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.5%us,  0.1%sy,  0.0%ni, 99.2%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   7645956k total,  1141728k used,  6504228k free,    39484k buffers
Swap:        0k total,        0k used,        0k free,   649556k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
23621 ubuntu    20   0 2292m 121m  11m S    1  1.6  21:24.45 java
  595 mysql     20   0  168m  26m 6916 S    0  0.3   0:09.45 mysqld
  611 root      20   0  236m  15m 8860 S    0  0.2  27:34.66 namecoind
24076 ubuntu    20   0 19352 1292  944 R    0  0.0   0:08.55 top
    1 root      20   0 24144 2208 1324 S    0  0.0   0:00.29 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd

There has to be a setting to kick those CGI guys to the curb.

If I add just ONE I repeat JUST 1 CGI miner with scantime of 1 I get this...

Code:

top - 18:48:56 up  2:27,  2 users,  load average: 0.98, 0.46, 0.33
Tasks: 116 total,   2 running, 114 sleeping,   0 stopped,   0 zombie
Cpu(s): 36.4%us, 19.0%sy,  0.0%ni, 32.5%id,  0.1%wa,  0.0%hi,  2.5%si,  9.5%st
Mem:   7645956k total,  1206340k used,  6439616k free,    41564k buffers
Swap:        0k total,        0k used,        0k free,   651896k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7895 ubuntu    20   0 2236m 178m  11m S   91  2.4   3:05.97 java
  611 root      20   0  236m  16m 8860 S   62  0.2  33:25.10 namecoind
  616 ubuntu    20   0  225m  63m 8924 S    4  0.9   4:20.02 bitcoind

If there is a LP it's over the system is down. (LP occurred shortly after I copied this.) Remember this is on a large server!
CGIMiner with scan settings set to 1 is like a DOS attack.

The reason no other pool is seeing this is because I told my uses to set CGIMiner scan rate to a low number, this reduces efficiency but also reduces the number of rejects people where seeing.  So please try not dismiss my issue as this would take down pools with just a few miners.

[shads]

There's definately something wrong here, one miner with scan=1 should request 1 getwork/second.  With 5 ECU I'd be surprised if psj had a hiccup at any less 500-1000/sec, *provided* the daemon is keeping up.

Does your daemon have the 4diff patch?

I'd like to see some running stats from the server.  Can you try running this bash script then zip up the log directory and send it to me.

Code:

#!/bin/bash

mkdir -p log
rm -f log/wget.log

INTERVAL=2s

while true; do
	echo poke...	
	wget -a log/wget.log -O log/getsourcestats-$(date +%Y%m%d-%H%M%S).log --connect-timeout=10 --read-timeout=10 http://localhost:8997/?method=getsourcestats
	sleep $INTERVAL
done

just save it to a file 'logger.sh' then
chmod +x logger.sh
./logger.sh

start it just after poolserverj comes online and keep it running until you have a longpoll.

Edit: Also it would be helpful if you could send me a copy of your properties file as well (remove your passwords).  PM me on IRC for an email address.

[DavinciJ15]

So I thought if I add a PSJ node to the round robin only the users that are using poclbm will roll over to the PSJ nodes as they stick to one server until there is a problem (and there are lots of problems with pushpoold) and the CGIMiners don't use the round robin domain name nmcbit.com.   As a result PSJ node is serving up requests over 10 per second with no problems on a micro server.

going back to the other servers the CGIMiners are HAMMERING it with 20+ requests per second. I don't know what they are doing but PSJ does not like it.

Code:

[2011-09-22 00:50:17.16420] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.71675] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.125674] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.181142] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.232956] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.288010] ::ffff:84.207.224.3 "/LP"
[2011-09-22 00:50:17.322250] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.375163] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.430217] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.484228] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.539632] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.592514] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.648209] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.701122] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.757177] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.809239] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.865398] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.917665] ::ffff:84.207.224.3 "/"
[2011-09-22 00:50:17.959539] ::ffff:84.207.224.3 "/"

I took out the user name but as you can see from the pushpool, the same user is hitting pushpool like crazy!
I will set up another node and point those guys to it and log what you are requesting.

Does your daemon have the 4diff patch?

I assumed the latest version of namecoind has those patches but I'm not sure.


Over all I'm impressed with how smooth PSJ is running.

[shads]

Does your daemon have the 4diff patch?

I assumed the latest version of namecoind has those patches but I'm not sure.

I don't know for sure but I'd be surprised if it's included in the stock namecoind.  Perhaps best to check with the devs.  I haven't got far enough to with native merged mining to need a patched namecoind yet so I haven't checked...

[DavinciJ15]

Does your daemon have the 4diff patch?

I assumed the latest version of namecoind has those patches but I'm not sure.

I don't know for sure but I'd be surprised if it's included in the stock namecoind.  Perhaps best to check with the devs.  I haven't got far enough to with native merged mining to need a patched namecoind yet so I haven't checked...

a quick grep search shows the patch is not applied I will I apply it after I modify 4diff.txt for namecoind source code changes unless you have a copy that google is hiding.

[shads]

a quick grep search shows the patch is not applied I will I apply it after I modify 4diff.txt for namecoind source code changes unless you have a copy that google is hiding.

Well the good news is after reviewing your logs I'm pretty sure 4diff will solve your problem.

If you look at the consolidated log I sent you you can see towards the end of the period the namecoin daemon suddenly starts sending masses of duplicate works and at one point reaches a duplicate rate of nearly 100%.  This is a known bug that the 4diff patch fixes.  The problem here is that poolserverj checks for duplicates incoming from the daemon and discards them if it finds one.  If this bug is happening you're likely to be only gettting one unique work/second from the daemon.  Because it can't fill it's cache psj keeps issuing getwork requests to the daemon and keeps discarding them when they come back duplicates.

If poolserverj wasn't behaving like this you would be issuing the same work to all your miners, you and they would think everything is fine but eventually you'd notice that your pool is not finding anywhere near as many blocks as it should be.  If you've got 10 miners all working on the same work you're effectively only getting 1 miners worth of work done.  IMHO it's better for the server to crash in this case then at least you'll know something is wrong.

Pushpool does not check incoming work for duplicates so this is why you saw it wasn't working nearly as hard.  It was feeding the duplicates to your miners then going back to sleep whereas psj was caning your daemon trying to get valid work and eventually everything went down in flames...

You will see massive improvements with a 4diff patched daemon.

[flower1024]

is it feasible to include the mining proxy (for merged mining) directly in PoolServJ?

if namecoin daemon dies bitcoin deamon could still deliever getworks (or vice versa)

[shads]

is it feasible to include the mining proxy (for merged mining) directly in PoolServJ?

if namecoin daemon dies bitcoin deamon could still deliever getworks (or vice versa)

I am working on that right now... You can thank Davinci for tempting me with a fat bounty or I probably wouldn't have.  I'm waiting on some detail from one of the namecoin devs before I can start implementing...

[flower1024]

is it feasible to include the mining proxy (for merged mining) directly in PoolServJ?

if namecoin daemon dies bitcoin deamon could still deliever getworks (or vice versa)

I am working on that right now... You can thank Davinci for tempting me with a fat bounty or I probably wouldn't have.  I'm waiting on some detail from one of the namecoin devs before I can start implementing...

+1 to davince and you ,)

[DavinciJ15]

a quick grep search shows the patch is not applied I will I apply it after I modify 4diff.txt for namecoind source code changes unless you have a copy that google is hiding.

Well the good news is after reviewing your logs I'm pretty sure 4diff will solve your problem.

If you look at the consolidated log I sent you you can see towards the end of the period the namecoin daemon suddenly starts sending masses of duplicate works and at one point reaches a duplicate rate of nearly 100%.  This is a known bug that the 4diff patch fixes.  The problem here is that poolserverj checks for duplicates incoming from the daemon and discards them if it finds one.  If this bug is happening you're likely to be only gettting one unique work/second from the daemon.  Because it can't fill it's cache psj keeps issuing getwork requests to the daemon and keeps discarding them when they come back duplicates.

If poolserverj wasn't behaving like this you would be issuing the same work to all your miners, you and they would think everything is fine but eventually you'd notice that your pool is not finding anywhere near as many blocks as it should be.  If you've got 10 miners all working on the same work you're effectively only getting 1 miners worth of work done.  IMHO it's better for the server to crash in this case then at least you'll know something is wrong.

Pushpool does not check incoming work for duplicates so this is why you saw it wasn't working nearly as hard.  It was feeding the duplicates to your miners then going back to sleep whereas psj was caning your daemon trying to get valid work and eventually everything went down in flames...

You will see massive improvements with a 4diff patched daemon.

Thanks I will apply the patch and document provide the public with edited version of 4diff as it does not patch the namecoind code correctly.

[shads]

if yr unable to get it working you could try this one from ArtForz:
https://github.com/ArtForz/namecoin/commit/127deb4aff13965741130dba7304073330a4adea

I think it's only for the duplicate work issue but at a quick glance it looks like it's applicable to the merged mining version.

[DavinciJ15]

if yr unable to get it working you could try this one from ArtForz:
https://github.com/ArtForz/namecoin/commit/127deb4aff13965741130dba7304073330a4adea

I think it's only for the duplicate work issue but at a quick glance it looks like it's applicable to the merged mining version.

It does not compile namecoind

Code:

root@bitcoinpool:/home/bitcoinpool/ArtForz-namecoin-127deb4/src# make -f makefile.unix namecoind
g++ -c -O2 -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -DNOPCH -DFOURWAYSSE2 -DUSE_SSL -o obj/nogui/namecoin.o namecoin.cpp
namecoin.cpp: In function âjson_spirit::Value name_history(const json_spirit::Array&, bool)â:
namecoin.cpp:616:30: error: âforeachâ was not declared in this scope
namecoin.cpp:617:9: error: expected â;â before â{â token
namecoin.cpp:1789:84: error: expected â}â at end of input
namecoin.cpp:1789:84: error: expected â}â at end of input
make: *** [obj/nogui/namecoin.o] Error 1

[DavinciJ15]

if yr unable to get it working you could try this one from ArtForz:
https://github.com/ArtForz/namecoin/commit/127deb4aff13965741130dba7304073330a4adea

I think it's only for the duplicate work issue but at a quick glance it looks like it's applicable to the merged mining version.

It does not compile namecoind

Code:

root@bitcoinpool:/home/bitcoinpool/ArtForz-namecoin-127deb4/src# make -f makefile.unix namecoind
g++ -c -O2 -Wno-invalid-offsetof -Wformat -g -D__WXDEBUG__ -DNOPCH -DFOURWAYSSE2 -DUSE_SSL -o obj/nogui/namecoin.o namecoin.cpp
namecoin.cpp: In function âjson_spirit::Value name_history(const json_spirit::Array&, bool)â:
namecoin.cpp:616:30: error: âforeachâ was not declared in this scope
namecoin.cpp:617:9: error: expected â;â before â{â token
namecoin.cpp:1789:84: error: expected â}â at end of input
namecoin.cpp:1789:84: error: expected â}â at end of input
make: *** [obj/nogui/namecoin.o] Error 1

Commenting out the code in name_history function (its not needed for mining) does the trick.

[DavinciJ15]

So PoolSeverJ is working perfectly with my patched namecoind.  The CGIMiner is unable to take down the pool with scantime set to 1. 

Thanks you have been a big help in fact you have helped me the most.

[shads]

Changelog:

[0.3.0.FINAL]
- partial implementation of worker cache preloading.  This is not active yet.
- fix: stop checking if continuation state is initial.  It can be if a previous Jetty filter has suspended/resumed the request.  In that case it immediately sends and empty LP response.  This might be the cause of a bug where cgminer immediately sends another LP.  This turns into a spam loop.  This only seems to be triggered under heavy load and only seems to happen with cgminer clients connected.
- added commented out condition to stop manual block checks if native LP enabled and verification off.
- remove warning for native LP when a manual block check is fired.  We want this occur in most circumstances.
- extra trace targets for longpoll empty and expired responses.
- fix: handle clients sending longpoll without trailing slash.  This can result in the LP request being routed through the main handler and returning immediately setting up request spamming loop.  This patch checks for the LP url from the main handler and redirects to the LP handler if it's found.
- add threadDump method to mgmt interface
- add timeout to notify-lp-clients-executor thread in case dispatch threads do not report back correctly and counters aren't updated.  Solved a problem where counter mismatch can prevent the thread from ever finishing thus hogging the executor and preventing future long poll cycles.
- add shutdown check to lp dispatch timeout

[DavinciJ15]

Changelog:

[0.3.0.FINAL]
- partial implementation of worker cache preloading.  This is not active yet.
- fix: stop checking if continuation state is initial.  It can be if a previous Jetty filter has suspended/resumed the request.  In that case it immediately sends and empty LP response.  This might be the cause of a bug where cgminer immediately sends another LP.  This turns into a spam loop.  This only seems to be triggered under heavy load and only seems to happen with cgminer clients connected.
- added commented out condition to stop manual block checks if native LP enabled and verification off.
- remove warning for native LP when a manual block check is fired.  We want this occur in most circumstances.
- extra trace targets for longpoll empty and expired responses.
- fix: handle clients sending longpoll without trailing slash.  This can result in the LP request being routed through the main handler and returning immediately setting up request spamming loop.  This patch checks for the LP url from the main handler and redirects to the LP handler if it's found.
- add threadDump method to mgmt interface
- add timeout to notify-lp-clients-executor thread in case dispatch threads do not report back correctly and counters aren't updated.  Solved a problem where counter mismatch can prevent the thread from ever finishing thus hogging the executor and preventing future long poll cycles.
- add shutdown check to lp dispatch timeout

What is "worker cache preloading"?

[shads]

What is "worker cache preloading"?

Well as it says it's not activated... you'd have to make some code mods and rebuild from source to use atm...

But in a nutshell... when a busy pool comes up it's worker cache is empty. It suddenly get's hit by a ton of requests which translates into a ton of single selects to the db.  Preloading dumps the worker id's from the cache to a file on shutdown.  Then on start up it grabs the worker id's and does a single bulk select to fill the worker cache.  Much more efficient but probably not an issue until you get to the terahash range.

[DavinciJ15]

What is "worker cache preloading"?

Well as it say it's not activated... you'd have to make some code mods and rebuild from source to use atm...

But in a nutshell... when a busy pool comes up it's worker cache is empty. It suddenly get's hit by a ton of requests which translates into a ton of single selects to the db.  Preloading dumps the worker id's from the cache to a file on shutdown.  Then on start up it grabs the worker id's and does a single bulk select to fill the worker cache.  Much more efficient but probably not an issue until you get to the terahash range.

This is where memcache can help.  A pool would have a memcache server that has all the usernames and passwords cashed when it reboots one of the servers the cache will have all the users assuming the users where connecting to another server before hand.

This allows for clustering so all servers have the same cached username and password.

What's your opinion of this design?

[eleuthria]

After hammering out the last few bugs we found at BTC Guild with PoolServerJ, I'm almost ready to completely remove pushpool from my servers.

While the CPU load of PoolServerJ is higher than pushpool, I would not call it inefficient since PoolServerJ is doing far more work than pushpool.  PoolServerJ is doing full difficulty checks internally, prioritizing getwork responses to known good clients [QoS filtering], organizing work from multiple bitcoind nodes [faster LP delivery], running a cache of work so miner requests are responded to from the server rather than the server proxying the request to bitcoind [faster getwork delivery].  In the end, as long as the servers have enough extra RAM, the performance is outstanding.

PoolServerJ's work caching means if bitcoind stutters for a second, you have work ready and available to send to your miners, whereas pushpool becomes useless until bitcoind can respond, and cause miners to complain.

The tradeoff is simple:  pushpool will run on minimal specs, but becomes slow to respond after a certain level of load, even though CPU & RAM are sitting idle.  PoolServerJ will use significantly more RAM, but as long as its there, it won't choke until either your bitcoind can't provide the work as fast as its being requested, or your CPU is at full utilization.  A small pool would probably stick with pushpool, due to its low footprint and reasonable performance.  Any pool that is starting to have growing pains (they start around 250 GH/sec but they aren't "a problem" til ~450 GH/sec) would benefit greatly from taking a look at PoolServerJ.

[eleuthria]

An update:  BTC Guild is running PoolServerJ for the entire pool.  We were able to push out 10 pushpool/10 bitcoind nodes with load balancing and replace them with a single PoolServerJ and 2 bitcoind nodes. 

[shads]

An update:  BTC Guild is running PoolServerJ for the entire pool.  We were able to push out 10 pushpool/10 bitcoind nodes with load balancing and replace them with a single PoolServerJ and 2 bitcoind nodes. 

you forgot to mention the whopping 16% cpu load... but I'm glad yo forgot to mention the memory usage

[DavinciJ15]

Quick question?

Is there a way to include the USER_ID found in the worker table when inserting into the shares table?

[BurningToad]

An update:  BTC Guild is running PoolServerJ for the entire pool.  We were able to push out 10 pushpool/10 bitcoind nodes with load balancing and replace them with a single PoolServerJ and 2 bitcoind nodes. 

you forgot to mention the whopping 16% cpu load... but I'm glad yo forgot to mention the memory usage

Hmm, could you post some configuration perhaps?  I've been getting a very high CPU load with PoolServerJ.  Really need a way to reduce it.

[urstroyer]

Adjusting maxWorkAgeToFlush and maxCacheSize helps a lot.

Try to raise maxWorkAgeToFlush to 300 and lower maxCacheSize to 1000 for example and take a look at your cpu usage again.

Make sure you got bitcoind patched with 4diff.

A more detailed documentation:
http://poolserverj.org/documentation/performance-memory-tuning/

[jamesg]

Hi Shadders, could you please comment on this post regard x-roll-ntime -> https://bitcointalk.org/index.php?topic=18567.msg560731#msg560731

[DavinciJ15]

shadders I will double my reward if you incorporate merge mining in the next 24 hours.  I will triple it if you get it to me in the next 12 hours.

[shads]

18hrs... I gotta sleep

[DavinciJ15]

18hrs... I gotta sleep

18 hrs then triple BTC, 24 hrs double.

After that my offer expires and I will stick to the original agreement.

[shads]

18hrs... I gotta sleep

18 hrs then triple BTC, 24 hrs double.

After that my offer expires and I will stick to the original agreement.

k well I better stop playing with merkle trees and get some sleep then

[DavinciJ15]

I'm giving Shadders a large bounty for adding Merged-Mine-Proxy any one want to donate to help me out the address is here..

17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX

It's a wallet I will not spend from for a long time so we can see the results here...
http://blockexplorer.com/address/17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX

[DavinciJ15]

4hrs left for 300 BTC 

[shads]

4hrs left for 300 BTC 

been going at this damn thing for 6 hrs straight now.  I remember why I've been putting it off for so long.

BTW I told Davinci that I wouldn't release this publicly for at least a week after he got it which I think is fair given the hefty bounty he's posted.  However he generously declined the offer and requested I make it available to everyone immediately so if you do use it please consider making a donation to Davinci because I certainly wouldn't have gotten off my lazy arse to finish if he hadn't dangled the carrot in front of me.

[flower1024]

I'm giving Shadders a large bounty for adding Merged-Mine-Proxy any one want to donate to help me out the address is here..

17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX

It's a wallet I will not spend from for a long time so we can see the results here...
http://blockexplorer.com/address/17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX

sent http://blockexplorer.com/tx/9717e933e0e2cfde85cf4af930f89afeaeebc96a231673ed13c2e18e2c1a36b7
btw. can't connect to balanceserver1 and 3...

[shads]

4hrs left for 300 BTC 

Committed...

Well we may have to wait a while to decide if that counts as working.  Of course I can actually test if winning blocks are accepted without winning one on my 50MH rig.

[DavinciJ15]

Awesome dude send me your address and I will send the first 100 right away.
Nodemaster has his own test net with the difficulty at crazy low value for namecoins
add this to yoru bitcoin.conf with noirc=1

addnode=78.47.40.55:18334

However I am not sure if he as taken it down.  Create a VM and check I'm busy trying testing your code.

[shads]

just the address in my sig is fine...

Just uploading rev .02 atm.  Found a deadlock in the work submit thread.  There's some awfully convoluted syncing that has to go on there.

about 6 mins to go on my lightning internet connection.

also, recommend that you set 'forceAllSubmitsUpstream' until it's proven that it detects winning shares.  There will be a log file created in yr launch directory called 'real-share-submits.log' that tracks the progress of a suspected winning share through the server and gives the info we need to prove it's going to work without the force option switched on.

[DavinciJ15]

just the address in my sig is fine...

Just uploading rev .02 atm.  Found a deadlock in the work submit thread.  There's some awfully convoluted syncing that has to go on there.

about 6 mins to go on my lightning internet connection.

100 BTC sent.
Any tips on compiling the code? Or is there a binary JAR I can download?  I have 600 MH for testing so I can see some results faster.

[urstroyer]

So what happens to the shares table with merged mining option enabled?

Can you confirm, that upstream result from each row refers to the parentchain (btc) only? So the upstream results from the merged chain is currently not in the database right?

[DavinciJ15]

So what happens to the shares table with merged mining option enabled?

Can you confirm, that upstream result from each row refers to the parentchain (btc) only? So the upstream results from the merged chain is currently not in the database right?

The upstream result should be true for both chains for now and you just check your wallet to see witch block chain got  it.  In the future shadders can add column in the database or a new table (preferable) that indicates witch block chain found a block.

[urstroyer]

So what happens to the shares table with merged mining option enabled?

Can you confirm, that upstream result from each row refers to the parentchain (btc) only? So the upstream results from the merged chain is currently not in the database right?

The upstream result should be true for both chains for now and you just check your wallet to see witch block chain got  it.  In the future shadders can add column in the database or a new table (preferable) that indicates witch block chain found a block.

Thanks DavinciJ15, this solution works fine for me.

May i ask if you found a 4diff patch which can be compiled with the current bitcoind version vom vinced?

Best regards,

Chris

[DavinciJ15]

Thanks DavinciJ15, this solution works fine for me.

May i ask if you found a 4diff patch which can be compiled with the current bitcoind version vom vinced?

Best regards,

Chris

Hi Chris,

I'm not a bastard like most pool operators and coin developers I will tell what I know.  I am using this fork...
https://github.com/ArtForz/namecoin/commit/127deb4aff13965741130dba7304073330a4adea

When it compiles it will give you an error you need to comment out the code as it's not needed for mining.

Good luck

Davinci

[saethan]

Quick question on poolserverj - is there by chance a config setting that will let me change the difficulty for 'accepted work'?  Haven't had a chance to take a closer look at the source yet, figured somebody else might know.

The reason I'm asking is I'm considering porting it to make a pool for litecoins, but of course would ask permission if I had to modify any of the source (I did manage to take a brief look at license.txt while perusing a bit of the poolserverj code ).

I'd prefer this over pushpool because I like Java.  

As far as I can tell anything other than share difficulty would have an easy workaround to get litecoind and poolserverj to work together (in terms of, if I run into an issue, I can make another program of my own to translate between the two, or modify litecoind).

[shads]

Quick question on poolserverj - is there by chance a config setting that will let me change the difficulty for 'accepted work'?  Haven't had a chance to take a closer look at the source yet, figured somebody else might know.

The reason I'm asking is I'm considering porting it to make a pool for litecoins, but of course would ask permission if I had to modify any of the source (I did manage to take a brief look at license.txt while perusing a bit of the poolserverj code ).

I'd prefer this over pushpool because I like Java.  

As far as I can tell anything other than share difficulty would have an easy workaround to get litecoind and poolserverj to work together (in terms of, if I run into an issue, I can make another program of my own to translate between the two, or modify litecoind).

No need to ask permission to modify psj.  It's GPL.  Earlier versions were no-derivs creative commons license but that's long gone.  However, I don't think you'll need to. Take a look at the very end of the sample properties file.  There is actually an option to set difficulty though no one's ever used it in production as far as I know.  I would recommend after setting you do a manual getwork and inspect the target to ensure the endianess is correct.

I don't know anything about litecoin but psj has been successfully used on several alt chains.  The only real requirement is that the alt chain uses the same hashing algorithm as bitcoin, if so poolserverj won't know the difference and will happily do it's thing.

One this to watch with alt chains though is whether or not they've got the duplicate work bug.  Have search back in this thread for an explanation of it...

[Keninishna]

4hrs left for 300 BTC 

been going at this damn thing for 6 hrs straight now.  I remember why I've been putting it off for so long.

BTW I told Davinci that I wouldn't release this publicly for at least a week after he got it which I think is fair given the hefty bounty he's posted.  However he generously declined the offer and requested I make it available to everyone immediately so if you do use it please consider making a donation to Davinci because I certainly wouldn't have gotten off my lazy arse to finish if he hadn't dangled the carrot in front of me.

This is just badass and props to both of you. I have merged mining going on my pool now. My only request now is it separate the winning shares between chains.

[DavinciJ15]

Shadders,

Our original deal was 100 BTC for the Merged-Mine-Proxy (MMP) to be incorporated into PoolServerJ (PSJ).  Then I requested you do it in 18hrs for 300 BTC or 24hrs for 200 BTC.  You worked very hard and released a version that did not work.

I paid you 100BTC before trying it assuming you did it, but your valiant efforts did not pan out as I hoped it would.  I agreed to give you access to a server on Amazon and 1.5 GHs of my miners, I have them currently set the domain name you gave me as the primary pool and they have not mined anything from your test domain.  You messaged me once asking where the miners where and I responded telling you they are still waiting for your pool to come alive and I have not heard from you since.

With that I am not sure where you stand with regards to incorporating MMP into PSJ, but here is where I stand... First off I agreed to pay you 100BTC to complete it and you did not.  You stated in the IRC that you would role it back and drop MMP code should I not continue helping you so I offered my assistance and I considered give you 1.5% of the revenue of nmcbit.com once the code was completed.

Well, this is my final offer for you, complete a releasable version with MMP incorporated by the next NMC or BTC difficulty change and I will pay you 200BTC that's on top of the 100 I gave you OR you can take 1.5% of the revenue generated from nmcbit.com until it's sold to someone else.  I have worked very hard to get this pool running and I have gotten up to 167GHs of miners partly thanks to PSJ, but just remember that no other pool has offered you anything for your efforts, I at least gave you 10BTC as a donation [before I offered and paid 100BTC for MMP] witch you told me it increased your BTC account by thousands of percents.

Finally, after the deadline has been reached I will find someone else to help me or figure it out on my own.  With that you should you not be able to complete it in time, it is my opinion that you should complete the incorporation of MMP into PSJ at your leisure for the community to use as I did pay you 100BTC to do so, however that's up to you and your conscience.

Best Regards

Davinci

[urstroyer]

I'am going to cover 25 BTC of davincis bounty for a working solution. It seems a little bit tiny but there are plenty pools out there running poolserverj who could be interested as well. Maybe someone else is going to add some btc, i think we should support shadders and davinci here.

After talking to shadders on irc, i understand that not only poolserverj has to do the job well. I would like to see a working 4diff patch for vinced's bitcoind/namecoind in order to eliminate the duplicate work bug and multithread support.

Maybe we should talk that through on irc?

[DavinciJ15]

I'am going to cover 25 BTC of davincis bounty for a working solution. It seems a little bit tiny but there are plenty pools out there running poolserverj who could be interested as well. Maybe someone else is going to add some btc, i think we should support shadders and davinci here.

After talking to shadders on irc, i understand that not only poolserverj has to do the job well. I would like to see a working 4diff patch for vinced's bitcoind/namecoind in order to eliminate the duplicate work bug and multithread support.

Maybe we should talk that through on irc?

Thanks if you are supporting the PSJ give the bitcoin to shadders if you are supporting MMP in PSJ you can donate here...
17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX
http://blockexplorer.com/address/17nk7MqLLNy9Kw3NGqpa4G6qsPtWkUTuUX
I appreciate the support.

[shads]

ok chill out for a minute Davinci...

Firstly you knew all along I didn't not have them means to fully test it.  A full test means mining a real block and it's well known by many including you that I only have 50MH at my disposal so there was never any chance that I could a release a fully tested version within 24 hrs.  It was at best a guess and hope that the I'd gotten every minor detail of the protocol right.  What I did release was a couple of thousand lines code with one wrong line (see below). That was 1/2 written over the previous few weeks and 1/2 written in a 12 hour coding marathon...

Secondly lets not forget that after this flurry of activity and urgency it took you several days before you even got around to testing it.  Unfortunately you seem to have a habit of coming online just before I go to bed so we haven't exactly had good communications.  You should probably be aware I've been working non-stop on this for the past 5 days.  You carry on as though I've abandoned the project whenever you can't get hold of me for a couple of days.  If I'm not answering on IRC it chances are I'm alseep or I've got head buried so deep in some code that I don't want to break my concentration.  3 other people have been helping me with various test/debug processes when I haven't been able to get a hold of you so you don't worry about idle hours...

I don't know why your miners haven't been able to connect... Other people have been able to.  I'm sorry about the message last night then immediately disappearing.  That was extenuating circumstance as some clown got drunk and drove into a power pole in my street and we lost power for several hours.

The good news is after rolling up sleeves and hacking at the namecoind code yesterday to dump some useful trace info and see what's going on from that end I think I've found the problem.  And as these things often turn out the be it was a single line of code.  I still need to prove it but all the tests I've done so far indicate it will work when a real nmc solution is submitted.

As to your offer, whilst I am a big fan of passive income your comment 'until it's sold to someone else' makes that a little vague.  It could be sold tomorrow for all I know.  So I will accept the other offer of the original bounty.  I feel this is reasonable considering 99% of the work was completed in the original timeframe and that it was impossible to complete the last 1% without outside assistance.

[DavinciJ15]

ok chill out for a minute Davinci...

Firstly you knew all along I didn't not have them means to fully test it.  A full test means mining a real block and it's well known by many including you that I only have 50MH at my disposal so there was never any chance that I could a release a fully tested version within 24 hrs.  It was at best a guess and hope that the I'd gotten every minor detail of the protocol right.  What I did release was a couple of thousand lines code with one wrong line (see below). That was 1/2 written over the previous few weeks and 1/2 written in a 12 hour coding marathon...

Secondly lets not forget that after this flurry of activity and urgency it took you several days before you even got around to testing it.  Unfortunately you seem to have a habit of coming online just before I go to bed so we haven't exactly had good communications.  You should probably be aware I've been working non-stop on this for the past 5 days.  You carry on as though I've abandoned the project whenever you can't get hold of me for a couple of days.  If I'm not answering on IRC it chances are I'm alseep or I've got head buried so deep in some code that I don't want to break my concentration.  3 other people have been helping me with various test/debug processes when I haven't been able to get a hold of you so you don't worry about idle hours...

I don't know why your miners haven't been able to connect... Other people have been able to.  I'm sorry about the message last night then immediately disappearing.  That was extenuating circumstance as some clown got drunk and drove into a power pole in my street and we lost power for several hours.

The good news is after rolling up sleeves and hacking at the namecoind code yesterday to dump some useful trace info and see what's going on from that end I think I've found the problem.  And as these things often turn out the be it was a single line of code.  I still need to prove it but all the tests I've done so far indicate it will work when a real nmc solution is submitted.

As to your offer, whilst I am a big fan of passive income your comment 'until it's sold to someone else' makes that a little vague.  It could be sold tomorrow for all I know.  So I will accept the other offer of the original bounty.  I feel this is reasonable considering 99% of the work was completed in the original timeframe and that it was impossible to complete the last 1% without outside assistance.

Well then I owe you an apology, I have yet to learn not to assume when dealing with people across the internet as I make an ASS out of U and ME.  LOL

Sorry about all of this and thanks for all of your efforts.

Yes, the agreement of 1.5% until sold is a bit vague however you you like I can put stuff in writing as I would cut you at 1.5% if I sell NMCBIT.

With that said I will just pay you the 200BTC unless you change your mind.

Cheers Mate and sorry for the misunderstanding.

Davinci

[Keninishna]

so is the single line of code updated on the poolserverj site?

[shads]

not yet... I'm waiting to prove it with an nmc block on testnet.

[shads]

should have checked before I posted.... it works! 

Code:

namecoind-mm listtransactions
[
    {
        "account" : "",
        "category" : "immature",
        "amount" : 50.00000000,
        "confirmations" : 3,
        "txid" : "682ab91c841886428c49070ba8bb26afc7809a8e5f1c5e49b7611bc9b291bf5d",
        "time" : 1318808850
    },
    {
        "account" : "",
        "category" : "immature",
        "amount" : 50.00000000,
        "confirmations" : 2,
        "txid" : "3952dfbdec1b4de1f14eb90394adbbeddf1cccad55029fb7c5b4f0be4d4872e9",
        "time" : 1318824415
    },
    {
        "account" : "",
        "category" : "immature",
        "amount" : 50.00000000,
        "confirmations" : 1,
        "txid" : "9f8a3477f178518cb5561036505ca7b0cc2623ebf27076d940be20c58db9b609",
        "time" : 1318824832
    }
]

[urstroyer]

So http://poolserverj.org/dist/mm-mini-binary.04.tar.gz is the magic peace of software?

Great work!

[shads]

No I haven't posted it up yet.  Give me a bit. I'll designate it .05 so look out for that one

[shads]

ok posted as .05.  You'll need the full .02 package then overlay the files in .05

[Keninishna]

yay, I'll give it a test.

[Keninishna]

Thanks DavinciJ15, this solution works fine for me.

May i ask if you found a 4diff patch which can be compiled with the current bitcoind version vom vinced?

Best regards,

Chris

Hi Chris,

I'm not a bastard like most pool operators and coin developers I will tell what I know.  I am using this fork...
https://github.com/ArtForz/namecoin/commit/127deb4aff13965741130dba7304073330a4adea

When it compiles it will give you an error you need to comment out the code as it's not needed for mining.

Good luck

Davinci

Is this namecoind or bitcoind? I ask because the vinced merged mining bitcoind crashes on me like hourly.

[Keninishna]

well it works I found 2 namecoin blocks.

[shads]

MM edition .06 is now on the downloads page.  This is first version of MM that I'm reasonably happy with.

It should now be sending longpolls if either block changes and the server is now actively monitoring the state of all block chains..  Issues with high CPU load caused by PSJ rejecting valid work should also be resolved.

If you are going to try it I'd appreciate if you could set the following:
debug=true
trace=true
traceTargets=blockmon

If anything odd happens this should give me some useful info to play with.

[Keninishna]

MM edition .06 is now on the downloads page.  This is first version of MM that I'm reasonably happy with.

It should now be sending longpolls if either block changes and the server is now actively monitoring the state of all block chains..  Issues with high CPU load caused by PSJ rejecting valid work should also be resolved.

If you are going to try it I'd appreciate if you could set the following:
debug=true
trace=true
traceTargets=blockmon

If anything odd happens this should give me some useful info to play with.

Its not quite working for me but I got a whole slew of errors - http://pastebin.com/eXtfLMRx

[shads]

You're missing a class that is in the 'core' snapshot file.  Did you copy the snapshot files into lib/lib_non-maven?

[Keninishna]

nope I just pasted everything to /bin derp. Working now.

[shads]

If I'd realised how little had to be changed I would have done this weeks ago but that's how it goes...

The most recent version of psj poolserverj-0.3.0.07.MM.tar.gz now has Scrypt support.  You'll have to remove all the merged mining config options from the properties file and set POWalgorithm=scrypt

Fair warning, this is completely untested as I don't have a scryptBrix/coin client.

[wtfman]

If I'd realised how little had to be changed I would have done this weeks ago but that's how it goes...

The most recent version of psj poolserverj-0.3.0.07.MM.tar.gz now has Scrypt support.  You'll have to remove all the merged mining config options from the properties file and set POWalgorithm=scrypt

Fair warning, this is completely untested as I don't have a scryptBrix/coin client.

sorry, but what is ScryptBrix? Google does not enlighten me

[terrytibbs]

If I'd realised how little had to be changed I would have done this weeks ago but that's how it goes...

The most recent version of psj poolserverj-0.3.0.07.MM.tar.gz now has Scrypt support.  You'll have to remove all the merged mining config options from the properties file and set POWalgorithm=scrypt

Fair warning, this is completely untested as I don't have a scryptBrix/coin client.

sorry, but what is ScryptBrix? Google does not enlighten me

Tenebrix, Fairbrix, and Litecoin are the only ones as far as I know.

[wtfman]

thx 4 info.


mm works really smooth so far, unfortunately the stale rate has dramatically increased from 0.3% to 3.3%. Do you only deem shares as valid when BTC + NMC results are up2date results? Only main chain would be enough imo if true.

[shads]

mm works really smooth so far, unfortunately the stale rate has dramatically increased from 0.3% to 3.3%. Do you only deem shares as valid when BTC + NMC results are up2date results? Only main chain would be enough imo if true.

I was just about to ask about this.  Is  anyone else seeing higher stale rates?  From what I've heard it's only cgminer clients that are getting it.  Any info people can report back will be helpful.

There's a numbers of issues at play here I think...

1/ There is a fundamental design clash between longpolling and merged mining.  Basically it works like this:  When there is a double block change (i.e. btc and nmc solved by one solution) the daemons won't see the new block at exactly the same time.  From what I've seen a delay of around a 1-2 seconds is not unusual.  What happens then is that one chain advances to the next block.  PSJ checks the other chains to see if they've updated, sees they haven't so starts sending out LPs.  Before the miner receives the LP and established a new LP connection the second chain advances so PSJ starts sending out another batch of LP's.  Those miners who haven't got their new LP connection registered in time miss out. 

2/ cgminer may be a bit too clever for it's own good.  It does it's own checks on whether work is valid and I presume it uses prev_block_hash to check or maybe the X-Blocknum header.  If an NMC block is found but it isn't a BTC solution then the pseudo block number will advance but the X-Blocknum header (which is for BTC block in psj) and the prev_block_hash won't change.  So cgminer may think it's the same block and carry on using it's cached work.

3/ The rules for accepting a 'partially stale' share are not defined.  If NMC block advances the previous work can still produce a valid BTC block although it will be rejected by the NMC daemon.  The default behaviour is if it's not the current block (block in psj is now defined as a pseudoblocknum which is sum of all chain's blocknums) reject it as stale.  This may have to change but I need some consensus from pool ops on the what the correct behaviour should be.  I'm not going to code every option under the sun coz I'm little bit sick of merged mining TBH.

4/ The code for detecting and switching to new blocks is still pretty raw.  It's possible the 'fireBlockChange' sequence is happening more often that it should which would put valid shares into the stale list.  I don't know if this is the case but the solution 3/ contains the fix for this issue if it exists...

[jamesg]

Is  anyone else seeing higher stale rates? 

I am running my equipment on yourbtc.net which is now running psj with merged mining and the stales are higher on phoenix 1.6.2 also. I am running at about 2% stales.

[shads]

what sort of rates were you seeing previously?

[jamesg]

what sort of rates were you seeing previously?

On yourbtc.net my stales rate was around .15% before merged mining.

[shads]

Thanks for the feedback... keep it coming...

ok I've just uploaded the latest revision .08.  Please note the base distribution was updated at .07 to apply .08 you will need to start with .07 and apply the contents of .08 mini-binary over the top.

This contains a number of changes that I hope will address most of the 4 issues I highlighted above...

- support for partially stale shares.  Where a single chain has advanced to the next block old work may still be valid for the other blocks.  This patch checks that the work is valid for at least one block giving three possible outcomes.  accepted, accepted partial-stale, stale

- longpoll passthru.  To address the double longpoll issue. There is a fundamental design clash between longpolling and merged mining.  Basically it works like this:  When there is a double block change (i.e. btc and nmc solved by one solution) the daemons won't see the new block at exactly the same time.  From what I've seen a delay of around a 1-2 seconds is not unusual.  What happens then is that one chain advances to the next block.  PSJ checks the other chains to see if they've updated, sees they haven't so starts sending out LPs.  Before the miner receives the LP and establishes a new LP connection the second chain advances so PSJ starts sending out another batch of LP's.  Those miners who haven't got their new LP connection registered in time miss out so continue working on the old block for as long as they would normally (probably about a minute).

This patch addresses this by setting a longpoll passthru period.  Where two block changes happen within a specified period (10 seconds).  After the 2nd block change and until 10 seconds after the 1st block change has passed, any longpolls received will have a short expiry of 1 second after which they'll return new work to the worker.  This will give any slow miners a few seconds to get their longpoll in and learn that there's a 2nd new block to work on. The reason for the 1 second delay is to prevent longpoll spam.  Most (but not all) miners will immediately send another LP request as soon as they get a response.  With 0 delay this sets up an LP spam loop which thrashes the server and causes high CPU usage and low response times.  They will still spam 1/sec until the passthru period is over which is why this is called a workaround rather than a fix.  There's really no way to fix this issue completely except to ditch either longpolling or merged mining alltogether.   

[shads]

2/ cgminer may be a bit too clever for it's own good.  It does it's own checks on whether work is valid and I presume it uses prev_block_hash to check or maybe the X-Blocknum header.  If an NMC block is found but it isn't a BTC solution then the pseudo block number will advance but the X-Blocknum header (which is for BTC block in psj) and the prev_block_hash won't change.  So cgminer may think it's the same block and carry on using it's cached work.

Ive just been looking through the cgminer source code and it looks to me like it does use prev_block_hash as an indicator of whether the block has changed.  I'm happy to stand corrected but if that's the case it's a fairly serious problem.  It means whenever an aux chain finds a new block but the parent chain doesn't that cgminer will ignore the new work (even if it gets it via longpoll).  I don't know how many other miners do this but pool ops should start looking into this.

With 'partial-stale' shares being accepted it effectively means the miner will get credited for both chains while only working on one.  Unless shares are actually assigned a 'credit value per chain' the choice is between crediting these partial shares or disallowing them unless the share is valid for all chains.  The 'credit value per chain' option will require and extra database column which will break pushpool compatibility completely (it had to happen sometime) and require all pool ops to make some mods in how they are calculating worker shares.

Need feedback from pool ops on how to deal with this.  This is a particularly significant issue for PPS pools where you'd end up paying out a full share where only part of the work is valid.

[DavinciJ15]

My suggestions to you had two parts be ether way the a partial share should tell the miner that it was accepted (IMO).

1) Insert into the reason column "partial-share" or "partial-stale", this allows the pool op to allow or deny the share as valid share an option in PSJ acceptPartialShares=true and reasonForPartial="" should be in the properties file.

2) Add an optional column called ChainMask where the bit marked block chain is placed into the column.  For example if you have 3 chains the value of Parent is 1, second is 2 and third is 4, if the share is valid for chain id 1 and 2 the value in the mask column is 3, if the share is valid for 1 and 4 the value in the mask column is 5 and all of them would be 7. 

The only issue with 2 is as you add chains the combo's are a factorial of the chains.

That's my 2 BTC cents.

I would like to see other ideas.


Davinci

[shads]

My suggestions to you had two parts be ether way the a partial share should tell the miner that it was accepted (IMO).

1) Insert into the reason column "partial-share" or "partial-stale", this allows the pool op to allow or deny the share as valid share an option in PSJ acceptPartialShares=true and reasonForPartial="" should be in the properties file.

2) Add an optional column called ChainMask where the bit marked block chain is placed into the column.  For example if you have 3 chains the value of Parent is 1, second is 2 and third is 4, if the share is valid for chain id 1 and 2 the value in the mask column is 3, if the share is valid for 1 and 4 the value in the mask column is 5 and all of them would be 7. 

The only issue with 2 is as you add chains the combo's are a factorial of the chains.

That's my 2 BTC cents.

I would like to see other ideas.


Davinci

your suggestion 1 is how the current implementation works.  A normal accept share will have our_result=true, reason=null.  A partial-share will have our_result=true, reason=partial-stale

re: 2.... Perhaps a bit mask would be an option.  Though this is limited and any combo value field tends to make efficient db queries difficult to achieve.

[Keninishna]

how often does a single share solve a nmc and btc block?

[shads]

if you mean a single share from yr own pool... btc diff / nmc diff ~= number of nmc only shares per btc+nmc share

If you mean across the whole network that's a bit more complicated since you've got to account for non-merged miners.

[urstroyer]

What do you think of adding a new table for all additional mm chains like this:



Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

[DavinciJ15]

What do you think of adding a new table for all additional mm chains like this:

Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I like large growth potential so that it does not need to be redone when it's time to grow.

[urstroyer]

What do you think of adding a new table for all additional mm chains like this:

Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I like large growth potential so that it does not need to be redone when it's time to grow.

I already added a third merged chain "stifflersmomchain" to the scenario.

We only need this table, you can query the db and set where statement to the desired chain on the column "chain".
Plus we already set the name of the merged chain into the psj property file (source.local.1.merged.chains=namecoin).

It's even possible to set an int instead of a string for the chain column, in case we wanna use the id (source.local.1.merged.namecoin.chainid=1).

[shads]

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I have to agree.  Writing shares to DB is already one the biggest bottlenecks in psj.   Doubling or tripling the number of rows written is definately not ideal, likewise with using a seperate table with a many-to-one relationship.

I'm wondering if perhaps a floating point share value column should be added.  There would need to be an interface so the pool could dynamically update the value of each chain.  The value field is then the sum of the solved chain's values.  The disadvantage of this is that values are changed frequently then it's retrospectively difficult to go back and try to work which chains each share solved for.

The remaining options are a single consolidated column like bitfield flags or a boolean column per chain.  I'm not sure what factorial planet davinci was on but the way you normally do it is allocate each bit to a flag.  so 1 = bitcoin, 2=namecoin, 4=scamcoin.  Then each is easy to extract with a simple logical &.

If you don't want to record which chains are solved you have a simple choice between accept or reject partials.  Accepting them provides the miner no incentive to fix any latency problems and hurts the network overall, not mention costing PPS operators a pretty penny.  Reject is going to fill yr inboxes with complaints (which I personally think you should be redirecting to the namecoin forum).

[DavinciJ15]

The remaining options are a single consolidated column like bitfield flags or a boolean column per chain.  I'm not sure what factorial planet davinci was on but the way you normally do it is allocate each bit to a flag.  so 1 = bitcoin, 2=namecoin, 4=scamcoin.  Then each is easy to extract with a simple logical &.

That was my original suggestion you shot down as too complex! Jack bean stalk!   

what i did not know is you could do this in SQL...

Code:

  select * from shares where reason IS NULL and chainmask & 2;

instead of this...

Code:

  select * from shares where reason IS NULL and chainmask in (2, 3, 6, 7, 10, 11, 14, 15);  

So I was on the SQL ignorance planet!  My problem is I don't think of SQL as a computer language, I think of it as a bad hack for accessing data and thus forget how robust it is.

OH well.  Learn something new everyday.

Anyhow...
I say having more than 32 block chains is crazy so unsigned INT is the way to go.

Thus my vote is for a bit wise column!

Davinci

[shads]

If anyone's downloaded .08 and found an odd looking file in there:  bitcoin-poolserverj-0.0.2-SNAPSHOT.jar

That's actually poolserverj.jar which I forgot to rename.  It should be in the bin directory and *no where else*.  I've uploaded a new version with the correct name. 

[shads]

If anyone has some spare time to play it would be really helpful if someone could download this version and run it for a while then send me the log:
http://poolserverj.org/dist/mm-mini-binary.09.debug.tar.gz

It looks like stales have been dramatically reduced but for some reason cgminer is sending back a few works that are registering as unknown.  The above version is identical to .08 but has a bit of extra logging in it that should help track down the source of the problem.

edit: updated post to point to .09 which has a minor fix for issue where in some cases partial-stales were treated as full shares.

[Keninishna]

im getting a 404 error http://poolserverj.org/dist/mm-mini-binary.09.tar.gz

[urstroyer]

im getting a 404 error http://poolserverj.org/dist/mm-mini-binary.09.tar.gz

http://poolserverj.org/dist/mm-mini-binary.09.debug.tar.gz

[DavinciJ15]

im getting a 404 error http://poolserverj.org/dist/mm-mini-binary.09.tar.gz

http://poolserverj.org/dist/mm-mini-binary.09.debug.tar.gz

+1

[shads]

use the link in the above post

[DavinciJ15]

use the link in the above post

What ever you did it looks good so far I only see the correct partial-stale,  no stales no unknown-work.  I will let it run until there is a double LP.

[jamesg]

use the link in the above post

What ever you did it looks good so far I only see the correct partial-stale,  no stales no unknown-work.  I will let it run until there is a double LP.

I second that. On http://yourbtc.net I have 70 stales out of 150,000 shares or .04%. Incredible.

[urstroyer]

@DavinciJ15: Are you running 0.09 right now? I run into a 1 million share nmc round and just wondering if you already succeeded in finding a block with this version?

Thanks for help, bro!

[DavinciJ15]

@DavinciJ15: Are you running 0.09 right now? I run into a 1 million share nmc round and just wondering if you already succeeded in finding a block with this version?

Thanks for help, bro!

No I am not running it right now but just about to set a server with it installed.  Msg shadders with your properties file and see if he can see anything wrong.

I hope it's just a settings issue cuz I'm going to turn off push to upstream.

[urstroyer]

@DavinciJ15: Are you running 0.09 right now? I run into a 1 million share nmc round and just wondering if you already succeeded in finding a block with this version?

Thanks for help, bro!

No I am not running it right now but just about to set a server with it installed.  Msg shadders with your properties file and see if he can see anything wrong.

I hope it's just a settings issue cuz I'm going to turn off push to upstream.

We already talked about it, couldn't find any unusual. I seems to be just bad luck or variance. PSJ is running smooth like always!

[DavinciJ15]

@DavinciJ15: Are you running 0.09 right now? I run into a 1 million share nmc round and just wondering if you already succeeded in finding a block with this version?

Thanks for help, bro!

No I am not running it right now but just about to set a server with it installed.  Msg shadders with your properties file and see if he can see anything wrong.

I hope it's just a settings issue cuz I'm going to turn off push to upstream.

We already talked about it, couldn't find any unusual. I seems to be just bad luck or variance. PSJ is running smooth like always!

The only other tip I can give you is that I have noticed luck improves if you have the right work cache and timeout  as it will take work that never gets done.

[shads]

Urstroyer I came across something last night that seemed a bit odd and Davinci's suggestion that your cache size might be too high could be on the money.

Somehow, I don't know when, the code that checks work age in cache has dissappeared.  When retrieving work from the cache it's supposed get retrieving and discarding until it finds one that's no expired.  That wasn't happening.  So this will only happen if your work cache size is quite large but you could have been passing out work that was so old the namecoin network was rejecting it.  It's just a guess.

In any case I've made a fix for the problem and released .10 on the website now.

As a backup could you do an sql dump of your solutions this round and send to me via email.   I'll run a script over them and check if any meet difficulty for either chain which should tell us if there's a problem detecting them or not.

[Keninishna]

would this bug affect normal poolserverj?

[shads]

would this bug affect normal poolserverj?

if you mean the pre mm versions (0.3.0.FINAL) then no.  If the problem even exists it's only in mm edition and probably only in the last couple of versions.

[shads]

would this bug affect normal poolserverj?

Turns out this bug doesn't actually exist *sigh of relief*... the pool in question was just having a looooong streak of bad luck.

[DavinciJ15]

would this bug affect normal poolserverj?

Turns out this bug doesn't actually exist *sigh of relief*... the pool in question was just having a looooong streak of bad luck.

PSJ takes work from the upstream and makes sure it's not duplicate work to put in it's cache right?

So, if that work expires and new work is requested would that not mean it wasted valid work that a miner could have found POW?  I understand that transactions could be occurring frequently and could be added into new work for a solution but I would think this would be harder on the miners to find a solution.

Just a guess.

[jamesg]

I'm not quite sure what shads did, but here is proof that psj is by far the best pool server.



For the quick math impaired, that a total of 129,601 shares submitted with 2 stales. Unbelievable!! 

I am donating all of my "world's best rig builder" winnings to psj.

[DavinciJ15]

I'm not quite sure what shads did, but here is proof that psj is by far the best pool server.

For the quick math impaired, that a total of 129,601 shares submitted with 2 stales. Unbelievable!! 

I am donating all of my "world's best rig builder" winnings to psj.

Yep no one tops Shadders! He is a kick ass developer.

[teukon]

Hi,

Just a simple question about when a Bitcoin block and Namecoin block are found simultaneously.

My original intuition was that a hash for a merged mining header which Generated a Bitcoin block would also be able to generate a Namecoin block due because the difficulty of Namecoin was lower than that of Bitcoin.

I was later told that this was in fact false.  While merged mining it is quite possible to find a Bitcoin block without a Namecoin block and that dual Bitcoin-Namecoin blocks, while possible, were relatively rare.

However, since urstroyer set up PoolServerJ for yourbtc.net all of the Bitcoin blocks (4 now) have come at the same time as Namecoin blocks.

Can anyone explain this?  Does PoolServerJ somehow do something clever with the header to guarantee that Bitcoin blocks always come with Namecoin blocks?  Is this just coincidence?

[jamesg]

Hi,

Just a simple question about when a Bitcoin block and Namecoin block are found simultaneously.

My original intuition was that a hash for a merged mining header which Generated a Bitcoin block would also be able to generate a Namecoin block due because the difficulty of Namecoin was lower than that of Bitcoin.

I was later told that this was in fact false.  While merged mining it is quite possible to find a Bitcoin block without a Namecoin block and that dual Bitcoin-Namecoin blocks, while possible, were relatively rare.

However, since urstroyer set up PoolServerJ for yourbtc.net all of the Bitcoin blocks (4 now) have come at the same time as Namecoin blocks.

Can anyone explain this?  Does PoolServerJ somehow do something clever with the header to guarantee that Bitcoin blocks always come with Namecoin blocks?  Is this just coincidence?

I think if you take a look at the yourbtc.net site, you'll see that the shares were not the same as the bitcoin share for this round are about 600 ahead of the namecoin round. I do find it quite interesting that we seem to find these blocks so close together though.

[shads]

A share that solves a block of difficulty n will always solve for all chains where difficulty is < n so your intuition is right so long as bitcoin difficulty remains higher.

The only instance where this might not occur would be where the miner is working on a work that is valid for only only the bitcoin block.  This is how it can happen:

a namecoin block is found but difficulty was below bitcoin so no bitcoin block is found.  All work previously issued is now no longer valid for namecoin but remains valid for bitcoin.  So if a bitcoin block is found very soon after before miners have refreshed work then it may be stale for namecoin but valid for bitcoin.

Under normal circumstance this should be very rare.  After a block is found LP should take less than a second or two to get the miners working on fresh work that is valid for both chains.  So you'd have to have to find a namecoin block then a bitcoin block (from a different getwork) within 1-2 seconds.

In practice this can actually happen more often.  cgminer ignores longpoll unless the prev_block_hash has changed.  This will only change when you find a bitcoin block.  So it will ignore longpolls that come straight after a namecoin-only block is found and continue working with old work.  This can go on for anywhere up to a minute before it refreshes it's work.  On average though call it 30 seconds.  So if namecoin block is found then within 30 seconds a bitcoin/namecoind block is found by a cgminer then you might see a bitcoin only block on a mm pool.

[urstroyer]

Hi,

Just a simple question about when a Bitcoin block and Namecoin block are found simultaneously.

My original intuition was that a hash for a merged mining header which Generated a Bitcoin block would also be able to generate a Namecoin block due because the difficulty of Namecoin was lower than that of Bitcoin.

I was later told that this was in fact false.  While merged mining it is quite possible to find a Bitcoin block without a Namecoin block and that dual Bitcoin-Namecoin blocks, while possible, were relatively rare.

However, since urstroyer set up PoolServerJ for yourbtc.net all of the Bitcoin blocks (4 now) have come at the same time as Namecoin blocks.

Can anyone explain this?  Does PoolServerJ somehow do something clever with the header to guarantee that Bitcoin blocks always come with Namecoin blocks?  Is this just coincidence?

I think if you take a look at the yourbtc.net site, you'll see that the shares were not the same as the bitcoin share for this round are about 600 ahead of the namecoin round. I do find it quite interesting that we seem to find these blocks so close together though.

In fact the total (valid + invalid) shares of btc and nmc rounds are the same if a "double block" has been found before and we are still hashing for new blocks. BTC share count on the website is 600 ahead because these are the valid shares. I will clarify on the pool website that these numbers are total shares - invalid shares.

The reason why nmc invalid shares are currently higher is, that cgminer don't care about longpoll on new nmc network blocks as far as i understood.

[shads]

Just had another look at the cgminer thread, looks like conman made a patch a couple of days ago.  I'm not sure if it's been released as a binary yet and even if so it'll probably be a while before the majority of cgminer user upgrade but at least now if pools get complaints to about the issue they can point to a solution that doesn't force miners to abandon their mining client of choice.

[teukon]

A share that solves a block of difficulty n will always solve for all chains where difficulty is < n so your intuition is right so long as bitcoin difficulty remains higher.

The only instance where this might not occur would be where the miner is working on a work that is valid for only only the bitcoin block.  This is how it can happen:

a namecoin block is found but difficulty was below bitcoin so no bitcoin block is found.  All work previously issued is now no longer valid for namecoin but remains valid for bitcoin.  So if a bitcoin block is found very soon after before miners have refreshed work then it may be stale for namecoin but valid for bitcoin.

Under normal circumstance this should be very rare.  After a block is found LP should take less than a second or two to get the miners working on fresh work that is valid for both chains.  So you'd have to have to find a namecoin block then a bitcoin block (from a different getwork) within 1-2 seconds.

In practice this can actually happen more often.  cgminer ignores longpoll unless the prev_block_hash has changed.  This will only change when you find a bitcoin block.  So it will ignore longpolls that come straight after a namecoin-only block is found and continue working with old work.  This can go on for anywhere up to a minute before it refreshes it's work.  On average though call it 30 seconds.  So if namecoin block is found then within 30 seconds a bitcoin/namecoind block is found by a cgminer then you might see a bitcoin only block on a mm pool.

Thank you!  I have a much better idea of what is going on now.  It seems that early teething problems with merged mining have created quite a few Bitcoin blocks with a stale Namecoin element and these blocks spawned a rumour about their likelyhood to which I fell victim.

By the way, thanks for the work on PoolServerJ.  I'm not a pool operator myself but I've been donating.

[shads]

I've added a new 'Guide to setting up poolserverj in Eclipse 3.7' page to documentation section of the website.  If you want to fork or make mods for your own pool this should get you going pretty quickly with a decent build environment.

http://poolserverj.org/documentation/guide-to-setting-up-poolserverj-in-eclipse-3-7/

[shads]

I've just posted two new versions of psj binaries on the website.  These contain a hotfix for a potential security issue.  I have reason to believe this issue has been exploited already.

The stable pre-merged mining version has been posted as 0.3.1
The merged mining branch contains the hotfix in .11 mini-binary.

There are two parts to this fix, one was fixed in the merged mining branch some time ago and I thought it was in main branch but was mistaken.  The was just discovered a couple of hours ago.

I'm not going to explain the issue in detail until pool ops have had a chance to update.  If you want to know please PM me or ask on IRC in private chat.  So long as I know you're a pool op that's running psj I'll give you the rundown.

I will post full detail here in a few days.  I have not committed these changes to the public repository.  I will do this in a few days also.  If anyone has concerns about this I'm happy to send you the new source code privately if you don't want to wait.

[Red Emerald]

Following

[shads]

Apologies to anyone experiencing stability issues with the 0.3.1 release.  I based it off a commit that further down the line that the 0.3.0 tag and it turns out it included a merge with a refactored branch.

I've redone using the exact 0.3.0.FINAL commit as the base.  The changes themselves are very minor and have been tested in production by a pool that builds from source so now that's it based of the right level stability should be identical to 0.3.0.FINAL.

I've uploaded the new pckage as 0.3.1.01 and I'll remove 0.3.1.

[shads]

[ANNOUNCE] Poolserverj WORKMAKER EDITION RELEASED - 0.4.0rc1

This is such a major release I thought it deserved it's own thread 

https://bitcointalk.org/index.php?topic=51226.0

[teukon]

A quick question about share difficulty.

Is it possible to allow different miners to use different share difficulties?  I'm thinking of something along the lines of allowing a miner to select an option at the pool, "golden shares" or whatever, which changes their share difficulty from 1 to 100 say.  I believe that, for a big miner at a small pool, this will make practically no difference to variance but could reduce pool workload significantly (perhaps the pool could reward miners appropriately for selecting "golden shares" with a reduced fee).

[Bit LC Inc.]

I'm having huge problems with psj started about an hour ago.

First day/two went by just smooth, then i started seeing idle miners and now it's 50-70% offline.

[18/01/2012 01:37:34] Result: 93cb2419 accepted
[18/01/2012 01:37:40] LP: New work pushed
[18/01/2012 01:37:41] Result: dababb02 accepted
[18/01/2012 01:37:51] Warning: work queue empty, miner is idle
[18/01/2012 01:38:01] Disconnected from server
[18/01/2012 01:38:09] Failed to connect, retrying...
[18/01/2012 01:38:21] Failed to connect, retrying...
[18/01/2012 01:38:36] Failed to connect, retrying...
[18/01/2012 01:38:51] Failed to connect, retrying...
[18/01/2012 01:39:06] Connected to server
[18/01/2012 01:39:06] Currently on block: 162689
[18/01/2012 01:39:25] Result: 67046925 accepted
[18/01/2012 01:39:27] Result: a0e9db30 accepted
[18/01/2012 01:39:37] Result: 76f2c530 accepted
[18/01/2012 01:39:41] Warning: work queue empty, miner is idle
[18/01/2012 01:39:50] Disconnected from server
[18/01/2012 01:39:56] Connected to server
[18/01/2012 01:40:10] Disconnected from server
[18/01/2012 01:40:14] Result: 0ba8e8fa rejected
[18/01/2012 01:40:19] Warning: work queue empty, miner is idle
[18/01/2012 01:40:19] Failed to connect, retrying...
[18/01/2012 01:40:34] Connected to server
[18/01/2012 01:41:01] Result: 777a0327 accepted
[18/01/2012 01:41:07] Result: 9619a765 rejected
[18/01/2012 01:41:09] Warning: work queue empty, miner is idle
[18/01/2012 01:41:18] Disconnected from server
[18/01/2012 01:41:45] Failed to connect, retrying...
[18/01/2012 01:42:15] Failed to connect, retrying...
[18/01/2012 01:42:33] Connected to server
[18/01/2012 01:42:43] Result: 3a8adbe1 accepted
[368.44 Mhash/sec] [11411 Accepted] [90 Rejected] [RPC (+LP)]

Logs shows nothing, dmesg/syslog doesn't either.

jine@pool:~$ netstat -an | grep ESTA | wc -l
1532

jine@pool:~$ netstat -an | grep TIME | wc -l
720

Used Bitcoin 0.5.1 until a few minutes ago, quickly updated to latest 0.5.99 from staging-git tree

Help us out!

--
Regards,
Jim "Jine" Nelin

[hoseen]

2 more features off the top of my head:

* Google app engine compatibility

I looked at google app engine but it has a couple of gotcha's.  

- Time per request: max 30 sec

Which effectively rules out longpolling.

- Memory cap 128MB.

Depends on how hard you are running it but if you run a large pool you may find this restrictive.  A smallish pool will comfortably run in about 50mb but once you start scaling you are consuming memory two ways, first is caching work.  Second is mapping work delivered to workers for current block and last block.  I can't actually remember why I did this, it was partly to ensure no duplicate work but there was one other reason.  

would you consider implementing poolserverj for appengine backends? requests can run indefinitely and they have 1GB ram. here is the spec: https://developers.google.com/appengine/docs/java/backends/overview

i'd like to add a bitcoin pool to my web app's community, but everything is already on appengine and i don't want to have to glue it together with ec2 AWS. if you could get it to run on appengine backends, i would be eternally grateful!

[rjk]

Instead of EC2 as such, Amazon also offers Elastic Beanstalk, which might be more appropriate. It is designed to run Java programs on Tomcat, and automatically scale up and down with usage.
They also recently added support for PHP and .NET applications, and I've always wondered how well it worked. Haven't heard from anyone that has tried it. But basically it takes care of all the "gluing together" that needs to be done.

[DavinciJ15]

Shadders or anyone reading this can fix 2 issues for me I will pay 50 BTC.

1.   I am seeing a slow down when long polling occurs form time to time.
2.   I would like to fix coin basing for Namecoin

[Bit LC Inc.]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

[Graet]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

poolserverj author seen once since December - he came on IRC and asked me to sell his Bitcoins - not heard of since

Unless someone takes it over it's time has come.

[DavinciJ15]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

poolserverj author seen once since December - he came on IRC and asked me to sell his Bitcoins - not heard of since

Unless someone takes it over it's time has come.

I agree.

[nyusternie]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

[jamesg]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

Poolserverj is also missing rollNTime support from the main repo. It would be great for someone to pick up this project again as poolserverj was far better option at the time it was being developed than other pool server options.

Seeing stratum and/or GBT support in poolserverj would be awesome.

[DavinciJ15]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

You came along 2 months too late.

[nyusternie]

You came along 2 months too late.

Yeah, that's what I figured. But thanks for getting back. BTW, what are you using now for your pool server?

Poolserverj is also missing rollNTime support from the main repo. It would be great for someone to pick up this project again as poolserverj was far better option at the time it was being developed than other pool server options.

Seeing stratum and/or GBT support in poolserverj would be awesome.

Looking back to last year, it really seemed to be an incredible server (pre-Stratum and GBT). But, considering the lack of a userbase, I'll probably work towards starting up a new project, based off of whatever I learn from PoolServerJ. This is academic for me at this point, so the more I can learn about the inner workings of Bitcoin the better for me. Trying to patch someone else code usually leads to too many headaches.

I'll certainly reach back out to this forum if anyone is interested in testing a Java-based server once its reached Beta.

Thanks

[13MSzfPPr4gXvusKJFAFR2EAN]

Can someone help me with poolserverj sources ?
In the sources which I have are some java files not included but needed.

I am looking for the class or source of com.shadworld.cache.ArrayDequeResourcePool.

Could you post it hier or send it to me on diamantminer@gmx.de

Please help.

Wily

[nyusternie]

where did you get the sources from?

the last working version that I know of is:
https://bitbucket.org/shadders/bitcoin-poolserverj

however, as the last update was over a year and a half ago, you may have your work cut out for you.

good luck

[13MSzfPPr4gXvusKJFAFR2EAN]

where did you get the sources from?

the last working version that I know of is:
https://bitbucket.org/shadders/bitcoin-poolserverj

however, as the last update was over a year and a half ago, you may have your work cut out for you.

good luck

Yes I got my Sorces from there. But there is something wrong with the sources. This file is failing in the sources com.shadworld.cache.ArrayDequeResourcePool
could you please send it to me if you have it ? diamantminer@gmx.de. It should be in a util jar.
If you have any scripts für Database initial creation please provide them also. Or are there some in sources ? Where ?
Any extra tests ?

Regards
Krastev

[13MSzfPPr4gXvusKJFAFR2EAN]

Hallo All and Shads,
I want to upgrade and change the poolserverj last version. I will name the new project diamant-bitcoin-poolserverj.
I will change the license too. I will use Creative-Commons-Licens. If you don't mine. Sometimes I will need also your help.
Falls was is contact me please at diamantminer@gmx.de.

PLEASE KEEP CHECKING THIS THREAD FOR NEWS ABOUT THE NEW PROJECT diamant-bitcoin-poolserverj !!!! COMMING SOON !!!! IN DEVELOPMENT !!!

Thank you.
Wily

[bindedraht]

what system requierments does it need?

[13MSzfPPr4gXvusKJFAFR2EAN]

As I said it will come soon. Be patient a bit.

Happy mining
Wily

[13MSzfPPr4gXvusKJFAFR2EAN]

Here is the thread for the new development of PoolserverJ keep watching the new thread for news.

https://bitcointalk.org/index.php?topic=602751.new#new

Happy Mining
Wily

[spring.yu]

Well then I owe you an apology, I have yet to learn not to assume when dealing with people across the internet as I make an ASS out of U and ME.  LOL Smiley

Sorry about all of this and thanks for all of your efforts.

Yes, the agreement of 1.5% until sold is a bit vague however you you like I can put stuff in writing as I would cut you at 1.5% if I sell NMCBIT.

With that said I will just pay you the 200BTC unless you change your mind.