[ATTN: POOL OPERATORS] PoolServerJ - scalable java mining pool backend

[shads]

Just had another look at the cgminer thread, looks like conman made a patch a couple of days ago.  I'm not sure if it's been released as a binary yet and even if so it'll probably be a while before the majority of cgminer user upgrade but at least now if pools get complaints to about the issue they can point to a solution that doesn't force miners to abandon their mining client of choice.

[1714904313]

1714904313

[1714904313]

1714904313

[teukon]

A share that solves a block of difficulty n will always solve for all chains where difficulty is < n so your intuition is right so long as bitcoin difficulty remains higher.

The only instance where this might not occur would be where the miner is working on a work that is valid for only only the bitcoin block.  This is how it can happen:

a namecoin block is found but difficulty was below bitcoin so no bitcoin block is found.  All work previously issued is now no longer valid for namecoin but remains valid for bitcoin.  So if a bitcoin block is found very soon after before miners have refreshed work then it may be stale for namecoin but valid for bitcoin.

Under normal circumstance this should be very rare.  After a block is found LP should take less than a second or two to get the miners working on fresh work that is valid for both chains.  So you'd have to have to find a namecoin block then a bitcoin block (from a different getwork) within 1-2 seconds.

In practice this can actually happen more often.  cgminer ignores longpoll unless the prev_block_hash has changed.  This will only change when you find a bitcoin block.  So it will ignore longpolls that come straight after a namecoin-only block is found and continue working with old work.  This can go on for anywhere up to a minute before it refreshes it's work.  On average though call it 30 seconds.  So if namecoin block is found then within 30 seconds a bitcoin/namecoind block is found by a cgminer then you might see a bitcoin only block on a mm pool.

Thank you!  I have a much better idea of what is going on now.  It seems that early teething problems with merged mining have created quite a few Bitcoin blocks with a stale Namecoin element and these blocks spawned a rumour about their likelyhood to which I fell victim.

By the way, thanks for the work on PoolServerJ.  I'm not a pool operator myself but I've been donating.

[shads]

I've added a new 'Guide to setting up poolserverj in Eclipse 3.7' page to documentation section of the website.  If you want to fork or make mods for your own pool this should get you going pretty quickly with a decent build environment.

http://poolserverj.org/documentation/guide-to-setting-up-poolserverj-in-eclipse-3-7/

[shads]

I've just posted two new versions of psj binaries on the website.  These contain a hotfix for a potential security issue.  I have reason to believe this issue has been exploited already.

The stable pre-merged mining version has been posted as 0.3.1
The merged mining branch contains the hotfix in .11 mini-binary.

There are two parts to this fix, one was fixed in the merged mining branch some time ago and I thought it was in main branch but was mistaken.  The was just discovered a couple of hours ago.

I'm not going to explain the issue in detail until pool ops have had a chance to update.  If you want to know please PM me or ask on IRC in private chat.  So long as I know you're a pool op that's running psj I'll give you the rundown.

I will post full detail here in a few days.  I have not committed these changes to the public repository.  I will do this in a few days also.  If anyone has concerns about this I'm happy to send you the new source code privately if you don't want to wait.

[Red Emerald]

Following

[shads]

Apologies to anyone experiencing stability issues with the 0.3.1 release.  I based it off a commit that further down the line that the 0.3.0 tag and it turns out it included a merge with a refactored branch.

I've redone using the exact 0.3.0.FINAL commit as the base.  The changes themselves are very minor and have been tested in production by a pool that builds from source so now that's it based of the right level stability should be identical to 0.3.0.FINAL.

I've uploaded the new pckage as 0.3.1.01 and I'll remove 0.3.1.

[shads]

[ANNOUNCE] Poolserverj WORKMAKER EDITION RELEASED - 0.4.0rc1

This is such a major release I thought it deserved it's own thread 

https://bitcointalk.org/index.php?topic=51226.0

[teukon]

A quick question about share difficulty.

Is it possible to allow different miners to use different share difficulties?  I'm thinking of something along the lines of allowing a miner to select an option at the pool, "golden shares" or whatever, which changes their share difficulty from 1 to 100 say.  I believe that, for a big miner at a small pool, this will make practically no difference to variance but could reduce pool workload significantly (perhaps the pool could reward miners appropriately for selecting "golden shares" with a reduced fee).

[Bit LC Inc.]

I'm having huge problems with psj started about an hour ago.

First day/two went by just smooth, then i started seeing idle miners and now it's 50-70% offline.

[18/01/2012 01:37:34] Result: 93cb2419 accepted
[18/01/2012 01:37:40] LP: New work pushed
[18/01/2012 01:37:41] Result: dababb02 accepted
[18/01/2012 01:37:51] Warning: work queue empty, miner is idle
[18/01/2012 01:38:01] Disconnected from server
[18/01/2012 01:38:09] Failed to connect, retrying...
[18/01/2012 01:38:21] Failed to connect, retrying...
[18/01/2012 01:38:36] Failed to connect, retrying...
[18/01/2012 01:38:51] Failed to connect, retrying...
[18/01/2012 01:39:06] Connected to server
[18/01/2012 01:39:06] Currently on block: 162689
[18/01/2012 01:39:25] Result: 67046925 accepted
[18/01/2012 01:39:27] Result: a0e9db30 accepted
[18/01/2012 01:39:37] Result: 76f2c530 accepted
[18/01/2012 01:39:41] Warning: work queue empty, miner is idle
[18/01/2012 01:39:50] Disconnected from server
[18/01/2012 01:39:56] Connected to server
[18/01/2012 01:40:10] Disconnected from server
[18/01/2012 01:40:14] Result: 0ba8e8fa rejected
[18/01/2012 01:40:19] Warning: work queue empty, miner is idle
[18/01/2012 01:40:19] Failed to connect, retrying...
[18/01/2012 01:40:34] Connected to server
[18/01/2012 01:41:01] Result: 777a0327 accepted
[18/01/2012 01:41:07] Result: 9619a765 rejected
[18/01/2012 01:41:09] Warning: work queue empty, miner is idle
[18/01/2012 01:41:18] Disconnected from server
[18/01/2012 01:41:45] Failed to connect, retrying...
[18/01/2012 01:42:15] Failed to connect, retrying...
[18/01/2012 01:42:33] Connected to server
[18/01/2012 01:42:43] Result: 3a8adbe1 accepted
[368.44 Mhash/sec] [11411 Accepted] [90 Rejected] [RPC (+LP)]

Logs shows nothing, dmesg/syslog doesn't either.

jine@pool:~$ netstat -an | grep ESTA | wc -l
1532

jine@pool:~$ netstat -an | grep TIME | wc -l
720

Used Bitcoin 0.5.1 until a few minutes ago, quickly updated to latest 0.5.99 from staging-git tree

Help us out!

--
Regards,
Jim "Jine" Nelin

[hoseen]

2 more features off the top of my head:

* Google app engine compatibility

I looked at google app engine but it has a couple of gotcha's.  

- Time per request: max 30 sec

Which effectively rules out longpolling.

- Memory cap 128MB.

Depends on how hard you are running it but if you run a large pool you may find this restrictive.  A smallish pool will comfortably run in about 50mb but once you start scaling you are consuming memory two ways, first is caching work.  Second is mapping work delivered to workers for current block and last block.  I can't actually remember why I did this, it was partly to ensure no duplicate work but there was one other reason.  

would you consider implementing poolserverj for appengine backends? requests can run indefinitely and they have 1GB ram. here is the spec: https://developers.google.com/appengine/docs/java/backends/overview

i'd like to add a bitcoin pool to my web app's community, but everything is already on appengine and i don't want to have to glue it together with ec2 AWS. if you could get it to run on appengine backends, i would be eternally grateful!

[rjk]

Instead of EC2 as such, Amazon also offers Elastic Beanstalk, which might be more appropriate. It is designed to run Java programs on Tomcat, and automatically scale up and down with usage.
They also recently added support for PHP and .NET applications, and I've always wondered how well it worked. Haven't heard from anyone that has tried it. But basically it takes care of all the "gluing together" that needs to be done.

[DavinciJ15]

Shadders or anyone reading this can fix 2 issues for me I will pay 50 BTC.

1.   I am seeing a slow down when long polling occurs form time to time.
2.   I would like to fix coin basing for Namecoin

[Bit LC Inc.]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

[Graet]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

poolserverj author seen once since December - he came on IRC and asked me to sell his Bitcoins - not heard of since

Unless someone takes it over it's time has come.

[DavinciJ15]

WARNING: With the 0.7.0 release of bitcoind - getmemorypool is removed, which cases the internal work-generation to fail.
Patches for poolserverj needed.

poolserverj author seen once since December - he came on IRC and asked me to sell his Bitcoins - not heard of since

Unless someone takes it over it's time has come.

I agree.

[nyusternie]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

[jamesg]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

Poolserverj is also missing rollNTime support from the main repo. It would be great for someone to pick up this project again as poolserverj was far better option at the time it was being developed than other pool server options.

Seeing stratum and/or GBT support in poolserverj would be awesome.

[DavinciJ15]

Greetings,

I would like to reach out to anyone that is still using PoolServerJ as an "active/production" pool server. I have about a month off during the holidays and I see it being spent in the Bitcoin community.

The last post on the official website "PoolServerJ 0.4rc1 WorkMaker Edition" was made on Nov 8 11'. I did see an update to the codebase on https://bitbucket.org/shadders/bitcoin-poolserverj/ on Oct 20 12', so I don't know where things stand right now. Does anyone know if this project is still active?

I've cloned the repo onto my GitHub account (just not a fan of Mercurial) to research the codebase, but it seems extremely outdated now (with Stratum and GetBlockTemplate being the norms now), so I'm trying to decide how to proceed with my own endeavors towards a Java-based mining server.

Any suggestions would be appreciated.

Thanks,
Shomari

You came along 2 months too late.

[nyusternie]

You came along 2 months too late.

Yeah, that's what I figured. But thanks for getting back. BTW, what are you using now for your pool server?

Poolserverj is also missing rollNTime support from the main repo. It would be great for someone to pick up this project again as poolserverj was far better option at the time it was being developed than other pool server options.

Seeing stratum and/or GBT support in poolserverj would be awesome.

Looking back to last year, it really seemed to be an incredible server (pre-Stratum and GBT). But, considering the lack of a userbase, I'll probably work towards starting up a new project, based off of whatever I learn from PoolServerJ. This is academic for me at this point, so the more I can learn about the inner workings of Bitcoin the better for me. Trying to patch someone else code usually leads to too many headaches.

I'll certainly reach back out to this forum if anyone is interested in testing a Java-based server once its reached Beta.

Thanks

[13MSzfPPr4gXvusKJFAFR2EAN]

Can someone help me with poolserverj sources ?
In the sources which I have are some java files not included but needed.

I am looking for the class or source of com.shadworld.cache.ArrayDequeResourcePool.

Could you post it hier or send it to me on diamantminer@gmx.de

Please help.

Wily