[ATTN: POOL OPERATORS] PoolServerJ - scalable java mining pool backend

[terrytibbs]

If I'd realised how little had to be changed I would have done this weeks ago but that's how it goes...

The most recent version of psj poolserverj-0.3.0.07.MM.tar.gz now has Scrypt support.  You'll have to remove all the merged mining config options from the properties file and set POWalgorithm=scrypt

Fair warning, this is completely untested as I don't have a scryptBrix/coin client.

sorry, but what is ScryptBrix? Google does not enlighten me

Tenebrix, Fairbrix, and Litecoin are the only ones as far as I know.

[1715079718]

1715079718

[1715079718]

1715079718

[1715079718]

1715079718

[wtfman]

thx 4 info.


mm works really smooth so far, unfortunately the stale rate has dramatically increased from 0.3% to 3.3%. Do you only deem shares as valid when BTC + NMC results are up2date results? Only main chain would be enough imo if true.

[shads]

mm works really smooth so far, unfortunately the stale rate has dramatically increased from 0.3% to 3.3%. Do you only deem shares as valid when BTC + NMC results are up2date results? Only main chain would be enough imo if true.

I was just about to ask about this.  Is  anyone else seeing higher stale rates?  From what I've heard it's only cgminer clients that are getting it.  Any info people can report back will be helpful.

There's a numbers of issues at play here I think...

1/ There is a fundamental design clash between longpolling and merged mining.  Basically it works like this:  When there is a double block change (i.e. btc and nmc solved by one solution) the daemons won't see the new block at exactly the same time.  From what I've seen a delay of around a 1-2 seconds is not unusual.  What happens then is that one chain advances to the next block.  PSJ checks the other chains to see if they've updated, sees they haven't so starts sending out LPs.  Before the miner receives the LP and established a new LP connection the second chain advances so PSJ starts sending out another batch of LP's.  Those miners who haven't got their new LP connection registered in time miss out. 

2/ cgminer may be a bit too clever for it's own good.  It does it's own checks on whether work is valid and I presume it uses prev_block_hash to check or maybe the X-Blocknum header.  If an NMC block is found but it isn't a BTC solution then the pseudo block number will advance but the X-Blocknum header (which is for BTC block in psj) and the prev_block_hash won't change.  So cgminer may think it's the same block and carry on using it's cached work.

3/ The rules for accepting a 'partially stale' share are not defined.  If NMC block advances the previous work can still produce a valid BTC block although it will be rejected by the NMC daemon.  The default behaviour is if it's not the current block (block in psj is now defined as a pseudoblocknum which is sum of all chain's blocknums) reject it as stale.  This may have to change but I need some consensus from pool ops on the what the correct behaviour should be.  I'm not going to code every option under the sun coz I'm little bit sick of merged mining TBH.

4/ The code for detecting and switching to new blocks is still pretty raw.  It's possible the 'fireBlockChange' sequence is happening more often that it should which would put valid shares into the stale list.  I don't know if this is the case but the solution 3/ contains the fix for this issue if it exists...

[jamesg]

Is  anyone else seeing higher stale rates? 

I am running my equipment on yourbtc.net which is now running psj with merged mining and the stales are higher on phoenix 1.6.2 also. I am running at about 2% stales.

[shads]

what sort of rates were you seeing previously?

[jamesg]

what sort of rates were you seeing previously?

On yourbtc.net my stales rate was around .15% before merged mining.

[shads]

Thanks for the feedback... keep it coming...

ok I've just uploaded the latest revision .08.  Please note the base distribution was updated at .07 to apply .08 you will need to start with .07 and apply the contents of .08 mini-binary over the top.

This contains a number of changes that I hope will address most of the 4 issues I highlighted above...

- support for partially stale shares.  Where a single chain has advanced to the next block old work may still be valid for the other blocks.  This patch checks that the work is valid for at least one block giving three possible outcomes.  accepted, accepted partial-stale, stale

- longpoll passthru.  To address the double longpoll issue. There is a fundamental design clash between longpolling and merged mining.  Basically it works like this:  When there is a double block change (i.e. btc and nmc solved by one solution) the daemons won't see the new block at exactly the same time.  From what I've seen a delay of around a 1-2 seconds is not unusual.  What happens then is that one chain advances to the next block.  PSJ checks the other chains to see if they've updated, sees they haven't so starts sending out LPs.  Before the miner receives the LP and establishes a new LP connection the second chain advances so PSJ starts sending out another batch of LP's.  Those miners who haven't got their new LP connection registered in time miss out so continue working on the old block for as long as they would normally (probably about a minute).

This patch addresses this by setting a longpoll passthru period.  Where two block changes happen within a specified period (10 seconds).  After the 2nd block change and until 10 seconds after the 1st block change has passed, any longpolls received will have a short expiry of 1 second after which they'll return new work to the worker.  This will give any slow miners a few seconds to get their longpoll in and learn that there's a 2nd new block to work on. The reason for the 1 second delay is to prevent longpoll spam.  Most (but not all) miners will immediately send another LP request as soon as they get a response.  With 0 delay this sets up an LP spam loop which thrashes the server and causes high CPU usage and low response times.  They will still spam 1/sec until the passthru period is over which is why this is called a workaround rather than a fix.  There's really no way to fix this issue completely except to ditch either longpolling or merged mining alltogether.   

[shads]

2/ cgminer may be a bit too clever for it's own good.  It does it's own checks on whether work is valid and I presume it uses prev_block_hash to check or maybe the X-Blocknum header.  If an NMC block is found but it isn't a BTC solution then the pseudo block number will advance but the X-Blocknum header (which is for BTC block in psj) and the prev_block_hash won't change.  So cgminer may think it's the same block and carry on using it's cached work.

Ive just been looking through the cgminer source code and it looks to me like it does use prev_block_hash as an indicator of whether the block has changed.  I'm happy to stand corrected but if that's the case it's a fairly serious problem.  It means whenever an aux chain finds a new block but the parent chain doesn't that cgminer will ignore the new work (even if it gets it via longpoll).  I don't know how many other miners do this but pool ops should start looking into this.

With 'partial-stale' shares being accepted it effectively means the miner will get credited for both chains while only working on one.  Unless shares are actually assigned a 'credit value per chain' the choice is between crediting these partial shares or disallowing them unless the share is valid for all chains.  The 'credit value per chain' option will require and extra database column which will break pushpool compatibility completely (it had to happen sometime) and require all pool ops to make some mods in how they are calculating worker shares.

Need feedback from pool ops on how to deal with this.  This is a particularly significant issue for PPS pools where you'd end up paying out a full share where only part of the work is valid.

[DavinciJ15]

My suggestions to you had two parts be ether way the a partial share should tell the miner that it was accepted (IMO).

1) Insert into the reason column "partial-share" or "partial-stale", this allows the pool op to allow or deny the share as valid share an option in PSJ acceptPartialShares=true and reasonForPartial="" should be in the properties file.

2) Add an optional column called ChainMask where the bit marked block chain is placed into the column.  For example if you have 3 chains the value of Parent is 1, second is 2 and third is 4, if the share is valid for chain id 1 and 2 the value in the mask column is 3, if the share is valid for 1 and 4 the value in the mask column is 5 and all of them would be 7. 

The only issue with 2 is as you add chains the combo's are a factorial of the chains.

That's my 2 BTC cents.

I would like to see other ideas.


Davinci

[shads]

My suggestions to you had two parts be ether way the a partial share should tell the miner that it was accepted (IMO).

1) Insert into the reason column "partial-share" or "partial-stale", this allows the pool op to allow or deny the share as valid share an option in PSJ acceptPartialShares=true and reasonForPartial="" should be in the properties file.

2) Add an optional column called ChainMask where the bit marked block chain is placed into the column.  For example if you have 3 chains the value of Parent is 1, second is 2 and third is 4, if the share is valid for chain id 1 and 2 the value in the mask column is 3, if the share is valid for 1 and 4 the value in the mask column is 5 and all of them would be 7. 

The only issue with 2 is as you add chains the combo's are a factorial of the chains.

That's my 2 BTC cents.

I would like to see other ideas.


Davinci

your suggestion 1 is how the current implementation works.  A normal accept share will have our_result=true, reason=null.  A partial-share will have our_result=true, reason=partial-stale

re: 2.... Perhaps a bit mask would be an option.  Though this is limited and any combo value field tends to make efficient db queries difficult to achieve.

[Keninishna]

how often does a single share solve a nmc and btc block?

[shads]

if you mean a single share from yr own pool... btc diff / nmc diff ~= number of nmc only shares per btc+nmc share

If you mean across the whole network that's a bit more complicated since you've got to account for non-merged miners.

[urstroyer]

What do you think of adding a new table for all additional mm chains like this:



Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

[DavinciJ15]

What do you think of adding a new table for all additional mm chains like this:

Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I like large growth potential so that it does not need to be redone when it's time to grow.

[urstroyer]

What do you think of adding a new table for all additional mm chains like this:

Keep the shares table clean and main chain (btc) only. That counts for all columns.

Add a new table for all merged chains and only enter a new row under 2 conditions:
- merged chain got our_result = 0
- merged chain got upstream_result = 1

In case a share is valid for all chains, don't even add a row to the shares_merged table.

Let me know what you think about that?

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I like large growth potential so that it does not need to be redone when it's time to grow.

I already added a third merged chain "stifflersmomchain" to the scenario.

We only need this table, you can query the db and set where statement to the desired chain on the column "chain".
Plus we already set the name of the merged chain into the psj property file (source.local.1.merged.chains=namecoin).

It's even possible to set an int instead of a string for the chain column, in case we wanna use the id (source.local.1.merged.namecoin.chainid=1).

[shads]

Doubling up the shares may be simple to search but a problem if you start adding more than one other block chain.  May I suggest a new column, repurposing of a current column, or a master/detail table (shares/chain) are the best options.

I have to agree.  Writing shares to DB is already one the biggest bottlenecks in psj.   Doubling or tripling the number of rows written is definately not ideal, likewise with using a seperate table with a many-to-one relationship.

I'm wondering if perhaps a floating point share value column should be added.  There would need to be an interface so the pool could dynamically update the value of each chain.  The value field is then the sum of the solved chain's values.  The disadvantage of this is that values are changed frequently then it's retrospectively difficult to go back and try to work which chains each share solved for.

The remaining options are a single consolidated column like bitfield flags or a boolean column per chain.  I'm not sure what factorial planet davinci was on but the way you normally do it is allocate each bit to a flag.  so 1 = bitcoin, 2=namecoin, 4=scamcoin.  Then each is easy to extract with a simple logical &.

If you don't want to record which chains are solved you have a simple choice between accept or reject partials.  Accepting them provides the miner no incentive to fix any latency problems and hurts the network overall, not mention costing PPS operators a pretty penny.  Reject is going to fill yr inboxes with complaints (which I personally think you should be redirecting to the namecoin forum).

[DavinciJ15]

The remaining options are a single consolidated column like bitfield flags or a boolean column per chain.  I'm not sure what factorial planet davinci was on but the way you normally do it is allocate each bit to a flag.  so 1 = bitcoin, 2=namecoin, 4=scamcoin.  Then each is easy to extract with a simple logical &.

That was my original suggestion you shot down as too complex! Jack bean stalk!   

what i did not know is you could do this in SQL...

Code:

  select * from shares where reason IS NULL and chainmask & 2;

instead of this...

Code:

  select * from shares where reason IS NULL and chainmask in (2, 3, 6, 7, 10, 11, 14, 15);  

So I was on the SQL ignorance planet!  My problem is I don't think of SQL as a computer language, I think of it as a bad hack for accessing data and thus forget how robust it is.

OH well.  Learn something new everyday.

Anyhow...
I say having more than 32 block chains is crazy so unsigned INT is the way to go.

Thus my vote is for a bit wise column!

Davinci

[shads]

If anyone's downloaded .08 and found an odd looking file in there:  bitcoin-poolserverj-0.0.2-SNAPSHOT.jar

That's actually poolserverj.jar which I forgot to rename.  It should be in the bin directory and *no where else*.  I've uploaded a new version with the correct name. 

[shads]

If anyone has some spare time to play it would be really helpful if someone could download this version and run it for a while then send me the log:
http://poolserverj.org/dist/mm-mini-binary.09.debug.tar.gz

It looks like stales have been dramatically reduced but for some reason cgminer is sending back a few works that are registering as unknown.  The above version is identical to .08 but has a bit of extra logging in it that should help track down the source of the problem.

edit: updated post to point to .09 which has a minor fix for issue where in some cases partial-stales were treated as full shares.

[Keninishna]

im getting a 404 error http://poolserverj.org/dist/mm-mini-binary.09.tar.gz