[1200 TH] EMC: 0 Fee DGM. Anonymous PPS. US & EU servers. No Registration!

[Meni Rosenfeld]

damn damn damn these rounds from hell!

Does anyone have a feel for how double geometric scoring affects round size based variance?

Out of interest, have you compared your average payout/share to a theoretical prop payout/share for the same round?

If anyone's not sure what I mean then just pick a previous long round and divide payout by your submitted shares. Then divide the Difficulty for that round by the total shares for that round (theoretical proportional pool payout). Compare and contrast. If you can be bothered I'd like to see the results.

You might need to compare a few sequential rounds rather than just one round. To do this, divide your actual total payout by the total shares you submitted. Then (assuming Difficulty didn't change between the rounds you are looking at) compare with (Difficulty*number of rounds)/(total shares in rounds).

It depends on the value of parameter o which every double geometric method pool chooses for his payout model:

o - Cross-round leakage. Increasing o reduces participants' share-based variance but increases maturity time. When o=0 this becomes the geometric method. When o->1 this becomes a variant of PPLNS, with exponential decay instead of 0-1 cutoff (note that "exponential" does not mean "rapid", the decay can be chosen to be slow). For o=1, c must be 0 and r (defined below) can be chosen freely instead of being given by a formula.

https://bitcointalk.org/index.php?topic=39497.msg481864#msg481864

Maybe Inaba will make the parameter o - cross-round leakage public.

If you are interested how round size based variance goes with o = 0.5 send me a pm!

Thanks urstroyer - but I was actually after what miner's experiences had been in the short term. In appendix D of Analysis of bitcoin pooled mining reward systems Meni shows how to calculate the variance and maturity time in the DGS payout system, but that doesn't give me the same insight as actual results do.

So I was interested in seeing what recent historical variance results had been after reading a post from someone complaining about a long round - had anyone seen a reduction in variance compared to a proportional payout?

EMC uses c=0.01, o=0.99 which means there is very little reduction in pool-based variance. In AoBPMRS I've only done Geometric so far, not DGS, and anyway I've only calculated share-based variance - deriving pool-based variance is much harder.

[1713878755]

1713878755

[1713878755]

1713878755

[1713878755]

1713878755

[Inaba]

The stats show your Proportional Differential on each block... basically what you would have recieved under a proportional system as opposed to the DGS.  It's listed in % and not absolute numbers, so just do the math and you'll have your BTC answer. 

Here's a last few blocks from my stats:

118   151835    2011-11-04 09:53:47   17:47:27   1822456   +51.43%   1203462   Valid    191909    5.46598395
(+3.67%)    0.00000000
117   151729    2011-11-03 16:06:20   19:15:37   2113683   +75.63%   1203462   Valid    219798    4.58462896
(-13.41%)    0.00000000
116   151627    2011-11-02 20:50:43   05:04:13   541474   -55.01%   1203462   Valid    54190    5.66392898
(+11.65%)    0.00000000
115   151600    2011-11-02 15:46:29   21:21:47   1948138   +61.88%   1203462   Valid    241323    6.08546998
(-1.78%)    0.00000000
114   151460    2011-11-01 18:24:42   02:32:38   269230   -77.63%   1203462   Valid    26681    5.58464397
(+11.27%)    0.0000000

Under the next to last column, the number in % is the prop differential.  I have miners go up and down on occasion, either I'm working on something, playing a game, etc... so sometimes I get negative differential.  However, block 117, Meni found a bug in  how stats are transferred from one block to the next and that block got recalculated without the "phatom shares" that were following some people, so it's showing lower than actual.  Going back to fix it would result in some people losing BTC, so I opted to leave it in, all other blocks than 117 are accurate.


I'd be happy to provide internal stats for whatever is needed (unless it compromised security or privacy).

[Inaba]

Catfish:

Very interesting about Poolserverj.  I will keep an eye on it.  I would need to do a lot of work on the internals to make it compatible with DGS and the way I have things running, so it's not something that would happen in the next week or two anyway. 

As far as "old" configs.  I use poclbm for all my miners, believe it or not.  None of this fancy phoenix or cgminer stuff!  None of my stuff starts up automatically, either... heh.  It's something that's been on my to-do list since April.  I do like that you have stats output in Pheonix, though... but I try to incorporate as much as that kind of stuff that I want to see into the pool itself.  What do your pages show that I might be able to add to the pool?

I'd be interested in your minimal install as well.  I almost tried to use the headless bitcoin miner guide someone else had posted, but there was a ton of extra, superfluous stuff that I didn't feel it offered a whole lot more than just a straight install.  Although, the last part of the guide did explain how to disable the desktop basically, which accomplishes what you're talking about. I may actually incorporate that into the guide.

AMD does have that questionnaire, but if you look at the actual download link once it starts downloading, you can script the download from AMD.  In fact, don't I already have it scripted from the new guide?

[cengique]

[...]I've trialled the latest (AFAIK) phoenix - 1.62 - and it works but doesn't offer any performance improvements to me, nor does it change the crazy monster-stdout verbosity (I redirect phoenix output to a file and then use that to extract performance statistics, after a week it's over a gigabyte per instance), so I haven't bothered updating globally.

I use screen to keep a tab on my miners, so no need to save them to files.

It won't be for everyone, but after installing my miners on full Ubuntu Desktop installs with all the patches, I was wondering why the hell I needed compiz, apparmor, and a load of fancy GUI eye candy (that *does* affect the 'first' processing GPU). I'm henceforth working on trying to get a happy, compatible miner build that starts with the Ubuntu Minimal installation. I've got a couple of GH/s lying idle whilst I faff about - hashing power that should be working for the pool - so I'm on the case...

Why not use linuxcoin? It's based on a minimal Debian system, and it is really minimal. I do understand that you already made a lot of scripts that work and you just want to share them. I do feel the same way, now that I have things running quite stable. However, like Inaba, I don't have automated startups and I like to watch my rigs manually. Not sure if many people would like that. Too much automation and things go out of hand. Although, it *is* on my to-do list.

Where is Inaba's guide by the way?

[WebMonkey]

Why not use linuxcoin? It's based on a minimal Debian system, and it is really minimal.

linuxcoin won't run on a 32bit system.  that's why i can't use it on 'little rigs'.

it does work fine on 64bit capable systems.

'monkey

[racerguy]

finally, that was brutal.

[cengique]

Time to finish off this pile of 5850s on the floor and get ALL my potential miners mining. Just need to find something to use as hard drive for the last rig... and need some more wood.

Tell me about it.. I still have two 5850s lying on the floor, too. To get them to work, I have to replace the capacitors on one motherboard 

You need wood? You wanna go out chopping some? 

[3phase]

Another MMC refugee here. Just like catfish, I really like medium-size pools. And PPS is boring (no names called) !!!!

Payout model looks very stable and fair. Still trying to work out the math though. I still mine GG from time to time, so I'm not a 24x7 miner, but the model seems to be very fair to such on/off behaviour. No pool hopping advantage, and no quick discounting of shares (like Slush as an example). And the more I mine on a block, my eventual share of the reward increases slowly. All this helps to give a good overalll feeling and help the emotions through the long rounds.

Very hot web design and theme. Really looks modern and "techy". To me at least it's much more closer aesthetically to the term "crypto currency mining" than most other pools .

I am getting an extremely low percent of rejected shares (eu server as primary, us server as 1st backup).
.
Thanks Inaba for your great work. I just increased my Donation % 

[cyberlync]

Hi peeps. Looks like there is more and more MMC refugees coming over, great to see that. Been with this pool since MMC went down, and I cannot complain, stability, features, communication are all excellent.

[Druas]

Block 121 says it took 3 days 17 hours and 47 minutes? Shouldn't it be 6 hours 20 minutes and 24 seconds or so?

[Inaba]

Fixed block 121.  Not sure why it thought the block started on the 3rd, but I think it may have had to do with the fact that was the day Meni found the bug and I probably left a stale share in the DB fixin' things. 

Glad to see you MMC folks!  If you (or anyone) has any suggestions or requests, I'm all ears.  I've got a long list of stuff, but I'm always up for more.  BF3 is cuttin' in the the dev time a little bit, though

[Druas]

At the moment EMC is well over 200 GH/s. I have never seen it break 200 and I hope it lasts/keeps going up. Would be nice to have a little less variance sometimes.

[shads]

I'm also considering switching to Poolserverj when that is a bit more stable.  Although if it ain't broke, I shouldn't fix it.  But I always do.

Hmmm... I'd refer you to BurningToad's comments re: poolserverj and arsbitcoin.com. I'm an old Mac Ach forum guy (as in back in the G4 days) so went with arsbitcoin after MMC closed their doors due to terrorist threats (regardless of arsbitcoin not being formally associated with ars technica, etc.) and it appears that poolserverj requires a fair amount of keeping on top of.

Let's put it this way - I wish them the best of fortune (because BT seems to be dealing with external issues that are of a higher priority) - but my miners were running at an average of 2-3 GH/s - and that's *with* me watching like a hawk and restarting miners when necessary. BT admitted that poolserverj and other software locked up after a while (memory leak sounds normal) and needed restarting, which he didn't always get informed of.

Fair enough if external issues are the main priority. But your system is clearly stable as a rock, and if the flakiness of arsbitcoin was down to poolserverj... I'd rather you not bother unless you are satisfied that it meets your currently (obviously very high) standards.

Catfish I didn't follow the Ars issues in closely because I was flat out with building merged mining support at the time.  But I would like to set a few things straight.

Firstly BT changed to PSJ for precisely those reasons of instability that Ars was experiencing with pushpool.  Whether it improved greatly or not I'm not sure, I know the stale rate dropped dramatically, however, the miner load compared to the server spec was pretty close to the limit for both PSJ and pushpool at the time.  I'll point out that BTC Guild was running double the hashrate on a single server using the same version of PSJ and had no stability or memory leaking issues.  Admittedly it was a higher hardware spec.

I will certainly concede that that the merged mining version of PSJ was wildly unstable for quite a period though it was clearly tagged as pre-alpha.  That was partly due to the fact I was offered a huge bounty for getting it out by a certain deadline so I did things in ways I would have preferred not to.  The new WorkMaker edition has only been released for a few days and despite the expected teething troubles is working in production at high loads with massively reduced resource usage.  It now several times faster than pushpool (and the 0.3.0 version of PSJ) on every metric and more than an order of magnitude faster on some.  Not to mention having some unique features specifically aimed at stability, i.e. able to continue operating seamlessly if the database goes down.

My point is, don't write it off due the Ars experience.  The evidence suggests the problems were not PSJ specific and unfortunately due to being so flat out with merged mining development I was not able to get as involved in working out the Ars issues as I would have liked.  I suspect if BT tried the new version and had the time to iron it out to to a working config he'd find a lot of his problems which are probably related to near-limit CPU usage would go away.  If stability is the prime directive of EMC then it is probably premature to look at PSJ workmaker edition due to it's very recent release and some known bugs. 

All of the issues reported so far with workmaker edition are typical  'dev forgot to account for this weird combo of config settings' none of them are stability related.  In fact PSJ has all sorts of code in it to deal with unexpected circumstance.  e.g. an aux daemon going down means the pool will revert to non-merged-mining mode.  Bad config combos will usually result in the server refusing to start with a warning message rather than allowing it continue into a potential fail scenario.  Failures in internal components are usually detected and trigger a restart of them if they are internal or a failback mode if they are external.  Of course it will go through periods of less than ideal stability during heavy innovation but overall it is designed and geared toward being able to achieve stability and has a good track record of that.

[Inaba]

Sounds reasonable shads.  Thanks for clearing some things up. 

My biggest hurdle with PSJ right now is that it has no support for non-static SQL schemas... not that my schema is dynamic, but it is not a pushpool schema either (and of course not the PSJ schema).  I have lots of internal tricks going on inside the getwork server to handle DGM and I would have to make those same changes to PSJ it would seem... I would really like like to be able to make changes to the SQL without having to muck about in the PSJ sourcecode... that is honestly the biggest issue that is stopping me from trialing PSJ.

[Inaba]

Ok folks, by popular demand, I have enabled the option to covert NMC to BTC.  Conversion happens at the lower of daily rate or last sale. For example, if the last sale of NMC was for .075 BTC per NMC and the daily average was .079, you would convert NMC to BTC at the .075 rate.

With this option, it allows you to effectively take your NMC payouts as NMC, BTC or Paypal.

Paypal fees have been reduced from 10% to 7.5%!  

Skyrim is released today and I can't find anyone to sell me a copy for BTC!  Arrgh... and no, I don't want a region locked copy that I have to use a VPN to play.  Just want a normal steam key or something!

[shads]

Sounds reasonable shads.  Thanks for clearing some things up.  

My biggest hurdle with PSJ right now is that it has no support for non-static SQL schemas... not that my schema is dynamic, but it is not a pushpool schema either (and of course not the PSJ schema).  I have lots of internal tricks going on inside the getwork server to handle DGM and I would have to make those same changes to PSJ it would seem... I would really like like to be able to make changes to the SQL without having to muck about in the PSJ sourcecode... that is honestly the biggest issue that is stopping me from trialing PSJ.

If that's the case the you might be interested in the new column mappings that are used in the latest version.  Previously it was built in a way such that if you wanted data field 9 included in your query you had to 1-8.  Which was very restrictive and inneficient if you happened to only need columns 1,2 and 9.  for example.

Now you can provide a mapping string which allows you to pick and choose data fields and map them to a position in your query.  This has opened up the possibility of adding any number of additional data fields and dynamic calculuations.  The current 'menu' includes:

Code:

###   1 - remote_host - TEXT or VARCHAR
###   2 - username - TEXT or VARCHAR
###   3 - our_result = BOOLEAN or INT(1), if pushpoolCompatibility mode: ENUM(Y,N) or TEXT or VARCHAR
###   4 - upstream_result = BOOLEAN or INT(1), if pushpoolCompatibility mode: ENUM(Y,N) or TEXT or VARCHAR
###   5 - reason - TEXT or VARCHAR
###   6 - solution - TEXT or VARCHAR (length 257)
###   7 - time - TIMESTAMP
###   8 - source - TEXT or VARCHAR
###   9 - block_num - INT or TEXT or VARCHAR
###  10 - prev_block_hash - TEXT or VARCHAR(65) - this is just solution.substring(8, 72) - may be useful for indexing
###  11 - useragent TEXT or VARCHAR
###  12 - unique_part TEXT or VARCHAR(88) - the part of solution that's unique in the block: merkleroot, time, difficulty, nonce
###  13 - nonce TEXT or VARCHAR(8) - nonce in hex
###  14 - hash TEXT or VARCHAR(64)
###  15 - unix_time BIGINTEGER
###
###  15 - 50 reserved

As well as per chain our_result_<chainname>.  You can expect this to expand considerably in the near future.

adding new ones is now trivial.  e.g. there's a number of stats that have been tracked internally by workers for some time, these can now be exposed to the DB engine easily.

Obviously if you want to do additional calculations for DGM inside the server you can't really avoid building from source.  Although it's no longer as daunting as it once was.  I published this step-by-step to getting a build environment ready a couple of weeks ago: http://poolserverj.org/documentation/guide-to-setting-up-poolserverj-in-eclipse-3-7/

I don't know a lot about DGM aside from what it stands for but if you can give a rundown on what you need to do I can probably tell you where the best place in the code would be to achieve it.  The DB API itself has an awful lot of data exposed to it.  In most cases it's a single method that needs modifying.

[cyberlync]

Ok folks, by popular demand, I have enabled the option to covert NMC to BTC.  Conversion happens at the lower of daily rate or last sale. For example, if the last sale of NMC was for .075 BTC per NMC and the daily average was .079, you would convert NMC to BTC at the .075 rate.

With this option, it allows you to effectively take your NMC payouts as NMC, BTC or Paypal.

Thanks for the feature! I was wondering if you could display the NMC exchange rate somewhere, perhaps under where the BTC exchange rate is at the moment.

[Inaba]

Sure, I will add that feature.

I also just added the ability to auto-payout to BTC from NMC.

Just to clarify - there is no fee charged for NMC to BTC conversion.

Shads:

Thanks for the info!  That's good to hear, I will look into PSJ soon, so I may be hitting you up with questions

[btcboston]

I don't think the NMC -> BTC conversion is working properly.  I just requested a payout of 20 NMC, selected pay as BTC.  The NMC was deducted from my account, but I never received anything at my bitcoin wallet and the transaction history on the site shows a record of a transaction for 0.0000 BTC.

My username on the site is barcafan if you want to check into it.

[village.idiot]

I also just added the ability to auto-payout to BTC from NMC.

Just to clarify - there is no fee charged for NMC to BTC conversion.

COOL!

I've been looking for Namecoin.org so I could set up a Namecoin wallet.  Now I don't need one.

Thanks, Inaba