Multipool - the pool mining pool (with source code)

[TeaRex]

NotaNumber (div by zero) just ruin everything! they're contagious and infect everything they contact.

Yeah... like the big red flashing "9.999999 99" that my dad's old TI-59 programmable calculator produced on divide by zero back in the seventies; they also propagated that value and then you'd have to tediously single-step to find the culprit, instead of the code just halting with an error immediately. Nice idea in principle but it can cause no end of headaches.

Can anyone explain what utility is? Efficiency is how many expected shares worth of coin your actual shares get you, but I can't figure out how to relate utility to shares.  And I don't understand what TeaRex means here (no offence mate) since I don't know how you could get an "expected efficiency".

As far as I understand it, utility is "the number of shares your submitted shares (including pending shares) are assumed to be worth by multipool's underlying algorithm", compared to solo mining or a hypothetical proportional pool without fees, and average luck. So basically if you'd have submitted that many shares to a proportional pool at random times, and the pool had been neither lucky nor unlucky, you'd get the same amount of money as Multipool expects you to receive for the shares you actually submitted there. I hope I haven't confused you any further now...

[1714155835]

1714155835

[organofcorti]

NotaNumber (div by zero) just ruin everything! they're contagious and infect everything they contact.

Yeah... like the big red flashing "9.999999 99" that my dad's old TI-59 programmable calculator produced on divide by zero back in the seventies; they also propagated that value and then you'd have to tediously single-step to find the culprit, instead of the code just halting with an error immediately. Nice idea in principle but it can cause no end of headaches.

Can anyone explain what utility is? Efficiency is how many expected shares worth of coin your actual shares get you, but I can't figure out how to relate utility to shares.  And I don't understand what TeaRex means here (no offence mate) since I don't know how you could get an "expected efficiency".

As far as I understand it, utility is "the number of shares your submitted shares (including pending shares) are assumed to be worth by multipool's underlying algorithm", compared to solo mining or a hypothetical proportional pool without fees, and average luck. So basically if you'd have submitted that many shares to a proportional pool at random times, and the pool had been neither lucky nor unlucky, you'd get the same amount of money as Multipool expects you to receive for the shares you actually submitted there. I hope I haven't confused you any further now...

I get NaN's propagating through matrices all the time. Is this is just because I suck at coding, or are NaNs some sort of digital life form? 

Well thx for trying to explain - but 

The reason for my  is that I'd thought that is what "Efficiency" already represents. Eg: atm if you solo 877000 shares/blocks, or do the same at a proportional pool at "luck" multiplier = 1.0, you expect 50 coins. You can check for yourself that efficiency is calculated as (received coins/expected coins), which means it would also be (utilised shares/accepted shares). But efficiency!=utility/shares - again, you can check this with a few of your own.

I prolly have this the wrong way around, and it'd be handy for stats purposes if it relates to eg. the total shares in that block. Then (I hope to) relate results to Raulo's original paper on hopping and not just calculate hopping efficiency but predict best hopping algos for each pool.

Cheers for the help!

[organofcorti]

Is Multipool the "crazy miner on eligius US" who has 68% of the hashrate?

http://forum.bitcoin.org/index.php?topic=6667.msg264031#msg264031

(eligius US has been down for the last week, everyone is on eligius EU)

[techwtf]

Sorry techwtf, I still don't get why you'd leave for losing some shares to btcmine - you get decaying shares at eligius-eu which zero out some rounds. But the overall efficiency for eligius-eu is ok, as is the overall efficiency for the pool.

Can you post your shares (without pending, the top number) and your earned (not collected) for all pools?  Efficiency of 1.086 is not just nice, but means you get 1.086 coins for every coin you'd earn elsewhere (averaged over quite often a loooong period of time).

Total:
Shares   18148
Earned   1.11115
Efficiency   1.074

---

I mean, even the worst case in deepbit(prop), 7-8 millions of shares goes to the 50 BTC(eff ~0.1), shares never become 0 BTC(I know there are invalid blocks, but not the case)...
I really hope, that is just a mistake.

---

Let's compare status of btcmine and multipool.

I'm not sure 132070 goes to which block, but both are confirmed in btcmine.
760   132071   06/20/2011 12:50:42    00:14:48   119823   confirmed
759   132070   06/20/2011 12:00:13    00:50:29   406284   confirmed
132070   Mon Jun 20 12:50:42 2011   162   159.564   0.00000000   0.000   0.00000000

Quite a long round. Zero.
761   132116   06/20/2011 13:05:30    04:53:55   2349995   confirmed
132116   Mon Jun 20 17:59:25 2011   636   619.802   0.00000003   0.000   0.00000000

Another round
766   132174   06/20/2011 21:41:43    02:50:34   1341979   confirmed
132174   Tue Jun 21 00:32:17 2011   246   258.809   0.00000000   0.000   0.00000000

since btcmine provides the status of the latest 25 blocks only... Is those shares really worth Zero?

[TeaRex]

Well thx for trying to explain - but 

The reason for my  is that I'd thought that is what "Efficiency" already represents. Eg: atm if you solo 877000 shares/blocks, or do the same at a proportional pool at "luck" multiplier = 1.0, you expect 50 coins. You can check for yourself that efficiency is calculated as (received coins/expected coins), which means it would also be (utilised shares/accepted shares). But efficiency!=utility/shares - again, you can check this with a few of your own.

I prolly have this the wrong way around, and it'd be handy for stats purposes if it relates to eg. the total shares in that block. Then (I hope to) relate results to Raulo's original paper on hopping and not just calculate hopping efficiency but predict best hopping algos for each pool.

Cheers for the help!

As I see it, utility is related to what the multipool algorithm expects the shares to be worth ahead of time, while efficiency is related to what their real worth turns out to be after the fact, once multipool knows how many coins it will actually collect for them. Confusingly the two are not expressed in the same way, one is expected total worth, while the other is real worth per share (both in relation to the average worth of one share at the current difficulty).

That's what I suggested should be changed, so that you'd have "expected worth per share" and "real worth per share" instead, allowing for a direct comparison. If the algorithm is correct, these numbers should more or less converge over time.

[organofcorti]

As I see it, utility is related to what the multipool algorithm expects the shares to be worth ahead of time, while efficiency is related to what their real worth turns out to be after the fact, once multipool knows how many coins it will actually collect for them. Confusingly the two are not expressed in the same way, one is expected total worth, while the other is real worth per share (both in relation to the average worth of one share at the current difficulty).

That's what I suggested should be changed, so that you'd have "expected worth per share" and "real worth per share" instead, allowing for a direct comparison. If the algorithm is correct, these numbers should more or less converge over time.

Ah, I think I get it. So any idea what it's based on? I see a correlation between efficiency and utility/shares, but it's not linear and I'm not sure what it means.

[Oldminer]

I've been getting a LOT of connection errors and invalids using this pool lately. Switching both my miners back to bitcoin.lc - will revisit once these issues are fixed.

[Coaster]

overall review on 10,000 shares and around .60 btc earned.. I think i started off really well on efficiency and all the solo mining mixed in with score based mining kinda brought that eff down

[organofcorti]

Multipool - any chance of putting deepbit higher up the preference list than BTCMine? Deepbit has an efficiency for me (over 1700 shares) of 1.18, with fees of 3%. BTCMine (22760 shares) on the other hand has an efficiency of 0.735. I'd rather lose 3% from Deepbit than 26.5% from BTCMine!

Great job, btw - this is fascinating stuff, and the data is great fun to play with.

[organofcorti]

overall review on 10,000 shares and around .60 btc earned.. I think i started off really well on efficiency and all the solo mining mixed in with score based mining kinda brought that eff down

10000 shares at a proportional pool would bring expected earnings of 0.57. So you're still ahead by 5%. Variance has a big effect on efficiency until about 40000 shares total when it starts to level off - at about 1.12, for me.

[iopq]

I've been getting a LOT of connection errors and invalids using this pool lately. Switching both my miners back to bitcoin.lc - will revisit once these issues are fixed.

same
also, my efficiency is down to 0.997, while paying 2% fee to the pool and a fee to the pools it mines :/

[organofcorti]

I've been getting a LOT of connection errors and invalids using this pool lately. Switching both my miners back to bitcoin.lc - will revisit once these issues are fixed.

same
also, my efficiency is down to 0.997, while paying 2% fee to the pool and a fee to the pools it mines :/

What is this 2% fee of which you speak?

[iopq]

I've been getting a LOT of connection errors and invalids using this pool lately. Switching both my miners back to bitcoin.lc - will revisit once these issues are fixed.

same
also, my efficiency is down to 0.997, while paying 2% fee to the pool and a fee to the pools it mines :/

What is this 2% fee of which you speak?

the "you got 2.0 efficiency on these shares and 0.6 efficiency on the next batch so you pay 5% on the 2.0 efficiency" fee

[Multipool]

NotaNumber (div by zero) just ruin everything! they're contagious and infect everything they contact.

Oh my, how did those got in the database! Eligius must have been sending out some crazy stats if they managed to get through all the layers of sanity checks. Cleaned up the database and slapped another sanity check, just to be sure. Besides that and btcmine glitches, the database has held up rather well over the past week.

The reason for my  is that I'd thought that is what "Efficiency" already represents. Eg: atm if you solo 877000 shares/blocks, or do the same at a proportional pool at "luck" multiplier = 1.0, you expect 50 coins. You can check for yourself that efficiency is calculated as (received coins/expected coins), which means it would also be (utilised shares/accepted shares). But efficiency!=utility/shares - again, you can check this with a few of your own.

I prolly have this the wrong way around, and it'd be handy for stats purposes if it relates to eg. the total shares in that block. Then (I hope to) relate results to Raulo's original paper on hopping and not just calculate hopping efficiency but predict best hopping algos for each pool.

As I see it, utility is related to what the multipool algorithm expects the shares to be worth ahead of time, while efficiency is related to what their real worth turns out to be after the fact, once multipool knows how many coins it will actually collect for them. Confusingly the two are not expressed in the same way, one is expected total worth, while the other is real worth per share (both in relation to the average worth of one share at the current difficulty).

That's what I suggested should be changed, so that you'd have "expected worth per share" and "real worth per share" instead, allowing for a direct comparison. If the algorithm is correct, these numbers should more or less converge over time.

TeaRex is exactly correct in the explanation. The reason I didn't display utility as "expected per share" to begin with, is that each share has unique utility. The database keeps count of the number of shares submitted by each user to each pool in each round, and also of the sum of the shares' utilities in that round. When a round is rewarded, each user receives a proportion of the round's reward equal to the proportion of the user's total utility against the total utility of all users in that round. To me it is easier to think of expected utility in these terms. Couldn't someone write a greasemonkey script if there really is demand for displaying utility as per share, rather than total?

Is Multipool the "crazy miner on eligius US" who has 68% of the hashrate?

http://forum.bitcoin.org/index.php?topic=6667.msg264031#msg264031

(eligius US has been down for the last week, everyone is on eligius EU)

So what's the deal with eligius? Eligius-us is back up, serving stats and getworks, and accepting shares, but it is "down"? I guess I'll keep it out of rotation while it decides its existential problem.

[organofcorti]

TeaRex is exactly correct in the explanation. The reason I didn't display utility as "expected per share" to begin with, is that each share has unique utility. The database keeps count of the number of shares submitted by each user to each pool in each round, and also of the sum of the shares' utilities in that round. When a round is rewarded, each user receives a proportion of the round's reward equal to the proportion of the user's total utility against the total utility of all users in that round. To me it is easier to think of expected utility in these terms. Couldn't someone write a greasemonkey script if there really is demand for displaying utility as per share, rather than total?

So shares -> (calculations) -> utility (calculations) -> efficiency? ok, got it.

I don't really care about showing utility/shares, but if you could publish the total shares it took the 'victim' pool solve the block our shares are from, that would be awesome. Although prob very hard. Are some of the shares getting grouped together with shares from other blocks?

So what's the deal with eligius? Eligius-us is back up, serving stats and getworks, and accepting shares, but it is "down"? I guess I'll keep it out of rotation while it decides its existential problem.

Yep, Eligius US accepts shares but forwards them on to Eligius EU. Sometimes, though, it's up and hashes for a little bit. But Eligius US is down for all intents:

http://eligius.st/~artefact2/

[iopq]

still getting RPC errors

[burp]

Why do I see no earnings for btcguild and for the other pools just after a very long time? Are you taking the funds from the pools manually?

[Mike 71]

What changed on wednesday in your utility calculations? mtred did really fine till Wednesday, then we started to just send 1 or 2 shares per block.

And what happend to all that eligius shares? That really hurt.

About profit, even if my stats look fine (efficiency 1.1), i havnt earend what i should have earned with standardvalue * 1.1. Not even 90 Percent of it. Maybe all those idle and RPC Probs one reason for that.

[superweb]

Trying for few days few with my one 6870

[organofcorti]

What changed on wednesday in your utility calculations? mtred did really fine till Wednesday, then we started to just send 1 or 2 shares per block.

And what happend to all that eligius shares? That really hurt.

About profit, even if my stats look fine (efficiency 1.1), i havnt earend what i should have earned with standardvalue * 1.1. Not even 90 Percent of it. Maybe all those idle and RPC Probs one reason for that.

Are you using shares+pending shares, or just shares? Mine are spot on using

earned = confirmed shares*Efficiency*50/877000, for 70000 confirmed shares