Bitcoin Forum
December 07, 2016, 08:49:20 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 ... 226 »
  Print  
Author Topic: [1200 TH] EMC: 0 Fee DGM. Anonymous PPS. US & EU servers. No Registration!  (Read 461445 times)
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
July 31, 2011, 09:17:46 PM
 #481

Simeonk83: Yep, I'm working on that.  The problem is to get an accurate estimate if the round ended right then, the amount of SQL query time is fairly substantial (has to add up all the rows) and it's not really feasible to do it on an interval on a per user basis, either... so I'm looking at the best way to handle it that puts the least load on the server.  Might be able to get it done today, I'm hoping.

GenTarkin: We are actually doing ok, running right at or just under difficulty which is good.  If you look at the blocks over all in the pool we are under difficulty by a decent margin so we are + luck as a whole.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481100560
Hero Member
*
Offline Offline

Posts: 1481100560

View Profile Personal Message (Offline)

Ignore
1481100560
Reply with quote  #2

1481100560
Report to moderator
1481100560
Hero Member
*
Offline Offline

Posts: 1481100560

View Profile Personal Message (Offline)

Ignore
1481100560
Reply with quote  #2

1481100560
Report to moderator
1481100560
Hero Member
*
Offline Offline

Posts: 1481100560

View Profile Personal Message (Offline)

Ignore
1481100560
Reply with quote  #2

1481100560
Report to moderator
simonk83
Hero Member
*****
Offline Offline

Activity: 896


View Profile
July 31, 2011, 11:19:40 PM
 #482

Simeonk83: Yep, I'm working on that.  The problem is to get an accurate estimate if the round ended right then, the amount of SQL query time is fairly substantial (has to add up all the rows) and it's not really feasible to do it on an interval on a per user basis, either... so I'm looking at the best way to handle it that puts the least load on the server.  Might be able to get it done today, I'm hoping.



No worries, I have no doubt it's not easy Smiley     What if you made it only update on request?   In that someone has to physically press a button to update the estimation.   That way the queries only have to run when requested, rather than every x minutes?  Not sure if that helps Cheesy
PoulGrym
Member
**
Offline Offline

Activity: 83


View Profile
August 01, 2011, 02:45:32 AM
 #483

That'd be a good solution. Then also limit to one check every 10 min.. not to get flooded with req's all the time..

If you found my post helpful, use my tip jar!
BTC: 1Q4um62DJ8kBRMzQ4VQqG6W7eLoPNfx6zn
NODE: 11993447274130959091 NXT: MINT:
rearwheels
Full Member
***
Offline Offline

Activity: 238


View Profile
August 01, 2011, 03:43:33 AM
 #484

Simeonk83: Yep, I'm working on that.  The problem is to get an accurate estimate if the round ended right then, the amount of SQL query time is fairly substantial (has to add up all the rows) and it's not really feasible to do it on an interval on a per user basis, either... so I'm looking at the best way to handle it that puts the least load on the server.  Might be able to get it done today, I'm hoping.



No worries, I have no doubt it's not easy Smiley     What if you made it only update on request?   In that someone has to physically press a button to update the estimation.   That way the queries only have to run when requested, rather than every x minutes?  Not sure if that helps Cheesy

I believe the SQL will take a fair bit of time to run, so if it is on-demand, after clicking, you will have to wait for a while before the information is displayed.

Secondly, if someone clicks on the link and the sql starts, but the user clicks to goto another page, the SQL will continue to run until it is completed or times out, replies to the web server which then does not display the information so the SQL processing time is wasted (similar to not having long poll).

Monitor multiple pools - www.btc-poolwatch.com
(forum)

If it's useful, please consider donating: 18bvrJ9Kouxwv8g4tLUemsv1GuxgGk8Kn7
(Pls PM me so I can thank you.)
simonk83
Hero Member
*****
Offline Offline

Activity: 896


View Profile
August 01, 2011, 03:45:40 AM
 #485

Fair enough, forget that then Wink
1bitc0inplz
Member
**
Offline Offline

Activity: 112


View Profile
August 01, 2011, 03:53:59 AM
 #486

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.

Mine @ http://pool.bitp.it - No fees, virtually 0 stales, what's not to love!
Chat with us @ #bitp.it on irc.freenode.net
Learn more about our pool @ http://forum.bitcoin.org/index.php?topic=12181.0
simonk83
Hero Member
*****
Offline Offline

Activity: 896


View Profile
August 01, 2011, 03:55:38 AM
 #487

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.


What have you switched to (and why)?
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
August 01, 2011, 03:56:27 AM
 #488

Can you elaborate on that?  And doing it correctly would be implementing it in SQL or in the case of EMC, most of it is in done on the getwork server.  Doing it in PHP is the absolute worst way to handle it.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
1bitc0inplz
Member
**
Offline Offline

Activity: 112


View Profile
August 01, 2011, 05:34:06 AM
 #489

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.


What have you switched to (and why)?

We switched to ESMPPS, and the reason (frankly) was that we disliked the increased variance of scoring systems.

Can you elaborate on that?  And doing it correctly would be implementing it in SQL or in the case of EMC, most of it is in done on the getwork server.  Doing it in PHP is the absolute worst way to handle it.

Since we wrote our own pushpool replacement, we did all the scoring calculations there. You are correct, PHP wouldn't be the place to do them. Neither is SQL though. To do it right you'll want to do it in your pool backend, which is pushpool I'm assuming? You'll want to keep the shares in memory, and do your scoring computations off of the in-memory copy.

Mine @ http://pool.bitp.it - No fees, virtually 0 stales, what's not to love!
Chat with us @ #bitp.it on irc.freenode.net
Learn more about our pool @ http://forum.bitcoin.org/index.php?topic=12181.0
GenTarkin
Legendary
*
Offline Offline

Activity: 1918


View Profile
August 01, 2011, 06:43:36 AM
 #490

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.


What have you switched to (and why)?

We switched to ESMPPS, and the reason (frankly) was that we disliked the increased variance of scoring systems.

Can you elaborate on that?  And doing it correctly would be implementing it in SQL or in the case of EMC, most of it is in done on the getwork server.  Doing it in PHP is the absolute worst way to handle it.

Since we wrote our own pushpool replacement, we did all the scoring calculations there. You are correct, PHP wouldn't be the place to do them. Neither is SQL though. To do it right you'll want to do it in your pool backend, which is pushpool I'm assuming? You'll want to keep the shares in memory, and do your scoring computations off of the in-memory copy.

Dude, do you run bitp.it? Cuz, thats similar to what they have. I so want a PPS type payout from emc....

GenTarkin's MOD Kncminer Titan custom firmware! v1.0.4! <--- CLICK HERE
Donations: bitcoin- 1Px71mWNQNKW19xuARqrmnbcem1dXqJ3At || litecoin- LYXrLis3ik6TRn8tdvzAyJ264DRvwYVeEw
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 1890



View Profile WWW
August 01, 2011, 08:20:44 AM
 #491

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.
What have you switched to (and why)?
We switched to ESMPPS, and the reason (frankly) was that we disliked the increased variance of scoring systems.
Did you make any attempt to adjust the parameters?

Anyway, you could have used PPLNS which has very little variance and isn't broken.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
1bitc0inplz
Member
**
Offline Offline

Activity: 112


View Profile
August 01, 2011, 11:20:54 AM
 #492

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.
What have you switched to (and why)?
We switched to ESMPPS, and the reason (frankly) was that we disliked the increased variance of scoring systems.
Did you make any attempt to adjust the parameters?

Anyway, you could have used PPLNS which has very little variance and isn't broken.

Yes, we did attempt to adjust the parameters. But the idea of giving the pool operator variance also wasn't for us. No offense, but I think you know very little about the specifics of ESMPPS but you seem very determined to think that it is broken. I, frankly, think you are basing your assumptions on ESMPPS from what you think you know about SMPPS.

But, that is besides the point because your bias to your own scoring method is quite clear.

I was just simply trying to point out to Inaba that it was possible to get real-time stats from your scoring system. I feared that speaking in this thread might provoke you, so I will cease offering my assistance.

Mine @ http://pool.bitp.it - No fees, virtually 0 stales, what's not to love!
Chat with us @ #bitp.it on irc.freenode.net
Learn more about our pool @ http://forum.bitcoin.org/index.php?topic=12181.0
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
August 01, 2011, 11:57:06 AM
 #493

Since we wrote our own pushpool replacement, we did all the scoring calculations there. You are correct, PHP wouldn't be the place to do them. Neither is SQL though. To do it right you'll want to do it in your pool backend, which is pushpool I'm assuming? You'll want to keep the shares in memory, and do your scoring computations off of the in-memory copy.

I'm a little unclear on how you'd achieve keeping the scoring information in memory in a long round without taking up an extraordinary amount of memory?  But regardless, how do you pass that information off to a web browser looking for it?  Additionally, how do you handle (or do you not) multiple getwork servers?  Each one would have differing information.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 1890



View Profile WWW
August 01, 2011, 12:39:22 PM
 #494

When we used to use Meni's scoring system, we were doing all the estimated in real-time. If implemented correctly (e.g., not doing the math in SQL) you guys shouldn't have a problem getting an estimate that is real-time as well.
What have you switched to (and why)?
We switched to ESMPPS, and the reason (frankly) was that we disliked the increased variance of scoring systems.
Did you make any attempt to adjust the parameters?

Anyway, you could have used PPLNS which has very little variance and isn't broken.

Yes, we did attempt to adjust the parameters. But the idea of giving the pool operator variance also wasn't for us. No offense, but I think you know very little about the specifics of ESMPPS but you seem very determined to think that it is broken. I, frankly, think you are basing your assumptions on ESMPPS from what you think you know about SMPPS.

But, that is besides the point because your bias to your own scoring method is quite clear.

I was just simply trying to point out to Inaba that it was possible to get real-time stats from your scoring system. I feared that speaking in this thread might provoke you, so I will cease offering my assistance.
I apologize for calling ESMPPS "broken" which was too confrontational. I resent your accusations. If my advice isn't welcome I'll stop wasting my time.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
August 01, 2011, 01:24:51 PM
 #495

I read the linked post in regards to how ESMPPS works and I'm a little confused - it seems like unless you have a large run of good luck, the pool will always run in the red and be behind in paying out shares?  As time goes on, the pool would go further and further into the red.  Does ESMPPS count on a run of good luck to even things out?


If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
GenTarkin
Legendary
*
Offline Offline

Activity: 1918


View Profile
August 01, 2011, 10:24:25 PM
 #496

grr another long block ... lol... hows the PPS stuff comin inaba? =)

GenTarkin's MOD Kncminer Titan custom firmware! v1.0.4! <--- CLICK HERE
Donations: bitcoin- 1Px71mWNQNKW19xuARqrmnbcem1dXqJ3At || litecoin- LYXrLis3ik6TRn8tdvzAyJ264DRvwYVeEw
1bitc0inplz
Member
**
Offline Offline

Activity: 112


View Profile
August 02, 2011, 12:42:26 AM
 #497

Since we wrote our own pushpool replacement, we did all the scoring calculations there. You are correct, PHP wouldn't be the place to do them. Neither is SQL though. To do it right you'll want to do it in your pool backend, which is pushpool I'm assuming? You'll want to keep the shares in memory, and do your scoring computations off of the in-memory copy.

I'm a little unclear on how you'd achieve keeping the scoring information in memory in a long round without taking up an extraordinary amount of memory?  But regardless, how do you pass that information off to a web browser looking for it?  Additionally, how do you handle (or do you not) multiple getwork servers?  Each one would have differing information.

I do not know much about your backend, but it sounds like we are two entirely different architectures. On my pool we use Redis (a no-SQL datastore) and Node.js. Additionally we have a single "getwork server". So, since Redis stores everything in memory anyway, we were able to really quickly re-perform the scoring algorithm on every share submission. Unless you centralize your share data in someplace other than SQL, I do not think our solution will be easy for you.

I am curious, why do you have multiple servers? Is it to allow for users to be closer to a server (e.g., geographic distribution), or are you doing this for performance reasons? If at all possible try and consolidate your getworks to a single server, and I think that will open up more possibilities to you.

I read the linked post in regards to how ESMPPS works and I'm a little confused - it seems like unless you have a large run of good luck, the pool will always run in the red and be behind in paying out shares?  As time goes on, the pool would go further and further into the red.  Does ESMPPS count on a run of good luck to even things out?

I am not aware of any single post that fully describes ESMPPS, unfortunately. I'd love to write one, but I've just been pretty swamped lately.

ESMPPS does not depend on luck to even things out. In fact, that is one of the key differences between it and SMPPS. SMPPS pays out older shares first. So, if the pool gets behind on payment, then it neglects newer shares to pay back older shares. This creates a problem where new miners may have to wait several rounds without any payment before receiving any of their earnings. Obviously nobody wants to work for "free", so it would be a very disastrous thing for a SMPPS pool to go red.

ESMPPS solves this problem by putting all shares into "buckets", allocated by their percent paid.  A bucket is never paid more (percentage wise) than the least paid bucket. This allows for all shares to be paid the maximum amount possible, while not favoring older or new miners.

Take the scenario where a pool is always lucky. The pool would therefore always have a positive buffer, and therefore pay 100% of what a 0% "true" PPS would. However, you and I both realize two problems with this scenario. A, no pool will ever be lucky 100% of the time. And B, no "true" PPS can sustain a 0% fee.

So, let's look a a difference scenario. Say you pay out on invalid blocks, and have a 0.5% block withholder rate (e.g., 0.5% of your pools hashing power is "attacking" you by withholding any block solutions they may find). Assume the probability of a block becoming invalid is ~1%. Also, assume that "luck" nets out to 0 over a long period of time.

In this scenario, you will eventually reach a future where no bucket is paid past the 98.5% bucket. However, unlike *MPPS systems old and new shares are paid immediately up to the 98.5% range. This underpayment is remembered by the system, and if the buffer goes positive, the backpay is made up. Interestingly enough, even in this scenario the ESMPPS pool still behaves "normal" and outperforms any true PPS pool, provided that true PPS pool charges more than a 1.5% fee.

IMHO it's a pretty interesting dynamic, a pool that "self regulates" itself. This is, ideally, what a true PPS pool tries to accomplish when the pool operator charges a fee. However, in the case of ESMPPS it set by the probabilities of the situation itself, and if the situation takes a turn for the better then everybody wins. If the situation stays the same, it still outperforms true PPS.

Mine @ http://pool.bitp.it - No fees, virtually 0 stales, what's not to love!
Chat with us @ #bitp.it on irc.freenode.net
Learn more about our pool @ http://forum.bitcoin.org/index.php?topic=12181.0
rearwheels
Full Member
***
Offline Offline

Activity: 238


View Profile
August 02, 2011, 02:33:24 AM
 #498


block stats were updated really fast this time round.

Monitor multiple pools - www.btc-poolwatch.com
(forum)

If it's useful, please consider donating: 18bvrJ9Kouxwv8g4tLUemsv1GuxgGk8Kn7
(Pls PM me so I can thank you.)
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
August 02, 2011, 03:17:18 PM
 #499

I do not know much about your backend, but it sounds like we are two entirely different architectures. On my pool we use Redis (a no-SQL datastore) and Node.js. Additionally we have a single "getwork server". So, since Redis stores everything in memory anyway, we were able to really quickly re-perform the scoring algorithm on every share submission. Unless you centralize your share data in someplace other than SQL, I do not think our solution will be easy for you.

Hmm, I'm not familiar with Redis - so are you saying you have direct access to the memory that your getwork backend is using from your webserver to gather those statistics?  That's mighty handy if so.

Quote
I am curious, why do you have multiple servers? Is it to allow for users to be closer to a server (e.g., geographic distribution), or are you doing this for performance reasons? If at all possible try and consolidate your getworks to a single server, and I think that will open up more possibilities to you.

I could easily consolidate to a single server, but you are correct in your geographical assumption.  There is simply no way to run an efficient server over oceanic links, the latency is to great.  Western Europe is at the very limit of acceptability, often falling into unacceptable latency, anything further east and it becomes untenable.  Thus multiple servers.

Quote
I am not aware of any single post that fully describes ESMPPS, unfortunately. I'd love to write one, but I've just been pretty swamped lately.

In this scenario, you will eventually reach a future where no bucket is paid past the 98.5% bucket. However, unlike *MPPS systems old and new shares are paid immediately up to the 98.5% range. This underpayment is remembered by the system, and if the buffer goes positive, the backpay is made up. Interestingly enough, even in this scenario the ESMPPS pool still behaves "normal" and outperforms any true PPS pool, provided that true PPS pool charges more than a 1.5% fee.

IMHO it's a pretty interesting dynamic, a pool that "self regulates" itself. This is, ideally, what a true PPS pool tries to accomplish when the pool operator charges a fee. However, in the case of ESMPPS it set by the probabilities of the situation itself, and if the situation takes a turn for the better then everybody wins. If the situation stays the same, it still outperforms true PPS.

Let me know if you do write something more definitive on it, I would be very interested in it.  Have you done any analysis on the timeframe for a pool to self regulate?  It seems that it might take a very long time to reach a steady state if you have a particularly bad run of blocks without a balancing good run extreme swing.  I can see how it would outperform a pure PPS, certainly, but it seems theres a lot of risk to the users, or operator if the operator wishes to shoulder that burden by paying out into the negative.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
August 02, 2011, 03:18:21 PM
 #500

grr another long block ... lol... hows the PPS stuff comin inaba? =)

I think I've come up with a good solution, but it's fairly complicated.  I need to complete a tenative design and I'll roll out a test server once I have something concrete in place. 

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 ... 226 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!