Bitcoin Forum
December 05, 2016, 08:42:43 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 [43] 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 ... 205 »
  Print  
Author Topic: bitHopper: Python Pool Hopper Proxy  (Read 332721 times)
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
July 23, 2011, 08:55:47 AM
 #841

It currently has to have the payout entered manually. with a bitcoind interface it could be automatic but I haven't written it yet.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
1480970563
Hero Member
*
Offline Offline

Posts: 1480970563

View Profile Personal Message (Offline)

Ignore
1480970563
Reply with quote  #2

1480970563
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480970563
Hero Member
*
Offline Offline

Posts: 1480970563

View Profile Personal Message (Offline)

Ignore
1480970563
Reply with quote  #2

1480970563
Report to moderator
burp
Member
**
Offline Offline

Activity: 98


View Profile
July 23, 2011, 10:51:14 AM
 #842

Seems bitHopper loses more and more flexibility. With this config file even more is gone, that is the possibility to write more complicated functionality to get the share counts, which I relied on for some pools I added myself. I will stick to older versions.
Clipse
Hero Member
*****
Offline Offline

Activity: 504


View Profile
July 23, 2011, 10:55:48 AM
 #843

Wouldnt it be possible to allow the config for "less knowledable" users and leaving the pure python code for those who want to tinker more in depth?

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
burp
Member
**
Offline Offline

Activity: 98


View Profile
July 23, 2011, 11:38:11 AM
 #844

My changes:

I don't think regex api_index is necessary if you write proper regular expressions to match, as in my example.
I also added api_method cmd which runs a command in the shell where you can use arbitrary pipe, curl,grep,sed,awk magic.

api_method:cmd
api_cmd:curl -s -k https://pool.bitp.it/leaderboard | … more shell magic that returns a number

api_method:cmd
api_cmd:echo 1234

Code:

diff -rN -u old-bitHopper_darcs/pool.cfg.default new-bitHopper_darcs/pool.cfg.default
--- old-bitHopper_darcs/pool.cfg.default    2011-07-23 13:34:45.675261168 +0200
+++ new-bitHopper_darcs/pool.cfg.default    2011-07-23 13:34:45.679261217 +0200
@@ -51,7 +51,7 @@
 mine_address: eu1.triplemining.com:8344
 api_address:https://www.triplemining.com/stats
 api_method:re
-api_key:-->\d+</td>
+api_key:<td>.*?(\d+)</td>
 api_index:3,-5
 role:mine
 #CHANGE THIS
diff -rN -u old-bitHopper_darcs/pool.py new-bitHopper_darcs/pool.py
--- old-bitHopper_darcs/pool.py 2011-07-23 13:34:45.675261168 +0200
+++ new-bitHopper_darcs/pool.py 2011-07-23 13:34:45.679261217 +0200
@@ -107,15 +107,15 @@

         elif server['api_method'] == 're':
             output = re.search(server['api_key'],response)
-            s,e = server['api_index'].split(',')
-            output = output.group(0)
-            s = int(s)
-            if e == '0' or e =='':
-                output = output[s:]
-            else:
-                output = output[s:int(e)]
-            round_shares = int(output)
+            round_shares = int(output.group(1))
             self.UpdateShares(args,round_shares)
+
+        elif server['api_method'] == 'cmd':
+            import subprocess
+            round_shares = int(subprocess.Popen(server['api_cmd'], shell=True,
+                stdout=subprocess.PIPE).communicate()[0])
+            self.UpdateShares(args,round_shares)
+
         else:
             self.bitHopper.log_msg('Unrecognized api method: ' + str(server))

flower1024
Hero Member
*****
Offline Offline

Activity: 854


luck is just a share away


View Profile
July 23, 2011, 11:43:54 AM
 #845

very nice idea!
what about just placing a command line in pools.conf which gets executed every time round-shares are updated?

that way we could completly decouple bithopper from pools - and even change them without an restart.

shell command could return sth like "poolhashrate,roundshares,false|true" (false|true: is the api lagging)
which could easily be parsed by python
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1946


Poor impulse control.


View Profile WWW
July 23, 2011, 12:27:09 PM
 #846

Anyone else have triplemining broken in the new bitHopper? Getting 'error in api'.

Also I love polmine, great service and good efficiency so far. But the new config broke the regex that ryouiki did. Anyone fix it? The old version (with what I assume is a commented out practice go) is:
Code:
            'polmine':{'shares':default_shares, 'name':'polmine',
    'mine_address':'polmine.pl:8347', 'user':polmine_user, 'pass':polmine_pass,
    'lag':False, 'LP':None,'api_address':'http://polmine.pl/?action=statistics',
    'role':'mine' },

    def polmine_sharesResponse(self, response):
        #info = re.search(r"stkich:</th><td> [0-9]*</td>", response).group(0)[4:-5]
        round_shares = int(re.search(r"stkich:</th><td>[ 0-9]*</td>", response).group(0)[16:-5].replace(' ',''))
        self.UpdateShares('polmine',round_shares)

worked fine, had some error at the start of bitHopper which I paid no attention to but worked.

So can anyone fix Triplemining and Polmine for me?

Cheers!

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
frisco
Full Member
***
Offline Offline

Activity: 176


View Profile
July 23, 2011, 01:38:14 PM
 #847

They changed the format again, new code for triple:

Code:
    def triple_sharesResponse(self, response):
        output = re.search(r"-->([0-9\s]+)</", response)
        if output != None:
            match = output.group(1)
            match = match.replace(' ','')
            round_shares = int(match)
            self.UpdateShares('triple',round_shares)

I don't think this can be done with the new config, so I am still at yesterday's version.

1obSpGygRiuss93u8X36Ex6GYmycgTacZ just in case you feel it
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
July 23, 2011, 01:47:27 PM
 #848

Other scoring methods
Many scoring systems are not statistically analyzed and hence still offer opportunity for hopping. For example, Pay-Per-Last-N-Shares is still quite hoppable (with typical N), as follows. Let's renormalize N by dividing by the difficulty, to give us the normalized last N shares, M. Using the same analysis as above, the speedup can be computed as shown below in Figure 3. PPLNS strongly deviates from proportional only during rounds which pay (per share) below the statistical average. Because the hopper "suffers" from PPLNS only during sub-par rounds, the absolute difference between proportional and PPLNS is small. One can calculated the expectation values of PPLNS shares (as a function of x and M), with the following formula (Wolfram link): exp(x) ( Gamma(0,x) - Gamma(0,M+x)) which is valid for x < M. A maximum speedup (at current difficulty) is still 13x with M=0.5. A quick'n'dirty way to implement PPNLS pools in hopping software is to penalize the PPNLS pool by multiplying round_shares by .43/.21. Slush's pool is also hoppable under the same mechanism, although the curves depend on hashing rate as well. Almost any proportional-based pool will be hoppable at x<.10


Figure 3. Comparison of expectation value from share submission from proportional and PPNLS (M=0.5, see text) payout schemes. Notice the intersection with y=1, where share expectation value is equal to solo mining. These intersections are respectively equal to ~0.21 and ~0.428 for PPNLS and proportional.

Conclusion
I hope this clears up some confusion about expectation values of shares and pool hopping. If anything is unclear feel free to reply or send me a PM.

Although this should REALLY go in a seperate topic... What's your understanding of PPLNS?
I got it like this, that you _always_ consider the last N shares and don't reset at block boundaries. Yes, this means you can get paid twice or more often for a single share but also that you can be not paid at all for some others. I suspect you do in your model reset your shares when a block was found. Is this true?

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
paraipan
Legendary
*
Offline Offline

Activity: 924


Firstbits: 1pirata


View Profile WWW
July 23, 2011, 02:11:53 PM
 #849

nice work streblo, just managed to put my concepts straight thanks. I added you on my future donations list if you don't mind

@c00w kudos to you to man, I was waiting for this to happen coz my python skills really suck and many times I was just breaking the script after editing, it's a more portable now... thanks really , btw I already have you on donation list

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1946


Poor impulse control.


View Profile WWW
July 23, 2011, 02:29:35 PM
 #850

I don't see how his model could include anything from previous rounds - I think it resets at boundaries. That just means the model gives a low bound for speedup/efficiency.

I'd really like some details (that i probably won't follow!) though. Or at least more details about
Quote
PPLNS strongly deviates from proportional only during rounds which pay (per share) below the statistical average. Because the hopper "suffers" from PPLNS only during sub-par rounds, the absolute difference between proportional and PPLNS is small.
because it's tantalising me. I'd like to know why the gamma functions allow us to model the PPLNS behaviour.


Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
bb
Member
**
Offline Offline

Activity: 84


View Profile
July 23, 2011, 03:00:23 PM
 #851

With the newest version I am getting this error:

Code:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1371, in dataReceived
    finishCallback(data[contentLength:])
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1585, in _finishRequestBody
    self.allContentReceived()
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1641, in allContentReceived
    req.requestReceived(command, path, version)
  File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 807, in requestReceived
    self.process()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 125, in process
    self.render(resrc)
  File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 132, in render
    body = resrc.render(self)
  File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 210, in render
    return m(request)
  File "/opt/bitHopper-4/bitHopper.py", line 370, in render_POST
    return bitHopper_Post(request)
  File "/opt/bitHopper-4/bitHopper.py", line 227, in bitHopper_Post
    bithopper_global.log_msg('RPC request [' + rep + "] submitted to " + str(pool_server['name']))
exceptions.TypeError: 'NoneType' object is not subscriptable

When this happened, current in line 212 in bitHopper.py was 'mtred', however, pool_server in line 213 turned out to be None.

The mtred entry in the config file looks ok.
streblo
Full Member
***
Offline Offline

Activity: 165


View Profile
July 23, 2011, 04:40:07 PM
 #852

Very nice work, Streblo!
Thanks, this pool math is really interesting to me because the results are sometimes bizzare!


edit: Streblo!
I love the explanation. And I'm adding ppln.
Sounds good. Remember, the quick'n'dirty method with the factor of ~2 for shares adjustment only works with M=0.5. Though, I (or anybody with the mathematica link) can easily calculate numbers for arbitrary M.


Amazing streblo. Really great stuff.
The thing I'm still unclear on though (and maybe I'm just dense) is:
Based on your analysis, given the situation of hopping two pools having different speeds, should I stay with the faster pool even though it is further from share 1 and then hop to slower later. Or do I hop to pool 2 immediately after it solves a block. It seems to me the downside is getting stuck in a slow pool and missing the spread of shares to other available pools ( even backup)
Assuming you switch pools with no increase stales and you have accurate round_shares information, ALWAYS send shares to the pool with least number of shares (unless, of course, all pools are above x=.428, in which case you should solo mine [read: use a "backup" pool]).


nice work streblo, just managed to put my concepts straight thanks. I added you on my future donations list if you don't mind
Thanks man!



Although this should REALLY go in a seperate topic... What's your understanding of PPLNS?
I got it like this, that you _always_ consider the last N shares and don't reset at block boundaries. Yes, this means you can get paid twice or more often for a single share but also that you can be not paid at all for some others. I suspect you do in your model reset your shares when a block was found. Is this true?
Hmm, should I start another topic? I started a similar thread in Mining->Pools, but that is more for a new payout system which I call normalized Proportional Payout (nPP), which is almost identical to proportional, except it cannot be exploited by hoppers.

I don't see how his model could include anything from previous rounds - I think it resets at boundaries. That just means the model gives a low bound for speedup/efficiency.

I'd really like some details (that i probably won't follow!) though. Or at least more details about
Quote
PPLNS strongly deviates from proportional only during rounds which pay (per share) below the statistical average. Because the hopper "suffers" from PPLNS only during sub-par rounds, the absolute difference between proportional and PPLNS is small.
because it's tantalising me. I'd like to know why the gamma functions allow us to model the PPLNS behaviour.
You are correct, this analysis resets at the pools found block boundaries. I could run Monte Carlo simulations of the PPLNS system, because handling multiple payouts per share (technically, upto N times!) would be quite difficult, I suspect.
Yes, this analysis does include reseting shares after the pool finds a block (not when the system finds a block). Whoops. I don't included: multiple blocks found per share or the decreased payout from including the last N shares (including previous round(s)). These effects work in opposite directions, although I don't know which one is stronger.

PP expectation value was calculated by taking the payout (1/(n+x)) multiplied by the distribution of number of shares (x) required to find a block, and integrating over all possibilities.
n shares are already submitted
the payout after x more shares are submitted (1/(x+n))
probability of a pool solving a block after x shares = exp(-x)
PP = Integrate exp(-x)/(x+n) from x=0 to Infinity = exp(x) E1(x) = exp(x) Gamma(0,y) (this is the incomplete gamma function)

PPLNS (when incorrectly resetting shares after every block)
n shares are already submitted
the payout after x more shares are submitted: 1/(x+n) (x < M)
PPLNS = Integrate exp(-x)/(x+n) from x=0 to M = exp(x) ( Gamma(0,x) - Gamma(0,M+x)) (x < M)


Cheers

1FQoC3zsos22QxZC35fMfP8JteMK9nEQPU
paraipan
Legendary
*
Offline Offline

Activity: 924


Firstbits: 1pirata


View Profile WWW
July 23, 2011, 05:10:30 PM
 #853

I'm trying to add polmine.pl to "pool.cfg" in last version (0dd8833) without success, I think the problem comes from the spaces in page source, just guessing

[polmine]
name: polmine.pl
mine_address: polmine.pl:8347
api_address:https://polmine.pl/?action=statistics
api_method:re
api_key:stkich:</th><td>\d+</td>
api_index:16,-5
role:mine
#CHANGE THIS
#https://polmine.pl/?action=signup
user: x
pass: x

any help would be appreciated



edit: minor changes

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
enmaku
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
July 23, 2011, 05:35:26 PM
 #854

Getting regex failures on triple with Ryouiki's latest. Did they change their stats page or something?

Aexoden
Jr. Member
*
Offline Offline

Activity: 53


View Profile
July 23, 2011, 05:38:51 PM
 #855

The expected value of a share in a (properly implemented) PPLNS pool is exactly 1.0 (relative to solo or PPS).

Once a share is submitted, the fundamental question of its value is: how many blocks will be found before N more shares have been submitted. This can be modeled as a Poisson process with the expected number of blocks equal to N / difficulty. (If N is the difficulty, you'd expect to find one block. If N is twice the difficulty, you'd expect to find two blocks, and so on.) If (for example), N is one-half the difficulty, you'd expect a share to be paid approximately as follows:

  • No payout: ~60.65%
  • 1 payout: ~30.33%
  • 2 payouts: ~7.58%
  • 3 payouts: ~1.26%
  • 4 or more payouts: ~0.18%

Regardless of the value of N or the difficulty, the expected value will sum out to 1, however.

To make it relevant for this thread, PPLNS pools should be used only as backups, and definitely not as hopping targets.

1JrEZbuiK1BBakhtVo9PiMctNCEhtcQAR
frisco
Full Member
***
Offline Offline

Activity: 176


View Profile
July 23, 2011, 05:39:59 PM
 #856

enmaku read my last post

1obSpGygRiuss93u8X36Ex6GYmycgTacZ just in case you feel it
enmaku
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
July 23, 2011, 05:44:40 PM
 #857

enmaku read my last post

Yeah, I see that now. I already fixed it myself, but in a different way - I added one more replace() to pools.py line 241:
Code:
parsed = response.replace('\n', '').replace('\r','').replace('\t','').replace(' ','')

Took care of the extra spaces they added in the middle of the number as well as the space between the </td> and <td>

Glancing over yours I can kind of see how it's working, regex isn't my strong suit so if I can fix it ANYWHERE but the regex I'll gladly do so  Grin

c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
July 23, 2011, 06:43:06 PM
 #858

1)Pool.cfg is less flexible?
Well you can still add methods if you need some custom python code.
see json_ec. So no. Its actually more flexible.

2)bb nothing is happening?
You need to mv pools.cfg.default to pools.cfg
I should probably have a error output telling people that.

3)Cmd api_method?
Awesome. Can I get a copyright release burp?

4)Different triple mining code?
I'm going to test that but that works? Really? What I have now for re works also.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
EskimoBob
Legendary
*
Offline Offline

Activity: 910


Quality Printing Services by Federal Reserve Bank


View Profile
July 23, 2011, 06:45:01 PM
 #859

Can you add http://mining.bitcoin.cz/ to the list?

While reading what I wrote, use the most friendliest and relaxing voice in your head.
BTW, Things in BTC bubble universes are getting ugly....
streblo
Full Member
***
Offline Offline

Activity: 165


View Profile
July 23, 2011, 06:46:46 PM
 #860

The expected value of a share in a (properly implemented) PPLNS pool is exactly 1.0 (relative to solo or PPS).

Once a share is submitted, the fundamental question of its value is: how many blocks will be found before N more shares have been submitted. This can be modeled as a Poisson process with the expected number of blocks equal to N / difficulty. (If N is the difficulty, you'd expect to find one block. If N is twice the difficulty, you'd expect to find two blocks, and so on.) If (for example), N is one-half the difficulty, you'd expect a share to be paid approximately as follows:

  • No payout: ~60.65%
  • 1 payout: ~30.33%
  • 2 payouts: ~7.58%
  • 3 payouts: ~1.26%
  • 4 or more payouts: ~0.18%

Regardless of the value of N or the difficulty, the expected value will sum out to 1, however.

To make it relevant for this thread, PPLNS pools should be used only as backups, and definitely not as hopping targets.
You're completely right, PPLNS isn't hoppable. (Although PPLNS is subject to a share-withholding attack)
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 [43] 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 ... 205 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!