Bitcoin Forum
November 11, 2024, 06:38:56 PM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 [42] 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 ... 205 »
  Print  
Author Topic: bitHopper: Python Pool Hopper Proxy  (Read 355786 times)
Xer
Member
**
Offline Offline

Activity: 99
Merit: 10


View Profile
July 22, 2011, 04:56:36 PM
Last edit: July 22, 2011, 05:11:47 PM by Xer
 #821

X8s.de now swapped to PPLNS

"Found block for Round 30 18 minutes ago · reply · retweet · favorite"

"After finding the next block we'll switch to PPLNS 5 hours ago · reply · retweet · favorite"

As for ryouiki's latest hopper

"Checking Database
Traceback (most recent call last):
  File "*\ryouiki-bitHopper-743b318\bitHopper.py", line 37
0, in <module>
    main()
  File "*\ryouiki-bitHopper-743b318\bitHopper.py", line 35
0, in main
    database.check_database(servers)
  File "*\ryouiki-bitHopper-743b318\database.py", line 22,
 in check_database
    curs.execute(sql)
sqlite3.OperationalError: no such column: accept
"
Edit. was a corrupted database.
ed64
Newbie
*
Offline Offline

Activity: 42
Merit: 0


View Profile
July 22, 2011, 05:32:47 PM
 #822

With time slicing and penalty around 3.5, hopping PPLNS can still be effective...
Clipse
Hero Member
*****
Offline Offline

Activity: 504
Merit: 502


View Profile
July 22, 2011, 05:39:17 PM
 #823

Speaking of timeslicing, flowers when will you merge your code into new updates from c00w github ?

Think most of your version is still heavily outdated since recent updates made to structure by c00w.

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
dewon
Newbie
*
Offline Offline

Activity: 55
Merit: 0


View Profile
July 22, 2011, 06:23:21 PM
Last edit: July 22, 2011, 07:26:11 PM by dewon
 #824

Edit: Nothing anymore  Smiley
hawks5999
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile WWW
July 22, 2011, 06:25:53 PM
 #825

Speaking of timeslicing, flowers when will you merge your code into new updates from c00w github ?

Think most of your version is still heavily outdated since recent updates made to structure by c00w.

flowerpower is now on Github:
https://github.com/flower1024/flowerPower

I'll leave it to someone who actually knows something about coding to merge into c00w's main source.

Right now it looks like everything echiu64 did has been merged into ryouki's master. and everything c00w has done has been merged into elitak's master. If elitak's work could pickup the latest ryouiki and merge into c00w that would be great!


It seems some of the main differences (and I may be missing a lot) lies in reporting. Could we just create a master index.html that has links for the various reporting views? I'm sure I'm missing that there are now different stat databases as well (lets' merge those).

Get all this back into the main source and then add some flowerpower to c00w for slicing and we should have a great release candidate!

Do we need to setup a bounty for this work?

■ ▄▄▄
■ ███
■ ■  ■               
LEDGER  WALLET    ████
■■■ ORDER NOW! ■■■
              LEDGER WALLET
Smartcard security for your BTCitcoins
■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Decentralized. Open. Secure.
ed64
Newbie
*
Offline Offline

Activity: 42
Merit: 0


View Profile
July 22, 2011, 06:38:22 PM
 #826

flowers diff isn't that big, shouldn't be too hard to merge.

Question is to merge with c00w or ryouiki to start? hmmm  Undecided

In the meantime, playing around with using fasthoop to adjust reported shares vs time last block found.

I've found that fasthoop doesn't seem to be reporting back solved blocks any faster than mtred/slush/bitclocker APIs, based on about 24hrs worth of data. Not sure how "realtime" it's block detection really is... looks like they rely on the same APIs that bithopper would poll.

Any other service that will show in _true_ realtime when blocks are solved by a pool?
enmaku
Hero Member
*****
Offline Offline

Activity: 742
Merit: 500


View Profile
July 22, 2011, 07:24:57 PM
 #827

Anyone still have invites to hoppersden?
paraipan
In memoriam
Legendary
*
Offline Offline

Activity: 924
Merit: 1004


Firstbits: 1pirata


View Profile WWW
July 22, 2011, 07:32:44 PM
 #828

Anyone still have invites to hoppersden?

dunno, ask around Tongue (just joking I feel fine today)

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
dewon
Newbie
*
Offline Offline

Activity: 55
Merit: 0


View Profile
July 22, 2011, 08:09:28 PM
 #829

X8s.de now swapped to PPLNS

"Found block for Round 30 18 minutes ago · reply · retweet · favorite"

"After finding the next block we'll switch to PPLNS 5 hours ago · reply · retweet · favorite"

I removed them. Personally I hate PPLNS, once you stop mining those pools your shares starts instantly falling no matter how much you mined. When they then hit a block during your downtime you'll get only pennies.. Would be interesting to know efficiency of eg. x8s for 24/7 mining because there will always be people who start mining earlier in round and then drop off. Their contribution helps finding block faster and that benefits 24/7 miners.
msb8r
Member
**
Offline Offline

Activity: 78
Merit: 10


View Profile WWW
July 22, 2011, 08:26:28 PM
 #830

Is it just me, or is updates on pool shares on triple mining a little excessive?

Code:
[22:27:35] triple: 921,102
[22:27:44] triple: 921,232
[22:27:54] triple: 921,377
[22:28:14] triple: 921,680
[22:28:24] triple: 921,837

I would think once every 60 seconds would be plenty fine.

Using c00w's hopper, latest version

New Bitcoin directory @ http://btcdir.org/
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1007


Poor impulse control.


View Profile WWW
July 22, 2011, 10:07:38 PM
 #831

With time slicing and penalty around 3.5, hopping PPLNS can still be effective...

proof? either a screen shot or a mathematical proof will be fine.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
flower1024
Legendary
*
Offline Offline

Activity: 1428
Merit: 1000


View Profile
July 22, 2011, 10:11:20 PM
 #832

as i think he speaks about my mod:
no pplns is not hoppable. the window ist just to big.
atm i am trying to hop slush @300% difficulty and stays until block is found; but i know already that this won't work (due to bitcoin poison process)
^^ i just want to learn it mysql the hard way
enmaku
Hero Member
*****
Offline Offline

Activity: 742
Merit: 500


View Profile
July 22, 2011, 10:26:27 PM
 #833

Anyone still have invites to hoppersden?

dunno, ask around Tongue (just joking I feel fine today)

Thanks for the invite, I'll try to poke around and absorb/contribute whatever I can this weekend.
streblo
Full Member
***
Offline Offline

Activity: 165
Merit: 100


View Profile
July 23, 2011, 03:01:41 AM
Last edit: July 23, 2011, 08:42:41 PM by streblo
 #834

I've calculated some statistics for everyone to help clear up some confusion. Poisson processes can be very non-intuitive. These numbers are the result of considering every possibility (and the probability of that possibility). Any situation which you can think of has been accounted for and (hopefully) accounted for. This analysis assumes one has an accurate share count for pools. That is to say, none of these numbers include things like delayed or falsified JSON stats

Proportional pools
Firstly, in a proportional pool the expectation value of a submitted share is dependent on one and only one variable: the number of shares submitted previously. Hashrate, timeslicing (as I understand it), number of shares you vs other other people submitted, etc. ALL DON'T MATTER. In fact, one can calculate the speedup. Define the normalized shares submitted, x, as the number of shares submitted divided by the current difficulty. Then the expectation value (average value) of a share (normalized to solo mining) is exp(x)*E1(x). E1 is a nonanalytic integral, but wolfram alpha will compute it for you. Alternatively, to a decent approximation, one can use speedup = (.428/x)^(0.65). This approximate formula can be handy for quickly calculating expectation value without resorting to numerical methods. The accurate speedup is shown in Figure 1. Notice the speedup is greater than one before x~0.428 and less than one after 0.428. This is where the magical 42.8% comes from: shares submitted to the pool when x<0.428 have an expectation value greater than one. This is the essence of pool-hopping.


Figure 1. Speedup vs normalized shares submitted (see above text)

Multiple proportional pools
As shown above when considering proportional pools, shares should always be submitted to the pool with the least number of shares or should be solo-mined if all pools are above x~.43. How much is to be gained from multiple pools? A lot If you go to the aforementioned Wolfram Alpha link, calculate the speedup for x=1/difficulty=1/1690000. It equals 13.7x! This means if every share you submitted was the first to a proportional pool, you would have over a 13x speedup! Hypotheticals aside, what about in practice? FIgure 2 shows the results of Monte Carlo calculations for the speedup vs number of proportional pools hopped which roughly follows the trend: speedup = ln(2.5+pools). Here we can see it is always QUITE advantageous to add more pools. The marginal return, although diminishing, decrease only slowly! Hence, a hopper should have as many pools as possible.


Figure 2. Speedup vs number of proportional pools. Note the famous ~28% increase for hopping one pool. Due to the steep slope of speedup(x) for small x, it is quite advantageous to add more pools.

Other scoring methods [Edit: I understood PPLNS incorrectly; the math is completely incorrect]
Many scoring systems are not statistically analyzed and hence still offer opportunity for hopping. For example, Pay-Per-Last-N-Shares is still quite hoppable (with typical N), as follows. Let's renormalize N by dividing by the difficulty, to give us the normalized last N shares, M. Using the same analysis as above, the speedup can be computed as shown below in Figure 3. PPLNS strongly deviates from proportional only during rounds which pay (per share) below the statistical average. Because the hopper "suffers" from PPLNS only during sub-par rounds, the absolute difference between proportional and PPLNS is small. One can calculated the expectation values of PPLNS shares (as a function of x and M), with the following formula (Wolfram link): exp(x) ( Gamma(0,x) - Gamma(0,M+x)) which is valid for x < M. A maximum speedup (at current difficulty) is still 13x with M=0.5. A quick'n'dirty way to implement PPNLS pools in hopping software is to penalize the PPNLS pool by multiplying round_shares by .43/.21. Slush's pool is also hoppable under the same mechanism, although the curves depend on hashing rate as well. Almost any proportional-based pool will be hoppable at x<.10


Figure 3. Comparison of expectation value from share submission from proportional and PPNLS (M=0.5, see text) payout schemes. Notice the intersection with y=1, where share expectation value is equal to solo mining. These intersections are respectively equal to ~0.21 and ~0.428 for PPNLS and proportional.

Conclusion
I hope this clears up some confusion about expectation values of shares and pool hopping. If anything is unclear feel free to reply or send me a PM.

Cheers

Donations greatly appreciated!
1FQoC3zsos22QxZC35fMfP8JteMK9nEQPU



organofcorti
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1007


Poor impulse control.


View Profile WWW
July 23, 2011, 05:08:09 AM
 #835

Very nice work, Streblo! Great explanation, and the first clear explanation of multiple pool hopping that I've seen.

As a big favour, could you show me how you derived the PPLNS function?

<\ttfn>

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
c00w (OP)
Full Member
***
Offline Offline

Activity: 196
Merit: 100


View Profile
July 23, 2011, 06:02:58 AM
Last edit: July 23, 2011, 06:24:18 AM by c00w
 #836

1) Flowers code?
I understand the idea of slicing between pools and will implement it. However I haven't had the time to look at his code and figure it out.

2)New pool config file?
I decided nobody should have to touch python source code. And its easier. For example fixing triple mining required only two lines of code to change.
And I am going to add a mine_ppln mode so we can hop ppln pools.

3) Where were you?
Canada.

4) Adding new pools?
I counted three at last count. Assuming they have json its pretty quick to add them. I'll try and do it tommorow...

edit: Streblo!
I love the explanation. And I'm adding ppln.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
hawks5999
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile WWW
July 23, 2011, 06:10:57 AM
 #837

Amazing streblo. Really great stuff.
The thing I'm still unclear on though (and maybe I'm just dense) is:
Based on your analysis, given the situation of hopping two pools having different speeds, should I stay with the faster pool even though it is further from share 1 and then hop to slower later. Or do I hop to pool 2 immediately after it solves a block. It seems to me the downside is getting stuck in a slow pool and missing the spread of shares to other available pools ( even backup)

■ ▄▄▄
■ ███
■ ■  ■               
LEDGER  WALLET    ████
■■■ ORDER NOW! ■■■
              LEDGER WALLET
Smartcard security for your BTCitcoins
■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Decentralized. Open. Secure.
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1007


Poor impulse control.


View Profile WWW
July 23, 2011, 06:16:44 AM
 #838

Amazing streblo. Really great stuff.
The thing I'm still unclear on though (and maybe I'm just dense) is:
Based on your analysis, given the situation of hopping two pools having different speeds, should I stay with the faster pool even though it is further from share 1 and then hop to slower later. Or do I hop to pool 2 immediately after it solves a block. It seems to me the downside is getting stuck in a slow pool and missing the spread of shares to other available pools ( even backup)


No.

Quote
. Hashrate, timeslicing (as I understand it), number of shares you vs other other people submitted, etc. ALL DON'T MATTER.

Different hashrate pools only affect variance, not speedup (=efficiency).

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
c00w (OP)
Full Member
***
Offline Offline

Activity: 196
Merit: 100


View Profile
July 23, 2011, 06:23:33 AM
Last edit: July 23, 2011, 07:33:18 AM by c00w
 #839

Explanation of the new pool system / how to add pools.

Pool.cfg is a config file.

Pools have the format
[internal name]
name: value
mine_address: address:port
role:disable/backup/mine/mineppln(Once I add it)
user: username
pass: Password

If you have a mining pool you have to specify
api_address:address
and
api_method: json/re

For json add
api_key: json_accesor1, json_accesor2, etc...

So if a pool has its roundshares at
{round_shares:1234}
then
api_key:round_share

if it is
{pool:{round_shares:1234}}
then api_key:pool,round_shares

For re (Regular expressions),
api_key is the expression to match.
api_index is how the resulting match should be sliced to get the value.
See triplemining for an example.

If you get a pool working then well put the code in an issue so it can be added.
Or the forums but I keep forgetting...

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1007


Poor impulse control.


View Profile WWW
July 23, 2011, 08:24:58 AM
 #840

@c00w: How does the efficiency/payout work? Is it something you have to do manually, or is it supposed to be automatic?

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 [42] 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 ... 205 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!