Bitcoin Forum
December 09, 2016, 02:12:58 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 [175] 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 »
  Print  
Author Topic: bitHopper: Python Pool Hopper Proxy  (Read 332986 times)
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
August 12, 2011, 11:15:20 PM
 #3481

I like aiming for 25% on db, but sometimes it skips over it and I'm there a little bit longer, but it's not exactly killing efficiency. I suggest fiddling with the penalties till you get what you're looking for.

Yea, I could do that.  I think an easier way would just be a timer though.  Deepbit is almost always 26-27 minutes per block average.  So just hop after 26.5*.43 minutes = 11.395 or say 11.5 minutes.

Damnit, bithopper missed out on a fast round on bitcoinpool because it was too busy mining at deepbit for hours on end.  Back to cherrypicker.

I drink it up!
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481249578
Hero Member
*
Offline Offline

Posts: 1481249578

View Profile Personal Message (Offline)

Ignore
1481249578
Reply with quote  #2

1481249578
Report to moderator
1481249578
Hero Member
*
Offline Offline

Posts: 1481249578

View Profile Personal Message (Offline)

Ignore
1481249578
Reply with quote  #2

1481249578
Report to moderator
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
August 12, 2011, 11:28:16 PM
 #3482

@r2edu
Are you seeing any errors on the front end?

Lp Penalty?
Sure.

Oh and there already is a mine_nmc...

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
r2edu
Member
**
Offline Offline

Activity: 68


View Profile
August 12, 2011, 11:36:51 PM
 #3483

@c00w: I don´t know if this is what you´re asking, but i think no, deepbit is in yellow and the share count of the round count normal, but my miner (GUIMiner x3vgas), just stop.

I´ll try to run BH with --debug so i can give you more details! (if it worth it, you tell me)
joulesbeef
Sr. Member
****
Offline Offline

Activity: 476


moOo


View Profile
August 13, 2011, 02:03:19 AM
 #3484

move bitminers union to the unhoppable part of your lists

mooo for rent
Endeavour79
Full Member
***
Offline Offline

Activity: 169



View Profile WWW
August 13, 2011, 02:11:42 AM
 #3485


Need an update anyway..

[bmunion]
name: BitMinersUnion.org
mine_address: pool.bitminersunion.org:8341
api_address: http://64.244.102.89/stats.php
api_method: re
api_key: Shares this round:([ 0-9]+)<br/>
api_strip: ' '
url: http://www.bitminersunion.org/

NSW, Australia - Rigs, Mining, Pools - Local help needed? Send me a message!
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
August 13, 2011, 02:27:05 AM
 #3486

@whoever wanted an lp_penalty option:
I added it. It is lp_penalty : seconds.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
r2edu
Member
**
Offline Offline

Activity: 68


View Profile
August 13, 2011, 02:33:11 AM
 #3487

^ you´re quick man!! hehe..

what is the beneffit of that, or what value do you recommend?
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
August 13, 2011, 02:39:28 AM
 #3488

You can put a penalty on deepbit if you've done the analysis for your location and determined it is getting too many blocks. Or any other site. Especially if there LP is super slow or delayed.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
August 13, 2011, 03:59:55 AM
 #3489

@whoever wanted an lp_penalty option:
I added it. It is lp_penalty : seconds.
Great! This feature adds X seconds to the LP receive time of the specified pool for figuring out who solved it. That means if you analyze the console log, and find one pool that is consistently faster in sending you it's LP block announce than others (especially if a pool responds to you before the one that actually found the block - causing the proxy to designate the wrong pool), then you can 'equalize' the time LPs are received by adding a delay to 'too fast' pools.

Block 140749 found by pool.bitp.it:

[19:28:09] received lp from: bitp
[19:28:09] New Block: 5ea4b4d35340f0a60c238a721e3b0bd72453873fe6a7cec6000005f700
000000
[19:28:09] Block Owner bitp
[19:28:09] LP Call pool.bitp.it:8334/longpoll
[19:28:10] received lp from: mmf
[19:28:10] LP Call mining.mainframe.nl:8343/LP
[19:28:10] received lp from: deepbit
[19:28:10] LP Call pit.deepbit.net:8332/listenChannel
[19:28:10] received lp from: mineco
[19:28:10] LP Call mineco.in:3000/LP
[19:28:10] RPC request [getwork] submitted to polmine
[19:28:11] received lp from: bclc
[19:28:11] LP Call bitcoins.lc:8080/LP
[19:28:12] received lp from: eclipsemc
[19:28:12] LP Call pacrim.eclipsemc.com:8337/LP
[19:28:12] received lp from: arsbitcoin
[19:28:12] LP Call arsbitcoin.com:8344/LP
[19:28:14] received lp from: eligius
[19:28:14] LP Call su.mining.eligius.st:8337/LP


Although this is not the best example (I actually had one block scroll by where mineco was the first to announce the block, beating the actual finding pool), we can see that pools take different amounts of time to respond. Bitp.it was almost beat in announcing their own block by mainframe. I can improve the results by tweaking like this (if I was only looking at the above results and not evaluating pool delays over many blocks):


[mmf]
lp_penalty: 4

[deepbit]
lp_penalty: 4

[mineco]
lp_penalty: 4

[bclc]
lp_penalty: 3

[eclipsemc]
lp_penalty: 2

[arsbitcoin]
lp_penalty: 2

[eligius]
lp_penalty: 0

[bitp]
lp_penalty: 3? (would need to evaluate blocks they didn't solve)


I will test this out and see if it can completely fix false pool block designation. Of course what would be wishful thinking would be if bithopper could learn for itself the average delays it sees from each pool.

organofcorti
Donator
Legendary
*
Online Online

Activity: 1960


Poor impulse control.


View Profile WWW
August 13, 2011, 04:18:49 AM
 #3490

@whoever wanted an lp_penalty option:
I added it. It is lp_penalty : seconds.
Great! This feature adds X seconds to the LP receive time of the specified pool for figuring out who solved it. That means if you analyze the console log, and find one pool that is consistently faster in sending you it's LP block announce than others (especially if a pool responds to you before the one that actually found the block - causing the proxy to designate the wrong pool), then you can 'equalize' the time LPs are received by adding a delay to 'too fast' pools.

Block 140749 found by pool.bitp.it:

[19:28:09] received lp from: bitp
[19:28:09] New Block: 5ea4b4d35340f0a60c238a721e3b0bd72453873fe6a7cec6000005f700
000000
[19:28:09] Block Owner bitp
[19:28:09] LP Call pool.bitp.it:8334/longpoll
[19:28:10] received lp from: mmf
[19:28:10] LP Call mining.mainframe.nl:8343/LP
[19:28:10] received lp from: deepbit
[19:28:10] LP Call pit.deepbit.net:8332/listenChannel
[19:28:10] received lp from: mineco
[19:28:10] LP Call mineco.in:3000/LP
[19:28:10] RPC request [getwork] submitted to polmine
[19:28:11] received lp from: bclc
[19:28:11] LP Call bitcoins.lc:8080/LP
[19:28:12] received lp from: eclipsemc
[19:28:12] LP Call pacrim.eclipsemc.com:8337/LP
[19:28:12] received lp from: arsbitcoin
[19:28:12] LP Call arsbitcoin.com:8344/LP
[19:28:14] received lp from: eligius
[19:28:14] LP Call su.mining.eligius.st:8337/LP


Although this is not the best example (I actually had one block scroll by where mineco was the first to announce the block, beating the actual finding pool), we can see that pools take different amounts of time to respond. Bitp.it was almost beat in announcing their own block by mainframe. I can improve the results by tweaking like this (if I was only looking at the above results and not evaluating pool delays over many blocks):


[mmf]
lp_penalty: 4

[deepbit]
lp_penalty: 4

[mineco]
lp_penalty: 4

[bclc]
lp_penalty: 3

[eclipsemc]
lp_penalty: 2

[arsbitcoin]
lp_penalty: 2

[eligius]
lp_penalty: 0

[bitp]
lp_penalty: 3? (would need to evaluate blocks they didn't solve)


I will test this out and see if it can completely fix false pool block designation.

Nice work! When you do get it done, could you paste your raw before and after data somewhere (eg google docs or pastebin)? I'd like to get an idea of mean time differences for the pools to your part of the world. Knowing your part of the world (eg Europe, Americas, Australasia - you don't have to be exact) might be handy since if a number of us do this and people from similar parts of the world have similar differences in LP announces then these penalties coud be built into bitHopper as a default,, and the penalties populated automatically.

Thanks for taking the time. I hope Sukrim is watching - he was keen to do something of the sort.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
August 13, 2011, 04:37:13 AM
 #3491


Nice work! When you do get it done, could you paste your raw before and after data somewhere (eg google docs or pastebin)? I'd like to get an idea of mean time differences for the pools to your part of the world. Knowing your part of the world (eg Europe, Americas, Australasia - you don't have to be exact) might be handy since if a number of us do this and people from similar parts of the world have similar differences in LP announces then these penalties coud be built into bitHopper as a default,, and the penalties populated automatically.

Thanks for taking the time. I hope Sukrim is watching - he was keen to do something of the sort.

This LP "who solved the block" procedure exploits an assumption, that a pool that finds a block will be sending notifications to miners earlier than other pools. This may not always be correct.

One has to understand these delays and what long polling is.

When a pool's bitcoin detects a new block (whether they found it themselves or found out about it over the p2p network), the the job of sending notifications goes to pushpool. Pushpool looks at all the open RPC connections that have long polling enabled, and pushes out a notification of the new block to the miners. Long polling keeps a connection open between the miner and the pool, so the miners get a quicker notification of the new block than if they were to just request new work every 15 seconds or so. This reduces obsolete share submissions ("stale" shares), improving pool efficiency.

Pushpool can't instantaneously notify everyone however, due to network bandwidth. Essentially, it goes through a list of all open connections and sends each the LP push consecutively. Small pools can send all their miners the notification quickly, but depending on the pool implementation and the number of open connections, this can take more than five seconds on busy pools. If you are on the start of the list, you get notification fast; at the end of the list, it can take longer. This means that the pool delay can be random per miner, much more significantly than packet transit times. Profiling your pool connection time may give different results from others, and may give different results even if you restart bithopper on a different day and make new RPC connections to the pool.

c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
August 13, 2011, 05:22:09 AM
 #3492

Its right about 50% of the time. And people are working to try and increase that.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
August 13, 2011, 06:09:20 AM
 #3493

Pushpool can't instantaneously notify everyone however, due to network bandwidth. Essentially, it goes through a list of all open connections and sends each the LP push consecutively. Small pools can send all their miners the notification quickly, but depending on the pool implementation and the number of open connections, this can take more than five seconds on busy pools. If you are on the start of the list, you get notification fast; at the end of the list, it can take longer. This means that the pool delay can be random per miner, much more significantly than packet transit times. Profiling your pool connection time may give different results from others, and may give different results even if you restart bithopper on a different day and make new RPC connections to the pool.
Thanks for the perfect explanation!

This is also the reason why ultimatively I want to have a stripped down Bitcoin client that watches block announcements and where they come from instead of timing longpolls in the longer run.

To get more data on Longpolls we would need to connect bitHoppers together (via IRC or something else) - something that might not appeal to some people. Also I'm not sure if it really can improve the prediction rate significantly if longpolls really differ by 5 seconds on the same pool. There are 2 interesting factors to consider: How often do we miss a block by a LP pool and how often do we think that an LP pool has solved a block when it in reality hasn't?
The first rate is not that critical, the second one can seriously harm business though.

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
August 13, 2011, 07:01:07 AM
 #3494

The 2nd one gets caught by our monitoring of API's.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
joulesbeef
Sr. Member
****
Offline Offline

Activity: 476


moOo


View Profile
August 13, 2011, 07:17:31 AM
 #3495


Need an update anyway..

[bmunion]
name: BitMinersUnion.org
mine_address: pool.bitminersunion.org:8341
api_address: http://64.244.102.89/stats.php
api_method: re
api_key: Shares this round:([ 0-9]+)<br/>
api_strip: ' '
url: http://www.bitminersunion.org/


what about

Quote
with built in hard coding protections to discourage pool hopping (not for people who change pools but for those who intentionally hop pools as a way to cheat the other miners of the pool


i saw it was you chating with him.. did you find out more or is the hard coding is that we need regex?

mooo for rent
anty
Jr. Member
*
Offline Offline

Activity: 40



View Profile WWW
August 13, 2011, 09:13:40 AM
 #3496

Need an update anyway..

[bmunion]
name: BitMinersUnion.org
mine_address: pool.bitminersunion.org:8341
api_address: http://64.244.102.89/stats.php
api_method: re
api_key: Shares this round:([ 0-9]+)<br/>
api_strip: ' '
url: http://www.bitminersunion.org/

I made the RegEx prettier:
Code:
[bmunion]
name: BitMinersUnion.org
mine_address: pool.bitminersunion.org:8341
api_address: http://64.244.102.89/stats.php
api_method: re
api_key: Shares this round:([0-9]+)<br/>
url: http://www.bitminersunion.org/

deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
August 13, 2011, 09:25:59 AM
 #3497

@whoever wanted an lp_penalty option:
I added it. It is lp_penalty : seconds.

The latest version seems to have a big bug, it basically seems seems to immediately change the block owner every time it gets a new block LP instead of evaluating which came first:

edit: fixed, removed log dump

lueo
Member
**
Offline Offline

Activity: 61


Bitcoin believer


View Profile WWW
August 13, 2011, 11:09:33 AM
 #3498

Anyway, I'm in!

Deepbit Team "bitHoppers": https://deepbit.net/teams/4e424d710691723831000002/join

Feel free to hop in! Wink

Donation: 1M1mB5BQX5QthTojfHxXxJQJr8ro5xLcKR
Real-time LR <-> MTGOX exchange! http://goo.gl/gJqZS
Internet Marketing Q&A in Chinese: http://qa.webcash168.com/
MaGNeT
Legendary
*
Offline Offline

Activity: 1050


Founder of Orlycoin | O RLY? YA RLY!


View Profile WWW
August 13, 2011, 11:28:00 AM
 #3499

This made me think:

Good news everyone! 43% is dead!

We no longer have to hop at 0.43*difficulty!

I haven't been around much lately because I spent the last week running and rewriting 'byteHopper', a multiple pool hopping simulator. When when I got results I didn't want, I rewrote it from scratch. Three times.

Although I made it run faster each time the results are the same, and the cumulative distribution functions follow real world bitcoin results. So I'm finally happy with the results it gives.

I'm only posting a summary here and I'll get around to making blog posts on hoppersden to give more detail, but the gist of it is: If there are 3 or more hoppable pools, you can stay on pools as long as you want and still get the same long term reward as if you hopped on 0.43, but without needing backup and without pissing pool operators as much. Keep in mind that this is not a claim about one particular round, but over, say, 100000 rounds (which I used for the graphs below).

Of course in any given round, after total shares=diff, your shares are worth less than one. But over time the shorter rounds make up for it. Even with only 3 Prop and backup, your efficiency will always be about 1.83, unless you hop off *too soon*. With ten other pools you get about 250% over PPS - sound familiar?

tl;dr BOLD CLAIM: As long as you always hop to the pool with the lowest shares - regardless of hashspeed - you don't need to hop to backup at 0.43.

The ran up some quick graphs which show byteHopper results up to hopping at 3*diff for 0 other pools (ie one proportional pools and one PPS), 1 other pools, 3 other pools and 5 other pools, all plus pps when needed. '0 other pools' mean the same as Raulo's example, and gives the same results as his equation.

 This is basically not 'hopping' much at all. PPS only really has positive effect if you hop only one or two other pools.

                 


I tried several schedulers now and I thought about the following (new "--HopLowestSharecount" scheduler):

- mine the 3 pools with lowest sharecount at any moment (where 3 can be changed to a custom value with "--hop_ x_pools=3" parameter)
- forget about the 0.43 threshold
- It won't need back-up pools, it will always hop between the 3 pools with lowest sharecount and with 18 pools, that should work without problems

It's a bit like the OldDefaultScheduler and SliceScheduler, without the threshold.



simonk83
Hero Member
*****
Offline Offline

Activity: 896


View Profile
August 13, 2011, 11:31:59 AM
 #3500

Er...interesting....

https://www.rfcpool.com/account

Quote
rfcpool has been shut down because the owner is a fucking idiot - that
should sum it up in brief.


If you were a miner, your money is all accounted for, and you will be paid as
soon as the most recent block matures. If you did not have a wallet address
in your account then email chris@rfcpool.com and after slinging your abuse
at me, tell me the username/email on account/wallet combination and you will
get paid.

So what happened? Well in short, running a pool and dealing with
people's money is a metric fucktonne harder than I ever imagined it would
be, and a series of fuckups relating to handling all said money has left me
about 150BTC out of pocket, so I'm pulling the plug before the situation
gets any worse.

Good luck to all other pool ops.
Pages: « 1 ... 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 [175] 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!