Bitcoin Forum
March 30, 2017, 06:55:33 AM *
News: Latest stable version of Bitcoin Core: 0.14.0  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 [180] 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 »
  Print  
Author Topic: bitHopper: Python Pool Hopper Proxy  (Read 340010 times)
flower1024
Legendary
*
Offline Offline

Activity: 924



View Profile
August 15, 2011, 05:10:27 PM
 #3581


Backup? We don't need no steenking backup!

Tongue  Grin

i'll ever have a backup server. i have seen to many days with all pools above 100% cdf. so i just don't care about "even out"-graphs

1490856933
Hero Member
*
Offline Offline

Posts: 1490856933

View Profile Personal Message (Offline)

Ignore
1490856933
Reply with quote  #2

1490856933
Report to moderator
1490856933
Hero Member
*
Offline Offline

Posts: 1490856933

View Profile Personal Message (Offline)

Ignore
1490856933
Reply with quote  #2

1490856933
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1490856933
Hero Member
*
Offline Offline

Posts: 1490856933

View Profile Personal Message (Offline)

Ignore
1490856933
Reply with quote  #2

1490856933
Report to moderator
1490856933
Hero Member
*
Offline Offline

Posts: 1490856933

View Profile Personal Message (Offline)

Ignore
1490856933
Reply with quote  #2

1490856933
Report to moderator
deepceleron
Legendary
*
Offline Offline

Activity: 1484



View Profile WWW
August 15, 2011, 05:19:06 PM
 #3582

I did some in-depth analysis of Bithopper Long-Poll timing method for identifying block-solving pools, initially to see how pool-delay corrections can be used to make round-finding results more accurate, I used the console log output of BH, which after some grepping, looks like this:

(round 141030 - found by deepbit)
[1295][23:39:34] LP Call mining.mainframe.nl:8343/LP
[1326][23:39:35] LP Call pit.deepbit.net:8332/listenChannel
[1333][23:39:35] LP Call mineco.in:3000/LP
[1334][23:39:35] LP Call eu1.triplemining.com:8344/LP
[1341][23:39:36] LP Call arsbitcoin.com:8344/LP
[1344][23:39:37] LP Call uscentral.btcguild.com:8332/LP
[1355][23:39:38] LP Call pool.bitclockers.com:8332/LP
[1356][23:39:38] LP Call pool.bitp.it:8334/longpoll
[1358][23:39:39] LP Call bitcoins.lc:8080/LP
[1392][23:39:42] LP Call su.mining.eligius.st:8337/LP


We see above, the first pool to respond is NOT the finding pool!

I have workers logging into 16 different pools; however only 11 of those reply with new block LP pushes. Current Bithopper logic simply guesses that the first LP is the finding pool; as we can see in above, the first LP was not from deepbit in this case. As I had seen this behavior before, I requested the addition of an lp_penalty option - a per-pool option in Bithopper, where you can "delay" the LP reponse time on fast-reporting pools.

Now, we must classify what that delay should be. I used the LP timestamps from the console log as shown above. I was originally expecting the finding pool to clearly stand out, with a definitive earlier LP push after correction, but even this turned out not to be the case, so I should have modded the output to be tenths of seconds for better analysis before I started..

Here is 11 rounds of data, with the actual finding pool verified with digbtc.com and pident.artefact2.com. The block delays for each pool and each round are normalized against the time of the finding pool, if present:


The time-delay table is the top, different analyses are in blue, and the actual correction I am using is on is on the far right. Note the lp_penalty setting would be the opposite of this correction - eligius might get lp_penalty:0 and mineco: 4.

The bottom table is the time delays after applying the best correction I could hand-tweak. We see that still the finding pool almost never is the fastest (highlighted in red). (This might improve with 1/10th second stats resolution).

However, we see that if we average all pool LPs after learning a delay correction, the finding pool does begin to stand out.

The conclusions I have to make:

-Per-pool LP delays are more random than one might expect; and some pools have more variation than others (likely related to number of miners, and their p2p distance from block-solving pool)

-Deepbit's block-find LPs do not clearly stand out from other non-finding LPs received from deepbit,

-The first reporting pool, even when testing many delay correction formulas, will not necessarily be the finding pool.

-Only with a time delay correction for each pool, and then comparison of each pool to the average of all pools can we hope to reliably identify the stand-out block-finding pool. We also need a confidence factor, so if no pools stand out, we can conclude none of LP-returning pools we check found the block.


We have also seen --p2pLP come to prominence, where LP block-finding information is shared on IRC, and then a vote helps further refine results. However this still is very rudimentary; and as nearly all bH users get LPs from deepbit vs only some getting other pools, deepbit tends to win votes undeservedly (and these biased voting results would be more wrong if it wasn't for DB being the largest pool).


So to improve BH LP block finding to the point of reliability, we need:

- Self-learning of per-pool LP delays (LP delay learning must remove the finding pool however, or else we bias deepbit's delays because they find so many blocks)

- Publishing of all local corrected new block LP results to IRC, not just winning pool (perhaps JSON format, like "** block 3fe33a: {"triple": 0.0, "bitp": 0.3, "deepbit": 0.5, "mineco": 1.3, "bclc", 1.7}"

- Averaging (not voting or first response) all IRC results (including pools we don't actually check ourselves), to determine if there is a clear LP block winner; a confidence level can be assigned.

For the super-advanced, it may even be able to profile a 'signature' of block delays to determine who found the block, since pools also seem to have a different LP delay depending on their p2p distance from the finding pool.

norulezapply
Hero Member
*****
Offline Offline

Activity: 481


View Profile
August 15, 2011, 05:34:09 PM
 #3583

I haven't been keeping up with this thread lately so I figured i'd just post and ask.

Is it worth using mine_deepbit with deepbit and/or BTCGuild?
At the moment it sounds like it's completely random and would be better off being disabled until it's been further refined.

Also can you use mine_deepbit with deepbit and btcguild at the same time?
Thanks

If my post helped, I'll happily accept a few bitmills!   15rGg6A1JFZV3b7TTbtpAaiYGdUD1e1oAm
paraipan
Legendary
*
Offline Offline

Activity: 924


Firstbits: 1pirata


View Profile WWW
August 15, 2011, 05:52:40 PM
 #3584

I haven't been keeping up with this thread lately so I figured i'd just post and ask.

Is it worth using mine_deepbit with deepbit and/or BTCGuild?
At the moment it sounds like it's completely random and would be better off being disabled until it's been further refined.

Also can you use mine_deepbit with deepbit and btcguild at the same time?
Thanks

atm it is worth mining deepbit and yes you can have various pools with mine_deepbit role at the same time, this role is very sensible regarding you geographic location to pools so that's the reason it doesn't work with a 100% accuracy


@deepceleron nice analysis man, I'm with this method (normalizing LP responses from pools) and in the future some automated learning would be awesome. I think manual correction you have posted are really different for every bH miner out there.

edit: dunno if it's a viable idea... could't we rely on a pident service to automatically have bH learn how to make it's LP adjustments ?

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
gentakin
Member
**
Offline Offline

Activity: 98


View Profile
August 15, 2011, 06:35:43 PM
 #3585

Actually, the manual correction is probably not only different for every miner out there based on their location, but also depends on how many other workers are currently connected to the various pools. It's probably a very unreliable method to determine a block finder.

Maybe it's possible to hook into bitcoind and see in which order connected nodes send the new block information. There might be patterns, but I suspect they are not very constant and need to be re-learned often. The learning would probably take a few hours at the begin of every session to recognize patterns of deepbit, BTC guild, and other pools, if there is any difference.

1HNjbHnpu7S3UUNMF6J9yWTD597LgtUCxb
bb
Member
**
Offline Offline

Activity: 84


View Profile
August 15, 2011, 06:39:48 PM
 #3586

Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?
Answer42
Newbie
*
Offline Offline

Activity: 9


View Profile
August 15, 2011, 06:40:06 PM
 #3587

I have been mining on a PPS pool, but this proxy has gained my interest.

What pools are still hopable?  Obviously proportional.  But don't some use anti-hoping techniques?  
joulesbeef
Sr. Member
****
Offline Offline

Activity: 476


moOo


View Profile
August 15, 2011, 06:53:14 PM
 #3588

Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Quote
I have been mining on a PPS pool, but this proxy has gained my interest.

What pools are still hopable?  Obviously proportional.  But don't some use anti-hoping techniques?


nothing to see here... move along. Let me guess you didnt want to read a 200 page thread? LOL.
Yeah a lot of pools still hoppable, some even like us, just wish we would stay longer. Some do use anti-hopping techniques. If you do use the proxy, you will want to keep up to date with your pools. They change, some add anti hop techniques, some change payout systems, some break, a few are buggy. The best anti hopping technique is a fair reward system. The rest have various levels of effectiveness.

mooo for rent
paraipan
Legendary
*
Offline Offline

Activity: 924


Firstbits: 1pirata


View Profile WWW
August 15, 2011, 06:55:33 PM
 #3589

I have been mining on a PPS pool, but this proxy has gained my interest.

What pools are still hopable?  Obviously proportional.  But don't some use anti-hoping techniques?  

hy, please edit your user.cfg and you will find info on every pool too in there

BTCitcoin: An Idea Worth Saving - Q&A with bitcoins on rugatu.com - Check my rep
GenTarkin
Legendary
*
Offline Offline

Activity: 1988


View Profile
August 15, 2011, 06:56:47 PM
 #3590

Actually, the manual correction is probably not only different for every miner out there based on their location, but also depends on how many other workers are currently connected to the various pools. It's probably a very unreliable method to determine a block finder.

Maybe it's possible to hook into bitcoind and see in which order connected nodes send the new block information. There might be patterns, but I suspect they are not very constant and need to be re-learned often. The learning would probably take a few hours at the begin of every session to recognize patterns of deepbit, BTC guild, and other pools, if there is any difference.

Off subject here but...um dude your name confused the hell outta me when I first saw it. I was like "I didnt make that post...WTF?!?!"   oh, his name is missing the r...haha

GenTarkin's MOD Kncminer Titan custom firmware! v1.0.4! <--- CLICK HERE
Donations: bitcoin- 1Px71mWNQNKW19xuARqrmnbcem1dXqJ3At || litecoin- LYXrLis3ik6TRn8tdvzAyJ264DRvwYVeEw
bb
Member
**
Offline Offline

Activity: 84


View Profile
August 15, 2011, 07:15:51 PM
 #3591

Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
August 15, 2011, 07:32:12 PM
 #3592

Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.

I drink it up!
Starlightbreaker
Legendary
*
Offline Offline

Activity: 1316



View Profile
August 15, 2011, 07:37:59 PM
 #3593

Quote
Can anyone hint me on why I am submitting shares to btlc, but not deepbit and polmine (all with mine_deepbit)?

that is what a lot of the discussion above is about. C00w is working on new methods to hop unhoppable pools. It works great for some, no so great for others. The debate above is about improving the methods. Obviously this is difficult as these pools were set up to not be hoppable. Most likely you have a worse connection to deepbit than you do for bclc. If you become concerned it is harming your rewards, then just disable the mine_deepbit pools, until the guys get the bugs worked out on the method.

Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.
same here.

the pools my hopper hops into was bclc & mtred pps, aside from db




           ▄▄▄▄▄ 
     ▄▄█████████████▄
   ▄█████▀       ▀█████▄
 ▄████▀             ▀████▄
 ███▀                 ▀███▄
███▀   █████████████   ▀███
███    ▀▀▀▀▀███▀▀▀▀▀    ███
███         ███         ███
███▄        ███        ████
▀███▄       ███       ▄███▀
 ▀████▄     ▀▀▀     ▄████▀
   ▀█████▄▄     ▄▄█████▀
      ▀█████████████▀
           ▀▀▀▀▀▀ 
WeTrust.io


             ▄██▄
     ▄      ▐████   ▄▄
   █████     ██████████
    █████████████████▀
 ▄████████████▀████▌
██████████     ▀████     
 ▀▀   █████     ██████████
      ▀████▌▄████████████▀
    ▄▄▄███████████████▌
   ██████████▀    ▐████
    ▀▀▀  ████▌     ▀▀▀
         ▀███▀


 ▄████████████████████▄
████████████████████████
██████████ ▀████████████
██████████    ▀█████████
██████████    ▄█████████
██████████ ▄████████████
████████████████████████
 ▀████████████████████▀

                 ▄████▄▄    ▄
██             ████████████▀
████▄         █████████████▀
▀████████▄▄   █████████████
▄▄█████████████████████████
██████████████████████████
  ▀██████████████████████
   █████████████████████
    ▀█████████████████▀
      ▄█████████████▀
▄▄███████████████▀
   ▀▀▀▀▀▀▀▀▀▀▀

  ▄██▄▄                ▄▄██▄
  ████████▄▄▄▄▄▄▄▄▄▄▄███████
  ██████████████████████████
  ██████████████████████████
▄████████████████████████████▄
██████████████████████████████▌
█████▀                  ▀█████▌
████    ███▄      ▄███    ████▌
████   ▐████      ████▌   ████
 ███    ▀██▀      ▀██▀    ███▀
  ▀██▄                  ▄██▀
    ▀▀██████████████████▀▀


hawks5999
Full Member
***
Offline Offline

Activity: 168



View Profile WWW
August 15, 2011, 08:54:45 PM
 #3594


Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.

yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.

■ ▄▄▄
■ ███
■ ■  ■               
LEDGER  WALLET    ████
■■■ ORDER NOW! ■■■
              LEDGER WALLET
Smartcard security for your BTCitcoins
■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
Decentralized. Open. Secure.
macboy80
Member
**
Offline Offline

Activity: 102


View Profile
August 15, 2011, 08:59:06 PM
 #3595

My bH, on my server, will only pull lp for one round. At home, on my pc, it keeps working regularly.
MaGNeT
Legendary
*
Offline Offline

Activity: 1050


Founder of Orlycoin | O RLY? YA RLY!


View Profile WWW
August 15, 2011, 08:59:35 PM
 #3596


Oh. I thought deepbit was working for everyone already. Especially with the IRC.

deepbit is working too good for me.  Bithopper thinks every block is from deepbit.

yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.

I'm still on the tuesday 9 august version Tongue
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
August 15, 2011, 08:59:47 PM
 #3597


yeah, the recent changes in bithopper basically make you a 24/7 miner at deepbit. Which is fine if you want to 24/7 mine at deepbit. Otherwise, though, it's best to just not even bother with deepbit.


About half of all of my shares are going to deepbit right now.  What would be awesome is if bithopper could check with pident about once an hour and confirm that solved block was actually from the same pool that bithopper received the LP from, and if not have bithopper auto adjust the penalties a little bit at a time until a certain accuracy is reached.  Of course like most ideas I am sure this is easier said than done.  

Where do we put the lp_penalty value for a pool?  Is it in user.cfg or pools.cfg?

I drink it up!
cirz8
Jr. Member
*
Offline Offline

Activity: 42


View Profile
August 15, 2011, 09:50:13 PM
 #3598

A crazy idea just popped up while I was messing around with some bash-scripts.
 
Would it be possible to have bh running multiple pools simultaneously?
And splitting the getwork requests of those based on a on-every-X-getwork.

If one sets a pool to 'on-every-X-getwork: 10' it would do a getwork every 10th getwork, getting 10% of the total hashspeed.

Why would anyone want something crazy like that?
Direct a constant portion of ones hashingpower towards a pool that one might like to support despite hopping or whatever reason.

Getting bh to re-read the user.cfg either by command or by internal filechange-checking would be a nice, but not required, addition to this.


Another idea came up while writing this post, but it can be shot down quickly if someone has the data:
Would it be favorable to mine, for example, two(things change if there are more?) sub 10%shares pools simultaneously, instead of one until it reaches the threshold and then jump to one of the others, if it/they are still below threshold, or is it better to try and get lucky as early on in a round as possible?

I'm not quite sure how the hopping is determined atm.
Will bh constantly try and find the pool with the lowest share-count and then jump to it?
or will it stay with a pool it has jumped to until the threshold is reached, then jump to the pool with the lowest share-count?
I'm guessing "--altslicesize" has a part in this, but I don't understand what a "slice" is, is it shares?

(The more I add to this post, the more confusing it gets, so I will just press "Post" now)  Huh

Mandatory?  123ABCcirz8CcieVh9UwThEX2vkoJF33Te
GenTarkin
Legendary
*
Offline Offline

Activity: 1988


View Profile
August 15, 2011, 10:48:54 PM
 #3599


I'm guessing "--altslicesize" has a part in this, but I don't understand what a "slice" is, is it shares?

(The more I add to this post, the more confusing it gets, so I will just press "Post" now)  Huh

A slice is simply a amount of time allotted to mining on a given pool, so its slices time, keeps track of slices and uses that to switch between 2 eligible pools

GenTarkin's MOD Kncminer Titan custom firmware! v1.0.4! <--- CLICK HERE
Donations: bitcoin- 1Px71mWNQNKW19xuARqrmnbcem1dXqJ3At || litecoin- LYXrLis3ik6TRn8tdvzAyJ264DRvwYVeEw
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
August 16, 2011, 01:25:32 AM
 #3600

What would be the best way of going about tweaking the lp_penalty so that I can get bithopper to make the most accurate guess that it can on who solved a block?  Does anyone have a method that is semi simple?

I drink it up!
Pages: « 1 ... 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 [180] 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!