Bitcoin Forum
December 08, 2016, 08:20:28 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 [22] 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 ... 205 »
  Print  
Author Topic: bitHopper: Python Pool Hopper Proxy  (Read 332916 times)
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 17, 2011, 09:37:36 PM
 #421

What kind of scoring are they using? May not be worth hopping. They also don't list any past shares. The lack of transparency isn't good.

Pretty sure its proportional right now.  They are apparently going to be switching to a different method soon though, but I highly doubt they would do that in a middle of a block.

I drink it up!
1481185228
Hero Member
*
Offline Offline

Posts: 1481185228

View Profile Personal Message (Offline)

Ignore
1481185228
Reply with quote  #2

1481185228
Report to moderator
1481185228
Hero Member
*
Offline Offline

Posts: 1481185228

View Profile Personal Message (Offline)

Ignore
1481185228
Reply with quote  #2

1481185228
Report to moderator
1481185228
Hero Member
*
Offline Offline

Posts: 1481185228

View Profile Personal Message (Offline)

Ignore
1481185228
Reply with quote  #2

1481185228
Report to moderator
The block chain is the main innovation of Bitcoin. It is the first distributed timestamping system.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
July 17, 2011, 11:49:46 PM
 #422

me too "thinks that", don't remember where i read something about a central hub patch for the pools to announce it's blocks very fast. We could connect to that hub instead of patching bitcoin, what do you think

Maybe that was my suggestion a few pages back?

Anyways:
For detecting (at least quite) reliably which pools solved which block we need an alternative implementation of the P2P protocol of bitcoind itself.

We need a pure listening node that can connect reliably and steadily to a lot of bitcoind clients and keep this connection alive.

We do NOT need direct connections to pools, what I suggested was to measure from which nodes you FIRST get a block announce (since we just listen and do not relay, we should get a "new block found" from node 23, then a few miliseconds later from node 42 and so on) and in which order. Still there might be a certain churn rate in the network, but with a little bit of luck (and enough connections/quickly enough found blocks...) it should be possible after an hour to say that blocks which are first announced by node 23 and then node 42 are from deepbit and blocks that are first announced by node 79 and then node 38 are from btcg for example.

Of course it would be easier if we connect directly or to neighbours of pools (with ~20 000 nodes online this is kinda hard to just randomly get out of luck though) but even a bit further away we still would hopefully see some patterns (which would naturally change over time as intermediate nodes leave and come who verify + relay blocks slower/faster) that would only shift slowly over time.

This entity should however in my opinion be open source (and hand out the acquired data via an API for free) BUT be hosted on Google App Engine or something similar, as I'm not 100% sure if it does not harm the network to have some well connected black holes in there (I seriously doubt it, but you never know...)

Advantages:
No need for ANY statistics besides eventually numbers of solved blocks (not even that is really needed as you get payouts that can be traced back to generations).

No way to delay or to develop countermeasures, as pools HAVE to rely on announcing their blocks to all neighbours as fast as possible. We just count on the fact that the pools have some different neighbours and this way new block announcements spread differently though the net.

Once in place this can be even useful beyond finally getting rid of proportional pools: It might be useful to find out if pool operators really don't steal blocks, detect hidden pools (or at least proof their existance) and/or huge solo miners and maybe other stuff I didn't think of yet.

Disadvantages:
Countermeasures from pools would involve hiding any kind of statistics to the users (noone is allowed to ever see which blocks are found by it's users) and additionally putting income of several pools in a mixing cascade before paying out. Even then we could probably guess that Pool X has found a block and by trying out deduct which one Pool X is after some time though. These countermeasures would be even worse than just faking/delaying stats for users there, but the fanbois would still praise them for their pool hopping fight.

It also might not be as reliable as APIs, but these fail also all the time (as you see).

Also it might be hard(er) to track small pools - hopping deepbit + btcguild should anyways be far more fun though, hm? Grin

Most likely some serious development work needed (probably, I have no idea how complex the P2P protocol of bitcoind really is)

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 17, 2011, 11:57:20 PM
 #423

earlier sharesare worth more than later shares.
you always (prop) want to go the pool which founds a block.

it gets a little tricky if two pools found a block very close. i made a patch for that - have to discussed with c00w (just see git if interested)

I will wait till c00w update bithopper if neccesary.

So what you suggest is, no matter the speed of a pool, if one pool moves above X shares in current difficulty, it is allways better to switch to another pool that has lower than X shares in current difficulty, even if X shares from first pool is less than the predefined hop percentage ie. 40% ?

this current version i published to discuss it only works with pools which has an more-or-less equal hashrate.

i don't say you should ever not switch to a pool with the most less shares.

but if there are two pools which found a block the same second you don't know which of them is better (means: who will solve this block first).

so best thing would be to divide your shares 50/50 (again: same hashrate atm only, could be improved)

i would prefer to switch pool whenever a share is found on a random basis (see my other version, not on git but in this forum - just for a look; it isnt really usable), but that just do only work if you have just one miner attached. with two or more you'll fuck up some getworks and get way more stales.

i am using my "time-slice" method since ten hours now (tweaked it a little more) and it seems good.

eg: the last mtred -> ozcoin switch. for you it switched immediatly. for me my shares has been splitted


It won't net you more coins, just a more consistent supply by dividing risk by two pools, so this mod is to reduce variation, right?


Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 12:19:24 AM
 #424

me too "thinks that", don't remember where i read something about a central hub patch for the pools to announce it's blocks very fast. We could connect to that hub instead of patching bitcoin, what do you think

Maybe that was my suggestion a few pages back?

Anyways:
For detecting (at least quite) reliably which pools solved which block we need an alternative implementation of the P2P protocol of bitcoind itself.

We need a pure listening node that can connect reliably and steadily to a lot of bitcoind clients and keep this connection alive.

We do NOT need direct connections to pools, what I suggested was to measure from which nodes you FIRST get a block announce (since we just listen and do not relay, we should get a "new block found" from node 23, then a few miliseconds later from node 42 and so on) and in which order. Still there might be a certain churn rate in the network, but with a little bit of luck (and enough connections/quickly enough found blocks...) it should be possible after an hour to say that blocks which are first announced by node 23 and then node 42 are from deepbit and blocks that are first announced by node 79 and then node 38 are from btcg for example.

Of course it would be easier if we connect directly or to neighbours of pools (with ~20 000 nodes online this is kinda hard to just randomly get out of luck though) but even a bit further away we still would hopefully see some patterns (which would naturally change over time as intermediate nodes leave and come who verify + relay blocks slower/faster) that would only shift slowly over time.

This entity should however in my opinion be open source (and hand out the acquired data via an API for free) BUT be hosted on Google App Engine or something similar, as I'm not 100% sure if it does not harm the network to have some well connected black holes in there (I seriously doubt it, but you never know...)

Advantages:
No need for ANY statistics besides eventually numbers of solved blocks (not even that is really needed as you get payouts that can be traced back to generations).

No way to delay or to develop countermeasures, as pools HAVE to rely on announcing their blocks to all neighbours as fast as possible. We just count on the fact that the pools have some different neighbours and this way new block announcements spread differently though the net.

Once in place this can be even useful beyond finally getting rid of proportional pools: It might be useful to find out if pool operators really don't steal blocks, detect hidden pools (or at least proof their existance) and/or huge solo miners and maybe other stuff I didn't think of yet.

Disadvantages:
Countermeasures from pools would involve hiding any kind of statistics to the users (noone is allowed to ever see which blocks are found by it's users) and additionally putting income of several pools in a mixing cascade before paying out. Even then we could probably guess that Pool X has found a block and by trying out deduct which one Pool X is after some time though. These countermeasures would be even worse than just faking/delaying stats for users there, but the fanbois would still praise them for their pool hopping fight.

It also might not be as reliable as APIs, but these fail also all the time (as you see).

Also it might be hard(er) to track small pools - hopping deepbit + btcguild should anyways be far more fun though, hm? Grin

Most likely some serious development work needed (probably, I have no idea how complex the P2P protocol of bitcoind really is)

Do you mean just one listening post, which would 'learn' after a while which pool announced blocks first, second, third etc? Or multiple listening posts?

I think multiple listening posts - as many as there are hoppers - would be a better bet since multiple sample will be more accurate more quickly than one sample multiple times.

Of course you then have to be able to trust the other contributors though. Any consistently very significant difference in estimated pool guess would let you brand the contributor as a plant, I suppose.

Wow, this really feels like we're on a war footing.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 18, 2011, 12:52:31 AM
 #425

Do we jump at 43.5%?  Isn't that 630k shares?  Seems to have jumped off of mtred onto arsbitcoin at around 550k shares.

I drink it up!
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
July 18, 2011, 12:59:54 AM
 #426

The nice thing is that we can only win, as there are only so many things "they" can hide... Wink

Multiple listening posts might be fine, the difficult task then would be aggregating them as anyone could sends bogus data.

Also the data from multiple listening nodes might be hard to combine, as geographical distance and other stuff makes network latency really uncomparable... all you could say is that block 1234 was likely from the same source as block 1230. If a lot of listening nodes agree, this might be a stronger indicator, that it's correct. BUT there's no guarantee that they don't all just send fake data, so the only listening node I would trust would be either my own or a big trusted one that had correct predictions in the past already (+ is open source).

In the end we need to find a way to be able to be individually spammed by as many nodes as possible when a block gets announced - preferrably in the hundreds, thousands or beyond. The more nodes send us that data, the finer that "fingerprint" would become - and the less we have to rely on certain nodes.

Specs are here btw.: https://en.bitcoin.it/wiki/Protocol_specification

Seems like you need to send a "version" packet first, then you get a similar one back from the other client (to gain the current block count, just send out a few ones and look at the answers, then report the max. value from there). It might be necessary to have the headers stored though, as the listener node might be asked for them (you can claim to not have any full block data) via "getheaders". I don't know what happens if you just ignore these request: in the worst case, the other node drops the connection/blacklists you. In the best case you just send nothing and the other client tries somewhere else.

On https://en.bitcoin.it/wiki/Protocol_rules you can see that there is a lot of computation going on, until a block is verified. It is only communicated to other clients AFTER this. This means a bigger block can take nearly half a second in reality to verify --> should show nice patterns!

According to https://en.bitcoin.it/wiki/Network we need to send something (maybe a "ping" message? https://en.bitcoin.it/wiki/Protocol_specification at the bottom...) every hour or so to our neighbours.
"Heartbeat
If thirty minutes or more has passed since the client has transmitted any messages it will transmit a message to keep the connection to the peer node alive.
If ninety minutes has passed since a peer node has communicated any messages, then the client will assume that connection has closed."

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 03:25:21 AM
 #427

Quote
Also the data from multiple listening nodes might be hard to combine, as geographical distance and other stuff makes network latency really uncomparable... all you could say is that block 1234 was likely from the same source as block 1230. If a lot of listening nodes agree, this might be a stronger indicator, that it's correct. BUT there's no guarantee that they don't all just send fake data, so the only listening node I would trust would be either my own or a big trusted one that had correct predictions in the past already (+ is open source).

Actually, I didn't explain myself well. I was thinking that you'd compare, for each listening node, arrival times and then compare that to which block actually was first. Collect say 24 hours of data for each listener. This will calibrate each listener to it's particular circumstance. Then the listener send its best guess to a central collecting website. If this is reliable, then you'll get a whole lots of listeners sending you their best guess. As long as:

a) Fakers don't make up a large percentage of the group;
b) non-faking listening nodes tend to agree most of the time
c) you can automatically remove fakers by their monitoring how consistently well or poorly they guess

then you'll get much better results than using one listening node alone.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
c00w
Full Member
***
Offline Offline

Activity: 196


View Profile
July 18, 2011, 05:29:12 AM
 #428

1) Where have you been?
Hiking traffic etc...

2) btcguild doesn't work!
I know I disabled it.

3) There is a wierd miner hanging issue?
I know I just saw it.

4) You need to merge flower's version into yours?
I'll talk to him. I literally just read the thread.

5) How are stats coming?
I did no work this weekend. So the same as friday.

1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 18, 2011, 05:30:26 AM
 #429

2) btcguild doesn't work!
I know I disabled it.


You can remove bitp.it too now.  They just disabled their share stats so miners will be mining with them all night long if they don't catch it.  Its a shame because they gave me some awesome payouts.

I drink it up!
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 05:35:39 AM
 #430

I'm still getting bitp.it's total shares in my user api. I didn't mine with them all night - although I did mine with btcg all night the night before, which killed my stats for them.

BTW, bitpit is on 25226 shares, only one hour from start of round. Jump on folks!

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 05:37:08 AM
 #431

1) Where have you been?
Hiking traffic etc...

......

5) How are stats coming?
I did no work this weekend. So the same as friday.

You mean (*looks incredulous*) you actually have some kind of *life*?
Huh. What that like, then?


Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
joulesbeef
Sr. Member
****
Offline Offline

Activity: 476


moOo


View Profile
July 18, 2011, 05:40:13 AM
 #432

bitp now too, we need new pools, our list if getting rather short.

mooo for rent
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 18, 2011, 05:42:10 AM
 #433

I'm still getting bitp.it's total shares in my user api. I didn't mine with them all night - although I did mine with btcg all night the night before, which killed my stats for them.

BTW, bitpit is on 25226 shares, only one hour from start of round. Jump on folks!

How are you getting that information?  On the website it says N/A and in bithopper is says 0 and in the bitp.it thread the last post is the operator talking about disabling the share stats.  I really want to keep mining at bitp.it because its been an awesome pool the last couple of days, but with no share stats that not a possibility . . .yet.

I drink it up!
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 05:45:58 AM
 #434

try using your user api instead of pool stats. Works for me.

edit wait, wrote too soon. User api total shares are down too now. If this is on purpose, why would they do that at the start of a round? Better off doing that at the end of a long round.

Ah well, little pool. Say good-bye to the 'hopper boost'! For the moment, eh Sukrim?  Wink


edit 2 Fools! They take away total shares, but they do not take away time since start of round and average hashrate! BWA-HA-HA-HA-HAAAAA!

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 18, 2011, 05:50:13 AM
 #435

try using your user api instead of pool stats. Works for me.

edit wait, wrote too soon. User api total shares are down too now. If this is on purpose, why would they do that at the start of a round? Better off doing that at the end of a long round. I reckon they're just having problems with their database.



Edit:  No point now that you have seen the change too.  Damn, was getting excited there for a second.  


I think he timed it so that miners in the US would be mining all night long on his pool without switching.  I think it was a coincidence that they found a block.  At least, that is what I would do if I was a pool operator.  Squeeze out a little bit more form that glorious hopper boost before fixing it for the users.

I drink it up!
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 05:54:46 AM
 #436

bitp now too, we need new pools, our list if getting rather short.

Did you add nofeemining and rfcpool? Bithasher too, although I'm having problems getting a log in to work there. Finally, I haven't gotten triplemining to work yet, but they're the ones to add if you can get them working. I haven't, Sukrim has.

I'm going to add slush tonight, just to see how bad the efficiency is with their scoring method. Might be ok. However multipool does it, it was always above 1.0


Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
July 18, 2011, 05:57:08 AM
 #437

I think c00w might need to add a 'don't hop if shares = 0' function. Then of course everyone will list their shares as = 1, and we have to change it to 'don't hop if shares = 1'. Then 2. Then 3. Wash, rinse, repeat.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
July 18, 2011, 06:04:36 AM
 #438

Ah well, little pool. Say good-bye to the 'hopper boost'! For the moment, eh Sukrim?  Wink

I give 'em a week max.

Doesn't look too difficult to write a basic bitcoind listener... Hopefully I outlined my ideas well enough so anyone can take on from there. I unfortunately(?) am coding other stuff at the moment and don't really have time for this.

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
muyoso
Member
**
Offline Offline

Activity: 84



View Profile
July 18, 2011, 06:05:01 AM
 #439

I think c00w might need to add a 'don't hop if shares = 0' function. Then of course everyone will list their shares as = 1, and we have to change it to 'don't hop if shares = 1'. Then 2. Then 3. Wash, rinse, repeat.

Add a function to consider a pool broken and disable it if the shares are unchanged between the 2 minute interval bithopper polls them.  And re-add the pool if later on the shares start changing again because it could be a minor glitch or temporary problem.  That would solve both problems most likely.

I drink it up!
joulesbeef
Sr. Member
****
Offline Offline

Activity: 476


moOo


View Profile
July 18, 2011, 06:08:15 AM
 #440

Quote
Did you add nofeemining and rfcpool? Bithasher too, although I'm having problems getting a log in to work there. Finally, I haven't gotten triplemining to work yet, but they're the ones to add if you can get them working. I haven't, Sukrim has.

i have nofee, i'll have to look into the others.

Tripplemining sounds a bit ffy to me,

mooo for rent
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 [22] 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 ... 205 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!