Bitcoin Forum
November 22, 2017, 06:07:10 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 [13] 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
Author Topic: [ANN] Stratum mining protocol - ASIC ready  (Read 143544 times)
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2352


Ruu \o/


View Profile WWW
October 26, 2012, 11:57:41 AM
 #241

That's a very generous donation and I only implemented unwittingly while implementing stratum, but I gladly accept it  Wink

No problem glad to help out, just not sure what you ment did you implement GBT as well when implementing stratum?
No I mean the sticking to an IP.

I have not begun to implement GBT as I think it's not as comprehensive a solution as stratum and has some serious weaknesses for pooled mining. I am in discussions with jgarzik to try and improve GBT to make it better and address one of its biggest weaknesses, but really, I'd much rather all the pools just moved to stratum instead.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
1511330830
Hero Member
*
Offline Offline

Posts: 1511330830

View Profile Personal Message (Offline)

Ignore
1511330830
Reply with quote  #2

1511330830
Report to moderator
1511330830
Hero Member
*
Offline Offline

Posts: 1511330830

View Profile Personal Message (Offline)

Ignore
1511330830
Reply with quote  #2

1511330830
Report to moderator
1511330830
Hero Member
*
Offline Offline

Posts: 1511330830

View Profile Personal Message (Offline)

Ignore
1511330830
Reply with quote  #2

1511330830
Report to moderator
Join ICO Now Coinlancer is Disrupting the Freelance marketplace!
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511330830
Hero Member
*
Offline Offline

Posts: 1511330830

View Profile Personal Message (Offline)

Ignore
1511330830
Reply with quote  #2

1511330830
Report to moderator
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
October 26, 2012, 11:03:53 PM
 #242

I wish I could read python code, it's not structured code, it's not object oriented, it's not even assembly language.  I can't follow control flow, I can't understand variable usage, nothing.

Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...

Quote
So why do you guys use Python?

Because I can focus to the target while programming and code on high level. Btw I understand weaknesses of Python quite good (those are generally bottleneck of current CPython implementation, not problem of language itself) and currently I'm learning Scala. But this discussion is a bit OT here :-).

makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
October 27, 2012, 08:40:20 AM
 #243

BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
kano
Legendary
*
Offline Offline

Activity: 2282


Linux since 1997 RedHat 4


View Profile
October 27, 2012, 09:04:13 AM
 #244

I wouldn't care about it either. If one is implementing Stratum in the first place, including difficulty in each job really doesn't help any.
Just thought I'd quote this ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
kano
Legendary
*
Offline Offline

Activity: 2282


Linux since 1997 RedHat 4


View Profile
October 27, 2012, 09:43:10 AM
 #245

BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.
Yeah they already are doing this - to get around the problem in the design Tongue

https://bitcointalk.org/index.php?topic=108533.msg1286637#msg1286637

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2352


Ruu \o/


View Profile WWW
October 27, 2012, 09:43:48 AM
 #246

To be fair, I had exactly the same discussion with Eleuthria but I had bigger fish to fry and stopped caring about it. He or slush can come and defend why they think it's ok.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
October 27, 2012, 09:44:52 AM
 #247

BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.

It's not a flaw in the protocol, it's a flaw in my current implementation.  It was significantly easier to tie difficulty to the miner connection object, rather than to keep a list of the miner's recent difficulties per work unit.  If I did that, it would allow you to submit "old difficulty" work, and be credited at your old difficulty.  I didn't do this because difficulty adjustments do not happen frequently enough for this to materially affect any miner unless they have a habit of reconnecting every 10 minutes.

I will be making that change to account for "stale olddifficulty" shares in one of the upcoming pool software updates.

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.

RIP BTC Guild, April 2011 - June 2015
kano
Legendary
*
Offline Offline

Activity: 2282


Linux since 1997 RedHat 4


View Profile
October 27, 2012, 09:49:02 AM
 #248

BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.

It's not a flaw in the protocol, it's a flaw in my current implementation.  It was significantly easier to tie difficulty to the miner connection object, rather than to keep a list of the miner's recent difficulties per work unit.  If I did that, it would allow you to submit "old difficulty" work, and be credited at your old difficulty.

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
... i.e. difficulty needs to be an attribute of the work ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
October 27, 2012, 10:14:22 AM
 #249

This has to be discussed many times,please read thread history and dont open this topic again.

kano
Legendary
*
Offline Offline

Activity: 2282


Linux since 1997 RedHat 4


View Profile
October 27, 2012, 10:29:52 AM
 #250

This has to be discussed many times,please read thread history and dont open this topic again.
Sweep it back under the carpet Smiley

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
October 27, 2012, 12:52:20 PM
 #251

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2352


Ruu \o/


View Profile WWW
October 27, 2012, 01:00:07 PM
 #252

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.
btcguild is the only stratum pool with variable diff, and it  sends out a clean work with every difficulty change meaning all work gets thrown out with each diff change. Even blocks and valid shares at the new diff get thrown out I might add.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2282


Linux since 1997 RedHat 4


View Profile
October 27, 2012, 09:47:44 PM
 #253

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.
btcguild is the only stratum pool with variable diff, and it  sends out a clean work with every difficulty change meaning all work gets thrown out with each diff change. Even blocks and valid shares at the new diff get thrown out I might add.
Quite logically, the implementation should be that difficulty should be part of the work, however, previous work on the same block should also be allowed since it is to the miners disadvantage for the pool to throw away valid work ... obviously Tongue

This does, however, lead to the issue that pools may be silly enough to send a 1TH/s miner 1 difficulty work.
Yeah it seems that it is impossible for the pool software programmers to handle this properly ... the solution seems to be that it's the miners problem, not the pools problem, so the miner should lose work because of it ... yet the protocol clearly states that it's the pool that has absolute control of this.

I guess setting a time limit on how much old work is valid (yes there was a reason I made that comment at the end of the post: https://bitcointalk.org/index.php?topic=108533.msg1287015#msg1287015 Tongue) and the pools will just have to put up with this time limit since they are unable to program a suitable solution without throwing away valid miners work.

Hmm - looking at the protocol in the light of this discussion, it really currently is a protocol that gives more power to the pool and takes it away from the miner.
The protocol, by definition, allows the pool to reject shares as it chooses and gives a way to do that, that the protocol defines as valid.
Giving the pool WAY less work to do, and passing that work reduction to the miner to do, yet the miner also loses rights about the pool accepting it's valid work ... hmm ... giving away rights and being made to do more work ... where have I heard that before Cheesy

The point of this protocol was a long term solution, but with getwork using roll-n-time and high difficulty shares, it won't be necessary quite yet.
A 1.5TH/s device can do ~233 nonce ranges a second, so with the roll-n-time limit set to 7000 as it is in cgminer at the moment, that's 30s worth of work ... so there is still some (small? amount) of time left before people are forced to give away rights and do more work ... and in the mean time there may be a better solution ...

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
DavinciJ15
Hero Member
*****
Offline Offline

Activity: 736


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 27, 2012, 11:24:44 PM
 #254

Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...
My bad.  I might as well interject into a conversation of Mensa members.  But "Twisted" is what I would call Python and I'm not the only one. Yet, I must agree you can do things in a very small amount of code in Python.  But I still think it bites.  (pun intended) Smiley

So the war is on.  VHS (Stratum) vs BetaMax (GBT)

It does not matter who is better, only the protocol that will be used will be the winner.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
October 27, 2012, 11:28:12 PM
 #255

Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...
My bad.  I might as well interject into a conversation of Mensa members.  But "Twisted" is what I would call Python and I'm not the only one. Yet, I must agree you can do things in a very small amount of code in Python.  But I still think it bites.  (pun intended) Smiley
Well, Twisted deviates from normal Python code quite a bit - almost bad enough that it's really its own language.
Also, Twisted doesn't work with any current Python versions (3.x).

So the war is on.  VHS (Stratum) vs BetaMax (GBT)

It does not matter who is better, only the protocol that will be used will be the winner.
Hopefully slush's recent steps toward putting Stratum through the BIP process will result in a new protocol that combines the capabilities of both.

DavinciJ15
Hero Member
*****
Offline Offline

Activity: 736


Bitcoin - helping to end bankster enslavement.


View Profile WWW
October 27, 2012, 11:50:03 PM
 #256

BIP process will result in a new protocol that combines the capabilities of both.

OK quick a quick and easy question...

Bitcoind 0.7 has GetBlockTemplate, does that mean I can use my BFL 1.5 TH Mini rig to solo mine using only bitcoind and BFGminer?
I want to underscore this one point, USING ONLY bitcoind and BFGminer, no proxy, no pool, and no other software.

NOTE: a yes, but... is a NO.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
October 28, 2012, 12:01:04 AM
 #257

BIP process will result in a new protocol that combines the capabilities of both.

OK quick a quick and easy question...

Bitcoind 0.7 has GetBlockTemplate, does that mean I can use my BFL 1.5 TH Mini rig to solo mine using only bitcoind and BFGminer?
I want to underscore this one point, USING ONLY bitcoind and BFGminer, no proxy, no pool, and no other software.

NOTE: a yes, but... is a NO.
Yes, this is supported by a beta BFGMiner branch and should be merged before the BFGMiner 3.0 release (which will introduce ASIC support).

optimator
Sr. Member
****
Offline Offline

Activity: 347



View Profile WWW
October 28, 2012, 12:06:23 AM
 #258

Quite logically, the implementation should be that difficulty should be part of the work, however, previous work on the same block should also be allowed since it is to the miners disadvantage for the pool to throw away valid work ... obviously Tongue

The reason this is an issue is because of the way the protocol has been implemented (not necessarily designed). Cleanjobs was designed to tell the miner to drop all work, presumably because a new block has been found. BTC Guild is pushing cleanjobs=true on a difficulty change.

There would not be an issue if difficulty changes were pushed with a new job (around every thirty seconds and with cleanjobs=false and with pools associating the difficulty of each connection with the job). It is trivial to track the associated difficulty of a connection and a new work request (with cleanjobs = false) and to ensure the work submitted matches the difficulty associated with the connection at the time the job was pushed.

A workable solution for the initial 30 seconds, which would not result in share loss, is to push connection specific jobs on difficulty changes for new connections.

Clearly miners benefit from efficient pools and pools benefit from efficient miners. Clearly, no one wants to lose a block because of a difficulty change!

Before we jump to the solution, I think we need to understand the problem. Losing shares is a problem for miners. Perhaps someone else can articulate the problem of difficulty changes between new jobs for the pools?

Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
November 01, 2012, 03:05:37 PM
 #259

I've been looking over the python code Smiley which works pretty well actually, just need to add DB work.

I wish I could read python code, it's not structured code, it's not object oriented, it's not even assembly language.  I can't follow control flow, I can't understand variable usage, nothing.  I could learn it but, I don't believe computer languages should be harder to use for all developers at all levels.  This is my opinion and I understand I may be wrong, I believe I'm right.  Tongue

So why do you guys use Python?
Python is object-oriented procedural code. Twisted twists it into something unreadable.
I can't speak for those who use Twisted, but I use Python generally to make it more accessible to less experienced developers.

DavinciJ15
Hero Member
*****
Offline Offline

Activity: 736


Bitcoin - helping to end bankster enslavement.


View Profile WWW
November 01, 2012, 03:38:09 PM
 #260

I've been looking over the python code Smiley which works pretty well actually, just need to add DB work.

I wish I could read python code, it's not structured code, it's not object oriented, it's not even assembly language.  I can't follow control flow, I can't understand variable usage, nothing.  I could learn it but, I don't believe computer languages should be harder to use for all developers at all levels.  This is my opinion and I understand I may be wrong, I believe I'm right.  Tongue

So why do you guys use Python?
Python is object-oriented procedural code. Twisted twists it into something unreadable.
I can't speak for those who use Twisted, but I use Python generally to make it more accessible to less experienced developers.

Thanks for clearing that up! I may check out regular python then.
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 [13] 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!