Bitcoin Forum
December 09, 2016, 09:43:59 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: [Claimed - 6 BTC bounty] New RPCProtocol for Phoenix  (Read 1655 times)
jedi95
Full Member
***
Offline Offline

Activity: 219


View Profile
July 12, 2011, 06:14:27 PM
 #1

For the past week I have been working on a new RPC implementation for Phoenix since the current one has several issues I have not been able to fix. It doesn't help matters that the original RPC code author, CFSworks, has been MIA for awhile now.

So I decided to rewrite the RPC code, without using the Twisted.web library. I think most of the issues with the current RPC protocol are due to the hacked-in support for keep-alive.

Anyway, what I have right now is basically an adaption of poclbm's RPC code to Phoenix's interface. It makes use of the Twisted library, but only for handling threads/deferreds.

The 6 BTC bounty will go to whoever can write code that meets the requirements below. You can use what I have or write it from scratch, so long as it meets the requirements. The current code successfully connects and gets work, but it doesn't correctly send shares back. Obviously the code must be distributed under the X11 license like the rest of Phoenix. The bounty is payable immediately to the address of your choice once I can confirm that the code meets the requirements.

Requirements:
1. Calls to requestWork() and sendResult() must not block while sending/receiving requests from the RPC server. (these don't need to be thread-safe, they are always called from the main thread)
2. For sent work, a boolean must be returned to the caller of sendResult() indicating accepted/rejected.
3. Sending and receiving work should use separate connections. (but only 1 each, to keep pool load down)
4. Sending and receiving work must not block each other.

Existing code:
http://xp-dev.com/svn/phoenix-miner/branches/NewRPC/RPCProtocol.py

Networking code isn't a strong point of mine, and I just don't have the time to do this myself.

Phoenix Miner developer

Donations appreciated at:
1PHoenix9j9J3M6v3VQYWeXrHPPjf7y3rU
1481276639
Hero Member
*
Offline Offline

Posts: 1481276639

View Profile Personal Message (Offline)

Ignore
1481276639
Reply with quote  #2

1481276639
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481276639
Hero Member
*
Offline Offline

Posts: 1481276639

View Profile Personal Message (Offline)

Ignore
1481276639
Reply with quote  #2

1481276639
Report to moderator
1481276639
Hero Member
*
Offline Offline

Posts: 1481276639

View Profile Personal Message (Offline)

Ignore
1481276639
Reply with quote  #2

1481276639
Report to moderator
1481276639
Hero Member
*
Offline Offline

Posts: 1481276639

View Profile Personal Message (Offline)

Ignore
1481276639
Reply with quote  #2

1481276639
Report to moderator
fpgaminer
Hero Member
*****
Offline Offline

Activity: 546



View Profile WWW
July 13, 2011, 02:27:54 AM
 #2

Quote
So I decided to rewrite the RPC code, without using the Twisted.web library. I think most of the issues with the current RPC protocol are due to the hacked-in support for keep-alive.
Keep Alive is only used for the Long Polling connections, as far as I know. So, do you suspect that whatever hack Phoenix uses to enable Keep Alive is also interfering with getwork requests and submissions? Or is Long Polling the only issue?

I'm just curious and trying to flesh out more information in this thread.

jedi95
Full Member
***
Offline Offline

Activity: 219


View Profile
July 13, 2011, 08:34:24 AM
 #3

Quote
So I decided to rewrite the RPC code, without using the Twisted.web library. I think most of the issues with the current RPC protocol are due to the hacked-in support for keep-alive.
Keep Alive is only used for the Long Polling connections, as far as I know. So, do you suspect that whatever hack Phoenix uses to enable Keep Alive is also interfering with getwork requests and submissions? Or is Long Polling the only issue?

I'm just curious and trying to flesh out more information in this thread.

The keep-alive connections are required for stable operation on some pools due to anti-DDoS protection. Without keep-alive the miner makes a new connection for each getwork() as opposed to re-using the existing one. A good example is Slush's pool, which many users were having problems with prior to adding keep-alive. The current RPCProtocol implementation uses the Twisted.web library, which does not support keep-alive. CFSworks "hacked-in" keep-alive support for Twisted.web so that we could continue to use it. However, I suspect that the idle issues people have been having with Phoenix are caused by a bug in this modification. All my miners run through MMP and I have yet to see a single idle problem, which eliminates the rest of the components as the source of the problem.

Basically, what I want is a new RPCProtocol that doesn't use Twisted.web.

Phoenix Miner developer

Donations appreciated at:
1PHoenix9j9J3M6v3VQYWeXrHPPjf7y3rU
fpgaminer
Hero Member
*****
Offline Offline

Activity: 546



View Profile WWW
July 13, 2011, 10:11:25 PM
 #4

Okay so, given your code, you do this for sending work:

Code:
d = threads.deferToThread(self.getwork, self.sendConnection, data)

Which means that it actually calls getwork. I guess you didn't implement a submitwork function and just used that as a placeholder?

EDIT: I take that back; getwork should be compatible with doing submitwork requests, because of the data parameter.


I don't know Phoenix's code well enough to immediately work on it myself. Seems overly complicated, compared to just having two threads (mining thread and network thread) and two Queues (workQueue and resultsQueue). I actually moved away from Phoenix because of the idle issues, after I hacked phatk into poclbm. poclbm's code is quite simple and indeed uses the two threads with Queues architecture.

fpgaminer
Hero Member
*****
Offline Offline

Activity: 546



View Profile WWW
July 13, 2011, 11:06:12 PM
 #5

Okay I've fixed your code so that it submits shares correctly, tested against eligius. Here are the changes:

Line 82 and 83 changed to:

Code:
        self.postdata = {"method": 'getwork', 'id': 1}
        self.headers = {"User-Agent": USER_AGENT, "Authorization": 'Basic ' + b64encode('%s:%s' % (url.username, url.password)), "Content-Type": 'application/json'}

Changed the id to 1. I doubt it makes a difference, but Phoenix uses 1 normally.
Added a Content-Type header, which is necessary for submitting work, but not getting work ... not sure why. That's just how the pool operates I guess.


Line 290:

Code:
        d = self.sendWork(result.encode('hex'))

You already wrap the data in an array, so that wasn't necessary, and we need to hex the whole result, not just the first byte.

And that's it! I have no tested if this solves the idle problem. Someone else who experiences the idle problem will probably have to test it out for an extended period of time to see if your code fixes it.

jedi95
Full Member
***
Offline Offline

Activity: 219


View Profile
July 14, 2011, 01:27:36 AM
 #6

Okay so, given your code, you do this for sending work:

Code:
d = threads.deferToThread(self.getwork, self.sendConnection, data)

Which means that it actually calls getwork. I guess you didn't implement a submitwork function and just used that as a placeholder?

EDIT: I take that back; getwork should be compatible with doing submitwork requests, because of the data parameter.


I don't know Phoenix's code well enough to immediately work on it myself. Seems overly complicated, compared to just having two threads (mining thread and network thread) and two Queues (workQueue and resultsQueue). I actually moved away from Phoenix because of the idle issues, after I hacked phatk into poclbm. poclbm's code is quite simple and indeed uses the two threads with Queues architecture.

I don't like how poclbm works, since it sends and recieves work in the same thread. This means that if sending results stalls for even a few seconds, the miner is unable to get more work. This is made worse by the fact that poclbm lacks a queue, so it doesn't have a buffer of work for when this occurs.

This is one of the main reasons I decided to make another miner.

Phoenix Miner developer

Donations appreciated at:
1PHoenix9j9J3M6v3VQYWeXrHPPjf7y3rU
jgarzik
Legendary
*
Offline Offline

Activity: 1470


View Profile
July 14, 2011, 01:30:22 AM
 #7


Oh good grief, let's not reinvent the wheel again.

Just use the efficient binary protocol already supported by pushpoold.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
fpgaminer
Hero Member
*****
Offline Offline

Activity: 546



View Profile WWW
July 14, 2011, 02:50:05 AM
 #8

Quote
I don't like how poclbm works, since it sends and recieves work in the same thread. This means that if sending results stalls for even a few seconds, the miner is unable to get more work. This is made worse by the fact that poclbm lacks a queue, so it doesn't have a buffer of work for when this occurs.
Good points. I'll have to add those features to my modified poclbm client  Grin

Did the changes I posted work for you, by the way?

jedi95
Full Member
***
Offline Offline

Activity: 219


View Profile
July 14, 2011, 03:13:24 AM
 #9

Okay I've fixed your code so that it submits shares correctly, tested against eligius. Here are the changes:

Line 82 and 83 changed to:

Code:
        self.postdata = {"method": 'getwork', 'id': 1}
        self.headers = {"User-Agent": USER_AGENT, "Authorization": 'Basic ' + b64encode('%s:%s' % (url.username, url.password)), "Content-Type": 'application/json'}

Changed the id to 1. I doubt it makes a difference, but Phoenix uses 1 normally.
Added a Content-Type header, which is necessary for submitting work, but not getting work ... not sure why. That's just how the pool operates I guess.


Line 290:

Code:
        d = self.sendWork(result.encode('hex'))

You already wrap the data in an array, so that wasn't necessary, and we need to hex the whole result, not just the first byte.

And that's it! I have no tested if this solves the idle problem. Someone else who experiences the idle problem will probably have to test it out for an extended period of time to see if your code fixes it.

Nice work! With these changes I was able to finish a complete RPCProtocol including long polling. Let me know what address you want payment sent to.

Phoenix Miner developer

Donations appreciated at:
1PHoenix9j9J3M6v3VQYWeXrHPPjf7y3rU
fpgaminer
Hero Member
*****
Offline Offline

Activity: 546



View Profile WWW
July 14, 2011, 04:34:04 AM
 #10

Quote
Nice work! With these changes I was able to finish a complete RPCProtocol including long polling. Let me know what address you want payment sent to.
Yay! I'm glad I was able to help, and thank you in advance for the bounty  Cool
1FEmGDo7Csi1S3w3pYuWwoV885UkiWSwx5

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!