Phoenix - Efficient, fast, modular miner

[bitclown]

No idle miners, but I did find a completely unrelated bug in long polling. If the server sends a message over long polling, then long polling will stop working until you either restart the miner or the connection to the server is lost/regained. This has been fixed in SVN already, and it will be included in the next release.

I've been running SVN rev 97 for a couple of hours now with BitcoinPool, and the idling issue still persists. At most I waited 10 minutes before restarting Phoenix, it then connected and fetched new work immediately.

Python 2.6.6
Twisted 11.0.0

[1714207106]

1714207106

[1714207106]

1714207106

[1714207106]

1714207106

[1714207106]

1714207106

[1714207106]

1714207106

[shivansps]

Your best option would be to modify ConsoleLogger to log only the information you want. The hashrate/accepted/rejected are reported separately from log messages so it should be easy to make it output exactly what you want and nothing else.

im not a python programer, but i think it can be made by adding:

   def updateStatus(self, force=False):
        #only update if last update was more than a second ago
        dt = time() - self.lastUpdate
        if force or dt > self.UPDATE_TIME:
            rate = self.rate if (not self.miner.idle) else 0
            type = " [" + str(self.connectionType) + "]" if self.connectionType is not None else ''
            status = (
                "[" + formatNumber(rate) + "hash/sec] "
                "[" + str(self.accepted) + " Accepted] "
                "[" + str(self.invalid) + " Rejected]" + type)
            self.say(status)
            fileHandle = open ( 'status.txt', 'w' )
            fileHandle.write(status)
            fileHandle.close()
            self.lastUpdate = time()

? i think i need to check how to add a new command line argument to use it for the file name..... mmm google here i go!

[jedi95]

No idle miners, but I did find a completely unrelated bug in long polling. If the server sends a message over long polling, then long polling will stop working until you either restart the miner or the connection to the server is lost/regained. This has been fixed in SVN already, and it will be included in the next release.

I've been running SVN rev 97 for a couple of hours now with BitcoinPool, and the idling issue still persists. At most I waited 10 minutes before restarting Phoenix, it then connected and fetched new work immediately.

Python 2.6.6
Twisted 11.0.0

Hmm, well I still can't fix it if I don't know what's wrong. This means I have to see it for myself or someone has to post highly detailed information. (run with -v to add verbose output, then post the log messages.)

EDIT:
Mostly what I need to know is if you have a "Disconnected from server" and then "Failed to connect, retrying...." after it goes idle. If my understanding of the issue is correct there is a problem with the disconnect being reported and thus it never tries to reconnect and just sits there. If you don't get these messages then I have the right idea. Otherwise I have been looking in the wrong place.

[bitclown]

Mostly what I need to know is if you have a "Disconnected from server" and then "Failed to connect, retrying...." after it goes idle. If my understanding of the issue is correct there is a problem with the disconnect being reported and thus it never tries to reconnect and just sits there. If you don't get these messages then I have the right idea. Otherwise I have been looking in the wrong place.

I don't get any error or diagnostics messages besides "Warning: work queue empty, miner is idle", even after sending SIGINT. Here's the full log:

Code:

$ svn up
At revision 97.
$ ./phoenix.py -u http://PonyBaloney:***@bitcoinpool.com:8334/ -v -k poclbm DEVICE=1 VECTORS BFI_INT AGGRESSION=12
[19/05/2011 02:05:04] Phoenix r86 starting...
[19/05/2011 02:05:05] Connected to server
[19/05/2011 02:05:05] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:05] New block (WorkQueue)  
[19/05/2011 02:05:05] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:16] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:16] Result 000000002dd85b62... accepted
[19/05/2011 02:05:16] Result 00000000eafda01c... accepted
[19/05/2011 02:05:27] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:27] Result 00000000d1f4cb13... accepted
[19/05/2011 02:05:28] Result 000000000d44f7cb... accepted
[19/05/2011 02:05:29] Result 0000000037d080df... accepted
[19/05/2011 02:05:33] Result 000000001ea49bfd... accepted
[19/05/2011 02:05:36] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:39] Result 000000002d3fc4f9... accepted
[19/05/2011 02:05:47] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:53] Result 000000008e7ae512... accepted
[19/05/2011 02:05:56] Server gave new work; passing to WorkQueue
[19/05/2011 02:05:59] Result 000000004da386a2... accepted
[19/05/2011 02:06:01] Result 00000000dfa6d714... accepted
[19/05/2011 02:06:09] Result 00000000583c8de4... accepted
[19/05/2011 02:06:16] Warning: work queue empty, miner is idle
[19/05/2011 02:06:19] Server gave new work; passing to WorkQueue
[19/05/2011 02:06:20] Server gave new work; passing to WorkQueue
[19/05/2011 02:06:21] Result 0000000090497502... accepted
[19/05/2011 02:06:29] Server gave new work; passing to WorkQueue
[19/05/2011 02:06:40] Server gave new work; passing to WorkQueue
[19/05/2011 02:06:50] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:01] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:09] Result 000000005a08c825... accepted
[19/05/2011 02:07:17] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:17] New block (WorkQueue)              
[19/05/2011 02:07:18] LP: New work pushed                
[19/05/2011 02:07:18] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:19] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:27] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:29] Result 00000000b913c353... accepted
[19/05/2011 02:07:38] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:49] Server gave new work; passing to WorkQueue
[19/05/2011 02:07:52] Result 0000000095161e35... accepted
[19/05/2011 02:07:55] Result 000000007ad2d6b7... accepted
[19/05/2011 02:07:58] Server gave new work; passing to WorkQueue
[19/05/2011 02:08:05] Result 0000000035d60b6f... accepted
[19/05/2011 02:08:08] Server gave new work; passing to WorkQueue
[19/05/2011 02:08:17] Result 00000000fa673e1c... accepted
[19/05/2011 02:08:19] Result 000000004a857f84... accepted
[19/05/2011 02:08:19] Server gave new work; passing to WorkQueue
[19/05/2011 02:08:29] Server gave new work; passing to WorkQueue
[19/05/2011 02:08:49] Warning: work queue empty, miner is idle
[0 Khash/sec] [19 Accepted] [0 Rejected] [RPC (+LP)]^C          <- Waited 5 minutes before interrupting.
$

Even if this turns out to be a pool failure, shouldn't there be some timeouts for retries?

[shivansps]

BTW, increasing queue size cant help those ppl having "miner is idle"? i think it worth a try.


I think this should do the hash/s log(export) thing.

phoenix.py

   def _parse(self):
        parser = OptionParser(usage="%prog -u URL [-k kernel] [kernel params]")
        parser.add_option("-v", "--verbose", action="store_true",
            dest="verbose", default=False, help="show debug messages")
        parser.add_option("-k", "--kernel", dest="kernel", default="poclbm",
            help="the name of the kernel to use")
        parser.add_option("-u", "--url", dest="url", default=None,
            help="the URL of the mining server to work for [REQUIRED]")
        parser.add_option("-q", "--queuesize", dest="queuesize", type="int",
            default=1, help="how many work units to keep queued at all times")
        parser.add_option("-a", "--avgsamples", dest="avgsamples", type="int",
            default=10,
            help="how many samples to use for hashrate average")
        parser.add_option("-lt", "--logtotext", dest="logtotext", default="none",
        
        self.parsedSettings, args = parser.parse_args()

   def makeLogger(self, requester, miner):
        if not self.logger:
            self.logger = ConsoleLogger(miner, self.parsedSettings.verbose, self.parsedSettings.logtotext)
        return self.logger

ConsoleLogger.py

   def __init__(self, miner, verbose=False, logtotext):
        self.verbose = verbose
        self.miner = miner
        self.logtotext = logtotext
        self.lastUpdate = time() - 1
        self.rate = 0
        self.accepted = 0
        self.invalid = 0
        self.lineLength = 0
        self.connectionType = None

       
    def updateStatus(self, force=False):
        #only update if last update was more than a second ago
        dt = time() - self.lastUpdate
        if force or dt > self.UPDATE_TIME:
            rate = self.rate if (not self.miner.idle) else 0
            type = " [" + str(self.connectionType) + "]" if self.connectionType is not None else ''
            status = (
                "[" + formatNumber(rate) + "hash/sec] "
                "[" + str(self.accepted) + " Accepted] "
                "[" + str(self.invalid) + " Rejected]" + type)
            self.say(status)
            if(self.logtotext != "none")
                fileHandle = open ( logtotext, 'w' )
                fileHandle.write(status)
                fileHandle.close()
            self.lastUpdate = time()
        

I gona try it once i figured out how to compile something in Python hahaha.

[benjamindees]

Python is an interpreted language.  There's no need to compile anything.  You can use py2exe to create an executable but it's just a wrapper.

[JorgePasada]

I just got this up and running on my rig.

1x 5970 and 2x 5850's /w another 5850 on the way.

Using SDK v 2.4, pain in the a$$ to find a couple errors caused by that. I'm pulling average numbers currently, lots of variance 900mhash to 1.2ghash depending on the time.

Can anyone recommend any settings I can tweak to decrease the level of variance for each GPU?

[jedi95]

Mostly what I need to know is if you have a "Disconnected from server" and then "Failed to connect, retrying...." after it goes idle. If my understanding of the issue is correct there is a problem with the disconnect being reported and thus it never tries to reconnect and just sits there. If you don't get these messages then I have the right idea. Otherwise I have been looking in the wrong place.

I don't get any error or diagnostics messages besides "Warning: work queue empty, miner is idle", even after sending SIGINT. Here's the full log:

Even if this turns out to be a pool failure, shouldn't there be some timeouts for retries?

This confirms that the cause is disconnects/timeouts not being detected. I think I know what I need to change, but it's somewhat harder for me at the moment because CFSworks wrote the entire RPC implementation and he hasn't been around for more than a week.

I just got this up and running on my rig.

1x 5970 and 2x 5850's /w another 5850 on the way.

Using SDK v 2.4, pain in the a$$ to find a couple errors caused by that. I'm pulling average numbers currently, lots of variance 900mhash to 1.2ghash depending on the time.

Can anyone recommend any settings I can tweak to decrease the level of variance for each GPU?

Use the phatk kernel, AGGRESSION between 7 and 12, VECTORS and BFI_INT. You might also see an increase with WORKSIZE=128, but the default of 256 is working optimally for me on 5870s.

[Internet151]

I can also confirm even with -v enabled phoenix 1.48 will sometimes not reconnect/keep going after a work queue is empty miner is idle message and no further info.

Sometimes this will cause all 9 of my GPU's in my 3 pc's to stop, and sometimes a few of them will pick back up.

[Eikill]

I need some help. I have a Sapphire 5770 card with no overclock running on SDK version 2.1 and driver version 10.11 (Because I've heard they are best for 5xxx cards, is this true?) and when I run the  miner I get this error,

Code:

C:\pminer>phoenix -u http://@deepbit.net:8332/
 -k poclbm DEVICE=1 VECTORS BFI_INT FASTLOOP=false AGGRESSION=11
Traceback (most recent call last):
  File "phoenix.py", line 123, in <module>
  File "Miner.pyc", line 75, in start
  File "phoenix.py", line 111, in makeKernel
  File "kernels\poclbm\__init__.py", line 22, in <module>
    import pyopencl as cl
  File "zipextimporter.pyc", line 82, in load_module
  File "pyopencl\__init__.pyc", line 3, in <module>
  File "zipextimporter.pyc", line 98, in load_module
ImportError: MemoryLoadLibrary failed loading pyopencl\_cl.pyd

I think there is something wrong with OpenCL or something, but I've tried to reinstall both SDK and drivers with no success.

[shivansps]

use 11.4/11.5 and sdk 2.1, anything lower than 11.3 does not work for mining for me, the GPU device cant be seen by miners, be phoenix or guiminer.

Funny thing is that openCL acually works on gpu using older drivers (directcompute benchmark incluides a opencl benchmark), so i have no idea of what is going on.

[JayC]

Anybody else having problems connecting to the SVN?

[jedi95]

I *THINK* I finally figured out the idling problem and why I can't reproduce it. The current behavior is to use persistent (keep-alive) connections to the server. If the connection is busy when another request is sent, a new connection is made. However, this is limited to 2 connections. If there is an attempt to create another connection when 2 already exist then it will block until one of the other connections is closed. This never happens though, so it blocks forever with no error messages.

This explains why jondecker76 was getting the issue VERY quickly on the slow wireless adapter, because with a slow internet connection there is a much higher chance of the connection being busy when a new request needs to be sent.

It also explains my inability to reproduce the problem because I am running on a very stable wired connection.

In any case, I have temporarily removed the connection limit in order to confirm that this is indeed the cause of the problem. If this fixes the issue, then I will do a more permanent fix later.

However, as explained above I can't reproduce the problem myself so I need users to test this fix. You can either download the binaries below or checkout the latest SVN revision. (98 or 99 is fine)

Download 1.48 debug build
Windows binaries
Source code/Linux release (requires Python, Twisted, and PyOpenCL)

SVN:
http://svn3.xp-dev.com/svn/phoenix-miner/

[bitclown]

Nice work! Testing r99 now. Will let you know how it turns out.

[Inaba]

I seem to be running into a strange problem with the latest build.  It seems to be dong the idle thing, but instead of locking up, just not processing anything - there's no error generated or any apparent problems communicating with RPC (at least it's not reporting it), but its acting like it and bringing the hash rate down.

[jedi95]

I seem to be running into a strange problem with the latest build.  It seems to be dong the idle thing, but instead of locking up, just not processing anything - there's no error generated or any apparent problems communicating with RPC (at least it's not reporting it), but its acting like it and bringing the hash rate down.

Can you post the log? Unless you get "Work queue empty, miner is idle" you probably have a different issue. That message is logged by WorkQueue so it's independent of any protocol changes. Basically it means the kernel requested work and there is nothing in the queue to give it. If this doesn't appear it means that either the queue isn't empty or the kernel is not requesting work. (this could be caused by a driver error, hardware issues, ect)

[jondecker76]

I ran revision 97 for about 24 hours on a wired connection and finally one of my 3 instances went idle...  Just updated to revision 99, will see how it does!

[marcus_of_augustus]

Anybody using the "phatk" kernel on HD 5970 ??

Did a quick test on one of my cores and didn't see any noticeable difference over pocblm (less if anything..... SDK 2.1, fglrx 11.2, clocks  800/300)

[jedi95]

Anybody using the "phatk" kernel on HD 5970 ??

Did a quick test on one of my cores and didn't see any noticeable difference over pocblm (less if anything..... SDK 2.1, fglrx 11.2, clocks  800/300)

It's only beneficial on SDK 2.4. For 2.1 use poclbm.

[charbo]

i have a 2 6950 (FLASHED to 6970) in crossfire i cant pass the limit of 355 Mhash/s using this command line for each device
[ start /dC:\phoenix phoenix -v -u http://mi******@gmail.com_0:*****@www.deepbit.net:8332 -k phatk device=0 WORKSIZE=128 VECTORS BFI_INT AGGRESSION=12 ]

i was hoping with this crossfire get closer to 375 Mhash/ per gpu.

thanks in advance