python OpenCL bitcoin miner

[jimbobway]

I went home for lunch and I am physically at my computer...back to work for me.  I have not tried puddinpop's client.

[1714215834]

1714215834

[1714215834]

1714215834

[1714215834]

1714215834

[1714215834]

1714215834

[1714215834]

1714215834

[m0mchil]

Jimbo, please try this and see if the tool shows OpenCL is supported. It would be helpful if you post a screenshot.

[GeorgeH]

I get a 2-3% increase on the new version. Not much, but a consistent increase. I have my hashes scaled back to about 150/m so this increases it to about 153/m.

It jumps around a little bit of course, but I don't really see it dip below 153/m


Edit-
Don't get me wrong either. 3000k/hs is an extra block a month so no complaints!

Wow! What hardware are you running?

On a more related note, I get this error when running m0m's release (8800 GTS, Driver 258.96)

Code:

C:\Users\Owner\Desktop\BitcoinOpenCL>poclbm
Traceback (most recent call last):
  File "poclbm.py", line 46, in <module>
  File "pyopencl\__init__.pyc", line 138, in program_build
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce 8800 GTS 512' at 0x2dd1600>:

Error: Code selection failed to select: 05102178: i32 = bswap 04EFFBA8

[BitLex]

i get similar results with my nvidia card

Code:

C:\Documents and Settings\Administrator\Desktop\bitcoin_stuff\poclbm_py2exe>poclbm
Traceback (most recent call last):
	File "poclbm.py", line 46, in <module>
	File "pyopencl\__init__.pyc", line 138, in program_build
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce GTX 260' at 0x24325c0>:

Error: Code selection failed to select: 0x4ea2128: i32 = bswap 0x516dca0

here's a screenshot of Caps Viewer and GPU-Z (for driver-info).

[jimbobway]

ok , i have no opencl support (see image) 

[GeorgeH]

Mine's an ATI 5870.

...
run that and see if it shows OpenCL is supported. Give us a screen shot. There are some trouble shooting steps on their website.

Wow, nice!

Here's my screenie:

[jimbobway]

Jim, I don't know where to help from here. Two other people have OpenCL support on their Nvidia card, and still can't get it to work. Without having a set in front of me, it's hard to troubleshoot it. I do have an Nvidia card, same one as you just about, but I don't know if I'm going to swap it out to test. I'm sure our set up varies greatly and I don't know if I would be able to make any beneficial advancements doing it, sorry.

It's ok.  I'm sure something awesome will come out, but I'm happy to test this for momchil.  thought my 8800 would have OpenCL support but it doesn't.  Thanks anyway.

[sandos]

On a more related note, I get this error when running m0m's release (8800 GTS, Driver 258.96)

Code:

C:\Users\Owner\Desktop\BitcoinOpenCL>poclbm
Traceback (most recent call last):
  File "poclbm.py", line 46, in <module>
  File "pyopencl\__init__.pyc", line 138, in program_build
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce 8800 GTS 512' at 0x2dd1600>:

Error: Code selection failed to select: 05102178: i32 = bswap 04EFFBA8

I get something very similar on a 9400GT, but in linux! No idea how to fix this.

[SmokeTooMuch]

has ANYONE got this working on a nvidia card ?

[bethel]

I have two machines that behave totally differently under this python client.

They both have the same hardware and OS:
AMD Semperon Single core
1GB RAM
ASUS ATI Radeon HD 5850 overclocked to 950mhz

One machine does 300-305mhash/s and stresses the gpu to 94-97%
The other does 282-289mhash/s and stresses the gpu to 89-91%

How do I increase my slow machine to get the extra 20mhash? Is there a command line way to tighten the processing loop?

[SmokeTooMuch]

Are both machines using the same driver version ?

[bethel]

They are the same.
8.14.10.0779 Catalyst 10.9 on Windows 7 64-bit

I would not expect them to be different because one is a clone of the other.

[m0mchil]

bethel, what are 5870s showing at stock frequency (850 MHz)? Perhaps one of the cards throttles down?

Latest version has new parameter, -w, to set work group size. I have best results on 5770 with -w 128 -f 35.

[m0mchil]

The IDs stay the same as long as there are no changes in hardware (new cards for example).

The -w parameter sets the number of 'work group size' (local threads). Unfortunately it's very difficult to determine the optimal value for this because it's different on different hardware platforms. The default is to use maximum reported by OpenCL for the specific hardware. This also was the behavior of previous versions. Last days I tried an optimization which tests two hashes in kernel run and it is 1-2% slower with default local threads (5770 - 256), but 5-6% faster with half of them (128) and global threads little bit more (-f 35). Generally, with default parameters one should achieve about the same or slightly worse performance than before.

[jef.blanc]

Hello All,

I tried without success to use the Python OpenCL Miner, here's what I did and what I got :

First my config, I run on Windows 7 Pro 32 bits with a NVIDIA GeForce 9800 GT.

I downloaded and installed CUDA Toolkit from :
http://developer.nvidia.com/object/cuda_3_2_toolkit_rc.html#Windows%20XP,%20Windows%20Vista%20and%20Windows7
-> cudatoolkit_3.2.7_win_32.msi

I downloaded Boost 1.44.0 and built it with :

Code:

bjam.exe toolset=msvc --with-python --with-date_time --with-thread threading=multi link=shared

as did davidonpda on the post #81.
Then I copied all the dll from H:\boost_1_44_0\stage\lib to system32.

I downloaded and built pyopencl-0.92 :
1) running > python configure.py
2) modifying siteconf.py :
(example from http://www.mail-archive.com/pyopencl@tiker.net/msg00349.html)

Code:

BOOST_INC_DIR = [r'H:\boost_1_44_0']
BOOST_LIB_DIR = [r'H:\boost_1_44_0\stage\lib']
BOOST_COMPILER = 'msvc'
BOOST_PYTHON_LIBNAME = ['boost_python-vc90-mt-1_44']
USE_SHIPPED_BOOST = False
CL_TRACE = False
CL_ENABLE_GL = False
CL_INC_DIR = [r'E:\NVIDIA GPU Computing Toolkit\CUDA\v3.2\include']
CL_LIB_DIR = [r'E:\NVIDIA GPU Computing Toolkit\CUDA\v3.2\lib\Win32']
CL_LIBNAME = ['OpenCL']
CXXFLAGS = ['/EHsc', '/DBOOST_PYTHON_NO_PY_SIGNATURES']
LDFLAGS = ['/FORCE']

3) running > ctags -R src || true
4) running > python setup.py build
 -> compilation without error
5) running > python setup.py install
 -> installation without error

I downloaded poclbm from http://github.com/m0mchil/poclbm
-> m0mchil-poclbm-cf33815.zip

And trying to run it : > python poclbm.py

Code:

Traceback (most recent call last):
  File "poclbm.py", line 5, in <module>
    import pyopencl as cl
  File "E:\Python26\lib\site-packages\pyopencl-0.92-py2.6-win32.egg\pyopencl\__init__.py", line 3, in <module>
    import pyopencl._cl as _cl
ImportError: DLL load failed: La procÚdure spÚcifiÚe est introuvable.

(the text in french with the fucked accents means : The specified procedure could not be found).

Note : I don't make all running together the first time, it took a long time and I can forget some step.

I also tried poclbm_py2exe.7z that I got from the board (post #87), first with the original OpenCL.dll in the archive, I got an error due to my Nvidia card I gess :

Code:

No device specified, you may use -d to specify one of the following

Traceback (most recent call last):
  File "poclbm.py", line 57, in <module>
  File "pyopencl\__init__.pyc", line 138, in program_build
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce 9800 GT' at 0x28d2be8>:

:37: error: incompatible type assigning 'int', expected 'uint2'
        work[5]=0x00000000;
               ^~~~~~~~~~~
:38: error: incompatible type assigning 'int', expected 'uint2'
        work[6]=0x00000000;
               ^~~~~~~~~~~
...

then I tried by replacing the dll with the one from the CUDA Toolkit, and I got the same error than while running poclbm I built :

Code:

  File "poclbm.py", line 5, in <module>
  File "pyopencl\__init__.pyc", line 3, in <module>
  File "pyopencl\_cl.pyc", line 12, in <module>
  File "pyopencl\_cl.pyc", line 10, in __load
ImportError: DLL load failed: La procÚdure spÚcifiÚe est introuvable.

So the problem come from OpenCL dll from Nvidia !

I checked _cl.pyd with dependency walker, if I'm correct (this is the first time I use this tool), some functions are missing in the OpenCL.dll from Nvidia :

Code:

clSetUserEventStatus (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1051,31))
clCreateUserEvent (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1062,20))
                  (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1063,32))
clCreateSubBuffer (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1165,22))
                  (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1168,36))
clEnqueueReadBufferRect (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1416,27))
clEnqueueWriteBufferRect (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1460,27))
clEnqueueCopyBufferRect (pyopencl-0.92\src\wrapper\wrap_cl.hpp(1496,27))

In the file OpenCL.dll in poclbm_py2exe.7z, those 6 functions exist !

I got the explication with a little more search on the net, with the confirmation on those two pages :
http://www.khronos.org/registry/cl/sdk/1.0/docs/man/xhtml/
http://www.khronos.org/registry/cl/sdk/1.1/docs/man/xhtml/

The 6 functions where added in OpenCL 1.1, and Nvidia OpenCL implements the version 1.0 !

So if I'm correct pyopencl won't work on Nvidia cards until Nvidia release an OpenCL 1.1 compliant Toolkit and driver.
See the post below.

Note from http://developer.nvidia.com/object/opencl.html

On the same day Khronos Group announced the new OpenCL v1.1 specification update (June 14th, 2010), NVIDIA released OpenCL v1.1 pre-release drivers and SDK code samples to all GPU Computing registered developers.

So you have to register as a "GPU Computing developer" to get the SDK and driver, I don't do it for now (you have to fill a big form which is verified before validation, and I'm not "GPU Computing developer"   ).

PS : I think you might have guessed, I'm French. sorry for mistakes and strange turns of phrase that can be  

[jef.blanc]

After writing the post above, I checked the file pyopencl-0.92\src\wrapper\wrap_cl.hpp to see the use of the 6 functions that are not in OpenCL 1.0, and I saw an interesting point :

The use of the 6 functions is "protected" by blocks #ifdef CL_VERSION_1_1 / #endif.
So I will check why CL_VERSION_1_1 was defined when I compiled the wrapper and will try to compil it without this macro defined.

More informations to come ... later (it is now 2h33 AM in France, time to go to bed )

[m0mchil]

jef.blanc, try to replace the rotate() function with some native rotate left. rotate() is compiled to bit_align AMD specific instruction on AMD. Also there are problems with casts... I switched to uint2 vectors and all operations with mixed types (vector, scalar) work fine on AMD, but Nvidia compilator has problems.

[GeorgeH]

Definitely hoping to see this on nvidia in the near future! I have 2 graphics cards now and can only use one with the CUDA client!

[jef.blanc]

After verification, the macro CL_VERSION_1_1 is defined in CL/cl.h from the Cuda Toolkit I installed, and this file declare the OpenCL 1.1 functions (it's logic in fact, I should have a compilation error otherwise) !

So the Toolkit I installed take in charge OpenCL 1.1, and my current Nvidia driver (that I didn't update) is still on OpenCL 1.0.

I checked on the "public" drivers on the net, the version is still 258.96 (the one I use), so I downloaded and installed the "Developer Drivers for WinVista and Win7 (260.61)" from http://developer.nvidia.com/object/cuda_3_2_toolkit_rc.html#Windows%20XP,%20Windows%20Vista%20and%20Windows7 (the page on which I download the CUDA Toolkit).

My OpenCL.dll file now provides the OpenCL 1.1 functions.
I tried to run poclbm.py, as m0mchil said there is a problem with the rotate function, but pyopencl seems to work*, that's a good point

* There are still some errors while running the tests provided with pyopencl, but I don't know if this is crucial.

Errors on rotate :

Code:

<program source>:204:2: error: no matching function for call to 'rotate'
        sharound(D,E,F,G,H,A,B,C,R(61),0xA4506CEB);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:4:103: note: instantiated from:
#define sharound(a,b,c,d,e,f,g,h,x,K) {h=(h+(rot(e, 26)^rot(e, 21)^rot(e, 7))+(g^(e&(f^g)))+K+x); t1=(rot(a, 30)^rot(a,
19)^rot(a, 10))+((a&b)|(c&(a|b))); d+=h; h+=t1;}
                                                                                                      ^
<program source>:2:19: note: instantiated from:
#define rot(x, y) rotate(x, (uint)y)
                  ^~~~~~
<built-in>:3025:26: note: candidate function
ulong16 __OVERLOADABLE__ rotate(ulong16, ulong16);
                         ^
<built-in>:3024:25: note: candidate function
ulong8 __OVERLOADABLE__ rotate(ulong8, ulong8);
                        ^
<built-in>:3023:25: note: candidate function
ulong4 __OVERLOADABLE__ rotate(ulong4, ulong4);
                        ^
<built-in>:3019:25: note: candidate function
ulong2 __OVERLOADABLE__ rotate(ulong2, ulong2);
                        ^
<built-in>:3018:25: note: candidate function
long16 __OVERLOADABLE__ rotate(long16, long16);
                        ^
<built-in>:3017:24: note: candidate function
long8 __OVERLOADABLE__ rotate(long8, long8);
                       ^
<built-in>:3016:24: note: candidate function
long4 __OVERLOADABLE__ rotate(long4, long4);
                       ^
<built-in>:3012:24: note: candidate function
long2 __OVERLOADABLE__ rotate(long2, long2);
                       ^
<built-in>:3011:25: note: candidate function
uint16 __OVERLOADABLE__ rotate(uint16, uint16);
                        ^
<built-in>:3010:24: note: candidate function
uint8 __OVERLOADABLE__ rotate(uint8, uint8);
                       ^
<built-in>:3009:24: note: candidate function
uint4 __OVERLOADABLE__ rotate(uint4, uint4);
                       ^
<built-in>:3005:24: note: candidate function
uint2 __OVERLOADABLE__ rotate(uint2, uint2);
                       ^
<built-in>:3004:24: note: candidate function
int16 __OVERLOADABLE__ rotate(int16, int16);
                       ^
<built-in>:3003:23: note: candidate function
int8 __OVERLOADABLE__ rotate(int8, int8);
                      ^
<built-in>:3002:23: note: candidate function
int4 __OVERLOADABLE__ rotate(int4, int4);
                      ^
<built-in>:2998:23: note: candidate function
int2 __OVERLOADABLE__ rotate(int2, int2);
                      ^
<built-in>:2997:27: note: candidate function
ushort16 __OVERLOADABLE__ rotate(ushort16, ushort16);
                          ^
<built-in>:2996:26: note: candidate function
ushort8 __OVERLOADABLE__ rotate(ushort8, ushort8);
                         ^
<built-in>:2995:26: note: candidate function
ushort4 __OVERLOADABLE__ rotate(ushort4, ushort4);
                         ^
<built-in>:2991:26: note: candidate function
ushort2 __OVERLOADABLE__ rotate(ushort2, ushort2);
                         ^
<built-in>:2990:26: note: candidate function
short16 __OVERLOADABLE__ rotate(short16, short16);
                         ^
<built-in>:2989:25: note: candidate function
short8 __OVERLOADABLE__ rotate(short8, short8);
                        ^
<built-in>:2988:25: note: candidate function
short4 __OVERLOADABLE__ rotate(short4, short4);
                        ^
<built-in>:2984:25: note: candidate function
short2 __OVERLOADABLE__ rotate(short2, short2);
                        ^
<built-in>:2983:26: note: candidate function
uchar16 __OVERLOADABLE__ rotate(uchar16, uchar16);
                         ^
<built-in>:2982:25: note: candidate function
uchar8 __OVERLOADABLE__ rotate(uchar8, uchar8);
                        ^
<built-in>:2981:25: note: candidate function
uchar4 __OVERLOADABLE__ rotate(uchar4, uchar4);
                        ^
<built-in>:2977:25: note: candidate function
uchar2 __OVERLOADABLE__ rotate(uchar2, uchar2);
                        ^
<built-in>:2976:25: note: candidate function
char16 __OVERLOADABLE__ rotate(char16, char16);
                        ^
<built-in>:2975:24: note: candidate function
char8 __OVERLOADABLE__ rotate(char8, char8);
                       ^
<built-in>:2974:24: note: candidate function
char4 __OVERLOADABLE__ rotate(char4, char4);
                       ^
<built-in>:2970:24: note: candidate function
char2 __OVERLOADABLE__ rotate(char2, char2);
                       ^
<built-in>:2967:24: note: candidate function
ulong __OVERLOADABLE__ rotate(ulong, ulong);
                       ^
<built-in>:2966:23: note: candidate function
long __OVERLOADABLE__ rotate(long, long);
                      ^
<built-in>:2965:23: note: candidate function
uint __OVERLOADABLE__ rotate(uint, uint);
                      ^
<built-in>:2964:22: note: candidate function
int __OVERLOADABLE__ rotate(int, int);
                     ^
<built-in>:2963:25: note: candidate function
ushort __OVERLOADABLE__ rotate(ushort, ushort);
                        ^
<built-in>:2962:24: note: candidate function
short __OVERLOADABLE__ rotate(short, short);
                       ^
<built-in>:2961:24: note: candidate function
uchar __OVERLOADABLE__ rotate(uchar, uchar);
                       ^
<built-in>:2960:23: note: candidate function
char __OVERLOADABLE__ rotate(char, char);
                      ^

Edit : I just see the rot function in poclbm.py (the one m0mchil was talking in fact I gess), I will try to see how to change the two rot functions, and perhaps bytereverse in btc_miner. cl ?

Edit2 : pfiuuu, it's hard !
first time I work on python and therefore on pyopencl, and using the nvidia toolkit. I don't even know from where the rotate functions come (the candidates functions), where are they defined (nividia toolkit, boost, standard c++, ... and from which include) ?

Edit3 : ...
I made some tries, and I don't succed for now:

Code:

Traceback (most recent call last):
  File "poclbm.py", line 57, in <module>
    miner = cl.Program(context, kernelFile.read()).build()
  File "E:\Python26\lib\site-packages\pyopencl-0.92-py2.6-win32.egg\pyopencl\__init__.py", line 138, in program_build
    "Build on %s:\n\n%s" % (dev, log) for dev, log in build_logs))
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce 9800 GT' at 0x2312eb8>:

Error: Code selection failed to select: 0x527c7f8: i32 = bswap 0x527cbb0

... I have no idea of what that means  

(for this test I just change #define rot(x, y) rotate(x, (uint)y) to #define rot(x, y) rotate(x, (uint2)y) in btc_miner.cl, trying to make it running first before checking the operations made).

Edit 4 :
a "funny" thing, the problem I have come from the end of the function search in btc_miner.cl

If I remove those lines

Code:

    if((H.x==0) && (bytereverse(G.x)<=target))
    {
        output[0] = 1;
        output[1] = nonce.x;
    }

    if((H.y==0) && (bytereverse(G.y)<=target))
    {
        output[0] = 1;
        output[1] = nonce.y;
    }

the script is built without error and start.

If I leave the tests without output[n] = ... it builds, and if I leave output[n] = ... without the tests, it builds also.
but if I put the test and output[n] = ... I have the following error :

Code:

Traceback (most recent call last):
  File "poclbm.py", line 57, in <module>
    miner = cl.Program(context, kernelFile.read()).build()
  File "E:\Python26\lib\site-packages\pyopencl-0.92-py2.6-win32.egg\pyopencl\__init__.py", line 138, in program_build
    "Build on %s:\n\n%s" % (dev, log) for dev, log in build_logs))
pyopencl.RuntimeError: clBuildProgram failed: build program failure

Build on <pyopencl.Device 'GeForce 9800 GT' at 0x2952eb8>:

Error: Code selection failed to select: 0x4f5a8d0: i32 = bswap 0x4f94670

[m0mchil]

Jef, at least it's now clear the problem is in the bytereverse(x) macro. Please try to replace it with:

uint bytereverse(const uint x)
{
   uint result;
   uchar* b = (uchar *)&x;
   uchar* l = (uchar *)&result;
   l[0] = b[3];
   l[1] = b[2];
   l[2] = b[1];
   l[3] = b[0];
   return result;
}