|
BTW, I am not going to accept a patch containing a custom concat setup. This is not C.
|
|
|
|
From my first run of profiling the miner, I saw that you were spending about 2% cpu time in just building strings (mainly StringBuilder copying char arrays internally). Using the + operator is inlined to StringBuilder, which can be pretty slow. I ran into this in my game engine here at work and had come across this post at StackOverflow from a guy that implements his own (albeit primitive) class for string concatenation. I forgot to save the profile for that one (the profiler automatically overwrites the output file every time and I'm lazy  ), but I reduced the CPU time spent on String building from 2% to <= .01% It's not much, but hey, it was easy and I knew how to do it  Anyway, here is the latest trace I ran. (a lot is left out, just is just the top 90% of cpu time) CPU TIME (ms) BEGIN (total = 2239712) Fri May 20 19:40:42 2011
rank self accum count trace method
1 17.47% 17.47% 21 306858 java.lang.Object.wait
2 17.46% 34.94% 828 306869 java.lang.ref.ReferenceQueue.remove
3 16.52% 51.46% 16 319564 sun.net.www.http.KeepAliveCache.run
4 15.74% 67.20% 7513448 319281 java.nio.DirectByteBuffer.getInt
5 4.05% 71.25% 210 318093 java.net.SocketInputStream.read
6 2.81% 74.05% 29347 319369 org.lwjgl.opencl.CL10.clEnqueueReadBuffer
7 2.70% 76.75% 7513448 319278 java.nio.Buffer.checkIndex
8 2.69% 79.44% 7513448 319279 java.nio.DirectByteBuffer.ix
9 2.64% 82.08% 7513448 319280 java.nio.DirectByteBuffer.getInt
10 1.80% 83.88% 675014 319312 org.lwjgl.opencl.CL10.clSetKernelArg
11 1.36% 85.24% 675014 319313 org.lwjgl.opencl.InfoUtilFactory$CLKernelUtil.setArg
12 1.01% 86.25% 675015 319298 java.lang.ThreadLocal.get
13 1.00% 87.26% 675016 311203 java.lang.ThreadLocal$ThreadLocalMap.getEntry
14 0.98% 88.24% 675015 319302 java.nio.DirectIntBufferU.put
15 0.68% 88.92% 29348 319351 org.lwjgl.opencl.CL10.clEnqueueNDRangeKernel
16 0.63% 89.55% 675015 319307 org.lwjgl.PointerWrapperAbstract.getPointer
17 0.63% 90.18% 675012 319315 java.lang.ThreadLocal$ThreadLocalMap.access$000
18 0.62% 90.80% 675015 319305 org.lwjgl.BufferChecks.checkBufferSize
Now I've started looking at some of the bigger stuff. The first 2 lines are from the garbage collector, so you can see that ~35% of the CPU time was spent on just garbage collecting, 17% of which was spent just blocking all the execution threads in order to do so. So I'm trying to figure out ways to improve that. I don't really think that the netcode can be much faster, but another ~20% of cpu time is spent on that. So if the netcode can be improved, that will get us back into the kernel faster. The third line there is the thread that is used for keeping the HTTP 1.1 session alive. I don't know much about that, but maybe it's a lead. Anyway, I'm done for now. Here is the new DiabloMiner.java with the new string builder. Also: So... ymm so fucking v.
I totally agree with that, but I love your code and bitcoin is fascinating. So digging through this code is a great joy for me! Great work so far man, and in Java too!  You need to get in the habit of using Github to push merge requests. Also, the thread pool is very important because it can keep HTTP connections open between getwork/sendworks (cutting down on network round trip). Further, I spawn 3 threads per GPU to cut down on the actual hit of blocking due to HTTP, and then on top of that, LP cuts down on needing to keep fetching work every 5 seconds (with LP it only fetches as it returns asynchronously , or when nonce saturation occurs). As for blocking on garbage collection, switching to Java 7 altogether would do a lot to improve that. |
|
|
|
I got the profiler working... that was a lot easier than I thought it would be. I haven't done too much Java outside of Google's DalvikVM, but it's not a true Java implementation so some things are done a little bit differently. Anyway, I'm running the first batch of samples now  Are you going to be modifying the kernel much? I'm curious as to how phatk reduced the operation count by that amount... I did a lot of examining of phatk. I can't tell where he thinks hes saving cycles. Not only that, it runs exactly the same on SDK 2.1 and SDK 2.4 on my 5850 vs phoenix's standard kernel. Plus, if he is in fact exploiting anything, it probably isn't exploiting it as much as -v 3 -w 128 on mine on 69xx. The key difference is not in the total number of instructions executed, but that they make better use of the 5-wide ALU design. Have a look at the ASM generated with AMD's KernelAnalyzer. Particularly the number of ALU ops. It's no faster than the poclbm kernel on 2.1, but for most people it eliminates the speed disadvantage of SDK 2.4. It's also designed with VLIW5 in mind, so it's obviously not going to be optimal on VLIW4 hardware. Well the big problem is on 2.4 phoenix-poclbm and phatk give near identical results... and both are still slower than real poclbm on both 2.1 and 2.4. And -v 18 and 19 give interesting results on 58xx on 2.4 which beats phatk's lackluster speed. So... ymm so fucking v. |
|
|
|
|
Update: Added all of Dustin's suggestions, and also added a timeout for non-LP connections.
|
|
|
|
:-(
Well, its why OpenCL was invented in the first place. Using GLSL for even generic computing tasks that don't fit the OpenGL workflow is problematic and really not worth it. I don't want to shoot down anyone's hopes, but Mesa already is growing OpenCL support for Gallium. What more could we possibly ask for? best wishes ? quicker Gallium3D adoption ? quicker Mesa development[along with Gallium3D] ? better drivers[esp free drivers. now about 10x times slower than proprietary counterparts] ? more suitable SDK' ? AMD/NVidia support for both developers and OpenCL itself[today Intel CPU's had better OpenCL support than Nvidia GPU's]. and yes OpenCL/OpenGL ES is cool. at least in theory. heil glorious OpenMAXdeveloper Mesa and Gallium are open source projects, you can always start developing for them. |
|
|
|
Is it normal for a 5850 to run at 900 mhz core stable at stock voltage?
...cause that's what mine is running at right now (doing ~340 mhash/s)
I have mine stable at 918 (as far as the slider goes) on stock voltage, but I also have it under 75c, so ymmv. |
|
|
|
I got the profiler working... that was a lot easier than I thought it would be. I haven't done too much Java outside of Google's DalvikVM, but it's not a true Java implementation so some things are done a little bit differently. Anyway, I'm running the first batch of samples now Are you going to be modifying the kernel much? I'm curious as to how phatk reduced the operation count by that amount... I did a lot of examining of phatk. I can't tell where he thinks hes saving cycles. Not only that, it runs exactly the same on SDK 2.1 and SDK 2.4 on my 5850 vs phoenix's standard kernel. Plus, if he is in fact exploiting anything, it probably isn't exploiting it as much as -v 3 -w 128 on mine on 69xx. |
|
|
|
:-(
Well, its why OpenCL was invented in the first place. Using GLSL for even generic computing tasks that don't fit the OpenGL workflow is problematic and really not worth it. I don't want to shoot down anyone's hopes, but Mesa already is growing OpenCL support for Gallium. What more could we possibly ask for? |
|
|
|
two 5870s, CC 11.5, SDK 2.1, on Debian testing.
i don't know yet how much faster it is than your pre-BFI_INT release.
but a lot.
i'm putting in some extra fans and a rheostatic fan speed controller - it's so damn fast that i have to clock it down right now to keep temps under 85.
so going from the old version, max volted at 300 MemClock and 900 GPUClock, to the new version down-volted by almost 0.2, MemClock at 315 and GPUClock at 850; i picked up a bit over 100 Mh/s.
i'll have the new fans and controller in tomorrow. i have another box that i've experimented with fans on - just a single 5870, but i've learned a bit. i'm hoping for a maxed-out setup on the dual box, running at well under 75 degrees. we'll see.
At stock 850, 2 5870 should be in the neighborhood of 740 using -v 2 -w 128 on SDK 2.1. BFI_INT adds around 10%. |
|
|
|
anyone with [deprecated]Render Monkey experience ?
note: using shader allow you to extract MORE from you GPU. also CAL implementation might me considerably faster than OpenCL one[why, how and some optimisation tricks was nuff said by BarsWF creator already].
and such experience also handy for porting solver in other areas[Linpack/Livermore shader ? Biology ? Meteorology ? Radiophysic]
ArtForz has a CAL miner that is slightly faster than the CL kernel poclbm and I based ours on. Its not particularly a huge win, especially when SDK 2.5 is getting rid of CAL support. Also, I have experimented with GLSL-based miners. The lack of real integer support kills the whole thing. |
|
|
|
Known issues: DiabloMiner exits without notification if the graphics card is unsupported. Hostnames cannot be set with a protocol prefix. (Should be supported in future versions.) Donations: 14V7RiwD3joqDF2Jv6zst3ZnQsR686u9iH  If you wanna use full urls, use -l http://user:password@hostname:port/Also, you should also include the donation addresses for all the miners you support with your script. |
|
|
|
is there any way I can use the miner without installing java?
can you put it in a warper and compile the whole thing including the java?
Java does not work that way. I call bullshit: http://gcc.gnu.org/java/I'm aware of gcj, and I do not consider something that cannot run quite a few apps, and be a shitload slower at it an actually valid Java implementation. Oh, and last time I noticed, they didn't do JNI yet, so you can't run my miner with it. |
|
|
|
I really like Diablo Miner but there is few problems with it:
1. When I run the new Diablo miner versions with .exe instead of .bat in it, it does not work. I get black console screen for maybe 25ms and it exits. Not enough time to even hit Pause button to see what's wrong. I need tu use .bat file instead.
2. No speed improvements on new version with BFI_INT. I even get speed decrease. I get 260 Mh/s with 2011-04-23 version and with 2011-05-19 I get 250 Mh/s
I use ATI HD5850 with Catalyst 10.11 and ATI SDK 2.1
You run the exe the same way as the bat. The exe does not magically read your mind on what arguments you want to use. The bat is probably running the old jar, which means, no, you're not running a new version of DiabloMiner. |
|
|
|
I'm not a programmer, so I don't know what I'm talking about here, but could such a linux binary permit graphics hardware too old to use the current miners to contribute at a respectable hash/watt rate even if they such at the hash/second rate?
No. They lack the hardware design to run programs like this, plus they would be amazingly slow. |
|
|
|
If that was true, then somebody would have collected the $3000+ in free money.
Not at all. Xorg nor FDO accept donations in BTC. |
|
|
|
Become an open source hero, and help bitcoin too!
OK, I think this project would see some real return (in BTC) on Linux, for all the miners out there. It would benefit open source as well.
The Project
-------------------------------------------------------
Successfully load and execute a sha256 "compute shader", using 100% open source video drivers on Linux (using closed source ATI tools to produce shader binary is permitted). Any Linux OS/distribution, as long as it's a recent version. Must work on r600/r700 ATI hardware.
This is pointless. R600 cannot run compute shaders of the kind we need, R700 (Radeon 4xxx) suck at it. Also, Mesa has a prototype OpenCL compiler for Gallium targets. Your "bounty" is technically already completed before you started. If you really want to help open source, go work on that project instead. |
|
|
|
is there any way I can use the miner without installing java?
can you put it in a warper and compile the whole thing including the java?
Java does not work that way. |
|
|
|
Just do a java -cp xyz -agentpath:wherever/your/profiler/lib/is com.diablominer.DiabloMiner.DiabloMiner Since classpath can be a pain, and Maven is quite good at handling dependencies just tell mvn to build the cp string: mvn package dependency:build-classpath and it will print the classpath. Edit: intended as general tip, not just for this case Or he can just look in DiabloMiner-Linux.sh and copypasta accordingly. |
|
|
|
Hey just a quick note on how much "Power" you need to finish one share from one block. This is only an "average" on how long it takes to solve one. I did a quick summary that I run at 345MHases/s it took me 57s to do 5 shares, will give you 3933MHashes/Share or on an avg 1Share/11.4s. This is just ball parking small numbers.. In your case.. CPU is very slow compared to an ATI card.. same goes for that GeForce.. Looking at https://en.bitcoin.it/wiki/Mining_hardware_comparisonI'd say your system puts about maybe for fun 10MHashes/s that would make one share that's 3933MHashes / 10 = 393.3s or 6min 33sec for one share. In that time my system would have computed about 34,5 shares aka 34,5 times faster.. Is it worth it?? I donno.. the system just jumped up another difficulty bump yesterday, from 152274 to 244139. So it will just take longer for full blocks to finish now. That means it take more shares on avg find the solution == less bitcoins for us all.. Cheers! You need approximately 2^33 hashes to find a block. At 75 mhash, you would complete 50 shares in roughly 45 minutes. |
|
|
|
As far as OSX having poor OpenCL implementation, that sure seems to be the case. DiabloMiner running in OSX gives me about 80 ~ 90 MHash / sec and switching over to Windows to run DiabloMiner gives me 185 ~ 205 MHash on the same gear. So it is certainly better to be running Windows.
However, your improvements will be marginal at best since you're running NVidia gear... if you really, really want to give it a try, you should definitely install Windows or Linux for maximum performance.
Its pretty bad. On Radeon 5xxx, you lose about 40% of your speed, on Radeon 4xxx and Nvidia, you lose about 25%. |
|
|
|
|
Show Posts
