Bitcoin Forum

So in the hopes that my findings will help others, I've decided to start this post and document all of the little glitches/oddities I've encountered while preparing my mining rigs.
Aside from the noble goal of helping others, I also hope a more knowledgeable miner can recognize these oddities and either provide confirmation that "Yeah - that's weird but normal" or perhaps even point me in the right direction on how to resolve it.

Before I begin, here's a simplified list of my setup for your comparison

• LinuxCoin 0.2a running persistent from a USB key
• Sapphire ATI 6990 video card

And here are the oddities I've found and learned about:

• Hashing:
  
  • When GPU0 is hashing alone, it hashes at a constant, steady pace. For the purposes of this example, let's say 350000 kh/s
  • When GPU1 is hashing alone, it also has a constant steady rate of 350000kh/s. (example value)
  • When GPU0 and GPU1 are hashing at the same time, GPU0 is a steady 350000kh/s while GPU1 has an erratic rate that jumps between 350000 and 200000kh/s.
  • The moment I'd kill GPU0, GPU1 would become stable at 350000kh/s. When I'd resume GPU0, GPU1 would become erratic again
  • The moment I'd kill GPU1, GPU0 would continue hashing stably at 350000kh/s. When I'd restart GPU1, GPU1 would be erratic
    
  • My first thought was temperature of GPU1 - perhaps it was being throttled for overheating? Nope. GPU1 temp is about 5 degrees cooler than GPU0 temp.
  • My second thought was power - with two 8pin PCIE inputs on the card, one was being fed by a 6-8 adapter. Nope, when the 2nd power lead was replaced with another dedicated 8pin PCIE, the same erratic hashing was observed
  I've come to accept that the 2nd GPU on my 6990 card just hashes erratically when the 1st one is in use.
• Overclocking with aticonfig:
  
  • When I attempt to use aticonfig to set clocks directly (within the peak limits) it does not work
  • When I use AMDOverdriveCtrl to set clocks (within the peak limits), it works great
  • If I launch AMDOverdriveCtrl and then shut it down, and THEN use aticonfig to set clocks, it works like a charm
  • I guess AMDODC does something behind-the-scenes that makes aticonfig work?
  I've come to accept that you just need to AMDODC first before aticonfig-ing.
• Overclocking GPU and Underclocking Memory:
  
  • If I try to overclock the GPU too high, it may hash fine for 5mins, 10mins, or even 2hrs until eventually the system freezes/halts
  • Overclocking a little bit doesn't produce any system halts!
  • When overclocking a little bit, you can have the Memory underclocked by more than 100MHz when compared with the GPU clock. (I had a 135MHz spread that worked great)
  • When overclocking a lot, you must keep the memory underclocked no more than 100MHz when compared with the GPU clock
  • If I attempt to overclock/underclock things that the 6990 doesn't like, it won't give me an error or tell me it failed. It'll silently reset the clocks to factory defaults. I need to check the clocks after setting them to see if the card took the settings.
  • I have read (but haven't tried) that editing the GPU BIOS with RBE allows more reliable overclocking for other cards, however from what I am able to find, 6990 users report that only the voltage can be changed with RBE?
  I've come to accept that overclocking/underclocking a 6990 has very little impact since you can't overclock the GPU too much without freezes, and you can't underclock the memory too much without it being undone. I wish there was a way to adjust the clocks in Linux more reliably.
• Monitoring and remote management:
  
  • If you try to run aticonfig or launch a miner over SSH, it won't work because the ssh session's "display" doesn't have a GPU
  • If you have a GNU screen session open on the machine, you can resume this session over SSH and have full control of everything that screen session had when it was first initialized
  I've come to accept that you need to start a GNU screen session first and then do everything inside it if you want to be able to remotely manage things
• Performance
  
  • My original goal was to have a headless, windowless environment for mining. Everything would exist in CLI land, and no window manager would be necessary. No KDE, no gnome, no "startx." I assumed it would use less GPU power to display a CLI environment than a window manager environment.
  • That doesn't seem possible without perhaps modifying drivers and other low-level things to make the GPUs available to command-line programs
  I've come to accept that you need to use a window manager in Linux in order to access the GPUs

I hope these nuggets of acceptance help some of you out there.
If you are able to provide any insight or assistance into these oddities, I'd be very happy to learn something new!

--TE

Wow bit of a pain. I can overclock both my 5830's on my motherboard in Windows and the hash rate is constant. Was thinking of switching to Linuxcoin today actually.

I tried ubuntu on my first miner, and I come to accept that linux is unsuitable for my mining purposes and I have been using Win7 ever since. So far I could overclock the core, underclock the mem, run reliably for weeks without reboot, and almost no problems so far on win7.

Have you looked at the CPU usage? I know with Folding@Home I would have similar issues in some cases where both GPU processes were fighting over the same CPU core, instead of each using a different one. Each one stable speeds on its own, but jumpy when both ran.

Are you enabling overdrive on the adapter first when you try to do it form the command line?

aticonfig --od-enable --adapter=all

Then try your setclocks.

I'm fairly certain I tried this, yes, but I will doublecheck tonight when I get home.

From memory, aticonfig replied with a message along the lines of "Unable to set clocks. Check that your values are valid and within the ranges" or something like that.
Of course the values were formatted correctly and within the acceptable ranges.

The exact same commandline works after AMDODC executes… I didn't retype the aticonfig line, I used UPARROW and ENTER, so it would not be a typo issue.

I'll re-try tonight and post here to confirm.
Thanks for offering a suggestion!

I run a dual 6990 box, running Ubuntu

It's generally stable (until I push them a little too hard of course...)

You will have to flash the bios on these cards in order to underclock the memory to any significant degree.  This helps.  A lot.  Your heat load will go down considerably.

Instructions are on the interwebs, but essentially create a DOS boot USB with atiflash.exe on it - back up all 4 GPU's (2 gpu's per card) and MAKE SURE you know which bios goes with which.  It matters.

Then move that USB stick over to a windows box, run the radeon bios editor, and set all clocks down to whatever you like for every power profile (I set mine to 150 myself, but you can do whatever).  Save these in a manner so you know which is which.

Back to the linux box.. reboot on dos USB stick, use atiflash to flash your new bios across each GPU.  Reboot into linux, run aticonfig --initial -f --adapter=all and you should now be able to set your memory clocks to whatever you like (after you reboot one last time).

I can PM you my simple .bat files and such if you like, and even give you my modified BIOS images (however, I would suggest you modify your own as there is no guarantee one bios from my card will work from yours).  Just let me know.  No guarantees you won't brick your cards though, of course!

-Phil

Be sure to try using the 11.6 catalyst drivers, they allow clock speeds outside of the displayed ranges.
If you let ubuntu install the ATI driver automatically, it's a bit of a mess to get the generic 11.6 driver to install.
About all I can offer on the topic was that even though I couldnt ever get the 11.6 to install properly, by the time I was done and went back to the driver installed by ubuntu, I was able to get bigger overclocks. 

My crossfired 5830's have the same issue as your 6990. One is pretty constant and the other is all over the place. It doesn't matter which is in the 1st PCIe slot it still happens. Annoying so I try not to look lol Another anomaly is intermittently getting "connection problems" with that second adapter. I'll get home from work to find out it hasn't been running for hours. I was hoping the switch to some form of linux would fix this?   :'(

I had the exact issues as the OP, I was doing the exact thing but in Ubuntu, I also noticed when executing a script to set AMDOverdriveCtrl profiles for each GPU it wouldn't work from screen session and will freeze during initialization. I have since decided go windows OS and use after burn which allowed me to overclock upto 1200/1500 with the overclock switch on the cards turned on.

I currently run 930/830 and am getting 400 MH/s per GPU with a stable headless windows OS using Realvnc to login to the box , I am happy enough with that and 2 card give me a total of 1600 MH/s. I have a 3rd card but took it out due to heat problems(might throw it into my everyday Pc for gaming), 3 6990's in on the same motherbord is too hot, I am going to buy a PCIe ribbon extender and place the 3rd card in the unused drive bays, maybe that will help with the heat problem... If I do this I will need to go back to linux to use all 6 GPU's, otherwise I will stick with windows as it seems stable enough and less hassles to get things working...

All you have to do when you get the "can't run without an X display" error, is add DISPLAY=:0 at the beginning of the command. So take an aticonfig command like setting the fan speed that needs to be run from within an X session. Just run DISPLAY=:0 aticonfig --pplib-cmd "set fanspeed 0 80" and it will work fine. Also, another pro-tip, for commands that use --pplib-cmd, if you want to set your second card, you have to put DISPLAY=:0.1 at the beginning or it will fail.

i've had almost all the same problems as described :D Those which i haven't, is simply because i've not tried to do the same thing.

I have erratic speeds with dual 5870s. Also noticed that screensaver will cause lower hash rate with low end CPU (in my case Celeron 2.66Ghz), which is wierd.
the system with Q9550 is about 0% CPU used, the celeron one is constantly at 50% or so, and causes lower than expected hash rates apparently. Erratic with even GPU0

under windows, stable hash rate on both GPUs. (tho also different GPUs)

There is an easy solution to the rate hopping on the second card.

Using Linuxcoin 0.2a with a machine that has 2x5830, I had the same problem.

The problem is the aggression setting on the first card.  While you lose a touch of speed off the first card, the second becomes rock steady.

Using pheonix I do something like:

Card 1 ( primary ):
DEVICE=0 AGGRESSION=7 FASTLOOPS=true

Card 2 ( primary ):
DEVICE=1 AGGRESSION=11 FASTLOOPS=false

( plus all the other standard options.

This gives me 298mh/s on card one and 301 on card2 ... both rock steady and shares submitted / accepted averages out to be almost exactly the same ( indicating similar hashing rates )

<edit>With identical overclocking settings ( 1010mhz, 300mhz ram, 1.125v )</edit>

Or, you could just add
to your .bashrc script that is sourced every time you login. All of my miners are offsite, and I have no problems launching mining clients on any of them without jumping through all those hoops! Unless there's something specific in LinuxCoin that requires all that; I personally use Ubuntu.

cent