Dual 6990. Underclocking Memory. Bios Flash Useless?

[CrazyGuy]

I started another thread about this in a linuxcoin environment. Basically it appears you cannot have a difference of more than 100mhz between core and memory clocks.
https://bitcointalk.org/index.php?topic=39895.0

After a bit of research I found a few people saying that they were able to surpass this limitation in Windows, so I gave it a shot. AOCLBF allows me to set the following core/mem clocks 830/250 but I know it isn't really sticking due to high temperatures. I confirmed my suspicions when changing to 830/730 and seeing a 5 degree temp drop. This leads me to believe there really is a limitation in the bios.

I've also seen quite a few posts in other forums stating that the 6990 does not take kindly to clock rate changes in the bios. So my question is, has anyone successfully modified their 6990 bios to get past these issues? Or even better, am I just using the wrong tool? I've also tried Afterburner(625mhz lower limit) and ATI tray tools(Only lets me control 2 cores)

[sje397]

Using AMDOverdriveCtrl on Linux, I can underclock the memory to 120mhz below the GPU frequency. I suspect the limit is 128mhz but I haven't tuned it any finer.

[CrazyGuy]

Yeh I was able to do the same in with OverDrive and ATIConfig but I didn't push it any further than 100. I suppose 28 mhz is better than nothing but surely someone here is running at a 940\330 or some other nice spread. I had my dual 6970 rig going at 960\330 and 426 mh/s at sub 75 C . With this rig, the best I can do is underclock to 700\600(311 mh\s per core) and still get 80 C + temps on my second cores. First cores stay around 65 C. I'm not going to run any of these cores over 78 C so until I find a solution my coin generating operation is dead in the water.

[sje397]

I had one card with pretty even temps across both cores, and ran at about 80-85 for a couple of months. The other one was much worse, with one core at around 75 and the other at between 95 and 100. I've since fixed it by watercooling, but I'm thinking that perhaps taking the fans off and re-doing the heat conducting paste/pads might balance the temps a bit better.

[CanaryInTheMine]

you can thank AMD for these restrictions.

if you flash the bios, the windows driver seems to recognize this and system will not work.

some people have been successful under linux after flashing bios.

see my posts here:
https://bitcointalk.org/index.php?topic=35792

and read this too:
https://bitcointalk.org/index.php?topic=37642

[Phil21]

BIOS Flash works perfectly.

You just can't boot the cards in Windows - the ATI driver will call a bluescreen on purpose, halting the machine.  Annoying.

When I get 6990's in, I flash them to 150mhz minimum RAM clock, and they tend to run anywhere from 915-950mhz at around 85C.  Some cores as mentioned by another poster in this thread just are horrible, such as one that maxes out at 910Mhz @ a running temp of 98C.  I'd say average is around 940mhz @ 89C - if you like pushing your gear that hard at least.  There is a lot of variation on these, but it seems cards of later manufacture do better than the earlier ones. 

I have not seen a difference clocking RAM higher than 150Mhz, but others report otherwise.  YMMV.

You absolutely *must* underclock RAM on these to get any form of thermal room for stable overclocks, I've found.  Also it saves about 50-75w per card.  I had issues keeping them stable prior to flashing them.

fwiw I expect 840Mhash out of each card, and tend to average 850ish it seems these days.

-Phil

[CanaryInTheMine]

BIOS Flash works perfectly.

You just can't boot the cards in Windows - the ATI driver will call a bluescreen on purpose, halting the machine.  Annoying.

When I get 6990's in, I flash them to 150mhz minimum RAM clock, and they tend to run anywhere from 915-950mhz at around 85C.  Some cores as mentioned by another poster in this thread just are horrible, such as one that maxes out at 910Mhz @ a running temp of 98C.  I'd say average is around 940mhz @ 89C - if you like pushing your gear that hard at least.  There is a lot of variation on these, but it seems cards of later manufacture do better than the earlier ones. 

I have not seen a difference clocking RAM higher than 150Mhz, but others report otherwise.  YMMV.

You absolutely *must* underclock RAM on these to get any form of thermal room for stable overclocks, I've found.  Also it saves about 50-75w per card.  I had issues keeping them stable prior to flashing them.

fwiw I expect 840Mhash out of each card, and tend to average 850ish it seems these days.

-Phil

so you are running something like 915-950/150 under linux?

[CrazyGuy]

BIOS Flash works perfectly.

You just can't boot the cards in Windows - the ATI driver will call a bluescreen on purpose, halting the machine.  Annoying.

When I get 6990's in, I flash them to 150mhz minimum RAM clock, and they tend to run anywhere from 915-950mhz at around 85C.  Some cores as mentioned by another poster in this thread just are horrible, such as one that maxes out at 910Mhz @ a running temp of 98C.  I'd say average is around 940mhz @ 89C - if you like pushing your gear that hard at least.  There is a lot of variation on these, but it seems cards of later manufacture do better than the earlier ones.  

I have not seen a difference clocking RAM higher than 150Mhz, but others report otherwise.  YMMV.

You absolutely *must* underclock RAM on these to get any form of thermal room for stable overclocks, I've found.  Also it saves about 50-75w per card.  I had issues keeping them stable prior to flashing them.

fwiw I expect 840Mhash out of each card, and tend to average 850ish it seems these days.

-Phil

Thanks for the info, those are really good hash rates.I chose not to flash and just put them up for sale in the marketplace forum. I do have a pair of XFX 6990s that have been on backorder through Amazon for the past 2 months that I may give it another try when they come in.
https://bitcointalk.org/index.php?topic=40055.0

[Phil21]

so you are running something like 915-950/150 under linux?

Yep, pretty much.  Here is one new set of 6990's under burn-in, about 36 hours of data:
[ALL (30s):1686.7  (avg):1686.9 Mh/s] [Q:18011  A:41153  R:270  HW:0  E:228%  U:22.95/m]
GPU 0: [418.6 / 419.4 Mh/s] [Q:3558  A:10141  R:58  HW:0  E:285%  U:5.69/m]
GPU 1: [424.4 / 423.9 Mh/s] [Q:4674  A:10368  R:72  HW:0  E:222%  U:5.81/m]
GPU 2: [420.1 / 419.6 Mh/s] [Q:3602  A:10050  R:71  HW:0  E:279%  U:5.63/m]
GPU 3: [423.7 / 424.0 Mh/s] [Q:4639  A:10361  R:67  HW:0  E:223%  U:5.81/m]

The cores are clocked at 935, 945, 935, and 945 respectively.  So an average of 421Mhash/core I guess (the second number is the total average since it started up).  Once put into "production" it will be in a cooler place with more airflow, so I'd expect clocks to go up 5-10mhz a core.  These two cards are performing better than average so far, but nothing exceptional.  Memory clocks are 150Mhz across the board.

Temps are:
Adapter 0 - AMD Radeon HD 6990
            Sensor 0: Temperature - 78.50 C
Adapter 1 - AMD Radeon HD 6990
            Sensor 0: Temperature - 74.50 C
Adapter 2 - AMD Radeon HD 6990
            Sensor 0: Temperature - 80.00 C
Adapter 3 - AMD Radeon HD 6990
            Sensor 0: Temperature - 75.50 C

No special cooling here - just running in a cool (68F ambient) basement, inside a large full ATX case.  This is used for burn-in, then the cards are put in a caseless environment with far more airflow on them.  I do run the fans @ 100% 24/7 - I have had good luck so far, but others report early fan death doing so.  Either way, I expect to swap out fans every 9-12 months in these, but have yet to have to do so.

The kill-a-watt has held steady at 880w, using a 1000W Kingwin PSU.  Prior to the memory underclocking, ramping up all GPU's at default clockspeed read 1090w before it crashed, so memory underclocking = will save you money on power, and PSU's!  Do it!

These temps have held steady and are typical.  When in production, this is also targeted at 85C per core, vs. the 80C/core during burn-in.  Some cores simply underperform/run hot of course, especially some of the earlier run units.  I have yet to see higher clocks than 955Mhz be stable, but I also don't spend a whole lot of time tuning this area.

There is lots of random old information/mis-information about these cards out there.  Wanted to get some actual data put forth based on documented data.

Hope it helps someone!

[AssemblY]

on windows, amd gpu clock tool does not work?

[CanaryInTheMine]

on windows, amd gpu clock tool does not work?

To the best of my knowledge it doesn't...

Does it work for you?

[CrazyGuy]

So, I put my 6970s back in today and it looks like they have the exact same restrictions under Linux. I was able to underclock mem 125 under core but anything past that reverts mem back to stock.

I have the same issue with all apps except atitraytools in windows. For some reason that application allows me to set really low clocks while others show low clocks but temps are 6-7 degrees higher so I know it's not sticking.

Unfortunately tray tools doesn't handle more than 2 GPUs very well so I couldn't get it to work with the 6990s. However, the fact that one tool works in windows leads me to believe there's got to be another tool out there that handles 3 or more GPUs...

[CanaryInTheMine]

So, I put my 6970s back in today and it looks like they have the exact same restrictions under Linux. I was able to underclock mem 125 under core but anything past that reverts mem back to stock.

I have the same issue with all apps except atitraytools in windows. For some reason that application allows me to set really low clocks while others show low clocks but temps are 6-7 degrees higher so I know it's not sticking.

Unfortunately tray tools doesn't handle more than 2 GPUs very well so I couldn't get it to work with the 6990s. However, the fact that one tool works in windows leads me to believe there's got to be another tool out there that handles 3 or more GPUs...

you'd have to flash the BIOS.  should work under linux fine.  if you flash the bios and try to use under windows, the drivers will crash.

I wish someone here could debug this and write a fixer app to trick the driver into thinking that the card was not flashed...  this is beyond my ability...

[AssemblY]

on windows, amd gpu clock tool does not work?

To the best of my knowledge it doesn't...

Does it work for you?

Yes, very well!