5970 Cooling Solutions

[Cablesaurus]

Those of you who use 1 or more 5970's with BitCoin, I'd like to inquire about your cooling solutions.

I am using the Sapphire 5970 in a Thermaltake case on a Gigabyte ep45-ud3p mobo and E8400 core2duo processor. With stock cooling on the GPU currently.

If I OC with some fairly standard OC settings the stock cooler runs fine at 80% for gaming, and keeps everything below the 70's even in a stress test. BitCoin, not so much, from overclocking the Core Voltage to 1.165 and 900mhz core speed (not touching mem for bitcoin) even with the fan at 100% eventually I push 82-84c and the card starts to downclock, not good.

So I'm considering an after market cooler of some kind and am looking for input from other folks in this situation

Thanks for any input!

[ArtForz]

It's throttling because the VRMs are hitting 120°C.
see http://www.anandtech.com/show/3590, bitcoin mining is about as bad as dnet.
Sadly, for aircooling stock is as good as it gets. The only 3rd party air cooler for 5970 is AC Accelero 5970, and its VRM cooling is *worse* than stock.
So a) go watercooling or b) fiddle around with more case airflow, fans pointed at rear of the card, ..., finally say fuck it and lower Vcore.

[Cablesaurus]

Thank you for the concise response. Do you (or anyone reading) have experience with the water cooling solutions to know which might be best? I have never done a water cooling setup before, but honestly the fan in this GPU is so loud, it would be a welcome change.

I wish I would have known about BitCoin about 3 months ago when I first purchased the card!

[sc8nt4u]

Are you planning on having more than 1 5970?

You're looking at more than half the cost of the 5970 for a good WC setup. Money would be better invested in another 5970 and lowering clocks on both until your board is full then go 3x120mm radiator setups for every 2 cards. Even then I don't know how well that kind of setup will cool 2 5970s as you will be putting out over 350w of heat per card. Plus you'll have space issues with 2 radiators that size.

[Cablesaurus]

So a) go watercooling or b) fiddle around with more case airflow, fans pointed at rear of the card, ..., finally say fuck it and lower Vcore.

ArtForz or anyone if I may ask, do you know if it's possible to bump up the core mhz speed with a slightly lower vcore? I tried lowering vcore from 1.1625 down to 1.1125 just to test... running at just 900mhz core with nothing else raised, sys still locked up on  a test.

I don't know a lot about voltages in GPU's, but does there exist any other sweet spot I could try for any halfway significant gain, aside from 1.1625vcore? I tried actually running 725mhz core and 1.1625vcore in testing (nothing else raised) and things still got too hot, so the issue does seem to be with the max vcore setting.

I see people claiming 650-700khash with their 5970's on here so if there's any suggestions I can tweak, I'll try them. Thanks for any input.

[sc8nt4u]

Your card is locking up because it's not getting enough volts for your core. Unsync them and have individual overclock profiles per core. I run 950core/1000mem at 1.1625v and get roughly 715 Mhash/s. I do not have the cores synced as one core is crappier than the other. So core crappy is @ 940, while the other one is at 960. I can also run 1.10v with core crappy at 900 and the other at 920. Unsyncing them will definitely help you balance core speeds with voltage.

[Cryptoman]

technopagan, which Thermaltake case do you have?  What I've found that works best is a bottom-mount fan of at least 100 cfm drawing cold air from underneath a mid-tower or tower case.  Take a look at this Cooler Master case:

http://www.coolermaster.com/product.php?product_id=6638

I have one 120 mm, 133 CFM Scythe Ultra Kaze drawing air in from the bottom just in front of the PSU and another mounted in the lower side mount.  They are both blowing directly on the GPUs.  I used to run 90+ degrees on the bottom card (with automatic throttling) and 87 degrees on the top card.  They are both now around 70 degrees.

I really don't recommend water cooling.  Trust me, I've played around with it a lot, and it's generally much more money and hassle than it's worth.  It does look cool with the UV coolant though. 

[Cablesaurus]

Your card is locking up because it's not getting enough volts for your core. Unsync them and have individual overclock profiles per core. I run 950core/1000mem at 1.1625v and get roughly 715 Mhash/s. I do not have the cores synced as one core is crappier than the other. So core crappy is @ 940, while the other one is at 960. I can also run 1.10v with core crappy at 900 and the other at 920. Unsyncing them will definitely help you balance core speeds with voltage.

I haven't tried unsynching the cores yet, I'll give that a shot and play around with it.

One thing to note though, that during testing, (synched of course at the time,) I set voltage to 1.1625 and left the core speed @ 725mhz. Even with 100% fan speed I was approaching the critical temperatures at 725mhz. Would unsynching make a difference then here?

[sc8nt4u]

Your card is locking up because it's not getting enough volts for your core. Unsync them and have individual overclock profiles per core. I run 950core/1000mem at 1.1625v and get roughly 715 Mhash/s. I do not have the cores synced as one core is crappier than the other. So core crappy is @ 940, while the other one is at 960. I can also run 1.10v with core crappy at 900 and the other at 920. Unsyncing them will definitely help you balance core speeds with voltage.

I haven't tried unsynching the cores yet, I'll give that a shot and play around with it.

One thing to note though, that during testing, (synched of course at the time,) I set voltage to 1.1625 and left the core speed @ 725mhz. Even with 100% fan speed I was approaching the critical temperatures at 725mhz. Would unsynching make a difference then here?

You do not need to run at 1.1625v on the cores for 725mhz. Stock volts would run that. Unsyncing would help with stabilizing your system at a particular voltage, and optimizing core clock for each core, as each core is slightly different and one will almost always be better than the other.

- I would start at 1.1625v @ 100% fan. I use a combination of Sapphire Trixx and MSI Afterburner. Use Trixx to set the voltage and sync the core speed to a core clock you think may work. Test this configuration and make sure m0mchill's client isn't throwing you errors. Scroll up in the prompt to check your logs. This will take several tries if you don't already know the cores’ limit.

- Using GPU-Z, check your VRM temps at your synced core speed and voltage and make sure it’s not throttling you at this speed; throttling kicks in around 120C for the VRM. If it is throttling, lower your clock speed and voltage. Repeat the previous step until it is not throttling you anymore. When it stops throttling, this particular voltage is very close to the most voltage you will be able to use. I let my VRM temps hang out around 110C so that I can account for room temperature fluctuations throughout the day.

- Once you figure out the maximum voltage and a core speed that works for both cores, use afterburner to unsync the cores and raise one of the core speeds up 5-10mhz, check that GPU’s log for errors. If this is the bad core, your system might crash or start throwing you errors in the log. If it is throwing you errors, revert back to the original core speed for this gpu and make a note of which one it was, GPU1 or GPU2. This is the bad core.
Note: Afterburner messes up my fan speed, I don’t know if it will do it to yours, but if it does, you will need to go back into Trixx to fix it.

- If it is not throwing you errors, then most likely this is the better core and you can keep bumping up the core speed in small increments and checking the log for errors. When the first error comes up or your system hangs, move down 5mhz and retry that core.

- Once you’ve determined the max clocks for each GPU @ 1.xxxx  volts, recheck your VRM temps as this higher clock rate as a higher clock rate at the same voltage will dump a little more heat.

Good luck with overclocking! Do it at your own risk!

[sturle]

So a) go watercooling or b) fiddle around with more case airflow, fans pointed at rear of the card, ..., finally say fuck it and lower Vcore.

ArtForz or anyone if I may ask, do you know if it's possible to bump up the core mhz speed with a slightly lower vcore? I tried lowering vcore from 1.1625 down to 1.1125 just to test... running at just 900mhz core with nothing else raised, sys still locked up on  a test.

Maximum stable with stock voltage and lowish temperature (at least below 80°C, lower is better down to about 60) is about 640 MHz with my card, but this will vary a bit from card to card.  The problem with overvolting, aside from higher temperature, is the fact that your performance per watt will fall significantly.  It would be more profitable in the long run to get another card than to invest in advanced cooling solutions, assuming you pay for the electricity.  A 5770 and a waterblock for a 5970 cost about the same, and the 5770 gives much better Mhash/$.  A stock 5770 gives better performance per watt than an overvolted 5970.  Power consumption is proportional to the frequency (linear) and square of the voltage.

I see people claiming 650-700khash with their 5970's on here so if there's any suggestions I can tweak, I'll try them. Thanks for any input.

I get 620 with mine overclocked to 840 MHz.  I think this is common.  mrb's miner will probably perform a few percent better.

[sc8nt4u]

So a) go watercooling or b) fiddle around with more case airflow, fans pointed at rear of the card, ..., finally say fuck it and lower Vcore.

ArtForz or anyone if I may ask, do you know if it's possible to bump up the core mhz speed with a slightly lower vcore? I tried lowering vcore from 1.1625 down to 1.1125 just to test... running at just 900mhz core with nothing else raised, sys still locked up on  a test.

Maximum stable with stock voltage and lowish temperature (at least below 80°C, lower is better down to about 60) is about 640 MHz with my card, but this will vary a bit from card to card.  The problem with overvolting, aside from higher temperature, is the fact that your performance per watt will fall significantly.  It would be more profitable in the long run to get another card than to invest in advanced cooling solutions, assuming you pay for the electricity.  A 5770 and a waterblock for a 5970 cost about the same, and the 5770 gives much better Mhash/$.  A stock 5770 gives better performance per watt than an overvolted 5970.  Power consumption is proportional to the frequency (linear) and square of the voltage.

I see people claiming 650-700khash with their 5970's on here so if there's any suggestions I can tweak, I'll try them. Thanks for any input.

I get 620 with mine overclocked to 840 MHz.  I think this is common.  mrb's miner will probably perform a few percent better.

How many more watts are needed to jump from 725 stock to 950? I know you'll get about 35% more performance.

He also needs to consider space requirements. If a 5770 is better at Mhash/$ than an OCd 5970, you still need to consider he has a finite amount of PCI-E slots and having a higher density of video cards would be better.

[Cablesaurus]

What's the minimum voltage required to hit 900 or 950 on the 'better' core?

[sc8nt4u]

What's the minimum voltage required to hit 900 or 950 on the 'better' core?

Card dependent. I use 1.1625v @ 955mhz on my good core and 925mhz on the crappier one. YMMV. The new m0mchill client increased my hash rate, but at the same time throws up more errors so I need to clock down.  Currently testing my 2nd 5970.

[Cablesaurus]

You do not need to run at 1.1625v on the cores for 725mhz. Stock volts would run that. Unsyncing would help with stabilizing your system at a particular voltage, and optimizing core clock for each core, as each core is slightly different and one will almost always be better than the other.

By the way, I understand you don't ened to run 1.1625 to run stock 725mhz, however what I meant was in testing, I set just the voltage up (to 1.1625) before increasing any core speed to see how the temperature handled. I still approached mid 70's without increasing the core.

This was of coursed synched, but moreover what I meant was that I essentially raised the core voltage without changing anything else, and was approaching the critical range of temperatures.

If this is useful info at all.

I'll try playing around with unsynching later tonight too.

[Cablesaurus]

I can't seem to actually unlink the cores with MSI Afterburner, newest version of the program, the option is greyed out :/

Edit: Interesting, I can control them individually with the ATI Control Panel, however, voltage isn't setting on the secondary core with MSI, and only with ATI Overvolt which doesnt have any choices between default and 1.1625. Sapphire Trixx locks up the machine and I'm wondering if I have the right version.

[sc8nt4u]

I can't seem to actually unlink the cores with MSI Afterburner, newest version of the program, the option is greyed out :/

Edit: Interesting, I can control them individually with the ATI Control Panel, however, voltage isn't setting on the secondary core with MSI, and only with ATI Overvolt which doesnt have any choices between default and 1.1625. Sapphire Trixx locks up the machine and I'm wondering if I have the right version.

Weird, I can individually individually set core/volts on each one separately but manual fan control doesn't work. I have to go back to CCC to set the fan speed after i apply my settings in afterburner.

Trixx now crashes on me, now that my 2nd 5970 has come. So I've change to use afterburner to set the voltages and use ccc to set the core speed because afterburner's fan control is broken for me. Scaled down all the voltages to 1.15v @ 900 for each one. I'm going to tweak each individual core later as every 25mhz or so you move up it only increases ~ 10 Mhash/sec between 900-950 core. I estimate a loss of about 3-4% of my theoretical Mhash if I don't unsync. Probably going to go solo mining as I'm having latency issues connecting to slush's mining pool and I'm losing 5 seconds or so on each core every couple minutes per core.

[Cablesaurus]

Anyone know enough about Afterburner to know why I wouldnt be able to set volts/corespeed on both cores? :/ I only see 1 slider.

Do you see 2 seperate sliders?

I unchecked the option to "sync like gpu's" and that didn't change a thing.

[sc8nt4u]

Anyone know enough about Afterburner to know why I wouldnt be able to set volts/corespeed on both cores? :/ I only see 1 slider.

Do you see 2 seperate sliders?

I unchecked the option to "sync like gpu's" and that didn't change a thing.

In the same menu did you pull down the box to select GPU1, GPU2, etc..

[Cablesaurus]

Found it, it was under the 'settings' screen
Played around with individual cvolts and cmhz, not really comfortable with the temperatures i'm getting sadly,

With the 'better' core clocked up to 900 / 1.149 , the un-oced core still hits 110c VRM temps and 82-84c+ core temp

Tried sticking an extra case fan pointed at the GPU and it totally didn't help, lol

[sc8nt4u]

I'd lower the voltage by 2 more steps i.e. 1.1625 --> 1.15 --> 1.1375 --> 1.125 etc.. Looks like 2 voltage steps down will get you temps of around 75C core and 100C VRM.

[Cablesaurus]

I have never heard of voltage stepping explained like that and you just made my life much easier lol! Much easier to test and troubleshoot if I have a meter to go on like that. A BIG THANKS to you, and everyone, for the assistance so far.

[Cablesaurus]

Well, I thoroughly tested both of my cores. One does great and the other is horrible!

On the better core, I can sit at a fairly comfortable 76-78c steady with ~98-102c vrm @ 1.1500v @ 950mhz, which brings that core from 270,000khash/s to 350,000khash/s.

While that's really nice, the 2nd core just doesn't hold up to much voltage increase, even stepping up to 1.0750 or 1.1000 @ 800 or 850mhz starts pushing temps on everything up without steadying out.

Not sure at this point if it's worth it to continue overclocking on this machine, that's about 60-80k khash which is maybe a block or two a month. That'd be a nice addition but at the same time running the fan @ 100% (which happens to be my workstation), as opposed to being able to have it at 70% otherwise is a little easier on my ears. Not sure.

[sc8nt4u]

Well, I thoroughly tested both of my cores. One does great and the other is horrible!

On the better core, I can sit at a fairly comfortable 76-78c steady with ~98-102c vrm @ 1.1500v @ 950mhz, which brings that core from 270,000khash/s to 350,000khash/s.

While that's really nice, the 2nd core just doesn't hold up to much voltage increase, even stepping up to 1.0750 or 1.1000 @ 800 or 850mhz starts pushing temps on everything up without steadying out.

Not sure at this point if it's worth it to continue overclocking on this machine, that's about 60-80k khash which is maybe a block or two a month. That'd be a nice addition but at the same time running the fan @ 100% (which happens to be my workstation), as opposed to being able to have it at 70% otherwise is a little easier on my ears. Not sure.

Sounds good . You might be breaking even with the machine with 60-80k in terms of electricity costs probably.

[dingus]

I have only just skimmed through this thread so forgive me if this has been said, but I would like to add that if I underclock memory to its lowest clock (in MSI afterburner) it shaves off a few degrees of heat, without losing any hashes/second.

[brocktice]

I am water cooling 3 5970s with a single 3x120mm radiator. I'm overvolting/overclocking to around 925-950 (varies per GPU). I was doing 1GHz, but when I added a third 5970 the power and heat requirements overwhelmed the system. My core temps are 59-69C.

If you want to water cool, you'll probably have to wait for 6990s. Nobody's making water blocks for 5970s, I got the last one I could find recently. I'll post pics/YouTube video in a moment.

[brocktice]

Thank you for the concise response. Do you (or anyone reading) have experience with the water cooling solutions to know which might be best? I have never done a water cooling setup before, but honestly the fan in this GPU is so loud, it would be a welcome change.

I wish I would have known about BitCoin about 3 months ago when I first purchased the card!

Here we go:

http://www.youtube.com/watch?v=zcYuxRde-30

And here's a pic if you don't like YouTube/videos.


Water cooled bitcoin miner, 3 5970s. by brocktice, on Flickr

It was about $500 for the water cooling parts, but it lets me overclock more, and more quietly, since it sits right next to my desk. I figure over time I get a better return from hardware, especially given my power and sound requirements.

[sc8nt4u]

Hey Brock, I noticed that you're using 3x5970s and it looks like your mobo is out of PCI-E slots. Is this the reason you have 3 and not 4? I only have 3 PCI-E slots and 2x5970s right now, but would like to stab another 5970 in there and get it on linux because windows is limited to 2x5970s(?). That's an interesting way to mount a hard drive .

What software are you using to increase your volts in linux or did you use something like RBE to flash to bios to get the clocks and volts you want.

[brocktice]

Hey Brock, I noticed that you're using 3x5970s and it looks like your mobo is out of PCI-E slots. Is this the reason you have 3 and not 4? I only have 3 PCI-E slots and 2x5970s right now, but would like to stab another 5970 in there and get it on linux because windows is limited to 2x5970s(?). That's an interesting way to mount a hard drive .

What software are you using to increase your volts in linux or did you use something like RBE to flash to bios to get the clocks and volts you want.

I have one more motherboard PCI-E slot, but it's too close to the other one so the last 5970 ('bottom' on the pic) blocks it.

The case is a 4u rackmount case, and it turns out there are no 4u rackmount cases with 8 card slot spaces, so without modding them down to only take 1 slot each, 4 cards is impossible in my setup. I also don't have the power supply capacity. If I wanted to go for 4/motherboard I'd do something like Art's custom cases.

For voltage, I'm using rbe and atiflash. No need to change the clocks, just the core voltages.

[brocktice]

That's an interesting way to mount a hard drive .

Both of my 4u rackmount cases came with hard drive cages on a crossbar like that, I didn't come up with it. I had to rip out all the drive bays to fit in the water cooling equipment, so that's the only place left.

[sc8nt4u]

That's an interesting way to mount a hard drive .

Both of my 4u rackmount cases came with hard drive cages on a crossbar like that, I didn't come up with it. I had to rip out all the drive bays to fit in the water cooling equipment, so that's the only place left.

Ah I see, I guess I'll have to try to find another 5970 then. I have a 1250w PSU only, so I think 3 5970s is about as much as it can handle without wiring a secondary PSU in. I'll go look at watercooling stuff now and see if I want to drop that much.

[brocktice]

Ah I see, I guess I'll have to try to find another 5970 then. I have a 1250w PSU only, so I think 3 5970s is about as much as it can handle without wiring a secondary PSU in. I'll go look at watercooling stuff now and see if I want to drop that much.

1250 is going to be just barely enough for 3 if you are overvolting/clocking.

[sc8nt4u]

Ah I see, I guess I'll have to try to find another 5970 then. I have a 1250w PSU only, so I think 3 5970s is about as much as it can handle without wiring a secondary PSU in. I'll go look at watercooling stuff now and see if I want to drop that much.

1250 is going to be just barely enough for 3 if you are overvolting/clocking.

I know. I'm probably going to underclock my CPU. I'm thinking the 3x5970s are going to be pushing close to 1100w or 92A on my 12V rail.

[sc8nt4u]

It's one of those 4U cases marketed by Compucase  and Codegen. All my gpu's are housed in those, but with air cooling. Nice cases, sturdy and simple and that top bar is very useful. 70-100£ each in UK

Mine is all cluttered in a HAF 932  w/ 6 hard drives too + 2 dvd drives.

[brocktice]

It's one of those 4U cases marketed by Compucase  and Codegen. All my gpu's are housed in those, but with air cooling. Nice cases, sturdy and simple and that top bar is very useful. 70-100£ each in UK

My workstation is in one that looks just like it, but from Rosewill. Those aren't sold anymore, which is too bad, because they're nicer.

[Cryptoman]

I don't see the logic of trying to cram more than two 5970s in a box.  The theory is that you only have to pay for one PSU, one case, one mobo, one HDD, etc.  However, your cost per watt for power supplies goes way up above 800-1000W.  Then, if your PSU takes a dump you have 3-4 GPUs offline.  Also, there aren't many 4 PCIe mobos or cases to hold them.  On the other hand, you can get a nice MSI 870-G45 mobo that has two PCIe slots with airspace between them for 70 USD.  Toss that in a garden-variety bottom-PSU mid tower with a nice fan and 1000W PSU and you're golden.  Overclock until the cows come home and it still runs cool.  Am I wrong?

[Cablesaurus]

Can anyone offer a little bit of input on what temperatures are considered "safe"?

i.e., is there a huge difference in stability (or anything to consider) having a stable 78c core/100c VRM, vs. a higher, 82c core/105c VRM?

[ShadowOfHarbringer]

If you want a really professional cooling, you should start building something like this:

http://www.pugetsystems.com/submerged.php
http://www.youtube.com/watch?v=PtufuXLvOok

Easiest to maintain, easy to cool (just make a loop with a pump and add a radiator from a car to it).

I planned to do something like this myself, if only i had the cash.