3 x 6970 in Windows...Help needed (.5 BTC bounty) [SOLVED]

[jake262144]

Perhaps the GPUs were drawing more juice than the mobo could feed them using the thin PCB lines?
It wouldn't be one of those mobos with an extra molex connector just for powering the PCI-E slots?

[1714118395]

1714118395

[1714118395]

1714118395

[-ck]

Perhaps the GPUs were drawing more juice than the mobo could feed them using the thin PCB lines?
It wouldn't be one of those mobos with an extra molex connector just for powering the PCI-E slots?

Maybe. That's what I was wondering about too. The perennial mining favourite, the GD70.

[jake262144]

Conman, there is a reason the new LGA2011 ATX-size boards all sport the additional molex connector.
Those are quite possibly the first consumer-grade boards designed with heavy multi-GPU action on the radar.

A traditional mobo gets only two 12V lines with the 24-pin connector.
The ATX standard defines up to 75W power usage per PCIE slot but the manufacturers never envisioned such a scenario for all the slots.
Two, perhaps even three devices but not a power hungry monster in each and every single PCIE slot.

There have also been reports of the 24-pin connector burning down.
Those pins can carry up to 13 A of juice (the HCS variation) but the el-cheapo phosphor bronze variety maxes out as 8 A.There are two of those so 16 A * 12 V = 192 W total.
With the HCS pins, you're safely at 26*12 = 312 W.

The question stands, however, what levels of electromigration does 300 W of juice subject the thin PCB traces to in 24/7 mode?
I'm betting high and I'm also betting that's what brought your board on its knees.
I went with Gigabyte boards for those double-thickness traces they're boasting about. When I need another board I'll do the research again - perhaps more manufacturers are using thicker PCB traces now.

The gd70 is the favourite because it's the most cost-effective board, not necessarily the best. Also, you're doing (like me) one thing differently than most miners seem to be doing: you're packing the rigs into PC cases and that means no extenders. More specifically, no POWERED extenders. All the juice flowing through the mobo.

[cypherdoc]

Take a good look at the pic of the case being used:
http://ck.kolivas.org/pictures/Mining/IMG_1048.JPG

(ignore the crossfire bridges, they've been removed).

All the heat in that case goes out vertically upwards with reference cards that blow in that direction.

on my cards the fans would blow to the left using your picture, not vertical.

The case fan underneath is 180mm (there are 2) and blows directly into the cards with no obstruction whatsoever when closed.
The PSU is not even effectively *in* the case, having its own section so not contributing to case heat nor having to suck any heat out of its own accord.
The back is massively ventilated as well.
The case is a full sized tower, not a mini or midi tower.
The cards have spacers between them as shown in the earlier photo.
All the cards have their memory underclocked to exactly 125 below their optimal engine clock speeds (the cards run engines from 950-1005).
The cards were ordered so that the fastest ones were put in slot 0 and 3 since they generated the most heat they needed the best ventilation.

what cards are you using?  is it possible to do this config with 4 6970's?

cgminer basically ran in all its auto settings to keep the fans running optimal.
Virtually all GPU fans ran in the 45-75% range during winter and most ran at 85% in summer (none ever needed to go to 100%), yet the temps were targetted and kept at 75 degrees.
This is all I can remember offhand.


...All in all this is a purpose built rig with a lot of thought into making it work within a case, and the case was carefully chosen. It is not just components slapped together. It ran 6 months without a single hiccup. What *has* taken this rig down, though, is a dead motherboard. Not sure what I could have done to prevent that one though.

do you usually leave the side panel off?  do you use any additional external fans?

edit:  just figured out one of my fans failed...

[RandyFolds]

I believe the standard defines 1.5cm of movement or something like that. This is only .5cm yet makes a massive difference to temperatures.

what are those pegs made of and where can i get some?  i'm having temp issues.

i got cgminer installed and working!

Great!

They're... pegs... as in clothes pegs, for a clothes line. I just pulled them apart. You get them... everywhere and these particular ones are made of... plastic.

i don't know how you're keeping your temps under control with cards that close.  i have 2 cards that close and the top one is running 102C using cgminer with fan at 100%. 

Vertical orientation on the motherboard, FTW. Those raven cases are pretty spiffy. Venting heat out of the top makes a hell of a lot more sense than trying to work it out the back, past a bunch of things that are  sucking it up again.

[cypherdoc]

I believe the standard defines 1.5cm of movement or something like that. This is only .5cm yet makes a massive difference to temperatures.

what are those pegs made of and where can i get some?  i'm having temp issues.

i got cgminer installed and working!

Great!

They're... pegs... as in clothes pegs, for a clothes line. I just pulled them apart. You get them... everywhere and these particular ones are made of... plastic.

i don't know how you're keeping your temps under control with cards that close.  i have 2 cards that close and the top one is running 102C using cgminer with fan at 100%. 

Vertical orientation on the motherboard, FTW. Those raven cases are pretty spiffy. Venting heat out of the top makes a hell of a lot more sense than trying to work it out the back, past a bunch of things that are  sucking it up again.

i agree.  where do you get Raven cases?

[jake262144]

Silverstone RV02 and RV03 "Raven" to be precise.
Both models rock; I'm very happy with my RV02's performance as Con is with his RV03's. RV03 is smaller and comes with one fan less but is also a tad cheaper.
You can jury-rig fans aplenty to either of those cases with a few zip ties and some duct tape.
Try NewEgg

[RandyFolds]

I've also got the RV02. It looks ultra sharp, and having the whole bottom lined with 180mm fans that have speed switches is awesome. It also has dust filters, which are like maintenance gold. I haven't found a single dog or cat hair in my box, I just pull the filters and shop-vac them off and I'm golden. Also, as jake pointed out, because of the vertical orientation, the PSU draws air from outside the case (with another filter!) and exhausts it outside the case. And there's a fancy window to show off you tri-fired liquid cooled goodness.

[cypherdoc]

with cgminer, how do i tell which card slot corresponds to GPU 0,1,2?

[jake262144]

Actually, there is no surefire way except for setting the GPU core speeds to something like "200,400,600,800" in the config file and looking at the actual hash rates and temperatures.
Another route would be setting one of the fans, say at GPU0, to 100% and observing which card is hyperventilating  

[cypherdoc]

Actually, there is no surefire way except for setting the GPU core speeds to something like "200,400,600,800" in the config file and looking at the actual hash rates and temperatures.
Another route would be setting one of the fans, say at GPU0, to 100% and observing which card is hyperventilating  

you'd think that such a comprehensive slick piece of software would have this.

[-ck]

Actually, there is no surefire way except for setting the GPU core speeds to something like "200,400,600,800" in the config file and looking at the actual hash rates and temperatures.
Another route would be setting one of the fans, say at GPU0, to 100% and observing which card is hyperventilating  

you'd think that such a comprehensive slick piece of software would have this.

Blame AMD for opencl and the ATI display library having no sure fire way of telling me so that I could tell you via cgminer.

[cypherdoc]

Actually, there is no surefire way except for setting the GPU core speeds to something like "200,400,600,800" in the config file and looking at the actual hash rates and temperatures.
Another route would be setting one of the fans, say at GPU0, to 100% and observing which card is hyperventilating  

you'd think that such a comprehensive slick piece of software would have this.

Blame AMD for opencl and the ATI display library having no sure fire way of telling me so that I could tell you via cgminer.

sorry; not complaining.  your software improved my hashrate by 33%.

btw, i'm sending you some BTC right now.

[RandyFolds]

This thread inspired me to fool with cgminer a bit. I used the config info from this thread and fixed it up for my purposes. Wow...I am friggin' impressed.


What I have found, though, is that it doesn't completely throttle to maintain the designated temperature. I ran it on a card that has a fan out and runs hot...~85-95c. I set the temp at 70 expecting the software to throttle the core to run at 70 and it worked for a while, but alas, when I left it for a few hours, it was running hot...

I am a little simple when it comes to command prompt stuff like this. I need my fat-american GUI. How can I verify that it is throttling as it should? I was hesitant to run another monitoring program with it after reading some stuff about ridiculous voltage spikes in some cards from running two sets of clocking tools.

[cypherdoc]

This thread inspired me to fool with cgminer a bit. I used the config info from this thread and fixed it up for my purposes. Wow...I am friggin' impressed.


What I have found, though, is that it doesn't completely throttle to maintain the designated temperature. I ran it on a card that has a fan out and runs hot...~85-95c. I set the temp at 70 expecting the software to throttle the core to run at 70 and it worked for a while, but alas, when I left it for a few hours, it was running hot...

I am a little simple when it comes to command prompt stuff like this. I need my fat-american GUI. How can I verify that it is throttling as it should? I was hesitant to run another monitoring program with it after reading some stuff about ridiculous voltage spikes in some cards from running two sets of clocking tools.

heh Randy, i have a card that has several blades knocked off the 2 fan rotors. don't ask how.  is it ok to run it anyway?  how would u cool it?

[RandyFolds]

This thread inspired me to fool with cgminer a bit. I used the config info from this thread and fixed it up for my purposes. Wow...I am friggin' impressed.


What I have found, though, is that it doesn't completely throttle to maintain the designated temperature. I ran it on a card that has a fan out and runs hot...~85-95c. I set the temp at 70 expecting the software to throttle the core to run at 70 and it worked for a while, but alas, when I left it for a few hours, it was running hot...

I am a little simple when it comes to command prompt stuff like this. I need my fat-american GUI. How can I verify that it is throttling as it should? I was hesitant to run another monitoring program with it after reading some stuff about ridiculous voltage spikes in some cards from running two sets of clocking tools.

heh Randy, i have a card that has several blades knocked off the 2 fan rotors. don't ask how.  is it ok to run it anyway?  how would u cool it?

Since you probably can't get warranty service on it with the blades busted, I would probably just replace it/them. Fans are cheap, and it beats a fried card. That MSI TwinFrozr-II 6870 that I was referring to in my post above doesn't seem to overheat with only one of the two fans running, but who knows...it could be cooking the VRAM or something that I can't monitor as easily. It's under warranty still, so I didn't just swap the fan out...figure I can get a card that is less beat to hell by RMA'ing it.

[cypherdoc]

Conman, there is a reason the new LGA2011 ATX-size boards all sport the additional molex connector.
Those are quite possibly the first consumer-grade boards designed with heavy multi-GPU action on the radar.

A traditional mobo gets only two 12V lines with the 24-pin connector.
The ATX standard defines up to 75W power usage per PCIE slot but the manufacturers never envisioned such a scenario for all the slots.
Two, perhaps even three devices but not a power hungry monster in each and every single PCIE slot.

There have also been reports of the 24-pin connector burning down.
Those pins can carry up to 13 A of juice (the HCS variation) but the el-cheapo phosphor bronze variety maxes out as 8 A.There are two of those so 16 A * 12 V = 192 W total.
With the HCS pins, you're safely at 26*12 = 312 W.

The question stands, however, what levels of electromigration does 300 W of juice subject the thin PCB traces to in 24/7 mode?
I'm betting high and I'm also betting that's what brought your board on its knees.
I went with Gigabyte boards for those double-thickness traces they're boasting about. When I need another board I'll do the research again - perhaps more manufacturers are using thicker PCB traces now.

The gd70 is the favourite because it's the most cost-effective board, not necessarily the best. Also, you're doing (like me) one thing differently than most miners seem to be doing: you're packing the rigs into PC cases and that means no extenders. More specifically, no POWERED extenders. All the juice flowing through the mobo.

so why exactly are u using cases?  an open architecture allows more spacing of cards and less chance of overheating right?  if you're not going to be moving the rig around why not make it open and save the money on buying a case?

[jake262144]

Eesthetics mostly, I think.
I don't have a huge garage to fill up with rigs and since I can't totally hide them from sight, I want the rigs to look decent.
The days I enjoyed seeing naked PCBs and wires sticking out of open cases have long been gone, along with my teenage years.

The 69xx cards are running at about 77°C with average 55% fan speed. Acceptable temps and bearable noise levels if you know what you're doing.
The Rv02, with its rotated mobo, seems to be just good enough. Mind you, I've yet to see my first mining-related GPU failure.
I'd never use a traditionally designed case, though, as barring some serious modding the middle cards get no airflow worth mentioning.

[cypherdoc]

Eesthetics mostly, I think.
I don't have a huge garage to fill up with rigs and since I can't totally hide them from sight, I want the rigs to look decent.
The days I enjoyed seeing naked PCBs and wires sticking out of open cases have long been gone, along with my teenage years.

The 69xx cards are running at about 77°C with average 55% fan speed. Acceptable temps and bearable noise levels if you know what you're doing.
The Rv02, with its rotated mobo, seems to be just good enough. Mind you, I've yet to see my first mining-related GPU failure.
I'd never use a traditionally designed case, though, as barring some serious modding the middle cards get no airflow worth mentioning.

would the rv03 be better since it has 3 180mm fans at the bottom?

[jake262144]

Uhmmm... you've got it wrong: the Rv03 has only two pre-installed intake fans as opposed to Rv02's three 180mm fans.
Basically, RTFM