Problems running 3+ GPUs (bounty)

[yochdog]

I seem to have consistent problems running more than 3 GPUs (3 cards that is....I have many 2x 5970's working) in a single system.

Whenever I add a 3rd graphics card to a system, they almost always blue screen on me.  It does not seem to matter what brand of MOBO, what type of GPU, etc.  I use Win7, mostly 64 bit. 

Has anyone else had issues like this? 

I would really like to figure out what is going wrong, as I could increase my density greatly by making use of the idle PCI slots. 

Thanks in advance. 


PS - as a sweetener, anyone who gives advice that leads me to getting a 3+ GPU system stable will get a 5 BTC bounty. 

[1715374032]

1715374032

[1715374032]

1715374032

[1715374032]

1715374032

[1715374032]

1715374032

[1715374032]

1715374032

[smracer]

have you tried setting the PCIE Latency to 96 in the Mobo bios? 

[yochdog]

have you tried setting the PCIE Latency to 96 in the Mobo bios? 

I have not.  But I will certainly give it a shot. 

[imsaguy]

I seem to have consistent problems running more than 3 GPUs (3 cards that is....I have many 2x 5970's working) in a single system.

Whenever I add a 3rd graphics card to a system, they almost always blue screen on me.  It does not seem to matter what brand of MOBO, what type of GPU, etc.  I use Win7, mostly 64 bit. 

Has anyone else had issues like this? 

I would really like to figure out what is going wrong, as I could increase my density greatly by making use of the idle PCI slots. 

Thanks in advance. 


PS - as a sweetener, anyone who gives advice that leads me to getting a 3+ GPU system stable will get a 5 BTC bounty. 

What drivers?  What SDK? What board?

[yochdog]

I seem to have consistent problems running more than 3 GPUs (3 cards that is....I have many 2x 5970's working) in a single system.

Whenever I add a 3rd graphics card to a system, they almost always blue screen on me.  It does not seem to matter what brand of MOBO, what type of GPU, etc.  I use Win7, mostly 64 bit. 

Has anyone else had issues like this? 

I would really like to figure out what is going wrong, as I could increase my density greatly by making use of the idle PCI slots. 

Thanks in advance. 


PS - as a sweetener, anyone who gives advice that leads me to getting a 3+ GPU system stable will get a 5 BTC bounty. 

What drivers?  What SDK? What board?

All over the board.  This is an issue I have had with all of the set-ups I have tried. 

I will get the specifics on one of the configs and post it. 

[Olly_K]

do they BSOD under load or while booting into windows ?

When I used to run windows for my miners I had 2 5970s in one rig and it would always crash CGMiner. Turned out to be the PSU.

[yochdog]

do they BSOD under load or while booting into windows ?

When I used to run windows for my miners I had 2 5970s in one rig and it would always crash CGMiner. Turned out to be the PSU.

They only BSOD once mining begins.  Otherwise they seem fine. 

[Olly_K]

that surely points to the PSU then ?

What wattage is it ? have you got another one you can try ?

[rjk]

that surely points to the PSU then ?

What wattage is it ? have you got another one you can try ?

I doubt yochdog would underpower his rigs.

But the crashing is odd, I've had 6 on 1 rig before, all on extenders, and 1 on a PCI to PCIe adapter.

I'd be interested in what the BSOD says. 11.6 with SDK 2.5 should be stable.

[Olly_K]

not saying he's under powering as much, more that the PSU might be faulty.

Or it can't handle 3 GPUs sucking power once mining starts, maybe "stage" the mining process by starting 1, then 2, then the 3rd instead of all at the same time.

What's the message on the BSOD ? atimag ?

[yochdog]

that surely points to the PSU then ?

What wattage is it ? have you got another one you can try ?

I doubt yochdog would underpower his rigs.

But the crashing is odd, I've had 6 on 1 rig before, all on extenders, and 1 on a PCI to PCIe adapter.

I'd be interested in what the BSOD says. 11.6 with SDK 2.5 should be stable.

I will try this.

The rig I am testing with has a 1200W PSU for 2x 5970, and whatever random card I throw in the 3rd slot.  I don't think power is the issue.  

[yochdog]

not saying he's under powering as much, more that the PSU might be faulty.

Or it can't handle 3 GPUs sucking power once mining starts, maybe "stage" the mining process by starting 1, then 2, then the 3rd instead of all at the same time.

What's the message on the BSOD ? atimag ?

I believe atimag is correct.  I will have to wait to look at it tonight. 

[Olly_K]

driver version will help too, had problems with the latest cats and had to take an earlier version

[TheHarbinger]

You already have 4 GPUs running, 2 in each 5970, your problem is getting the 5th GPU running.  Please remember, the number of graphics cards is not always the same as the number of GPUs.

First, I would use catalyst 11.12 with win7.  Anything lower version than that I couldn't get past 4 GPUs to mine with on win 7, same issue as you are having, it would blue screen at the start of mining.  Since you have 2 5970s in there already, that is 4 GPUs, anything you add brings it over that.  Took me forever to figure that out with my 6 x 5870 rig.

If that doesn't work then use cgminer to start with no cards/GPUs active.  Bring each GPU up to mining speed one at a time.  If each GPU(core) ramps up to mining this way, your power supply can't handle the inrush when they all start.  If one of them doesn't ramp up, then you have a borked GPU(core).

[yochdog]

You already have 4 GPUs running, 2 in each 5970, your problem is getting the 5th GPU running.  Please remember, the number of graphics cards is not always the same as the number of GPUs.

First, I would use catalyst 11.12 with win7.  Anything lower version than that I couldn't get past 4 GPUs to mine with on win 7, same issue as you are having, it would blue screen at the start of mining.  Since you have 2 5970s in there already, that is 4 GPUs, anything you add brings it over that.  Took me forever to figure that out with my 6 x 5870 rig.

If that doesn't work then use cgminer to start with no cards/GPUs active.  Bring each GPU up to mining speed one at a time.  If each GPU(core) ramps up to mining this way, your power supply can't handle the inrush when they all start.  If one of them doesn't ramp up, then you have a borked GPU(core).

I will try this also tonight and report back.

Ya, I know the GPU count was off, but I was more thinking it was a problem loading the 3rd physical slot, as I had the same issue trying to run 3 5850's.  2 would work great, then the 3rd would crap everything out. 

[TheHarbinger]

You already have 4 GPUs running, 2 in each 5970, your problem is getting the 5th GPU running.  Please remember, the number of graphics cards is not always the same as the number of GPUs.

First, I would use catalyst 11.12 with win7.  Anything lower version than that I couldn't get past 4 GPUs to mine with on win 7, same issue as you are having, it would blue screen at the start of mining.  Since you have 2 5970s in there already, that is 4 GPUs, anything you add brings it over that.  Took me forever to figure that out with my 6 x 5870 rig.

If that doesn't work then use cgminer to start with no cards/GPUs active.  Bring each GPU up to mining speed one at a time.  If each GPU(core) ramps up to mining this way, your power supply can't handle the inrush when they all start.  If one of them doesn't ramp up, then you have a borked GPU(core).

I will try this also tonight and report back.

Ya, I know the GPU count was off, but I was more thinking it was a problem loading the 3rd physical slot, as I had the same issue trying to run 3 5850's.  2 would work great, then the 3rd would crap everything out. 

Well, a quick way to check if it's a hardware or software issue is to try starting it up with BAMT or Xubuntu 12.04.  Both are pretty fast to get up and mining from a downloaded image, BAMT can be up and going in about 5 minutes if you are familiar with it, Xubuntu takes a bit longer.

But loading up a different OS with different drivers will tell you if it is a hardware problem, or a driver/OS problem.

[imsaguy]

Use GPUz and compare the BIOS of the gpus.  Basically, mixed bioses will often cause the crashing on Windows.  Make sure you're running 11.12.  If anything, wipe the machine and start from scratch.  Put the drivers on a thumb drive and install 11.12 before letting Windows connect to the internet, otherwise it will download AMD drivers from Microsoft Update and you'll be on something later than what you want. 

[check_status]

Put the drivers on a thumb drive and install 11.12 before letting Windows connect to the internet, otherwise it will download AMD drivers from Microsoft Update and you'll be on something later than what you want.

I had a laptop with Vista using a custom ATI driver because the one Windows was installing kept screwing up the display, the latest update. After I installed the custom driver, Windows still tried to update to the latest and I had to disable the ATI driver install option in the update manager after I turned off auto updates. I did a lot of wiping and reinstalling.

Well, a quick way to check if it's a hardware or software issue is to try starting it up with BAMT or Xubuntu 12.04.  Both are pretty fast to get up and mining from a downloaded image, BAMT can be up and going in about 5 minutes if you are familiar with it, Xubuntu takes a bit longer.

12.04 or 11.04? I vote 11.04 because most of the guides around here are based on that version of Linux. I haven't seen a guide yet on 12.04. Are the Kernels different? But yeah, build a Linux based flash drive for just such emergencies, will help in diagnosing (and maybe turn you on to linux full time).   Kano's guide is pretty fool proof for USB creation.
If you don't like Kano's guide there are more here: How To's and Guides Mega List

It sounds a lot like a power issue, the draw is too much. Try keeping the memory clocks lowered but don't overclock the cards and see how that works. Windows ACPI drivers are hidden from the kernel and they monitor the hardware, and then talk to the kernel as needed. I'm not sure if you can turn off the ACPI in Windows. (ACPI driver is a Genuine Windows rootkit. )

[padrino]

On Windows I had better luck running 5+ GPUs using the 12.x drivers but it was never stable for long periods of time if at all.

You may not want to hear it but if you want to run more then 4 reliably you might have better luck jumping to Linux. I have run 8 for weeks on end with anywhere between 4 (4x5970)and 6 (2x5970 4x5870) physical cards and everything works great.

[imsaguy]

On Windows I had better luck running 5+ GPUs using the 12.x drivers but it was never stable for long periods of time if at all.

You may not want to hear it but if you want to run more then 4 reliably you might have better luck jumping to Linux. I have run 8 for weeks on end with anywhere between 4 (4x5970)and 6 (2x5970 4x5870) physical cards and everything works great.

I run many, many 6 GPU rigs under Windows x64, 11.12 drivers.  They run stable for weeks at a time.  They only time they get restarted is to update cgminer when I need to.  I don't think the OS is the issue, rather its some combination of either the drivers, gpu bios, and/or board.