Six Card Rig x6 7950 MSI TWIN FROZR 3GD5/OC

[gregyoung14]

Need as much feedback and advice as I can get!!

I'm trying to get a x6 7950 up and running smoothly. I've been doing some preliminary testing on different, OS's, tried, BAMT, CentOS, Xubuntu, Ubuntu, Windows, I've cycled through all of these a million times. Originally I had a 850w corsair and 600w corsair, but that was a no go. Tried a 1000w corsair and 600w corsair still no go, tomorrow I give it a run with a 1600w Lepa, that'll solve my power promblems hopefully.

I'm running MSI 7950's the R7950 TWIN FROZR 3GD5/OC to be specific. Now with this card under volted at 1.087 and many other settings in cgminer, I can easily hit 680, with close to 12% reject shares in Windows. In Xubuntu it was about 615 12% stale, in Ubuntu with a higher clock/memory about 650, with again 12% stale, and so this was just testing one card, but as soon as I plugged in a second or third card Windows would lock up with driver issues. Ubuntu/Xubuntu would have power issues. (with the 1000w) with the 850w I could see two fine in any OS, and could mine with both pulling 680 with 12% reject shares.

I was using 12.7, 12.8, 13.1 drivers, tried all three, 13.1 seems to be doing good right now. And for the SDK 2.8

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

As far as cooling goes, I can keep any card at 680 hashing constant, and keep the card below 65c with fans at under 60% (Not bad right?   ) So cooling isn't an issue. I have an open rig design, custom aluminum rig that I build, all the cards spaced out and bolted in and elevated. Excellent air flow/cooling etc. So heat from the GPU's isn't an issue.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine. What OS would you recommend for the best hashing? I really enjoy Xubuntu and Ubuntu, I just need to be sure to get up there with the hash rate and low rejected shares. I'm mining a lot of new little scrypt coins on small pools so I thought maybe it was the pool? I threw in some settings into cgminer, scan time, query, and one more I forgot, combined with a lower intensity of 18 opposed to 20, that were able to push the rejected shares  down to close to 5% however I was only mining at 650 (this is in windows)

So what are recommendations on OS, PSU, pcie slots/extenders, drivers?

Thanks,

Greg

[nightbladexx]

What motherboard are you using?

I run Windows 8 w/ 13.1 & 2.8

Im running a Lepa G1600 and working on trying to get 5 - 7950s working, up to 4 ATM, on it

[Slander]

I honestly think you'd be better off just running 4 cards in one rig, way more stable. Trying to push to the limit is fun but you soon get tired of constantly rebooting your rig etc. I like sleeping well and not worrying if it crashed 3 minutes after I went to bed 

[Eastwind]

Theoretically, GD65 should run 7 cards.
https://bitcointalk.org/index.php?topic=186877.0
https://bitcointalk.org/index.php?topic=193695.0

Can you undervolt the memory as well? Maybe it is too hot.

[gregyoung14]

What motherboard are you using?

I run Windows 8 w/ 13.1 & 2.8

Im running a Lepa G1600 and working on trying to get 5 - 7950s working, up to 4 ATM, on it

Ok good to know, keep me posted when you get 5 or more running stable, I'd like to know about the details.

I honestly think you'd be better off just running 4 cards in one rig, way more stable. Trying to push to the limit is fun but you soon get tired of constantly rebooting your rig etc. I like sleeping well and not worrying if it crashed 3 minutes after I went to bed 

Yes it would be more stable, but a simple script to start cgminer when the computer reboots, and a setting in motherboard bios for it to always be on takes care of constant reboots, though I am trying to avoid that all together. And it is extremely more profitable to run x6 cards or more in one rig. The more cards in one rig, the less you spend on infrastructure (mobo,cpu,ram,shell) per card. Which improves the ROI a lot as well.

If I can get 6 stable and it takes me a whole weekend of testing to get it, it'll be worth it. But I don't see your point, it is becoming a huge headache and I've put in(wasted) a lot of time as it is.

Theoretically, GD65 should run 7 cards.
https://bitcointalk.org/index.php?topic=186877.0
https://bitcointalk.org/index.php?topic=193695.0

Can you undervolt the memory as well? Maybe it is too hot.

I actually have thought about that, I would imagine the bios has an option to downclock it, which would use less volts if I couldn't force a lower constant voltage. That is if were speaking about the memory for the system and not each cards actual memory, the cards stay plenty cool. The mobo/cpu/ram also stay pretty cool. So I don't think its that, but I suppose its worth trying, have you seen info/post anywhere that recommend this? or confirm any kind of results?

 

[gregyoung14]

What motherboard are you using?

I run Windows 8 w/ 13.1 & 2.8

Im running a Lepa G1600 and working on trying to get 5 - 7950s working, up to 4 ATM, on it

Oh sorry forgot the mobo model. Its this one GA-F2A85X-UP4

http://www.amazon.com/Gigabyte-Motherboard-Triple-Monitor-support-Windows/dp/B009L1EOKQ/ref=sr_1_1?ie=UTF8&qid=1370547565&sr=8-1&keywords=gigabyte+fm2

[Hephaestus]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

[gregyoung14]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

[Hephaestus]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

Sorry for the confusion, if you're using x16 to x16 risers, you shouldn't need to do the presence short for the x16 slot like I did.  I had to do it because I had the graphics card there on a riser like this one: http://www.amazon.com/Micro-SATA-Cables-PCI-E-Riser/dp/B005G4SADY/ref=sr_1_1?ie=UTF8&qid=1370553268&sr=8-1
So it was only connected as x1 even though it's plugged into an x16 slot.

And yes, the farthest x1 slot from the CPU is the one that needs the short.  I actually just peeled the presence lines from the risers, clipped and soldered them together because I didn't have any wire that fit very well.

Undervolting is harder in Linux, or so I've heard.  If cgminer can't do it you're pretty much screwed...
The real advantages of Linux are 1) free and 2) can be installed to a flash drive.

[gregyoung14]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

Sorry for the confusion, if you're using x16 to x16 risers, you shouldn't need to do the presence short for the x16 slot like I did.  I had to do it because I had the graphics card there on a riser like this one: http://www.amazon.com/Micro-SATA-Cables-PCI-E-Riser/dp/B005G4SADY/ref=sr_1_1?ie=UTF8&qid=1370553268&sr=8-1
So it was only connected as x1 even though it's plugged into an x16 slot.

And yes, the farthest x1 slot from the CPU is the one that needs the short.  I actually just peeled the presence lines from the risers, clipped and soldered them together because I didn't have any wire that fit very well.

Undervolting is harder in Linux, or so I've heard.  If cgminer can't do it you're pretty much screwed...
The real advantages of Linux are 1) free and 2) can be installed to a flash drive.

Right ok so then I just followed this diagram

http://blog.zorinaq.com/images/pcie-short-schematic.png

and did that to the third pcie 1x slot, thats the only thing thats need to be presence shorted

Yeah windows just craps out in so many departments though, remote desktop client wouldn't work, updates forever and ever, system instability, I'll give both a go and see what happens. Which drivers did you use in Ubuntu?

And what BIOS settings did you tweak? Also, are you using dummy plugs? actual plugs? no plugs?

Thanks

[Hephaestus]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

Sorry for the confusion, if you're using x16 to x16 risers, you shouldn't need to do the presence short for the x16 slot like I did.  I had to do it because I had the graphics card there on a riser like this one: http://www.amazon.com/Micro-SATA-Cables-PCI-E-Riser/dp/B005G4SADY/ref=sr_1_1?ie=UTF8&qid=1370553268&sr=8-1
So it was only connected as x1 even though it's plugged into an x16 slot.

And yes, the farthest x1 slot from the CPU is the one that needs the short.  I actually just peeled the presence lines from the risers, clipped and soldered them together because I didn't have any wire that fit very well.

Undervolting is harder in Linux, or so I've heard.  If cgminer can't do it you're pretty much screwed...
The real advantages of Linux are 1) free and 2) can be installed to a flash drive.

Right ok so then I just followed this diagram

http://blog.zorinaq.com/images/pcie-short-schematic.png

and did that to the third pcie 1x slot, thats the only thing thats need to be presence shorted

Yeah windows just craps out in so many departments though, remote desktop client wouldn't work, updates forever and ever, system instability, I'll give both a go and see what happens. Which drivers did you use in Ubuntu?

And what BIOS settings did you tweak? Also, are you using dummy plugs? actual plugs? no plugs?

Thanks

I just installed the drivers from the settings menu under "Restricted Drivers", it will download and install them automatically from the repository.  I had to install the beta ones for my 7870 XTs though.  I'm not sure what version number any of them are...so I feel kinda dumb about it. 

I haven't actually tweaked the BIOS yet, and no dummy plugs needed.  Nothing is plugged into any of my cards, I do everything over SSH with byobu so I can detach sessions.

[gregyoung14]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

Sorry for the confusion, if you're using x16 to x16 risers, you shouldn't need to do the presence short for the x16 slot like I did.  I had to do it because I had the graphics card there on a riser like this one: http://www.amazon.com/Micro-SATA-Cables-PCI-E-Riser/dp/B005G4SADY/ref=sr_1_1?ie=UTF8&qid=1370553268&sr=8-1
So it was only connected as x1 even though it's plugged into an x16 slot.

And yes, the farthest x1 slot from the CPU is the one that needs the short.  I actually just peeled the presence lines from the risers, clipped and soldered them together because I didn't have any wire that fit very well.

Undervolting is harder in Linux, or so I've heard.  If cgminer can't do it you're pretty much screwed...
The real advantages of Linux are 1) free and 2) can be installed to a flash drive.

Right ok so then I just followed this diagram

http://blog.zorinaq.com/images/pcie-short-schematic.png

and did that to the third pcie 1x slot, thats the only thing thats need to be presence shorted

Yeah windows just craps out in so many departments though, remote desktop client wouldn't work, updates forever and ever, system instability, I'll give both a go and see what happens. Which drivers did you use in Ubuntu?

And what BIOS settings did you tweak? Also, are you using dummy plugs? actual plugs? no plugs?

Thanks

I just installed the drivers from the settings menu under "Restricted Drivers", it will download and install them automatically from the repository.  I had to install the beta ones for my 7870 XTs though.  I'm not sure what version number any of them are...so I feel kinda dumb about it. 

I haven't actually tweaked the BIOS yet, and no dummy plugs needed.  Nothing is plugged into any of my cards, I do everything over SSH with byobu so I can detach sessions.

You've got it to easy my friend lol, all 6 cards are plugged in, pci slot is presence shorted, popped in my ubuntu disk, lets see where I am in an hour.

[gregyoung14]

I've got two of the exact same motherboard you have, and I'm running six cards each as well.  Using Ubuntu 12.04.2 LTS.  I can't really say anything about Windows vs Linux, I've only been mining in Linux.

Now for the issue of PCIe slots, all three of my 16x slots handle non powered and powered extenders fine. my 1x slots one and two are good, but the third one I think might need a presence short, it was being picky. As far as powered extenders go, I was planning to do something like all the 16x powered(three) and 1x (three) non powered. This should be ok I think, or do pcie 1x pull less power through the board and would benefit more from a powered extender? And or should I just power extend all 6 of them?

Yes, you do need a presence short on the 3rd PCIe x1.  I've got all my cards on x1 to x16 risers (even on x16 slots) and had to do the presence short on that last x1 slot and the x16 slot just under it.
There aren't any additional power lines on the longer PCIe slots, x16 won't have any more than x1 does.  Take a look: http://en.wikipedia.org/wiki/PCI_Express#Pinout - all of the power pins are on the little section before the notch.
On my boards, I use three unmodified and three powered risers.  With my powered ones though, I sever the +12V lines going to the motherboard so that it only pulls power from the molex.  I actually have those powered risers powered off the second power supply.
If you don't sever the lines, you have to plug those molexes into the same power supply that's powering the motherboard.

Now do you think a 1600w 80 plus gold psu the Lepa one, that should be able to handle x6 cards just fine.

1600 watts should be enough, I measured one of my rigs with a Kill-A-Watt at about 1500 watts with all six cards running (before playing with voltage).  I actually over-spec my power supplies, each rig has two in the 900-1000 watt range.

-my gf wrote this while staring at my rig sitting on the toilet

There's a mining rig in your bathroom?

You have the same motherboards!!! YES!!!! With 6 running?! *hallelujah* That literally just relieved me so much, because I really didn't want to go through the hassle of having to order a different Mobo/CPU setup.

Ok, presence shorts on the thrid x1 slot assuming your counting the northbound (coming from cpu moving down the board) one as #1 then pcix16 then pcix1 # 2 and pcix1 # 3 and pcix16 pci 16 end of board

So I need to presence short pcix1 #3 and pcix16 just under it? Got it. Could you send me pictures of your presence short? I've seen pics on line for a 1x slot, but not of a 16x, or if you could link me to a good pic/diagram.

I have powered x16 to x16 extenders, I have powered x1 to x16 as well, but they were done incorrectly and don't provide the extra 12v I think... None the less i have the 16 to 16 which I know were perfectly, and since I'm using one PSU and its the same one for the mobo I don't need to sever any of the lines.

Yeah I have these MSI cards and they undervolt just fine in windows so hopefully it won't be much of an issue in Ubuntu, any advantages to Ubuntu or Xubuntu?

Any chance you mind emailing or IMing me or some kind of instant communication and we can discuss a little more?

Oh and in regards to the song my gf wrote lol, we live in these new apartments for college students and our walk in closet is actually through our bathroom, so you walk into the bathroom and then theres another door to the closet which is directly in front of the toliet and often open, and in there on the top shelf under an AC vent are all my rigs Lol so she had a clear view of them

Sorry for the confusion, if you're using x16 to x16 risers, you shouldn't need to do the presence short for the x16 slot like I did.  I had to do it because I had the graphics card there on a riser like this one: http://www.amazon.com/Micro-SATA-Cables-PCI-E-Riser/dp/B005G4SADY/ref=sr_1_1?ie=UTF8&qid=1370553268&sr=8-1
So it was only connected as x1 even though it's plugged into an x16 slot.

And yes, the farthest x1 slot from the CPU is the one that needs the short.  I actually just peeled the presence lines from the risers, clipped and soldered them together because I didn't have any wire that fit very well.

Undervolting is harder in Linux, or so I've heard.  If cgminer can't do it you're pretty much screwed...
The real advantages of Linux are 1) free and 2) can be installed to a flash drive.

Right ok so then I just followed this diagram

http://blog.zorinaq.com/images/pcie-short-schematic.png

and did that to the third pcie 1x slot, thats the only thing thats need to be presence shorted

Yeah windows just craps out in so many departments though, remote desktop client wouldn't work, updates forever and ever, system instability, I'll give both a go and see what happens. Which drivers did you use in Ubuntu?

And what BIOS settings did you tweak? Also, are you using dummy plugs? actual plugs? no plugs?

Thanks

I just installed the drivers from the settings menu under "Restricted Drivers", it will download and install them automatically from the repository.  I had to install the beta ones for my 7870 XTs though.  I'm not sure what version number any of them are...so I feel kinda dumb about it. 

I haven't actually tweaked the BIOS yet, and no dummy plugs needed.  Nothing is plugged into any of my cards, I do everything over SSH with byobu so I can detach sessions.

Hey So I tried just installing the proprietary drivers like you did and running cgminer, and cgminer spits out GPU 0 failure, disabling, for all the cards excpet one, and so then they all list in cgminer as OFF. I followed a few different guides on getting cgminer up and running in ubuntu/xubuntu and no luck. All the cards just show as off, expect for one. And then a few seconds in my display will go blank and I'll loose picture. Computer still on and running, no display. An ideas whats going wrong? I don't think theres anyway that 1600w wouldn't be enough for all 6 but is it possible?

Suggestions? I really just want to get this damn thing stable. I've had all the parts for almost 7 days now -___-

[broken_pixel]

My 5 R7950 rig runs great, GA-990FXA-UD5, NEX1500 12.6 drivers, Win7 64 bit @ 1165 watts total system.
[/URL]

[gregyoung14]

My 5 R7950 rig runs great, GA-990FXA-UD5, NEX1500 12.6 drivers, Win7 64 bit @ 1165 watts total system.
[/URL]

Looks good! 12.6 drivers, I'll try those. Any SDK install you did as well? and bios flashing? msi overburner? Any install details? or you just plugged everything in, clean install of windows 7 64bit, installed 12.6 drivers. booted up cgminer and off you went?

Thanks

[broken_pixel]

Thanks, I use the latest AB beta, SDK 2.8. The GPUs are stock & undervolted on powered risers 4x 16x slots, 1 1x slot with 2 1x risers connected together powered closest to the GPU. 

1070/1250 @ 1.063v using --thread-concurrency 22400 -I 20 for 655Kh/s x5.

[Hephaestus]

Hey So I tried just installing the proprietary drivers like you did and running cgminer, and cgminer spits out GPU 0 failure, disabling, for all the cards excpet one, and so then they all list in cgminer as OFF. I followed a few different guides on getting cgminer up and running in ubuntu/xubuntu and no luck. All the cards just show as off, expect for one. And then a few seconds in my display will go blank and I'll loose picture. Computer still on and running, no display. An ideas whats going wrong? I don't think theres anyway that 1600w wouldn't be enough for all 6 but is it possible?

Suggestions? I really just want to get this damn thing stable. I've had all the parts for almost 7 days now -___-

I was setting up three 7950s today and actually found that I need the beta drivers (from the additional drivers in the settings menu) in order to use multiple cards.  That fixed most of my problems.
After all the cards were working properly I frequently got reboots right after launching cgminer, I managed to fix that with a powered riser...
1600 watts on the output side of the power supply would be cutting it really, really close.  My three 7950s right now are pulling 940 watts from the wall (pre-tweaking).

[broken_pixel]

Use 12.6 drivers if you are running more than 5 GPUs per mainboard. Most other drivers will not utilize but only 4 and will disable the others. The 13.1 drivers should work but over at the AMD Developer forums people have posted the 13.1 drivers have a memory leak.

Some mainboards will need to be shorted with a thin wire on certain PCIe slots. Some boards need specific PCIe slots populated to run said amount of GPUs. There is a list of those boards somewhere on the forums. 

[gregyoung14]

Use 12.6 drivers if you are running more than 5 GPUs per mainboard. Most other drivers will not utilize but only 4 and will disable the others. The 13.1 drivers should work but over at the AMD Developer forums people have posted the 13.1 drivers have a memory leak.

Some mainboards will need to be shorted with a thin wire on certain PCIe slots. Some boards need specific PCIe slots populated to run said amount of GPUs. There is a list of those boards somewhere on the forums. 

I'll give the 12.6 drivers a go. I was using the 13.1 previously and would start cgminer and then just loose the display shortly after. They would report as mining but on the pool side it didn't show anything.

[gregyoung14]

Hey So I tried just installing the proprietary drivers like you did and running cgminer, and cgminer spits out GPU 0 failure, disabling, for all the cards excpet one, and so then they all list in cgminer as OFF. I followed a few different guides on getting cgminer up and running in ubuntu/xubuntu and no luck. All the cards just show as off, expect for one. And then a few seconds in my display will go blank and I'll loose picture. Computer still on and running, no display. An ideas whats going wrong? I don't think theres anyway that 1600w wouldn't be enough for all 6 but is it possible?

Suggestions? I really just want to get this damn thing stable. I've had all the parts for almost 7 days now -___-

I was setting up three 7950s today and actually found that I need the beta drivers (from the additional drivers in the settings menu) in order to use multiple cards.  That fixed most of my problems.
After all the cards were working properly I frequently got reboots right after launching cgminer, I managed to fix that with a powered riser...
1600 watts on the output side of the power supply would be cutting it really, really close.  My three 7950s right now are pulling 940 watts from the wall (pre-tweaking).

Thats interesting, I cycled over to an Xubuntu instal but it seems that even my clean installs are getting corrupted, I open firefox and terminal and the whole system just locks up and craps out. I've pulled a card and only using x5 now to eliminate the possibility of using to much power. Same issues still occuring. I Had windows 8 running on another hard drive and it saw all 5 cards fine, installed drives, booted up cgminer conncects, reports hash rate, and screen goes blank no signal all control lost, no idea.