|
gigabytecoin (OP)
|
 |
May 01, 2011, 07:45:30 PM
Last edit: May 01, 2011, 10:39:52 PM by gigabytecoin |
|
I am having a hard time setting up my first multi GPU system here... I have 6 x 5870s (all different brands) all plugged into one MSI 890-FXA-GD70 with a PCIe x1 extension cables that have been modified by myself to accept x16 sized cards. When they are all plugged in to the board directly, and I plug a video cable from my monitor into one of them.... I see the boot screen and the operating system loads... When they are plugged into the board with the modified x1 to x16 cables...you cannot see any video through them... Any ideas?! EDIT: - I am running ubuntu 10.10 - will update as I figure things out! |
|
|
|
|
goatpig
Legendary
 Offline
Activity: 3752
Merit: 1364
Armory Developer
|
|
May 01, 2011, 08:08:24 PM |
|
You're not supposed to see boot infos on anything but the x16 slots I think.
Let it get into your OS and see if the card is identified.
|
|
|
|
|
gigabytecoin (OP)
|
|
May 01, 2011, 08:13:04 PM |
|
You're not supposed to see boot infos on anything but the x16 slots I think.
Let it get into your OS and see if the card is identified.
That was the other problem... when I did get a boot screen showing and loaded ubuntu 10.10... I could only find 2 of the 6 cards... Might this have something to do with the fact that there is only 4GB of installed ram on this machine? |
|
|
|
|
|
trentzb
|
|
May 01, 2011, 08:44:12 PM |
|
I have 7 x 5870s (all different brands) all plugged into one MSI 890-FXA-GD70 with a PCIe x1 extension cables that have been modified by myself to accept x16 sized cards. How in the world are you able to connect 7 x 5870 into that board? I am seeing 6 PCIe slots unless I am looking at the wrong board. Using a PCI -> PCIe adapter also? You're not supposed to see boot infos on anything but the x16 slots I think.
In the BIOS settings of some mainboards you can select to output BIOS video via x16 or x1 or IGP and/or set the order to light them up in. At a shell prompt do a lspci | grep -i vga and see what you see. If you got all of them listed you are getting there. If you don't or are interested in how the kernel is probing the PCI/PCIe busses do a dmesg at a prompt and look for fglrx output. |
|
|
|
|
|
shivansps
|
|
May 01, 2011, 09:48:14 PM |
|
you may need to do the pci-e x1 bridge trick in order to make cards work at x1.  |
|
|
|
|
gigabytecoin (OP)
|
|
May 01, 2011, 10:35:42 PM |
|
you may need to do the pci-e x1 bridge trick in order to make cards work at x1.
These are just typical 5870s I am using in this setup, from a few different manufacturers. Is this a necessary thing to do with 5870s in general? All 5870s are x16 by default and are a fairly popular board... |
|
|
|
|
|
gigabytecoin (OP)
|
|
May 01, 2011, 10:41:37 PM |
|
I have 7 x 5870s (all different brands) all plugged into one MSI 890-FXA-GD70 with a PCIe x1 extension cables that have been modified by myself to accept x16 sized cards. How in the world are you able to connect 7 x 5870 into that board? I am seeing 6 PCIe slots unless I am looking at the wrong board. Using a PCI -> PCIe adapter also? You're not supposed to see boot infos on anything but the x16 slots I think.
In the BIOS settings of some mainboards you can select to output BIOS video via x16 or x1 or IGP and/or set the order to light them up in. At a shell prompt do a lspci | grep -i vga and see what you see. If you got all of them listed you are getting there. If you don't or are interested in how the kernel is probing the PCI/PCIe busses do a dmesg at a prompt and look for fglrx output. Woops! I meant 6, changed that! Nope, not using a PCI -> PCIe adapter... not really economical. I cannot even get to the shell is the problem. When all of the cards are plugged in via x1 PCIe extension cables... I cannot even get a video signal coming out of any of them. I have tried each one. When they are plugged into the board directly they work fine. Which is why I ask if there is something I am doing wrong or do x1 connections simply not transmit video signals due to lack of bandwidth or something?? |
|
|
|
|
|
shivansps
|
|
May 01, 2011, 10:44:03 PM |
|
BRIDGE the pci-e slots like in the picture i posted.
Some mbs have troubles detecting videos cards at x1 slots, using a simple cable to bridge 2 pins will fix it.
|
|
|
|
|
gigabytecoin (OP)
|
|
May 02, 2011, 12:02:31 AM |
|
BRIDGE the pci-e slots like in the picture i posted.
Some mbs have troubles detecting videos cards at x1 slots, using a simple cable to bridge 2 pins will fix it.
Sounds like a task easier said then done... There is not a lot of room to work with on an x1 extension cable. Would probably be simpler to purchase x16 cables for $5 each, no? |
|
|
|
|
|
shivansps
|
|
May 02, 2011, 12:22:28 AM |
|
 man you cant do that?  Take 5cm of ANY cable and just put it like the image... |
|
|
|
|
trentzb
|
|
May 02, 2011, 12:59:02 AM |
|
Are you meaning to hot wire the x1 extender or the slot on mainboard?
|
|
|
|
|
|
shivansps
|
|
May 02, 2011, 01:08:54 AM |
|
its the same thing.
|
|
|
|
|
trentzb
|
|
May 02, 2011, 01:47:06 AM |
|
He is having an issue with his x16 slots unless I am misreading. Are you saying he should jumper his x16 slot using x1 presence detect?
|
|
|
|
|
chungenhung
Legendary
Offline
Activity: 1134
Merit: 1005
|
|
May 02, 2011, 01:57:46 AM |
|
man you cant do that? Take 5cm of ANY cable and just put it like the image... In the picture, it looks like the wire is just placed inside the PCI-e slot. |
|
|
|
|
|
shivansps
|
|
May 02, 2011, 02:24:33 AM |
|
He is having an issue with his x16 slots unless I am misreading. Are you saying he should jumper his x16 slot using x1 presence detect?
his using x1 risers, even if the slots are x16, if his using a x1 riser it NEEDs to jumper the slot -or the riser-, depends on the mb and/or the video card combination, in some cases you dont need to jumper anything. BTW... 6x5870, there is a limit on how many VGA cards can be placed on a single MB? |
|
|
|
|
bitjet
|
|
May 02, 2011, 03:05:14 AM |
|
BTW... 6x5870, there is a limit on how many VGA cards can be placed on a single MB?
OS limitations. win7 = 4gpu's , linux 8 gpus. |
|
|
|
|
|
gigabytecoin (OP)
|
|
May 02, 2011, 03:24:18 AM |
|
with a PCIe x1 extension cables that have been modified by myself to accept x16 sized cards.
1. I assume you did this by cutting the back of the x1 socket. You should check that you didn't eliminate the connections closest to the cut. 2. Some x16 cards need other leads shorted to work. 3. If you're seeing two of the cards working, I'd mark cards as "known good" and start swapping them through. If they work in all ports, then it's card related, especially if the other cards don't work in the slots where the working cards are currently seated. 4. You're going to need to titrate this, regardless. You have a combination of cards and slots, some of which work and most of which don't, and all the cards are different brands that could have their own issues and electrical incompatibilities that could cause everything to go to heck. Start with your two good cards in any slots in which they work, then start adding cards in a methodical manner until you reach your maximum. PCIe x1 itself is definitely capable of transmitting video, and you can buy (a few) x1 cards, primarily to go into rackmount servers. Just set your expectations of video performance accordingly. Remember that PCIe, even with one lane, is faster than ISA, and we played lots of bad games in VGA and definitely saw Linux/X output. Thanks for your insight! I know that all of the cards work since I have tested them all individually. It's just that whenever I plug then into an x1, x8 or x16 slot with an x1 extension cable they fail to be recognized. I like your idea of trying all of the same brand of cards on one mobo. I have a bunch of 5870s all made by Gigabyte so I will lump them together and get back to you guys here. |
|
|
|
|
|
shivansps
|
|
May 02, 2011, 03:27:37 AM |
|
Thanks for your insight!
I know that all of the cards work since I have tested them all individually. It's just that whenever I plug then into an x1, x8 or x16 slot with an x1 extension cable they fail to be recognized.
I like your idea of trying all of the same brand of cards on one mobo. I have a bunch of 5870s all made by Gigabyte so I will lump them together and get back to you guys here.
again, your problem is VGA detection at x1... just jumper the slots or the risers as showed on the picture. |
|
|
|
|
gigabytecoin (OP)
|
|
May 02, 2011, 03:27:58 AM |
|
man you cant do that? Take 5cm of ANY cable and just put it like the image... So you are talking about bridging the motherboard, I thought you meant for me to bridge the pcie x1 extension cable originally. And I had not thought of doing it that way. You're right it seems quite simple! I am not specifically having problems with plugging the x1 extension cables into an x1 slot. I am also having trouble reading cards that are plugged into an x16 slot via an x1 extension cable. Do I simply have to use the correct size of extension cable? (I mean.. x16 extension cables for an x16 slot?) I am buying some x16 extension cables right now... I thought I could save a buck by buying x1s and making the necessary cuts myself... stupid mistake! Thanks for your help shivansps! |
|
|
|
|
|
gigabytecoin (OP)
|
|
May 02, 2011, 03:29:24 AM |
|
He is having an issue with his x16 slots unless I am misreading. Are you saying he should jumper his x16 slot using x1 presence detect?
his using x1 risers, even if the slots are x16, if his using a x1 riser it NEEDs to jumper the slot -or the riser-, depends on the mb and/or the video card combination, in some cases you dont need to jumper anything. BTW... 6x5870, there is a limit on how many VGA cards can be placed on a single MB? Is there a limit? I am using ubuntu 10.10... I heard that windows imposed limits but not ubuntu..? |
|
|
|
|
|
