Considering developing a 6 GPU Riser/Backplane, thoughts?

[FyrBridd]

I've been considering developing a 6 GPU riser/backplane using a PLX chip allowing the 6 GPUs to be hosted by a x4 PCIe 2.0 Slot, and giving each a x1 PCIe 2.0 Slot using an off the shelf x4 riser to link the motherboard and backplane. What are peoples thoughts on this? A worthwhile project? Would people be interested in such a product?
It would allow practically any motherboard to hit it's GPU hard limit but would cost more than 6 of the normal x1 risers due to the extra electronics involved (the IC alone is about £40...), however it wouldn't bypass 4G decoding support.

The idea is to give GPUs 3 slots each meaning the board size would be approximately 360mm wide to fit neatly behind 3 fans and/or in a rackmount chassis. It would use 1 6pin PCIe connector for every 2 GPUs.

Other additional features could include:
Support for 6 PWM high power fans requiring an additional 1 or 2 6 pin PCIe connectors with manual adjustment with a breakout board.
Hotswap/Overcurrent/Overvoltage protection/indication for the GPUs (1 IC does it all).
Debug lights.
Possibly others?

I am an digital electronic engineer by profession so I'm not jumping into this completely blind.

Any thoughts, feelings or feedback would be appreciated.

[Dotem]

Sounds interesting. IMO, worth checking price feasibility. I just ordered a set of risers and was thinking something similar would be handy.

[warrior333]

How can you load 2 GPU on a single 6 pin connector? That isn't possible. It will melt. Do you think you're smarter than electronic engineers at work? Lol. I saw the dividers 3 PCI connector on the riser card. It's nonsense. Such equipment will not work for a long time. I am opposed to expanding the capabilities of the equipment by such methods.

[HansHagberg]

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

[philipma1957]

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

HansHagberg

okay  spacing spacing spacing.

go big or go home.

3 or 3.5 slot spacing

I have been running a  server mobo with a bridge to a 16 pci board.

https://www.youtube.com/watch?v=94IX_fjlpao&t=1s

https://www.youtube.com/watch?v=b4gA7QPP3CA

https://www.youtube.com/watch?v=9gAysiHOWBE

https://www.youtube.com/watch?v=68EGwGnhk1g


the 16 card board is nice  but requires special cooling.

a 12 card board  with room would have been nice


same for you a six card with space would be nice

[Eyedol-X]

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

HansHagberg

okay  spacing spacing spacing.

go big or go home.

3 or 3.5 slot spacing

I have been running a  server mobo with a bridge to a 16 pci board.

https://www.youtube.com/watch?v=94IX_fjlpao&t=1s

https://www.youtube.com/watch?v=b4gA7QPP3CA

https://www.youtube.com/watch?v=9gAysiHOWBE

https://www.youtube.com/watch?v=68EGwGnhk1g


the 16 card board is nice  but requires special cooling.

a 12 card board  with room would have been nice


same for you a six card with space would be nice

In addition, if you create a custom backplane, you can probably hit up Spotswood to design a chassis to support virtually any motherboard plus the backplane with enough spacing + 1-2 PSU's. -- For the right price it could make for some very competitive open air rigs IMO.

[FyrBridd]

How can you load 2 GPU on a single 6 pin connector? That isn't possible. It will melt. Do you think you're smarter than electronic engineers at work? Lol. I saw the dividers 3 PCI connector on the riser card. It's nonsense. Such equipment will not work for a long time. I am opposed to expanding the capabilities of the equipment by such methods.

It will only load the PCIe socket draw of 2 GPUs, not the whole GPU, will be on 1 PCIe 6 Pin connector so 150W @12V, I'm quite happy to incorporate 1 per GPU but people will just use splitters anyway which are far more dangerous than a PCB based solution. Each contact of PCIe connector is rated at 9A, equating to 27A total draw limit for the connector or a 324W @12V limit, the PCIe standard is very conservative in this matter.

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

4-6 slot spacing isn't really possible for a single PCB solution, it would be far too big. Exhaust fans and sufficient air extraction are the best way of preventing warm exhaust air from getting to any other GPUs. Intelligent purchasing of GPUs for the particular situation will always be a factor with noise to cooling capability the main factor, and I'd argue that GPUs with horizontally aligned heatsinks are sufficient as well as blower style to keep decent airflow at low noise levels, however testing would obviously be necessary. However, only the most extreme coolers tend to go over 2 slots these days.

It could possibly be increased to 3.5 slot but would no longer neatly fit behind 3 120mm fans.

[QuintLeo]

It will only load the PCIe socket draw of 2 GPUs, not the whole GPU, will be on 1 PCIe 6 Pin connector so 150W @12V, I'm quite happy to incorporate 1 per GPU but people will just use splitters anyway which are far more dangerous than a PCB based solution. Each contact of PCIe connector is rated at 9A, equating to 27A total draw limit for the connector or a 324W @12V limit, the PCIe standard is very conservative in this matter.

PCI-E spec only uses 2 pins for +12VDC on a 6 pin connector, but that's still 192 watts of RATED draw.
The pins are rated at 8A not 9A when used in the connector body per MOLEX spec sheet.

[philipma1957]

How can you load 2 GPU on a single 6 pin connector? That isn't possible. It will melt. Do you think you're smarter than electronic engineers at work? Lol. I saw the dividers 3 PCI connector on the riser card. It's nonsense. Such equipment will not work for a long time. I am opposed to expanding the capabilities of the equipment by such methods.

It will only load the PCIe socket draw of 2 GPUs, not the whole GPU, will be on 1 PCIe 6 Pin connector so 150W @12V, I'm quite happy to incorporate 1 per GPU but people will just use splitters anyway which are far more dangerous than a PCB based solution. Each contact of PCIe connector is rated at 9A, equating to 27A total draw limit for the connector or a 324W @12V limit, the PCIe standard is very conservative in this matter.

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

4-6 slot spacing isn't really possible for a single PCB solution, it would be far too big. Exhaust fans and sufficient air extraction are the best way of preventing warm exhaust air from getting to any other GPUs. Intelligent purchasing of GPUs for the particular situation will always be a factor with noise to cooling capability the main factor, and I'd argue that GPUs with horizontally aligned heatsinks are sufficient as well as blower style to keep decent airflow at low noise levels, however testing would obviously be necessary. However, only the most extreme coolers tend to go over 2 slots these days.

It could possibly be increased to 3.5 slot but would no longer neatly fit behind 3 120mm fans.

it would fit if  you use  120mm fans to a 140mm shroud

gpu 140mm 120mm fan>

2 gpus behind each shroud

 https://www.amazon.com/Bgears-Cooling-Fan-Adapter-140mm-Black/dp/B0043GMY1U/ref=sr_1_3?

now this means  3 x 140mm = 420mm Plus a little more   but 420 mm = 16.535 inches


so say  the full build is 451mm that =  17.756 inches   which means it fits in a standard rack

that  extra spacing allows  1080ti's  at 200 to 225 watts    which = beast

a 19 inch rack mount with 6 1080ti's that  can do 1200 to 1350 watts  is a monster

the fan adapter will actually cut fan noise down


you could use a lessor delta  like this one

https://www.newegg.com/Product/Product.aspx?Item=N82E16835213015

if you used blower 1080ti's like this one

https://www.newegg.com/Product/Product.aspx?Item=9SIAAZU6HW6961

[pickleburglar]

chinese estores already sell 1 PCI-e 1x -> 4 PCI-e 16x pcb's with 2 slot spacing for $25 so if you can beat that price, go for it

[hack4love]

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

HansHagberg

okay  spacing spacing spacing.

go big or go home.

3 or 3.5 slot spacing

I have been running a  server mobo with a bridge to a 16 pci board.

https://www.youtube.com/watch?v=94IX_fjlpao&t=1s

https://www.youtube.com/watch?v=b4gA7QPP3CA

https://www.youtube.com/watch?v=9gAysiHOWBE

https://www.youtube.com/watch?v=68EGwGnhk1g


the 16 card board is nice  but requires special cooling.

a 12 card board  with room would have been nice


same for you a six card with space would be nice

From where did you get this rig in the first video

[philipma1957]

3 slots is not enough space for cooling.
Advanced GPUs all use 2.5 - 3 slots of space so it would only work for blower style cooling solutions.
With a spacing 4 - 6 slots and some provisions for compartmentalizing the GPUs so exhaust air is not cross-fed, it will be much more useful.
Cooling is the big problem to solve and a solution that takes this into account will be worth more on the market.

HansHagberg

okay  spacing spacing spacing.

go big or go home.

3 or 3.5 slot spacing

I have been running a  server mobo with a bridge to a 16 pci board.

https://www.youtube.com/watch?v=94IX_fjlpao&t=1s

https://www.youtube.com/watch?v=b4gA7QPP3CA

https://www.youtube.com/watch?v=9gAysiHOWBE

https://www.youtube.com/watch?v=68EGwGnhk1g


the 16 card board is nice  but requires special cooling.

a 12 card board  with room would have been nice


same for you a six card with space would be nice

From where did you get this rig in the first video

I did a review thread for it

https://bitcointalk.org/index.php?topic=3015786.0

kjs provided it to me

https://bitcointalk.org/index.php?action=profile;u=336359

[FyrBridd]

*snip*

PCI-E spec only uses 2 pins for +12VDC on a 6 pin connector, but that's still 192 watts of RATED draw.
The pins are rated at 8A not 9A when used in the connector body per MOLEX spec sheet.

Ah yes had forgotten that, and was going off memory for the rating. Maybe I'll go with 2x 6pins for 3 GPUs for safety, giving 384W of rating for 225W of draw.

*snip*

it would fit if  you use  120mm fans to a 140mm shroud

gpu 140mm 120mm fan>

2 gpus behind each shroud

 https://www.amazon.com/Bgears-Cooling-Fan-Adapter-140mm-Black/dp/B0043GMY1U/ref=sr_1_3?

now this means  3 x 140mm = 420mm Plus a little more   but 420 mm = 16.535 inches


so say  the full build is 451mm that =  17.756 inches   which means it fits in a standard rack

that  extra spacing allows  1080ti's  at 200 to 225 watts    which = beast

a 19 inch rack mount with 6 1080ti's that  can do 1200 to 1350 watts  is a monster

the fan adapter will actually cut fan noise down


you could use a lessor delta  like this one

https://www.newegg.com/Product/Product.aspx?Item=N82E16835213015

if you used blower 1080ti's like this one

https://www.newegg.com/Product/Product.aspx?Item=9SIAAZU6HW6961

Yeh, I think I'll go for 3.5 slot spacing giving a PCB size of roughly 425mm (any bigger and you will struggle with screws, tolerances, etc), which would fit neatly behind 3x140mm fans as you describe. Thanks for the helpful feedback!

chinese estores already sell 1 PCI-e 1x -> 4 PCI-e 16x pcb's with 2 slot spacing for $25 so if you can beat that price, go for it

As you say, 2 slot, meaning you then need 4 PCIe risers adding $32 more to sufficiently space them bringing the total to $57 dollars, whilst I won't get near that I will hopefully not laugh in the face of the PCIe standards that these risers do and be far more reliable... and these still don't allow you to run 12+ GPUs off the most bog standard motherboards.

Started putting together a schematic which will allow me to build a BOM and cost it up. Will try and add '+£' figures for any options to get peoples opinions on them.

Does anyone know the sort of PCIe bandwidth mining with a GPU actually uses, as in would a single x1 slot be sufficient for 6 GPUs or is that pushing it a bit? If so price could definitely be decreased and even lower end motherboards utilised.

[pickleburglar]

As you say, 2 slot, meaning you then need 4 PCIe risers adding $32 more to sufficiently space them bringing the total to $57 dollars

..no, 2 slot means I can comfortably fit 99% of the cards out there. I mostly use them in pairs for 1050ti/rx550/rx560 rigs paired with cheap uATX boards for 8 card rigs, more or bigger cards gets less cost efficient per slot since bigger PSU's cost too much per watt and I'm avoiding multi PSU setups as much as I can.

If you really need spacing they also sell 4x single riser hubs that along with the risers come to $35 total