Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS]

[rjk]

Hi, rjk. It's a great pity that this interesting project was finally failed.
You should have made more progress if you had asked help from AMD directly.
I heard there was an informal "Multi-GPU" project proposed by AMD last year and finally
they got 16 GPUs (8x FirePro S10000s) to work in a single rig successfully.
So I think AMD might have already developed a proprietary driver to overcome the 8-gpu limit.
See:
http://devgurus.amd.com/thread/158863
http://fireuser.com/blog/8_amd_firepro_s10000s_16_gpus_achieve_8_tflops_real_world_double_precision_/

Hey if you wanna have a go, I've got all the parts except the video cards... Just name your price.

[420]

Hello rjk,

Your rig is looking really nice!

Quick question on your setup. Which driver(s) are you using? Is the driver causing the 11 GPU limitation? I believe I read somewhere that the AMD drivers could only handle 8 GPUs but your rig has proven otherwise, kudos!

I don't think it's a driver limit because I don't have any drivers loaded. I am just listing the PCI devices, which as far as I know does not require a driver. That's why I'm pretty sure it's a BIOS limit. There will of course be a driver limit, likely still of 8 devices, which is why I was planning on using VT-d and PCIe passthough with some virtual machines, but I didn't get very far on that.

A research group from the University of Antwerp managed to get 13 GPUs on one ASUS motherboard. They confirm the limitation is BIOS related as you mentioned. The group ended up customizing the BIOS and hacking the Linux kernel.

kickass is there a video of this?

Hello 420,

Sorry for my late reply,

The system is called Fastra II: http://fastra2.ua.ac.be/

under construction

[barwizi]

Hi, rjk. It's a great pity that this interesting project was finally failed.
You should have made more progress if you had asked help from AMD directly.
I heard there was an informal "Multi-GPU" project proposed by AMD last year and finally
they got 16 GPUs (8x FirePro S10000s) to work in a single rig successfully.
So I think AMD might have already developed a proprietary driver to overcome the 8-gpu limit.
See:
http://devgurus.amd.com/thread/158863
http://fireuser.com/blog/8_amd_firepro_s10000s_16_gpus_achieve_8_tflops_real_world_double_precision_/

Hey if you wanna have a go, I've got all the parts except the video cards... Just name your price.

let me  get it straight, you have managed to get it working at 11 GPUs?
if so then expoanding the rest of the way should be simple.

If it works with 11 GPUs i am willing to buy it, but shipping is international, get a quote and i will pay.

For simplicity i can pay in BTC.

[Biggen]

Rjk,

I was looking to do basically what you did but I only wanted it to work with 6 GPUs. Do you know if there is a case available in 4u rackmount that will accept the backplane and SBC? Id like to do a six gpu setup but i need to put it into a rack enclosure. If so ill buy it from you.

[rjk]

All the parts and accessories for this build have now been sold. Locking thread.

EDIT 7/4/2017: I keep getting asked where to find the connectors for the Dell power supply. I got them from onlinecomponents.com, but Mouser has them in stock. The Molex part page is here: http://www.molex.com/molex/products/datasheet.jsp?part=active/0755425000_PCB_RECEPTACLES.xml