Bitworks 8 GPU 4U rack mount mining rig

[SpectrosKop]

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

How have you started 10 GPUs? My personal record is 8 at the moment. And I want to start 12. What OS?

[IOTUSA]

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

How have you started 10 GPUs? My personal record is 8 at the moment. And I want to start 12. What OS?

You can run 32 cards from one Motherboard using Linux Zcash miner under Ubuntu and a combination or risers and PCIE-X3 splitters. Not sure why you would want to, but you can.

[m0niker]

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

How have you started 10 GPUs? My personal record is 8 at the moment. And I want to start 12. What OS?

You can run 32 cards from one Motherboard using Linux Zcash miner under Ubuntu and a combination or risers and PCIE-X3 splitters. Not sure why you would want to, but you can.

Even with pcie switches how are you compiling the kernel for 8+ gpus? Much less bypassing current mobo restrictions to get over 16 gpus?

[padrino]

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

How have you started 10 GPUs? My personal record is 8 at the moment. And I want to start 12. What OS?

You can run 32 cards from one Motherboard using Linux Zcash miner under Ubuntu and a combination or risers and PCIE-X3 splitters. Not sure why you would want to, but you can.

Even with pcie switches how are you compiling the kernel for 8+ gpus? Much less bypassing current mobo restrictions to get over 16 gpus?

There are multiple things in play to have systems support large numbers of any PCIe card, GPUs of various types require different addressing so the specifics depend on what you are doing but in short it's a combination of the motherboard itself, the OS' ability to handle the cards/addressing, the driver (fglrx or amdgpu), Xorg, and the application using the GPUs. Currently I have 16 GPU systems as my maximum in development, the 8 GPU system has been on the market for a few months now.

[SpectrosKop]

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

How have you started 10 GPUs? My personal record is 8 at the moment. And I want to start 12. What OS?

You can run 32 cards from one Motherboard using Linux Zcash miner under Ubuntu and a combination or risers and PCIE-X3 splitters. Not sure why you would want to, but you can.

Even with pcie switches how are you compiling the kernel for 8+ gpus? Much less bypassing current mobo restrictions to get over 16 gpus?

There are multiple things in play to have systems support large numbers of any PCIe card, GPUs of various types require different addressing so the specifics depend on what you are doing but in short it's a combination of the motherboard itself, the OS' ability to handle the cards/addressing, the driver (fglrx or amdgpu), Xorg, and the application using the GPUs. Currently I have 16 GPU systems as my maximum in development, the 8 GPU system has been on the market for a few months now.

I'm using mobo Supermicro X10SRL in my tests. Already done folowing:
1. Enabled option "Access above 4G"
2. All the PCIe ports are set to Gen2. I suppose, it's because of risers speed. Default speed Gen3 don't allow to work with more than 4 GPUs.

Maybe I lost something important?

In my tests I've used Sapphire R9 380's. So, driver is fglrx. Os is EthOs (based on Ubuntu).

Give me a catch please, what could be a limitation of the system?

Thanks a lot in advance for your help!

[PanneKopp]

I've been manufacturing an 8 GPU rig for a few months now, as a follow-on to 4 and 6 GPU open frame rigs I've been making for a few years... Initially this unit used Radeon 380s and now Radeon 470/480s. Some may have seen it listed on ebay, I sell some there but it's primarily for the exposure, most units are sold direct to large farm operators who are running 50+ units at a time.

I do sell to individual miners as well and have been doing that for some time as well, posting this to share some details on the system as I have had a few people from the forums ask me about them...

The units run a fully custom Linux load, or a customized version of ethOS depending on the end users coin, they can mine most anything as with any GPU rig, given how many GPUs are in the system I sometimes need to customize the miner or it's launcher so it runs properly.

Prices start at $3600 USD for single systems and go down under $3,000 as the quantity goes up.

ETH: 230Mh/s
ZEC: 1200H/s
Power Usage ~1150W on 220V+, 1250W On 110V

These shots are from my 380 unit, 470 unit almost identical minus change in cards, and some strengthening of the chassis in my latest model revision to better handle shipping...

Airflow does not look very intelligent.

Nearly impossible with actual Intel x1y0 Chipsets (design-issue).

... just my 2 Satoshis.

  PanneKopp (all made in germany)

[padrino]

Airflow does not look very intelligent.

Nearly impossible with actual Intel x1y0 Chipsets (design-issue).

... just my 2 Satoshis.

  PanneKopp (all made in germany)

Good questions, on the airflow the temps stay quite good actually, the fans are high static pressure fans and push air through the system without much problem, the highest ambient they have run in this far is about 95F, not recommended but they handled it fine..

You are right on the chipset, which is why it does not use any chipsets from that series.

[QuintLeo]

Is it just me, or does the front view of that case look almost identical to the Backblaze Pod designs?

[Qumiu]

These shots are from my 380 unit, 470 unit almost identical minus change in cards, and some strengthening of the chassis in my latest model revision to better handle shipping...

I saw the distance between GPU too close together, does not heat?

[QuintLeo]

These shots are from my 380 unit, 470 unit almost identical minus change in cards, and some strengthening of the chassis in my latest model revision to better handle shipping...

I saw the distance between GPU too close together, does not heat?

 Probably a little, it appears to be the same or very similar spacing to a "1-4-7" 3-card motherboard setup.
 High-flow fans blowing air into the cards should help quite a bit though.

[SweaterJacket]

As all you can see there are 6x risers are going from MB - so it is standard one, but additional are connected through PCI splitter. And how to get them married together - it is his know-how.

And this is why it can be done only under *nix system

Just buy it - you cannot make it

its probably a Z97 gaming 5 MSI board = 7 PCIE slots - 7 GPUs
with one M.2 to PCIE ? - 8th GPU
and ubuntu should work fine.... dont need to waste money on a ubuntu wrapper

that is exactly what this rig is. I have one just like it.

[Walrusbonzo]

No i mean the converters do exist, but on the msi gaming 5 if you are using the M2 slot the PCI-e 5 is disabled, so that wouldn't work i think.

this is what I was not sure about, if using m.2 disabled a PCIE lane or just some sata ports.
still might get one to try for the fun of it though

Anyone figure out how to get 8 GPUs on the MSI Gaming 5?

My power supply supports 9 PCIe power cables, be awesome If I could fit 2 more GPUs.

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

Mind if I ask what method you used to get 8th GPU running? I've tried PCIe multipliers, but it still maxes out at 7 GPUs.

In Windows at a guess? That'll be your problem, i'm sure in Linux 8+ will be fine with a splitter.

[padrino]

Is it just me, or does the front view of that case look almost identical to the Backblaze Pod designs?

It sure does, since the owner of the Backblaze design pulled together my case with me.

[padrino]

No i mean the converters do exist, but on the msi gaming 5 if you are using the M2 slot the PCI-e 5 is disabled, so that wouldn't work i think.

this is what I was not sure about, if using m.2 disabled a PCIE lane or just some sata ports.
still might get one to try for the fun of it though

Anyone figure out how to get 8 GPUs on the MSI Gaming 5?

My power supply supports 9 PCIe power cables, be awesome If I could fit 2 more GPUs.

You cant off the shelf. Im the one that designed this system with Padrino. My personal prototype systems are running 10 GPUs of a Gigabyte board. It requires very low level changes to how the board BIOS operates.

Mind if I ask what method you used to get 8th GPU running? I've tried PCIe multipliers, but it still maxes out at 7 GPUs.

In Windows at a guess? That'll be your problem, i'm sure in Linux 8+ will be fine with a splitter.

You need to modify the addressing on the system to handle 8 GPUs on Linux or Windows, it won't boot out of the box that way but with some work it can be done.

[batko]

You need to modify the addressing on the system to handle 8 GPUs on Linux or Windows, it won't boot out of the box that way but with some work it can be done.
how? I have 7x480 works fine, but 8xRx 480 black screen.

[QuintLeo]

Is it just me, or does the front view of that case look almost identical to the Backblaze Pod designs?

It sure does, since the owner of the Backblaze design pulled together my case with me.

 Interesting statement, since Backblaze has been using that front design for quite a few YEARS now on all 6 of their Pod designs.

[padrino]

Is it just me, or does the front view of that case look almost identical to the Backblaze Pod designs?

It sure does, since the owner of the Backblaze design pulled together my case with me.

 Interesting statement, since Backblaze has been using that front design for quite a few YEARS now on all 6 of their Pod designs.

I'm not sure how you read my statement given your response... Yes they have been using it for years, and they pointed at that design as a starting point for my case so that is what we did...

[QuintLeo]

Is it just me, or does the front view of that case look almost identical to the Backblaze Pod designs?

It sure does, since the owner of the Backblaze design pulled together my case with me.

 Interesting statement, since Backblaze has been using that front design for quite a few YEARS now on all 6 of their Pod designs.

I'm not sure how you read my statement given your response... Yes they have been using it for years, and they pointed at that design as a starting point for my case so that is what we did...

 Ah, that makes sense.

 Your original statement was ambiguous enough I wasn't sure how you meant it.

[PeaMine]

Is it possible to get it with only 6 GPU for example?
or 4?
I have some of my own cards I'd love to add to this, then get rid of my more open air design.

[padrino]

Is it possible to get it with only 6 GPU for example?
or 4?
I have some of my own cards I'd love to add to this, then get rid of my more open air design.

I've been thinking about that as of late, hit me up in PM with exactly what you are looking for and maybe we can work it out..