Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS]

[twoBitBasher]

how about guest operating systems? (bamt/linuxcoin)

[1714136717]

1714136717

[1714136717]

1714136717

[1714136717]

1714136717

[rjk]

how about guest operating systems? (bamt/linuxcoin)

Likely BAMT, if I can make it virtualize, otherwise I'm not sure.

[oldDirty]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

[rjk]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

I wonder if I should consolidate all the information into a new topic, since no one ever reads the posts in this one.

[Red Emerald]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

I wonder if I should consolidate all the information into a new topic, since no one ever reads the posts in this one.

It is kinda sad.  I mean you are talking about virtualizing on this very page.  It's not like they had to even read the whole topic...

[Gomeler]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

I wonder if I should consolidate all the information into a new topic, since no one ever reads the posts in this one.

Threads of this length become hard to parse and maintain interest through the entire thread. Especially with the last few pages consisting of the same "how will you run that many GPUs" without any progress on that front. Might be worth closing this thread, linking at the last post to a new thread with updates.

[oldDirty]

sry i dont read 20 pages, maybe the answer was allready writen to page 2, 3, 4 or where ever.  -_-
nevermind.
virtualizing, ok thx.
But i mean is there a standalone OS that can handle more than 5-6 GPU's?

[rjk]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

I wonder if I should consolidate all the information into a new topic, since no one ever reads the posts in this one.

Threads of this length become hard to parse and maintain interest through the entire thread. Especially with the last few pages consisting of the same "how will you run that many GPUs" without any progress on that front. Might be worth closing this thread, linking at the last post to a new thread with updates.

I guess I'm probably taking it a bit more personally than I should, since this is my baby. But perhaps I will start a new thread when I get the new host board.

[Gomeler]

Can they handle more than 5-6 GPU's? I heard that a Windows Cat/Forceware Driver cant handle more than 5 ore 6 GPU's, not sure.

I wonder if I should consolidate all the information into a new topic, since no one ever reads the posts in this one.

Threads of this length become hard to parse and maintain interest through the entire thread. Especially with the last few pages consisting of the same "how will you run that many GPUs" without any progress on that front. Might be worth closing this thread, linking at the last post to a new thread with updates.

I guess I'm probably taking it a bit more personally than I should, since this is my baby. But perhaps I will start a new thread when I get the new host board.

Understandable. Keep in mind though that it is a serious time investment to read a 20 page thread if you haven't been following this from the start. Keep at it, we want to see this thing completed.

[DeathAndTaxes]

But i mean is there a standalone OS that can handle more than 5-6 GPU's?

Both Windows & Linux can handle 8 GPUs but no more.

[oldDirty]

But i mean is there a standalone OS that can handle more than 5-6 GPU's?

Both Windows & Linux can handle 8 GPUs but no more.

thx a lot.

[Garr255]

But i mean is there a standalone OS that can handle more than 5-6 GPU's?

Both Windows & Linux can handle 8 GPUs but no more.

thx a lot.

Additionaly, -- as I understand -- it is not the OS that applies the limit, but the ATI drivers.

[DeathAndTaxes]

Additionaly, -- as I understand -- it is not the OS that applies the limit, but the ATI drivers.

There are multiple limits.
AMD driver has hard limit of 8 GPUs
Unmodified linux kernel does support more than 8 GPUs.
xorg (and likely aero) is incompatible with >8 GPUs
x86 OS don't support >7 GPUs due to memory mapping limitations.
Windows XP (even x64) has never worked w/ 8 GPUs.
Windows Vista/7 didn't support >4 GPUs until driver 11.5 (or was it 11.4?)
Most (all?) BIOS don't properly support >8 GPUs w/o modification (I had to turn off some unused components to even get 8 GPUs to post).

The only non-virtualized system with >8 GPUs that  I am aware of uses Nvidia (no hard driver limit and no dependency on xorg).  Even then it required a modified x64 Linux kernel and support from both NVidia and motherboard manufacturer to write a custom BIOS which properly mapped the 12 GPUs.

So it isn't simply AMD release a new driver.  The tight coupling w/ xorg likely means AMD will never support >8 GPUs unless they redesign their drivers from scratch (which IMHO they should).

[Garr255]

The tight coupling w/ xorg likely means AMD will never support >8 GPUs unless they redesign their drivers from scratch (which IMHO they should).

Agreed. And thanks for the info.

[twoBitBasher]

Did you guys know Microsoft has this product called "multipoint server" it allows, if I'm not mistaken, up to 10 monitors(desktops/sessions) on one computer. This of course is not the same as 10 GPUs but my point is that at least windows os is capable of handling a great number of screens so maybe it is only the driver limitation that applies to windows. It would be interesting to test these limits, but I don't have 12 monitors and my current rig won't accept any more cards either. 

[Garr255]

I was wondering what that was called, thanks. My PC is good enough to easily run a few extra windows instances for friends

[rjk]

If loaded to the gills with big GPUs, this rig needs a lot of power. Fortunately, PDUs that can handle it are cheap these days. I got this massive sucker for less than a hundred bucks with shipping:


Rated for 62 amps at 250 VAC. It has a 25 amp circuit breaker on each outlet, although each one is rated for a max of 16 amps, and each section of 2 receptacles is rated for a max of 16 amps total. It has current monitoring on each outlet, Ethernet and serial ports for monitoring, and includes an environmental probe for temperature monitoring. The plug is massive, although I'll probably end up hard wiring it. The cable is 3 conductor 6 AWG.

[rjk]

The total expenditure so far on the project is $2,865.86, not including time spent or video cards since I had them already. (13x 5870s from Bensoutlet)

[twoBitBasher]

Did you get the new backplane yet?

[rjk]

Did you get the new backplane yet?

I think you mean the host card (SHB), and no that will arrive on Tuesday.