Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS]

[The-Real-Link]

  Ok man sorry... forgive me for merely speculating as to what could physically fit on this board.  Of course I know there's the 8 GPU limit now that I've read about this stuff .  Now if or how that can be overcome, I do not know.  Good luck!

[BTC guy]

cool toy

[shakaru]

Seems like you could do 5 5970s in that with 7 single slot cards. 

Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.

And that is?

[shakaru]

Man, I had no clue that showing you this crap would make you buy it. Love of god, you need help, I got your back. This solves alot of problems if it works.

32 cards waterstation he I come!

[DeathAndTaxes]

Seems like you could do 5 5970s in that with 7 single slot cards. 

Might want to read the thread. We still have to deal with 8 gpu driver limit.

This can be worked around quite reasonably.

Really? Also please don't say virtualization these are non VT-d capable CPUs.

[jake262144]

Really? Also please don't say virtualization these are non VT-d capable CPUs.

I don't feel like digging through spec sheets, doesn't it take a Faildozer^WBulldozer-based CPU to get amd-vi (AMD equivalent of VT-d)? Ugh...
Intel is even worse as even some of their server-grade CPUs lack VT-d support.

[BinaryMage]

Do tell? If it is anything with running multiple X sessions, I am enough of a linux noob to not know how this works or where to begin. If you mean by using VT-d and some virtual machines to overcome the driver limits, sure sounds good but my hardware is not capable (no VT-x or VT-d unless I get a more expensive host board). Or, if you mean modifying the drivers in any way, or using specific versions without a limit, perhaps you will enlighten me in that regard?

EDIT: Spoke on IRC, and it seems that VT-d was the idea, but unfortunately my current hardware doesn't support it

Seems like VT would work though; how much are VT-d capable host boards?

[RandyFolds]

I am tagging along to watch the progress on this beast. Someone's datacenter is gonna get a lot louder...

[rjk]

EDIT: Spoke on IRC, and it seems that VT-d was the idea, but unfortunately my current hardware doesn't support it

Sorry, you guys must not have seen my edit.

Intel had VT-d, (my SHB is too old to support it, see end of page 1), and AMD's equivalent is IOMMU. They both do the same thing.

Seems like VT would work though; how much are VT-d capable host boards?

Starting at more than $1600 new, and progressing up to more than 3 grand for a dual quad core xeon beast.

[hmblm1245]

i looked at these b/c i have some rack space,but the price was too high. I'll also tag along to see where this goes. 

[Red Emerald]

watching this

[rjk]

UPDATE:

I was having software issues with one of my existing 6-card rigs, and I had a spare card around, so I whipped this together. The frame and mobo next to the backplane aren't doing anything; I just didn't feel like taking the PSUs out of the frame to hook this up. I ran out of molex connectors, so I had to use a third PSU. The main PSU is a 1300 watt Rosewill Lightning, and the other 2 are Cooler Master GX750s.

So this is seven (7) HD 5870s. Not a record breaker, I know, but just proving that the system works so far. It was a simple plug-n-play experience, after configuring BAMT. No major hurdles, it just worked. Clocks are currently 930/160 on all cards, fan speed is 50%, and everything else is using BAMT defaults. I would have gone 950/160, but the airflow is rather restricted.


Setting up, not started yet.


Mining!


Sorry, the flash reflected on the screen, but you can read most of it.


The PDU that I am using won't plug into my Kiilawatt, because it has a high-amperage plug. But its readout shows 11 amps at 120 VAC.

[DeathAndTaxes]

Very pretty but will 5870s get enough air like that?

[rjk]

Very pretty but will 5870s get enough air like that?

So far they are happy and stable at 930/160 50% fan - doing between 61 and 82 degrees. The 82 degree card is squashed in tight because I wasn't able to insert a quarter-inch shim (too tight). I'm going to leave it mining for 24 hours and see what happens.

Actually I just tried again to see, and I was able to get a shim in partially, which made the card drop down to 78 degrees. I'll leave it like that.

EDIT: Down to 75 now. What a difference a quarter inch makes.

[fitty]

That's cool.

Are you going to add 8? 9? 10? I thought I read the issue with 8+ is the linux ati drivers? Anyway this is freaking badass, please let us know how it goes adding more.

UPDATE:

I was having software issues with one of my existing 6-card rigs, and I had a spare card around, so I whipped this together. The frame and mobo next to the backplane aren't doing anything; I just didn't feel like taking the PSUs out of the frame to hook this up. I ran out of molex connectors, so I had to use a third PSU. The main PSU is a 1300 watt Rosewill Lightning, and the other 2 are Cooler Master GX750s.

[rjk]

That's cool.

Are you going to add 8? 9? 10? I thought I read the issue with 8+ is the linux ati drivers? Anyway this is freaking badass, please let us know how it goes adding more.

The 8 GPU limit is still alive and well, as far as I know. I have not yet tried more than 8 GPUs.

[malevolent]

These are 5870s? Shouldn't you be getting ~415-425MHash/s with these clocks?

EDIT: now I see, it is because of 160MHz VRAM clocks. Raise them to 260MHz to get ~70MHash/s more.

[rjk]

This doesn't have good perspective, so it's hard to see the issue, but this image shows where the molex connectors on the board are, and where they get in the way of the 5870.
My next project is to get some 14 gauge wire and some terminal blocks with bus connectors, and wire up those black terminal strips, so I don't have to use the white molexes at all.

These are 5870s? Shouldn't you be getting ~415-425MHash/s with these clocks?

EDIT: now I see, it is because of 160MHz VRAM clocks. Raise them to 260MHz to get ~70MHash/s more.

Would 260 really get me that much more hash? brb

[rjk]

These are 5870s? Shouldn't you be getting ~415-425MHash/s with these clocks?

EDIT: now I see, it is because of 160MHz VRAM clocks. Raise them to 260MHz to get ~70MHash/s more.

From around 2.5 ghash to almost 3. woot.

[malevolent]

From around 2.5 ghash to almost 3. woot.

You might get some more mhash/s by increasing VRAM clock to 300-350MHz, but it shouldn't be more than ~2-4MHash/s per card also increasing temp by ~1.5-2 more degrees (that was my case).