Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS]

[rjk]

To properly mount those fans, you could remove one of the "beams" in the top, placing it in the front in between the posts.

Well, the problem is that the fancy wiring job that I did with that terminal strip just coincides with the fan placement. They need to be down low, which is why they are in a stairstep. If I mounted them all at the same height, they would be a little high, and all the air would go over the GPUs instead of through them. But I can't go all they way down since the terminal strip is there. I am going to try to get everything hooked into the terminal strip, and then push it down out of the way since it is sticking up right now.

Vimeo is saying it wants to wait half an hour before my video gets converted, so I have uploaded it to cloudfront here: <removed>
I think the Vimeo link is this: http://vimeo.com/40200641 and presumably that link will start working after it is converted.

The cloudfront link expires in a couple hours, and I'll remove it later.

Vimeo link here: http://vimeo.com/40200641

[JL421]

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

[rjk]

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

There's tons of room, and they prevent damage to the back of the board due to clumsy mishandling, which I am prone to. Removing the plastic gave me a huge amount of space, and the only thing I really need to do is mod/remove/replace the front brackets with higher flow pieces.

[bulanula]

WOW. Those fans are incredible.

Sounds like a jetpack.

But, do they provide enough power so the air goes through all the cards and there are no hot spots ?

[rjk]

WOW. Those fans are incredible.

Sounds like a jetpack.

But, do they provide enough power so the air goes through all the cards and there are no hot spots ?

Short answer - yes.

Longer answer - I might have to do some fancy ducting to be sure that the air doesn't just blow over the top of the heatsink instead of through it, but for right now the sheer tornado force negates that into a non-issue. The issue will be when I turn them down to a reasonable level. Also, I don't want to mod the existing brackets and remove some resale value, so I'll need to see about getting some custom ones fabricated.

[JL421]

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

There's tons of room, and they prevent damage to the back of the board due to clumsy mishandling, which I am prone to. Removing the plastic gave me a huge amount of space, and the only thing I really need to do is mod/remove/replace the front brackets with higher flow pieces.

Sorry, I meant the slot mounting bracket...right now they're like a wall to your airflow...I removed them on all of my cards and my temps dropped ~5C...back plates are fine.

[ionbasa]

If you need any help with setting up KVM on linux, HowToForge has this guide for setup and install on CentOS if you need any help:

http://www.howtoforge.com/virtualization-with-kvm-on-a-centos-6.2-server

[DiabloD3]

You are going to want to remove those back plates on the video cards...you'll impede so much airflow with them still attached...

There's tons of room, and they prevent damage to the back of the board due to clumsy mishandling, which I am prone to. Removing the plastic gave me a huge amount of space, and the only thing I really need to do is mod/remove/replace the front brackets with higher flow pieces.

Sorry, I meant the slot mounting bracket...right now they're like a wall to your airflow...I removed them on all of my cards and my temps dropped ~5C...back plates are fine.

Not entirely they aren't. Hes building a closed air channel that is as large as the cards that go in it. The only way for air to escape, if he properly built it, is through the vents in the card backplates. The worst he'll get here is a pocket of hot air right behind the heatsink, but the heatsink will still perform correctly. The airflow dynamic is no different then how the card would have operated normally.

However, I suggest he removes the backplate and use zip ties on the now empty screw holes to secure the card semi-firmly. Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

[rjk]

Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

This is the part that I want to get fabricated, so I can keep the original brackets for resale. Unless someone wants to sell me a bunch of brackets? They could actually be from any reference card model, as long as it has the 2 screws at the top, since the rest would be getting chopped off.

[rjk]

If you need any help with setting up KVM on linux, HowToForge has this guide for setup and install on CentOS if you need any help:

http://www.howtoforge.com/virtualization-with-kvm-on-a-centos-6.2-server

Thanks, that looks like a good clear guide to follow.

Anyone want to sell me 18 AWG or better (preferably 16 AWG) 6-pin and 8-pin connectors? Chop them off of dead PSUs, or make them from parts from digi-key -  I don't feel like crimping all those terminals, but if the price is right I'll pay someone else to do it.

[bulanula]

Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

This is the part that I want to get fabricated, so I can keep the original brackets for resale. Unless someone wants to sell me a bunch of brackets? They could actually be from any reference card model, as long as it has the 2 screws at the top, since the rest would be getting chopped off.

Damn it !

Now you tell me.

I had like 8 of those things from my 8 reference 5870s and I threw them away

[Spotswood]

Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

This is the part that I want to get fabricated, so I can keep the original brackets for resale. Unless someone wants to sell me a bunch of brackets? They could actually be from any reference card model, as long as it has the 2 screws at the top, since the rest would be getting chopped off.

Would this single wide bracket work?  EDIT: More info here.

Available here for ~$2 each.

[rjk]

Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

This is the part that I want to get fabricated, so I can keep the original brackets for resale. Unless someone wants to sell me a bunch of brackets? They could actually be from any reference card model, as long as it has the 2 screws at the top, since the rest would be getting chopped off.

Would this single wide bracket work?  EDIT: More info here.

Available here for ~$2 each.

Well, the top of the card/bracket has 2 screws, and it would be ideal to attach both, since I don't want the card flopping around on one screw. Also I would have to drill some holes in those - is there a double-slot version of the same thing that i could chop the top off of?

[DiabloD3]

Either that, or cut the backplanes and only leave the very top screw mount and the overhang (so it balances on the beam).

This is the part that I want to get fabricated, so I can keep the original brackets for resale. Unless someone wants to sell me a bunch of brackets? They could actually be from any reference card model, as long as it has the 2 screws at the top, since the rest would be getting chopped off.

Would this single wide bracket work?  EDIT: More info here.

Available here for ~$2 each.

Well, the top of the card/bracket has 2 screws, and it would be ideal to attach both, since I don't want the card flopping around on one screw. Also I would have to drill some holes in those - is there a double-slot version of the same thing that i could chop the top off of?

You know, you wouldnt really need the double slot version. You could trim these, and then use the backplane screw that you normally use when plugging cards into real machines (which exist to do exactly that: keep it from flopping around).

[Garr255]

I got your fans today, along with a bunch of x6500s and risers. Christmas at Garrett's!

Anyway, I'm going to get started on that controller tonight.

[rjk]

I got your fans today, along with a bunch of x6500s and risers. Christmas at Garrett's!

Anyway, I'm going to get started on that controller tonight.

Sweet!

Yeah, maybe I'll have a go at the single slot bracket. What you guys may not have noticed is the ingenious system that Rich used to secure the video cards. Instead of pre-drilling a bunch of holes, he just made an aluminum bar "sandwich" with some felt or soft foam in between, and screws to squeeze it together. The brackets get trapped in between and don't move at all. (only 1 small issue is that the middle screws don't line up with the holes in the bracket tabs, so I can only use the ones on the end - but this works out fine)

[twoBitBasher]

Do you guys know what is the limit of GPUs that a typical mobo(bios) can initialize? The fastra2 had 13 GPUs and they needed a custom bios to boot.
I have tried to look into this virtualization business and damn there's a little of good howtos available. The esxi could have been an easy solution but 5970s won't play nice with that.

[DiabloD3]

Do you guys know what is the limit of GPUs that a typical mobo(bios) can initialize? The fastra2 had 13 GPUs and they needed a custom bios to boot.
I have tried to look into this virtualization business and damn there's a little of good howtos available. The esxi could have been an easy solution but 5970s won't play nice with that.

Depends on the CPU arch's address space (hint: 64 bit processors, internally, do not have 64 bits of address space, its usually something like 40 or 48 bit). It also depends on how the BIOS allocates space for the PCI-E GART window, some blindly select huge address spaces no matter what the device will actually use leaving you enough room for 8-10 devices.

[Global BTC]

Very impressive!

[rjk]

Do you guys know what is the limit of GPUs that a typical mobo(bios) can initialize? The fastra2 had 13 GPUs and they needed a custom bios to boot.
I have tried to look into this virtualization business and damn there's a little of good howtos available. The esxi could have been an easy solution but 5970s won't play nice with that.

Depends on the CPU arch's address space (hint: 64 bit processors, internally, do not have 64 bits of address space, its usually something like 40 or 48 bit). It also depends on how the BIOS allocates space for the PCI-E GART window, some blindly select huge address spaces no matter what the device will actually use leaving you enough room for 8-10 devices.

My BIOS has an option that I can enable that allows for the use of a 64-bit address table to map devices with, instead of the usual 32-bit table. However, it is an option that I have never seen on any other device, and I do not yet know whether it will work correctly, or even what operating systems support it. When I turned it on and attempted to boot BAMT, the screen filled with corruption and I didn't get too far. Similarly, Windows refused to install, citing some unknown hardware capabilities.

I think a 64-bit flavor of linux should work though.