Mining rig extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS]

[DiabloD3]

Well there you go.  I will have to pick some AS cleaner up then.  Thanks Diabolo! 

I am a Diablo, not a Diabolo.

[bulanula]

I don't know about you guys but I have been using isopropyl alcohol and rubbing alcohol and even acetone very successfully and got very low temperatures.

If you know how to do it, then anything is good. The difference between thermal compounds is a few degrees at best. The difference between a failed application and a professional application can be huge.

[trouserless]

alcohol works but it doesn't really dissolve the TIM.

If found this works good.  Yes a ripoff @ $7 for small bottle (good for 10 GPU at least) but it works very well.
http://www.newegg.com/Product/Product.aspx?Item=N82E16835100010

For cleaning up bad thermal paste jobs which got some of the resistors I have found a method which works well.  Using the #1 cleaner in the link above, let it dissolve some of the thermal paste, mop it up with qtips.  You aren't really scrubbing it (which just smears the paste around to other resistors) more using the qtip like a sponge.  It will take 3 or 4 passes but you can get any surface perfectly clean.

I just watched the video on the newegg link above and was surprised at the "grain of rice" application of the thermal paste.  I've been reading that "less is more" when applying this stuff in other posts here, but had not seen how little the "less" reference is.  I'm assuming that a GPU application is the same as the CPU application in the video above?  I've got a 5970 that is requiring close to 100% fan to keep at 72C and think I will see what this could do to drop the temps/fans to be more in line with the other two cards that I have.

[DeathAndTaxes]

I have found the grain of rice method doesn't work well on GPUs.

With a CPU you have a single heatsink which mounts to the flat heat spreader and applies a lot of pressure.

With a GPU the heatsink covers multiple components and is tightened down with multiple screws.  Very hard to get the same amount of direct pressure on the GPU die.

The other advantage of CPU is the heat spreader is larger than the chip surface so if thermal compound doesn't get all the way to the corners it doesn't really matter because there is no heat source there.  With GPU what you see is the actual chip package and even the corners are putting out significant amount of heat.

For CPU the single grain of rice method works fine.  For GPU I use the thin line and spread with a credit card.  It doesn't need to be perfect the heatsink will apply some pressure and spread it out. 

Still regardless of the method you want as little as possible.  I try to get it thin enough that it looks almost translucent.

[Roadhog2k5]

I have found the grain of rice method doesn't work well on GPUs.

With a CPU you have a single heatsink which mounts to the flat heat spreader and applies a lot of pressure.

With a GPU the heatsink covers multiple components and is tightened down with multiple screws.  Very hard to get the same amount of direct pressure on the GPU die.

The other advantage of CPU is the heat spreader is larger than the chip surface so if thermal compound doesn't get all the way to the corners it doesn't really matter because there is no heat source there.  With GPU what you see is the actual chip package and even the corners are putting out significant amount of heat.

For CPU the single grain of rice method works fine.  For GPU I use the thin line and spread with a credit card.  It doesn't need to be perfect the heatsink will apply some pressure and spread it out. 

Still regardless of the method you want as little as possible.  I try to get it thin enough that it looks almost translucent.

Nope... Grain of rice method does not work for any bare dies. It leaves the edges bare like you said.

I use an X method which works fantastic. Two thin lines in an X shape from each corner of the die. Spread method works good too. Really, any method will work as long as the die is covered.

[bulanula]

I have found the grain of rice method doesn't work well on GPUs.

With a CPU you have a single heatsink which mounts to the flat heat spreader and applies a lot of pressure.

With a GPU the heatsink covers multiple components and is tightened down with multiple screws.  Very hard to get the same amount of direct pressure on the GPU die.

The other advantage of CPU is the heat spreader is larger than the chip surface so if thermal compound doesn't get all the way to the corners it doesn't really matter because there is no heat source there.  With GPU what you see is the actual chip package and even the corners are putting out significant amount of heat.

For CPU the single grain of rice method works fine.  For GPU I use the thin line and spread with a credit card.  It doesn't need to be perfect the heatsink will apply some pressure and spread it out. 

Still regardless of the method you want as little as possible.  I try to get it thin enough that it looks almost translucent.

Nope... Grain of rice method does not work for any bare dies. It leaves the edges bare like you said.

I use an X method which works fantastic. Two thin lines in an X shape from each corner of the die. Spread method works good too. Really, any method will work as long as the die is covered.

I used big grain of rice and my temperatures were fantastic ...

As said before YMMV depending on how much you apply and how much you press etc.

[Cablez]

For GPUs I generally use (2 rice grain) size and press as hard and even as I can while screwing cooler back on using opposite corners.  Has worked well so far, no bubbles and good coverage.

[tnkflx]

How many GPU's are you hoping to use in this box?

As many as I can cram in... I count 18 slots, which gives me 36 GPUs IF I am able to use all the slots, and IF I can get software to support them. At least one, and maybe even 2 of the slots appear to be "reserved" or may require an expansion board in order to be useable. The other 16 are fair game though. Dual GPU cards would then mean 32 GPUs total.

I have been experimenting with removing the plastic shroud on my 5870s - as you may know, 5870s (and most other graphics cards) fit very close together if you use adjacent slots. There is usually no gap to speak of, which makes airflow impossible unless you use shims to wedge the cards apart. When the plastic is removed, there is tons of space between the cards. See pic:

I have discovered that doing this leaves enough of a gap to fit in an x16 extender, which means that I could mount 8 cards directly to the board, and an additional 8 to 10 cards one level up via x16 extenders, double stacker style. Combined with some special cooling apparatus, this should allow me to air cool all the cards, while still fitting within a 7 or 8 U chassis.

Watching this thread with interest I'm currently running 4x 6990 under Win7, but I might be going to 8x 7990 once they come out (if that's even possible...)

[minero1]

i've found it more efficient to even the TIM with a small scrapper like this one i use to evenly apply flux in reballing of BGA packages, they have a nice curvy shape that really helps you do the job, just apply a thin layer.

[1l1l11ll1l]

i've found it more efficient to even the TIM with a small scrapper like this one i use to evenly apply flux in reballing of BGA packages, they have a nice curvy shape that really helps you do the job, just apply a thin layer.

That makes me worry of scratching the bare die. I'd consider maybe a nylon spudger though .

[minero1]

That makes me worry of scratching the bare die. I'd consider maybe a nylon spudger though .

silicon is 7 in the mohs hardness scale

steel varies according to its quality from 4 to 6.5

you'll have a hard time scratching it unless you actually hammer the die with the tool also there is going to be a paste between the scrapper and the die, although you may scratch it if you don't clean it first and there are sand particles present but that would happen with a nylon spudger too

[rjk]

That makes me worry of scratching the bare die. I'd consider maybe a nylon spudger though .

silicon is 7 in the mohs hardness scale

steel varies according to its quality from 4 to 6.5

you'll have a hard time scratching it unless you actually hammer the die with the tool also there is going to be a paste between the scrapper and the die, although you may scratch it if you don't clean it first and there are sand particles present but that would happen with a nylon spudger too

Yeah but the actual packaging isn't silicon is it? The logic gates inside consist of silicon, but the die packaging/heat spreader is some kind of shiny metal.

[Joshwaa]

Usually Copper coated with zinc.

[rjk]

So I found a good price on some fans, and have ordered them. They are far more loud and powerful then your average Delta screamers. Link: http://www.ebay.com/itm/220832473067 (Got 8 )

To go with these, I need someone to build me a fan controller. See this thread: https://bitcointalk.org/index.php?topic=74402.0

Just the fact, ma'am: Delta PFC1212DE-PWM [PDF link], 252 CFM, 48 watts at 12 volts DC, 5500 RPM, 66dBA (!), and 35 mmH²O static pressure.
EDIT: fixed link, sorry.

[Garr255]

I know this has probably been covered in the 14 pages of this thread, but:

Why won't risers work to use the inaccessible slots?

[hmblm1245]

So I found a good price on some fans, and have ordered them. They are far more loud and powerful then your average Delta screamers. Link: http://www.ebay.com/itm/220832473067 (Got 8 )

To go with these, I need someone to build me a fan controller. See this thread: https://bitcointalk.org/index.php?topic=74402.0

Just the fact, ma'am: Delta PFC1212DE-PWM [PDF link], 252 CFM, 48 watts at 12 volts DC, 5500 RPM, 66dBA (!), and 35 mmH²O static pressure.
EDIT: fixed link, sorry.

I guess this is going underground to dampen that noise.

[rjk]

I know this has probably been covered in the 14 pages of this thread, but:

Why won't risers work to use the inaccessible slots?

They will. On the previous page, I discuss how I can remove the plastic shroud and blower fan on the video cards, allowing me to use my own fan setup and allowing more space between them. The additional space between them will allow me to use risers to have a 2 level design.

[rjk]

DeathAndTaxes, DiabloD3, thank you for your suggestion of ArctiClean for cleaning off the thermal paste from my GPUs. This stuff is awesome.

ArctiClean is citrus based, and this makes it really pleasant to work with as opposed to harsh chemicals and/or alcohol. It comes in a bottle with a metered drip tip that lets out a drop at a time, and the $7 I spent on it is going to last forever. Cheap as hell for what you get. You drip on a few drops of the first cleaner part, and the existing thermal paste dissolves instantly, just waiting for you to wipe it up with a soft lint-free cloth. On some of my GPUs, it took 2 applications to completely clean it. Then, drop on a drop of the second part to finish the job, and admire the mirror finish on the GPU die!

The thermal paste that I chose (Tuniq TX-4) comes with a plastic scraper to spread the stuff around real thin, but it ends up being somewhat uneven anyway. I decided that having the paste so thin that I could read the lettering on the die through it was probably not enough, and ended up using an even layer about as thick as some heavy paper. I haven't tried it yet, but it looks far better than Arctic Silver that I used on some other GPUs.

Finally, the project is sort of on hold at the moment until Spotswood finishes my custom frame - it should be here in a few weeks though. If anyone needs something custom, talk to him - he can make whatever you want. The best part about it is that he whips up the design in CAD, and will discuss exactly what is needed as part of the design phase first, while sending you renders of what it will look like as the design goes on. The final frame should look something like this:



That includes a tray to fit the custom hole pattern on this backplane. The PSUs will go on top, and the side sections are removable. It is really wide for testing with a large number of GPUs, but I eventually hope to compress this project into a 7-8U 19" rack formfactor. The fans are Delta PFC1212DE-PWM 120mm models with 252 CFM airflow.

As usual, all the project costs are in the second post, and I will keep that updated. I also keep adding to the imgur album, available here: http://imgur.com/a/0oist

[amazingrando]

First of all - this has been really interested to watch it develop.  The more rigs I've accumulated the more management of rigs becomes an issue and higher density setups look more appetizing.  Can't wait to see how it turns out.  Specifically how well it works and what the final bill is.

Second, what app are you using to model the rig frame?  I need to do something similar but don't know what would be good software to use for mocking up ideas.

[Gomeler]

I'm guessing there still isn't a solution to the lame 8 GPU limit with AMD drivers? I've read through a few of the pages and it sounds like short of AMD adjusting the limits in the drivers you'll be stuck at 8 cores.