GPU Rig PORN

[mewhoyou]

Thanks all - glad everyone likes it!  In answer to a couple of the questions;

The things sticking out of the cards are HDMI dummy plugs to simulate monitors - if you see in the background, most of the time I only have one in a system (on the card in the 16x slot).  But sometimes when I'm tuning or tweaking things it's convenient to have it in each device.

The PSU is the DPS-2000BB with these breakouts:

https://bitcointalk.org/index.php?topic=1667131.0

along with these adapters:

https://bitcointalk.org/index.php?topic=1627191

This gives WAY more power than the system needs, even the older ones with R9's - plus all in you're still talking sub $200 for 2000 watts of Platinum rated power, pretty hard to beat.

I'll probably be making one more slight mod to it - basically rotating the motherboard 90 degrees and changing the height of the unit from 14" to 11".  This will give better airflow from the large fans (they're staggered right now), and I originally did it so I could run full size boards but the reality is I only run one rig like that, so I'd rather have better fan coverage.  Because I'm overkill on cooling, the machines can run in temperature conditions that are much higher than normal.

I also think it's not really accurate to call it an 'open frame' case, as basically all sides except the back are enclosed - it's just a much more spacious case than you typically see.  

Oh, and for those interested in the 'Mines' signs, I got them here:

https://bitcointalk.org/index.php?topic=32469.0

wow .. i see you have done wonders for the mining community ... question again about the PSU. Are you powering each miner with 2000w? and for your breakers... since I am quite ignorant on this ... does your breaker comes with 8pin? or is it safe to use a 6pin to 8pin setup? Does your breaker also convert from 6pin to mobo power?

[QuintLeo]

Just a quick question for all you guys running big rigs at home, do you have any trouble/concern about the power to the outlets in your home? I have 2 rigs one with 5x1070 and one with 3x 480 and 2x 470 and all together they pull 1200 to 1500w at the wall depending on whats being mined. A standard residential circuit is 15A so maximum 1800w, I'm just wondering if pulling 1200+ through a single circuit 24/7 is a recipe for disaster

 1800 watts ASSUMES 120 volts - common residential power delivery is more likely to be 115-117 volts at the outlet and I've seen as low as 110 in some areas NOT counting "brownouts".

 (Power companies don't normally do "deliberate" brownouts any more, as they tended to fry some industrial equipment then the industrial company would sue the power company for the damages - and when you have BIG INDUSTRIAL COMPANY suing they tend to be able to afford high-capability lawyers and WIN some of those lawsuits costing the power companies more than they saved by doing the deliberate brownout).
 
 Also, the outlet itself has a 15 amp maximum rating (assuming the common NEMA 5-15 type used almost everywhere in the US for residential circuits) so even if the CIRCUIT is 20 amps the outlet isn't.

 I generally aim to keep draw from a single outlet to 1200 watts or a BIT over at most, to leave leeway for voltage sag, but in most cases 1320 watts should meet the "80% continuous usage" derating guideline safely.

[bucketofsocks]

I'm pulling 1260 from the single outlet, but in the same room/circuit I also have another 250w computer, 3 monitors, router and modem, NAS etc, I haven't had any problems yet, just slightly worried about whats going on inside the walls, but at the same time pulling a whole new circuit, which would be the best plan, isn't practical

and I constantly monitor my voltage and 119 is about the lowest I've ever seen

[dagarair]

Just a quick question for all you guys running big rigs at home, do you have any trouble/concern about the power to the outlets in your home? I have 2 rigs one with 5x1070 and one with 3x 480 and 2x 470 and all together they pull 1200 to 1500w at the wall depending on whats being mined. A standard residential circuit is 15A so maximum 1800w, I'm just wondering if pulling 1200+ through a single circuit 24/7 is a recipe for disaster

The simple answer is run dedicated 20A 240 V lines.  That way you can run 4 current generation 6 card miners on 1 outlet and still be at 16 Amps.  You increase the voltage = you lower the amp usage.  I run 30 Rigs at my house pulling about 120 amps.  watts / voltage = Amps is the formula.  I am lucky that I have 400 Amp service at my house, most don't.  I can run up to about 40 miners without disrupting house functions and getting wife aggro.

[leroy627]

Thanks all - glad everyone likes it!  In answer to a couple of the questions;

The things sticking out of the cards are HDMI dummy plugs to simulate monitors - if you see in the background, most of the time I only have one in a system (on the card in the 16x slot).  But sometimes when I'm tuning or tweaking things it's convenient to have it in each device.

The PSU is the DPS-2000BB with these breakouts:

https://bitcointalk.org/index.php?topic=1667131.0

along with these adapters:

https://bitcointalk.org/index.php?topic=1627191

This gives WAY more power than the system needs, even the older ones with R9's - plus all in you're still talking sub $200 for 2000 watts of Platinum rated power, pretty hard to beat.

I'll probably be making one more slight mod to it - basically rotating the motherboard 90 degrees and changing the height of the unit from 14" to 11".  This will give better airflow from the large fans (they're staggered right now), and I originally did it so I could run full size boards but the reality is I only run one rig like that, so I'd rather have better fan coverage.  Because I'm overkill on cooling, the machines can run in temperature conditions that are much higher than normal.

I also think it's not really accurate to call it an 'open frame' case, as basically all sides except the back are enclosed - it's just a much more spacious case than you typically see.  

Oh, and for those interested in the 'Mines' signs, I got them here:

https://bitcointalk.org/index.php?topic=32469.0

I can see how you're powering the GPUs and risers, what about the cpu and mobo though?

[sezyumx]

So nice rigs.

[topgeek]

The things sticking out of the cards are HDMI dummy plugs to simulate monitors - if you see in the background, most of the time I only have one in a system (on the card in the 16x slot).  But sometimes when I'm tuning or tweaking things it's convenient to have it in each device.

Sorry for the rookie question - What does the HDMI monitor simulator actually do?  Why do you need it?

cheers and thanks!

[Shiroslullaby]

Great pics.
I would love some information about what coins you are mining, monthly electricity costs, investment costs, and predicted time to ROI on some of these setups.

[suggsy89]

The things sticking out of the cards are HDMI dummy plugs to simulate monitors - if you see in the background, most of the time I only have one in a system (on the card in the 16x slot).  But sometimes when I'm tuning or tweaking things it's convenient to have it in each device.

Sorry for the rookie question - What does the HDMI monitor simulator actually do?  Why do you need it?

cheers and thanks!

It's a possibility that the graphics card hardware fails to initialise properly so you can’t run tasks that need the help of the GPU accelerator.

[xleejohnx]

The things sticking out of the cards are HDMI dummy plugs to simulate monitors - if you see in the background, most of the time I only have one in a system (on the card in the 16x slot).  But sometimes when I'm tuning or tweaking things it's convenient to have it in each device.

Sorry for the rookie question - What does the HDMI monitor simulator actually do?  Why do you need it?

cheers and thanks!

Windows doesn't like to startup correctly without an active display sometimes
Also most overclocking software needs an active video to work and set settings
Dummy plugs fix those issues

[MarkAz]

Some more answers - I don't need to answer about the HDMI dummy plugs because a couple other people described their use perfectly.

As far as motherboard, I use the ASRock H81 BTC, like most people do I think - most are the older version, but I have two of their newer ones (basically the same).  I have a couple random boards when the H81's weren't available, but now that they are again that's my goto board.

On the Memory/CPU, most all have 16GB memory - it's cheap enough, so why not.  On the CPU side, I just pick whichever CPU has the best bang for the buck at the time, so I'm not overly concerned about that either.  Since I don't mine with the GPU and really just have it run the system, any of them seem to work equally well.

OS-wise I run Windows 10, I don't bother licensing it because unlicensed gives me all the functionality I need, and I went through and locked down the image so it's basically purpose built for mining.  Runs off an SSD, usually 120GB just to be safe, but you could easily use smaller.

Mining-wise I use Claymore/WattTool/GPU-Z/PolarisBiosEditor/ATIFlash, and then a couple of random custom things I wrote to make rebooting and doing scheduled tasks easier to deal with.

There were questions about power - and a couple people gave great answers, I'll just throw in my specific solution.  In the case of where I have these miners, I have ~300 amps available at 240v, so everything runs off of that.  The machines themselves don't even come close to using 2000w anymore - the older ones with R9's did though.  Either way, there is some slight loss of efficiency in the PSU to be running basically at 60% of max - but not enough for me to really care.

When you say breaker, I think you're asking about the breakout - and that comes with 6 pin plugs.  I buy custom cables from Sidehack, so he makes me 6-pin to 8-pin cables that are great quality and perfect for mining - I highly recommend his cables.  As far as whether it's safe to run an 8-pin card off of a 6-pin connector, the answer is yes if you're using high-quality cables, and you know you're not exceeding the connector/AWG max power ratings, which basically none of the GPU's even come close to.  Since they assume you're connecting with PSU 6-pin wires, which are generally not that good, you're well within safety when using some miner-grade cables.

For powering the motherboard, I just run a custom 6-pin cable to one of these:
 
http://amzn.to/2gU8pWP

They work great, and all it really powers is the motherboard and the SSD, which typically pull well under 50w, so it's more than enough.

I dug up an old picture of one of my earlier designs (these fans were worthless compared to what I use now, older risers, I was still doing the 5v stepdown myself, etc) - but it gives a good idea how everything is setup:

[armazingerz]

I have a question,

how do you evacuate so much heat from the room? I intend to build 25 rigs, but that's over 20.000W, how could I keep the room moderately cold? Here there can be 38º celsius on the street in summer

Regards

[nitromining]

I have a question,

how do you evacuate so much heat from the room? I intend to build 25 rigs, but that's over 20.000W, how could I keep the room moderately cold? Here there can be 38º celsius on the street in summer

Regards

Step 1- Calculate the volume of the room (L’xW’xH’).

Step 2- Multiply the volume of the room by the required room air changes.

Step 3 - Divide the answer by 60 minutes per Hour to find the required room CFM

Take that CFM and find a blower system to do so.

Computer room cooling reference doc:

http://services.geant.net/cbp/Knowledge_Base/Physical_Infrastructure/Documents/gn3-na3-t4-ufs108.pdf

[MarkAz]

how do you evacuate so much heat from the room? I intend to build 25 rigs, but that's over 20.000W, how could I keep the room moderately cold? Here there can be 38º celsius on the street in summer

Nitromining gave you the math behind it and that gives you a good rough idea of how it will perform - but there's a fair bit of variables that go into the mix.

#1; Whether you're going to control exhaust (such as using a plenum), or just going to have it be uncontrolled
#2; Determine whether you're designing a system to be push or pull for fresh air
#2a; Make sure exhaust and intake are far enough away from eachother to not short-circuit the airflow
#3; Based on where you live, if you can process the air (cool) in some way
#4; Do baffles or other things need to be done to control noise

So in my case, I've got the flexibility that in this room (my testing area), I can basically do almost all of those things - I have a huge amount of filtered air grills for input air.  I've experimented with both tightly controlled exhaust air (even more controlled than the typical hot/cold channel you see in datacenters) - here's an example of my Avalon 6 Plenum build:



All the exhaust air is vented out through the ceiling and additional ducting... I also have a direct/indirect evap cooler that can feed the system, but this requires you to be in a dry climate (which I am, in Arizona).  Processed air via evap, or industrial misting, or even air conditioning can make a huge difference on performance - so if you are in an arid environment, definitely explore buying or building your own evap solution (it's super simple).  The other thing to keep in mind is if you design the system from the start, you can do clever things like have unpowered (except for pump) evap on your intake side, then machines, the powered exhaust fans - so you get the benefit of evap but you're also separating things enough that you don't get the more traditional issues you'll run into with evap and moisture.  Plus evap media when running works even better than air filters in many cases...

With the GPU miners, I stopped doing the highly controlled airflow because the thermal load was so much less than BTC that I didn't need to make something so custom - and I like the additional flexibility that the modular cases bring.

[nitromining]

Very nice!

When summer hits if it's still profitable I'm going to vent everything out a stack in my roof as well.

[MarkAz]

Very nice!

When summer hits if it's still profitable I'm going to vent everything out a stack in my roof as well.

One cool trick about doing that, if you're careful about total CFM and the static pressure of the plenum, you can remove almost all of the fans that are typically used.  In my Avalon 6 build, I actually unplugged the 120mm fans on them (they were still mounted, as I needed them in place to effectively reduce the intake size) and I just used the extraction fans to pull the air through them.  Nice because my biggest failure point on the BTC miners was usually fans - and when I built the exhaust I used two large extraction fans each behind their own auto-close louvers, so if one failed it wouldn't short-circuit the airflow (although the rig as a whole would certainly run hotter).

[mettalmag]

Here some of my rigs, total rigs 85 cards 292.
In a week or so we are starting the preparation process for hot summer, more photos to come ..

[w00tcl]

MarkAz
what two large extraction fans you use in your build?

[AriesIV10]

I had many people ask me to share my 7 GPU mining RIG so, here it is:

MintMiner I:
•   1101 SOLs ZCash, 860W +/- 10%
•   160 MH/s Ethereum, 907W +/- 10%

MintMiner II
•   1284 SOLs ZCash, 906W +/- 10%
•   175 MH/s Ethereum, 955W +/- 10%

MintMiner III
•   1500 SOLs ZCash, 1075W +/- 10%
•   205 MH/s Ethereum, 1200W +/- 10%




Packages Available for purchase:: https://bitcointalk.org/index.php?topic=1676763.0


The most important reason for me developing this RIG is to have the ability to adjust the temperature and control the airflow.  I can now direct this air into my furnace or out the window.

Let me know what you think, suggestions and recommendations:  https://bitcointalk.org/index.php?topic=1676763.0

[klondike_bar]

Here some of my rigs, total rigs 85 cards 292.
In a week or so we are starting the preparation process for hot summer, more photos to come ..

looks great, but why would you not use an ATX mobo that can handle 6-7 cards?

your mine would be using ~40 fewer motherboards, 40 fewer CPUs, ~320GB less ram, and 40 fewer SSDs. Thats like $5000-6000 savings, and less time spent tuning 1 rig at a time