Mining Farm Cooling

[jamesg]

I have been doing more work to rectify my "heat situation" and have consulted multiple hvac / ventilation specialists. The consensus seems to be to either vent the hot air out of the building or condition the air. Don't do both.

Another point they kept bringing up is the very little amount of insulation between the roof and the air inside the building. So I am having extra insulation put on the ceiling early next week to see just how much it helps. I will be going for the highest R value I can get and will follow up once it is installed. Insulating 800sqft should be about a $1 a sqft, a lot cheaper than other solutions. 

[CanaryInTheMine]

I have been doing more work to rectify my "heat situation" and have consulted multiple hvac / ventilation specialists. The consensus seems to be to either vent the hot air out of the building or condition the air. Don't do both.

Another point they kept bringing up is the very little amount of insulation between the roof and the air inside the building. So I am having extra insulation put on the ceiling early next week to see just how much it helps. I will be going for the highest R value I can get and will follow up once it is installed. Insulating 800sqft should be about a $1 a sqft, a lot cheaper than other solutions. 

extra insulation will help retain more heat.

[RobertRibbeck]

I have been doing more work to rectify my "heat situation" and have consulted multiple hvac / ventilation specialists. The consensus seems to be to either vent the hot air out of the building or condition the air. Don't do both.

Another point they kept bringing up is the very little amount of insulation between the roof and the air inside the building. So I am having extra insulation put on the ceiling early next week to see just how much it helps. I will be going for the highest R value I can get and will follow up once it is installed. Insulating 800sqft should be about a $1 a sqft, a lot cheaper than other solutions. 

extra insulation will help retain more heat.

not really true
depends where the heat is coming from
IE it will keep the suns heat out

[Big Time Coin]

Problem is, in Florida, you're going to have to do both.  Condition the air AND exhaust the heat.  Like this:



Except, since its only a 800 sq. ft. data center, instead of recirculating the hot air back into the CRAC you can draw fresh outside air into the CRAC intake and blast the hot air to the sky.

[jamesg]

Problem is, in Florida, you're going to have to do both.  Condition the air AND exhaust the heat.  Like this:



Except, since its only a 800 sq. ft. data center, instead of recirculating the hot air back into the CRAC you can draw fresh outside air into the CRAC intake and blast the hot air to the sky.

Thanks for this diagram. Actually just had another hvac guy out and he was the most competent of all that have come out so far. This is pretty much the direction he is pushing. Smaller room, high air flow from one side of the room to the other and enough cooling capacity to cool the 17Kw of equipment (5 ton).

[Reckman]

Hvac spec that sounds expensive...make sure your spending your money well

[STP]

If you really want to cool then think like a grower. Mini ductless AC unit and inline ducting for intake and exhaust. Use an environmental control and when temps go higher than ducting can handle they shut off and mini split system kicks in. The inline venting alone can do the job minus those really hot days. I highly recommend the Panasonic whisperline duct fans. Do the math, get the right size, x 2 since you want intake and exhaust unless you have passive intake like an open door to suck air in from.

You can find lots of growers on craigslist selling equipment all the time.

[RandyFolds]

If you really want to cool then think like a grower. Mini ductless AC unit and inline ducting for intake and exhaust. Use an environmental control and when temps go higher than ducting can handle they shut off and mini split system kicks in. The inline venting alone can do the job minus those really hot days. I highly recommend the Panasonic whisperline duct fans. Do the math, get the right size, x 2 since you want intake and exhaust unless you have passive intake like an open door to suck air in from.

You can find lots of growers on craigslist selling equipment all the time.

You are talking closet scale equipment. He's got the equivalent of a 20-lamp grow in 800 square feet, not sure of the shape, and 20ft ceilings.

I worked on a 16x1000w op in about 2000ft^2 and we kept temps below 80 with a 5-ton AC and 2 12" CanFans (good brand, but un-throttlable) and two 10" centrifugal fans in and out (with a fair amount of filtration resistance). We upped it to 20 lamps and it started running about 85, just over the capacity of the AC. The power bill was phenomenal, and 2/3rds of the room was on 12/12. Goddamn that place made some money. Puts bitcoin at $30 to shame. Then the sheriffs came and cut down all our children and ended up dismissing the charges after doing about a hundred grand in damage. Good ole' fuckin' amuuurica.

[jamesg]

extra insulation will help retain more heat.

During the day, the sun is producing a pretty serious amount of heat within the warehouse since the sun shines a great deal here in FL. Installing the spray form insulation will allow me to only need to cool the heat that is produced from the equipment during the day and night which *should* help considerably. I have been reading about the spray foam insulations and they have a much greater effect on reducing conductive heat from ceilings even though they may have a smaller R value than traditional insulations.

Installation is happening wednesday so I will soon be able to give some definitive results. I have monitoring equipment for the temps in the building so we will have before and after results with 19.5Gh running in the facility.

[cicada]

Then the sheriffs came and cut down all our children and ended up dismissing the charges after doing about a hundred grand in damage. Good ole' fuckin' amuuurica.

Please tell me you sued the city afterward

As for the insulation, the point is to keep the environments isolated - yes it keeps the hot air in, but it also keeps ambient hot air out.  I brought my attic insulation from R30 up to R60 last fall and drastically reduced my heating costs in the winter, and my cooling costs in the late spring/summer.

Then I started mining and threw those savings away

[RandyFolds]

Then the sheriffs came and cut down all our children and ended up dismissing the charges after doing about a hundred grand in damage. Good ole' fuckin' amuuurica.

Please tell me you sued the city afterward

As for the insulation, the point is to keep the environments isolated - yes it keeps the hot air in, but it also keeps ambient hot air out.  I brought my attic insulation from R30 up to R60 last fall and drastically reduced my heating costs in the winter, and my cooling costs in the late spring/summer.

Then I started mining and threw those savings away

Who's got the coin to sue Los Angeles? We wrote it off as a 'well, that sucks, but at least we're not in jail' loss.

BTW, raid was not in response to heat/smell/power consumption...it was dispute with a landlord (who signed a lease that stated we were using the property to cultivate medical cannabis and then narced on us after two years because he got a better offer for the space).

[RobertRibbeck]

extra insulation will help retain more heat.

During the day, the sun is producing a pretty serious amount of heat within the warehouse since the sun shines a great deal here in FL. Installing the spray form insulation will allow me to only need to cool the heat that is produced from the equipment during the day and night which *should* help considerably. I have been reading about the spray foam insulations and they have a much greater effect on reducing conductive heat from ceilings even though they may have a smaller R value than traditional insulations.

Installation is happening wednesday so I will soon be able to give some definitive results. I have monitoring equipment for the temps in the building so we will have before and after results with 19.5Gh running in the facility.

Well whats the results

[jamesg]

Well whats the results

Ok. Here's what I ended up doing.

1. Added the foam insulation onto the ceiling. This leveled out the temps in the building, cutting down the high temp by about 7 degrees. We had a high inside of 103 a couple of days and how the highest the temps have been is 96. This also cut out the lower temps we were experiencing in the evening. Instead of dropping to 78, we now drop to around 85.

2. I have redirected the entire 3 tons of AC to point directly at the equipment through a single 16" air duct. This has helped lower the temps at the GPU cores without lowering the temp in the building itself. I also added an LG portable AC unit in the office area which cut out 315 sqft of space that the 3 ton AC handled before.

Next I am going to take the 16" air duct and attach smaller ducts to direct cold air directly onto the computers which should help even more. I am also going to build an exhaust manifold like in the video but instead of venting it to the outside, it will send air into the return of the 3 ton AC unit. This way I will not lose all of the conditioned air the AC worked to condition and the hot air will not mix with the cooler air in the building.

The exhaust manifold will also be double sided, basically sandwiched between two shelves. The manifold will basically become the hot isle in the hot/cold isle  diagrams above.

[sveetsnelda]

Float them all in Mineral Oil: http://www.pugetsystems.com/mineral-oil-pc.php

You first. 

Mineral Oil is neat, but the thermal transfer sucks.  Getting rid of the heat from any serious rig using mineral oil would me *much* more trouble than it's worth.  You'd need radiators with 4.3X more surface area than what you'd need to cool them with water (or non-conductive fluid made for this purpose...  which is $20 a liter).

From the same page that you linked:

Additional Cooling Development Halted

We tried a number of different extreme cooling techniquies, including dry ice, phase change cooling, aquarium chillers, and liquid cooling blocks with TEC coolers. Ultimately, the snag was always the same: The thermal conductivity of oil is not as good as that of water, so all products designed to cool water do not have the sheer surface area necessary to cool the oil. It is possible that you can build your own aquarium chiller, taking care to dedicate a large amount of extra copper coils to the oil side of the heat exchanger. However, we have spent a lot of time on this, and have to get back to our main jobs -- building high quality computer systems! We encourage those with experience in phase change cooler design to pick up this ball and run with it, and if anyone can achieve a sub-zero cooling technique, we'd love to talk about it and post it here. Our standard radiator setup does a great job of cooling, so we're happy leaving our project there...but sub-zero would be pretty amazing!

[kirax]

Float them all in Mineral Oil: http://www.pugetsystems.com/mineral-oil-pc.php

You first. 

Mineral Oil is neat, but the thermal transfer sucks.  Getting rid of the heat from any serious rig using mineral oil would me *much* more trouble than it's worth.  You'd need radiators with 4.3X more surface area than what you'd need to cool them with water (or non-conductive fluid made for this purpose...  which is $20 a liter).

From the same page that you linked:

Additional Cooling Development Halted

We tried a number of different extreme cooling techniquies, including dry ice, phase change cooling, aquarium chillers, and liquid cooling blocks with TEC coolers. Ultimately, the snag was always the same: The thermal conductivity of oil is not as good as that of water, so all products designed to cool water do not have the sheer surface area necessary to cool the oil. It is possible that you can build your own aquarium chiller, taking care to dedicate a large amount of extra copper coils to the oil side of the heat exchanger. However, we have spent a lot of time on this, and have to get back to our main jobs -- building high quality computer systems! We encourage those with experience in phase change cooler design to pick up this ball and run with it, and if anyone can achieve a sub-zero cooling technique, we'd love to talk about it and post it here. Our standard radiator setup does a great job of cooling, so we're happy leaving our project there...but sub-zero would be pretty amazing!

I was looking into this before I redesigned my case full of five computers for better airflow: The thing is, they care if it looks pretty, where if this is out in an industrial area like it sounds, he doesnt really have to care about four old car radiators outside the window :p Also, they are cooling with ambient room temp: Depending on your geographical location (like here, for example, where delightful winter with -40 degree weather is coming), the heat exchange should be a whole lot more.

[sveetsnelda]

I would do it first! I have thought about doing this myself many times for the cool factor.

Don't get me wrong...  I'd love to do this with my main machine.  It'd work fine for daily use and the occasional game with 1 mid-grade video card.  Start throwing a hundreds of watts in a tank with multiple GPUs or with GPUs that are being stressed for Bitcoin mining and you're going to have a *very* difficult time keeping temperatures under control.

I just crack eggs on my wifes back to make breakfast in the morning.

haha

Why make a bunch of little tanks? Couldn't you sit all five motherboards in a 55 gallon aquarium. Fill it up just enough to cover the equipment and pump the heat to an outside radiator! In the above experiment they said the setup was still working over two years later. Way cool.

You could, but their rig didn't really produce a lot of heat (relatively anyways).  Their rig worked because it was made of glass.  That had just enough surface area to transfer the heat away.

To put things into perspective -- with water, you'd need a 3x120mm radiator to cool a CPU, power supply, and a couple video cards that were Bitcoin mining.  With mineral oil, you'd need more than 4 of those radiators to get the same cooling capacity.  A 4x GPU mining rig with 200 watt cards *might* be able to be cooled with a large car radiator (depending on the fans used and the outside air temperature).  If I wanted to cool all of my mining rigs, I'd need radiators with more combined surface area than the outside of my house.  

You'd need some wicked pumps to move that fluid around too...

[P4man]

No experience, but just thinking out loud; if you'd install a ground water well and put a sizeable container with water (and/or long PVC pipes going back and forth to act as makeshift heat exchanger )  in the datacenter and constantly pump water through the containers, it should remove quite a bit of heat at relatively low cost. Basically cooling the room with ground water.  Where I live ground water has a constant 15C temperature, but I think its the same everywhere. Drilling water wells is surprisingly cheap. Water containers can be found cheap. PVC piping costs next to nothing.  A power efficient pump with sufficient flow will cost you a bit, but the savings compared to AC Im sure would make it worth it.

Oh, and should you have a pool.. use the warm water to heat your pool. You could swim all winter

[Swishercutter]

Has anyone that has a water cooled rig tried to use the waterless coolant in it, I use it in my car and it did benefit cooling as specified by the manufacturer.  If anyone feels like experimenting I use the Evan's NPG+ coolant, I have been wondering for a while if it would help in computer applications...it certainly helps in cars.

Here's a link:  http://www.evanscooling.com/

I have been running this in my car for at least 7 years with no issues.  The only problem is you cannot have any water in the system before you add the coolant, there is an installation method that you have to follow.

[kirax]

No experience, but just thinking out loud; if you'd install a ground water well and put a sizeable container with water (and/or long PVC pipes going back and forth to act as makeshift heat exchanger )  in the datacenter and constantly pump water through the containers, it should remove quite a bit of heat at relatively low cost. Basically cooling the room with ground water.  Where I live ground water has a constant 15C temperature, but I think its the same everywhere. Drilling water wells is surprisingly cheap. Water containers can be found cheap. PVC piping costs next to nothing.  A power efficient pump with sufficient flow will cost you a bit, but the savings compared to AC Im sure would make it worth it.

Oh, and should you have a pool.. use the warm water to heat your pool. You could swim all winter

I have also considered something like this, but I am currently renting, so drilling wells in the backyard might cause the landlord to be a little flustered and stern.

[Swishercutter]

Has anyone that has a water cooled rig tried to use the waterless coolant in it, I use it in my car and it did benefit cooling as specified by the manufacturer.  If anyone feels like experimenting I use the Evan's NPG+ coolant, I have been wondering for a while if it would help in computer applications...it certainly helps in cars.

Here's a link:  http://www.evanscooling.com/

I have been running this in my car for at least 7 years with no issues.  The only problem is you cannot have any water in the system before you add the coolant, there is an installation method that you have to follow.

That's an interesting idea but a standard Koolance EXT-440CU system with a full gpu waterblock keeps my temp readings at full load less than 55c using their coolant. The downside is the heat is still transferred to the house. The card is cool but i'm on fire.

The upside is Koolance coolant is non-conductive and that's important. I had a small leak develop and didn't hurt any components. It was messy as hell but did not cost me anything to fix it. Engine coolant isn't designed to be non-conductive so you could end up with a very expensive experiment if your not careful.

I was meaning to test the conductivity, just never got around to it.