Novec 7000 Project [immersive evaporating cooling]

[tntdgcr]

Mega waste of time and money.

Gratz

not at scale it's not.

[1715591915]

1715591915

[user27]

Mega waste of time and money.

Gratz

This guy has clearly done his reading thoroughly; I can't see any reason to question his conclusion/

[mangodream]

Update, I just received 6L of Novec 649 but tomorrow i'll go to Deutschland for a couple of days, next week i'll make you some pics of all the stuff working. See you there

[allcoinminer]

Update, I just received 6L of Novec 649 but tomorrow i'll go to Deutschland for a couple of days, next week i'll make you some pics of all the stuff working. See you there

Whats the life expectancy of this liquid?

[user27]

Whats the life expectancy of this liquid?

It is stable so in theory there is no limit; however depending on your setup you will always see some loss from either opening the tank or from leaks, expansion, contraction of the gas in the process. All these things can be reduced with better seals, bellows to allow expansion without venting and so on but it is very difficult to eliminate all losses/

1% per month is a number I have heard banded around but I have a buddy running this stuff and his losses are higher that that so far.

u27

[antirack]

With 1% per month there is definitely something wrong with the design of the system, especially for Bitcoin mining where power is constantly on full throttle, tanks don't actually need to be opened often and hardware goes in and out very seldom.

Close to zero (as in 0.0x%) losses is what we achieve with our systems, but they are our third generation built on industrial grade and we have spent a considerable amount of engineering since early 2012. And only if you ignore the "spilling" when people handle the fluid, but that's down to human errors.

What's also more important is the loss after a hardware refresh, for instance when you retire some 40nm hardware and move to 28nm. When everything is said and done, how much fluid did you "throw out" with the old hardware? We've been working hard to minimize that - and the spilling of course  

Getting fluid loss under control is what consumed most of our resources. Large systems would mean large losses if that is not handled properly.

(contrary to popular belief we have to pay 3M for the Novec so every little bit counts)

[user27]

With 1% per month there is definitely something wrong with the design of the system, especially for Bitcoin mining where power is constantly on full throttle, tanks don't actually need to be opened often and hardware goes in and out very seldom.

Close to zero (as in 0.0x%) losses is what we achieve with our systems, but they are our third generation built on industrial grade and we have spent a considerable amount of engineering since early 2012. And only if you ignore the "spilling" when people handle the fluid, but that's down to human errors.

What's also more important is the loss after a hardware refresh, for instance when you retire some 40nm hardware and move to 28nm. When everything is said and done, how much fluid did you "throw out" with the old hardware? We've been working hard to minimize that - and the spilling of course  

Getting fluid loss under control is what consumed most of our resources. Large systems would mean large losses if that is not handled properly.

(contrary to popular belief we have to pay 3M for the Novec so every little bit counts)

Are you running 7100 or what?

[antirack]

All of them, including 649, 7000, 7100, 7200 and some older fluids on the smaller scale/tests (we use some of them for vapor phase soldering). Depends on the chip mainly.

[user27]

All of them, including 649, 7000, 7100, 7200 and some older fluids on the smaller scale/tests (we use some of them for vapor phase soldering). Depends on the chip mainly.

It is my understanding that the efficiently recovered energy in the form of hot water is then wasted with your systems? For instance I recall seeing a couple of external cooling units at asicminer?

Have you looked into using the energy?

[antirack]

Have you looked into using the energy?

It heats our office in the winter for a month or two, but apart from that there is not a lot you can do in Hong Kong with the warm water (reuse in restaurant kitchen maybe, or warm some fish pond, if you'd have that in your vicinity).

We've been heating with the immersion system since the FPGA clusters.
http://www.allied-control.com/blog/heating-office-with-fpgas-and-servers

The DataTanks have an option for a heat exchanger for reuse as well.

With higher boiling points you could theoretically run reverse chillers, or even better produce electricity.

Look at the Panasonic developments for thermoelectric tubes, it's early but promising if true (820W/m3 with waste heat of 96°C water). I think the catch is that you also need cold water at the same time (river?) to create a high temp difference.

http://techon.nikkeibp.co.jp/english/NEWS_EN/20140416/346700/
http://panasonic.co.jp/corp/news/official.data/data.dir/en110630-4/en110630-4.html

With a chip that likes it hot (some 28nm that was just released for instance), you could run it at 110-125C and create your electricity back. One can only dream I guess.

[antirack]

Are you guys all building this as a hobby, to build a large mine, or for professional use (job/company)?

We've been playing with the thought of doing some sort of internship program for a long time (we have done that in the past) but this needs a bit of planning as training and visas are involved. We certainly have plenty of development tanks sitting around and experiments to be conducted.

Just curious, this is not really related to Bitcoin.

[allcoinminer]

Have you looked into using the energy?

It heats our office in the winter for a month or two, but apart from that there is not a lot you can do in Hong Kong with the warm water (reuse in restaurant kitchen maybe, or warm some fish pond, if you'd have that in your vicinity).

We've been heating with the immersion system since the FPGA clusters.
http://www.allied-control.com/blog/heating-office-with-fpgas-and-servers

The DataTanks have an option for a heat exchanger for reuse as well.

With higher boiling points you could theoretically run reverse chillers, or even better produce electricity.

Look at the Panasonic developments for thermoelectric tubes, it's early but promising if true (820W/m3 with waste heat of 96°C water). I think the catch is that you also need cold water at the same time (river?) to create a high temp difference.

http://techon.nikkeibp.co.jp/english/NEWS_EN/20140416/346700/
http://panasonic.co.jp/corp/news/official.data/data.dir/en110630-4/en110630-4.html

With a chip that likes it hot (some 28nm that was just released for instance), you could run it at 110-125C and create your electricity back. One can only dream I guess.

I heard many people are heating their house with the heat generated from miners.
How are you heating you office? Direct or via any heat exchanger?
Isn't it a very unhealthy or charged air ? Any heat concerns?

[user27]

Are you guys all building this as a hobby, to build a large mine, or for professional use (job/company)?
We've been playing with the thought of doing some sort of internship program for a long time (we have done that in the past) but this needs a bit of planning as training and visas are involved. We certainly have plenty of development tanks sitting around and experiments to be conducted.
Just curious, this is not really related to Bitcoin.

I just figured that running 4kW+ of Neptunes (when the other five cubes arrive), expending energy to waste the heat generated by them and then paying to run a 3kW immersion heater at my house for our hot water is kinda stupid.

No reason I can't run the lowest coil in an evap. cooling system through a big tank of water 24/7 and then only waste the energy if the 60 degree 7100 vapor gets to the next coil up; at least then I get free hot water for running miners.

I would have it up and running by now if I wasn't so busy with work. Likewise I would love to get involved with what you guys are doing; but I just don't have the time.

u27

[soy]

Why not just build a pressure chamber since you have to condense the working fluid and use R134a as its a fraction of the cost?  Would only take 80-100PSI to hit a similar range.

I personally would be afraid of building a pressure vessel to contain the R134a. I suppose you could reuse something like a boiler or other already existing pressure vessels that would be sufficiently large. Immersive cooling with fluids that change phase at low temperatures brings all sorts of issues with safety.

It would be very interesting to see if any chips could scale in performance with temperature. Eventually, when chips are cheaper, I'd like to do such testing but sadly now is the time for that to potentially make economic sense. Use 1-10x more power to achieve 1-2x higher hash rate = more profit given the high fixed costs of the chips and general low power requirements. I imagine in the future when I can afford to blow up dozens/hundreds of chips in testing, the fixed costs of the chips will be a fraction of their expected energy consumption over their lifetime and thereby make the process of chilling them a waste of money.

Also the R134A available in auto parts stores 30 lb. cylinders for $199 probably contain lubricant and who knows if the lubricant is conductive.  Those 30 lb. cylinders are sold with a clear stipulation that the R134A contained is not to be use in electric vehicles.  There are research articles describing R134A cooling of electronic car propulsion electronics.

[Biffa]

LOL soy you dug up a thread from Aug 2013???

[sjenja]

lol but seriously any updates on the rig?

[tvasbn]

The best liquid cooling agents i think is the 3M Fluorinert, Novec is use more in fire extinguisher sytems.

http://solutions.3m.com/wps/portal/3M/en_US/Electronics_NA/Electronics/Products/Product_Catalog/~/3M-Fluorinert-Electronic-Liquid-FC-770?N=4294412737+5153906&Nr=AND%28hrcy_id%3AMHLSRZTBD8gs_0B7JCKFWS6_N2RL3FHWVK_GPD0K8BC31gv%29&rt=d

[aerobatic]

The best liquid cooling agents i think is the 3M Fluorinert, Novec is use more in fire extinguisher sytems.

http://solutions.3m.com/wps/portal/3M/en_US/Electronics_NA/Electronics/Products/Product_Catalog/~/3M-Fluorinert-Electronic-Liquid-FC-770?N=4294412737+5153906&Nr=AND%28hrcy_id%3AMHLSRZTBD8gs_0B7JCKFWS6_N2RL3FHWVK_GPD0K8BC31gv%29&rt=d

no, novec is their newer stuff and is the one they intend to use for liquid cooling.  there are multiple versions of novec that boil at different temps and you can mix them together to dial in almost any temp that you wish it to boil at.  there are even green novecs that are less polluting in the atmosphere.  fluorinert is their older product.. as used in cray supercomputers.  the one you linked to boils at 95 degrees which is a little too hot for most electronics, which wouldn't be the best solution for liquid cooling as we WANT it to boil, to utilise the two-phase nature of Novec.  if you didn't want it to boil you'd use a much cheaper liquid, like mineral oil!

the whole point of using Novec is that you decide what temperature you want your boards to run at, e.g., say 75 degrees...  and you dial in the right novec liquids to boil at that temperature... and then above the liquid you have coils to cool the vapour back into liquid form.  it rains back down into the novec tanks, and you carry the heat away in the coils to somewhere where you can use it or lose it.

[sloopy]

I first saw people doing the mineral oil thing in the early 90s.
I built a peltier water cooled system and fell in love with the idea of submersion cooling, but did not pursue it due to R&D cost, but now it seems for a large scale mining operation with the right backing this could be viable. Not the mineral oil per say but the 3M products.

Is anyone successfully  running a mining rig setup long term this way?
If so, what is the cost versus air?
Does the density provide the savings?

I could see this as being most useful for higher ambient mining. Say some home miner must shut down int he summer due to heat, and are limited anyway due to noise. Take a fish tank and mine year round?

Just curious...

[aerobatic]

I first saw people doing the mineral oil thing in the early 90s.
I built a peltier water cooled system and fell in love with the idea of submersion cooling, but did not pursue it due to R&D cost, but now it seems for a large scale mining operation with the right backing this could be viable. Not the mineral oil per say but the 3M products.

Is anyone successfully  running a mining rig setup long term this way?
If so, what is the cost versus air?
Does the density provide the savings?

I could see this as being most useful for higher ambient mining. Say some home miner must shut down int he summer due to heat, and are limited anyway due to noise. Take a fish tank and mine year round?

Just curious...

there are several people doing immersion cooling for bitcoin mining rigs, and the most large scale to date (that i know of) is Allied Control in hong kong, most recently acquired by bitfury, but have previously done their own bitcoin mining and also are the guys who setup the asicminer immersion mining project.