DataTank Mining: 1.2MW 3M Novec Immersion Cooled 2PH Mining Container

[jimmothy]

Once again, i'm forced to educate the great unwashed

Why do you insist on lecturing people about topics you know nothing about?

When I was a little kid, I wondered why resistors weren't simply labeled in volts, so if i wanted to drop the voltage by 2V, all I'd have to do is use a 2V resistor.
I was playing with scraps of wire I stripped with my little teeth.  I was about five years old.

So what you're basically saying is that your technical knowledge of electricity has not increased since you were five?

Asking for a 300W heatsink is the equivalent of asking for a 2V resistor, or "a pipe that flows 2 gallons of water per minute.
That's why CPU heatsinks are not rated in Watts.

No it's more like asking for a 300w resistor. Heatsinks dissipate heat in watts which is probably why they are all in fact rated in watts. But don't take my word for it: http://en.m.wikipedia.org/wiki/Thermal_design_power

That's why you need a bigger heatsink to keep your CPU @45 degrees Celsius in a room @30 degrees Celsius than you would need to keep it @50 degrees Celsius in a room @15 degrees Celsius.

Nice for you to acknowledge one of the drawbacks of air cooling. I'm also happy you are actually trying to have a debate instead of exclusively slinging shit and grade school insults as usual.

[1713501688]

1713501688

[1713501688]

1713501688

[1713501688]

1713501688

[NotLambchop]

@Collider:
Lol, and let ignorance and confusion reign?  
It is mah noblesse oblige to enlighten the huddled masses, "The wretched refuse of your teeming shore," as someone once so eloquently put it.
How about you go instead?

@Jimmothy:  Stop being retarded.  Of course air cooling has drawbacks, air's thermal conductivity is very low compared to, oh, water and copper.  To counter that, it also happens to be outrageously cost-effective.

Finally (and this will no doubt be news to you), both water and immersion cooling are simply an intermediate stage of air cooling, since the final heat exchange is still done between cooling tower and air.

Now stop being a nuisance

[Collider]

Welcome, you are the very first noble person on my ignore list.

[NotLambchop]

[jimmothy]

@Jimmothy:  Stop being retarded.  Of course air cooling has drawbacks, air's thermal conductivity is very low compared to, oh, water and copper.  To counter that, it also happens to be outrageously cost-effective.

This is why I asked how much you think a 300w heatsink would cost. I would guess ~$30 for the heatsink and $10 for the fan and another ~$10 for the case/rack.

Finally (and this will no doubt be news to you), both water and immersion cooling are simply an intermediate stage of air cooling, since the final heat exchange is still done between cooling tower and air.

Yes but unlike immersion cooling, most air cooled DCs are indoors and require inefficient heat pumps to cool/move the air. There are operations like zhaos which are practically outdoors but it still requires a few thousand inefficient fans/heatsinks vs 1 giant pump and a few giant fans/radiators.

Immersion cooling also does not require such a large temperature difference like air cooling does.

[NotLambchop]

Re: 300wW cooler:  Again, this is meaningless.  You need to specify the chip (die size (surface area), package type/size, etc., etc.)  Bitfury chips, for instance, need no heatsink other than the PC board copper, and convection cooling is good enough (no fan) for low ambient temps.

Forget the TDP article, the heatsinks are rated in watts based on different givens:  Intel/AMD chip (so die size/spreader size is given), ~70 degrees average working temp, average workload, average ambient temp.  Had you been trying to cool a GPU chip instead of a CPU, the TDP Watt rating would be different.  Also, when you are pricing heatsinks and fans, don't go with the consumer pricing, go with wholesale quantity lots.  Because you can't buy a DataTank for just one board.
Hope this helps.

If it's 2-phase cooling that you like, it is cool--conventional heat pipe heatsinks use it.  



Edit:  If you know basic electronics, check out THIS LINK, and scroll down to "Analogies."  Seriously helpful, not trolling.

[LeanSixSigma]

Useful lifespan of mining hardware is less than a year.  If "high sulfur content" corrodes them in 2 years, nothing of value is lost.

A prime example for many fake claims made here, which are not based on any facts and lack sources. "Thus it was a surprise to the industry when electronics in high sulfur industrial environments would fail rather quickly, some within 4 weeks of being put in service; replacement systems would do the same. Most failures would occur in 2-4 months." - my point with potentially not being able to reuse PSUs, switches, etc. is still valid, as well the risk on reliability of mining operations.
http://www.dfrsolutions.com/pdfs/2007_08_creep_corrosion_on_lead-free_pcb_in_high_sulfur_environments.pdf

As for not installing in high-sulfur areas: Well, what experience do you have with building mines? Have you ever created a weighted multi-factor comparison list? It's not only electricity costs and climate. There are local labor costs, logistics / customs duties for importing hardware produced in China for example, general tax situation, or even as simple as that you don't want to move to another city/state/country, etc. I'm not saying that China is an ideal location, but there are many factors at play when choosing a mining location. So when you posted an open air 'chicken farm' mining cluster in China as example for cheap mining facility, then I was merely pointing out the related drawbacks specifically on that example. Open air will inevitably reduce reliability due to dust, pollution, humidity, etc. Then again, setting up a real building will cost more. Why do you have to restrict yourself from potentially lucrative locations if there is a container solution which is globally usable? Oil-rich middle-eastern countries have very cheap electricity and often zero tax. But too bad - open air facilities wouldn't work there with electronics getting the occasional sand blasting with the next storm, next to 50C/122F ambient air temperatures in summer.

@Jimmothy:  Stop being retarded.  Of course air cooling has drawbacks, air's thermal conductivity is very low compared to, oh, water and copper.  To counter that, it also happens to be outrageously cost-effective.

Finally (and this will no doubt be news to you), both water and immersion cooling are simply an intermediate stage of air cooling, since the final heat exchange is still done between cooling tower and air.

Yes, walking is cheaper than driving a car - but it depends on your specific application: It may be just not suitable for anything more than 50 miles and/or if you need to be anywhere faster than walking and/or transport stuff. Same with air cooling, water cooling, etc. The higher the density and TDP, the more air or water flow are required due to the previously mentioned limits on heat transfer per medium - it gets more difficult, more effort/energy/costs required. I even had to run computational fluid dynamics simulations using COMSOL on multiple heat sinks in series to see effects on pre-heated air from front rows of heatsinks on heatsinks at the back - such setups require much more air flow than small single row setups. Studies like these show how insane it gets if you approach 20kW per rack:

http://www.apcmedia.com/salestools/sade-5tnrk6/sade-5tnrk6_r7_en.pdf
"Remove 2,500 cfm (1,180 L/s) of hot exhaust air from the enclosure [...] For reference, carrying 2,500 cfm (1,180 L/s) in a 12” (30 cm) round duct requires an air velocity of 35 miles per hour (56 km/hr)." - that's like holding a dinner plate out of the window while driving, just imagine the energy wasted to establish that... Yes, there are big industrial fans. But we need to consider total cost of ownership, including CapEx AND OpEx - inevitably, you will need to provide a very high air flow and maybe even chilled air to cool down sufficiently - both requiring a lot of electricity. As jimmothy pointed out already, average PUEs (and OpEx) of air conditioned installations or even cheap fan installations are MUCH higher than 2-phase immersion cooling.

I have as well calculated the pressure drop losses for multiple closed loop water cooling systems myself using Darcy-Weisbach equations, incl. all pressure losses per every elbow, valve, reducer, etc. Just running water in a big pipe won't get you anywhere, due to lack of surface area to mount your chips on. So water blocks are needed as interface to isolate water from electronics at the same time (but introducing more thermal resistance as well). And to get down from big pipes, branching out into multiple smaller water blocks, will lead to big pressure losses. This again requires not only very high water flow, but as well much stronger pumps using much more electricity. It's not just theory, I could calculate it down to fractional kW for given water cooling system. Even the heat pipe you are referring to has its limitation due to multiple thermal resistances before the heat even reaches the working fluid. And again, the higher the TDP, the bigger those heatsinks with heat pipes (e.g. notebook vs. desktop CPU).

With 2-phase cooling you can easily transport a lot of heat away from the small chips even with extremely high TDP. THAT is the key difference and why W/m²K matters. The secondary stage with the water system (no one was hiding/omitting this) is then going big scale away from hardware with a lot more surface area to reject the heat using only single phase. If you refer to the often quoted 4kW simulator on the PCB size of only two postcards, then we are talking about 16 GPUs or high performing mining chips in 20nm or similar, each putting out 250W (simplified analogy). Try cooling this with heatsinks and air or single phase water cooling with water blocks requiring sufficient space to be mounted on - all introducing additional multiple thermal resistance layers (silicon to package to heatsink or water block to air or water). The amount of air (hurricane speeds?) or water flow required will be immense and/or the energy to cool down air/water will be as well higher.

On top of that, several manufacturers having employed water cooling blocks noticed that it's now actually the power components like buck regulators, etc. which contribute more and more heat with increasing currents required for the mining chips which still needs to be rejected (e.g. fans despite water cooling blocks). And there is no elegant way to mount a water block on those due to lack of sufficient co-planarity over different components' heights. With 2-phase immersion cooling, the fluid simply surrounds everything, even the side of the chip packing to cool it almost all around, no matter which shape. Again, multiple of the biggest mining hardware manufacturers are seriously considering immersion cooling. Why should they still do that if 2-phase immersion cooling doesn't make much sense according to you? So yes, maybe we should all simply sit back, relax and wait as 2112 mentioned: "Time will show".

[Ozziecoin]

Some parts (PCBs etc) need to be ordered in advance (weeks) for larger quantities. At that time the choice of chip/manufacturer needs to be clear. We have a good solution for that.

Let me know if that helps.

Yes good info. Assuming manufacturing is ready to go, what is the estimated timeline from chip purchase to shipping to immersion tanks? Reasonable approximation is fine.

Obviously will also depend on location of tanks etc.. But some kind of idea of timeline would be nice. Thanks.

[NotLambchop]

Useful lifespan of mining hardware is less than a year.  If "high sulfur content" corrodes them in 2 years, nothing of value is lost.

A prime example for many fake claims made here, which are not based on any facts and lack sources. "Thus it was a surprise to the industry when electronics in high sulfur industrial environments would fail rather quickly, some within 4 weeks of being put in service; replacement systems would do the same. Most failures would occur in 2-4 months." - my point with potentially not being able to reuse PSUs, switches, etc. is still valid, as well the risk on reliability of mining operations.
http://www.dfrsolutions.com/pdfs/2007_08_creep_corrosion_on_lead-free_pcb_in_high_sulfur_environments.pdf
...
[dissertation-sized wall of text]

Bro, what you're doing here is a prime example of fake claims.  The article you link to, titled CREEP CORROSION ON LEAD-FREE PRINTED CIRCUIT BOARDS IN HIGH
SULFUR ENVIRONMENTS, [unsurprisingly] describes corrosion in one type of lead-free solder and a new type of surface finish (ImAg).  The described corrosion took place INSIDE of a paper mill, a rubber factory and a pottery plant.

The "study data" was collected thusly:



The study then goes on to suggest PCB revisions, alternate lead-free solder pastes, plating and coatings to minimize the condition.

TL;DR:  Everyone who fired up a soldering iron or stuck a PCB in a frying pan knows lead-free solder sucks ass.  Industrial electronics (of the type to fill your gigawatt mining containers) are not required to use PB-free solder.  Use normal solder, or don't build your mine inside of a rubber factory/pottery plant.  Problem solved.

NotLambchop:  Simple solutions to complex problems^TM

[LeanSixSigma]

Bro, what you're doing here is a prime example of fake claims.  The article you link to, titled CREEP CORROSION ON LEAD-FREE PRINTED CIRCUIT BOARDS IN HIGH
SULFUR ENVIRONMENTS, [unsurprisingly] describes corrosion in one type of lead-free solder and a new type of surface finish (ImAg).  The described corrosion took place INSIDE of a paper mill, a rubber factory and a pottery plant.

Please read again: The key argument of linking to this document were the very short time frames of corrosion observed in high sulfur environments. Those time frames were listed in the general Introduction, and NOT yet part of the tests performed and mentioned later. So your counter argument pointing out the test environments and indicating that those were the abnormal conditions doesn't apply. Here some more info on data centers (including banks, financial services companies, etc.) observing corrosion despite data center and server rooms - not industrial environments. Note how much corrosion increases if outside air is involved: http://www.wescorhvac.com/What%20is%20Creeping%20Around%20in%20Your%20Data%20Center.pdf

Furthermore, this study has been made by DELL with environments known in the USA, or maybe even only specifically Texas. Have you traveled like me for many dozens of times to China and know that about 75% of electricity in China is still produced by burning high-sulfur coal? Have you read the news with many Chinese cities completely covered in the worst possible smog you can imagine with visibility less than a house block and pointing out the extremely high sulfur content, next to other contaminants known to promote corrosion? Safe WHO levels of PM2.5 are at 25. China has seen levels of 1,000 with even the capital of Beijing at 755 and not just some heavy industry areas: http://thinkprogress.org/climate/2013/10/21/2810581/air-pollution-shuts-chinese-city/

Another interesting quote: "High airflow appears to increase creep corrosion, as it is most severe in the direct path of air intake (likely due to more sulfur being made available for reaction)." - this in combination with quantification of sulphide levels (http://www.era.co.uk/case-studies/impact-of-gaseous-sulphides-on-electronic-reliability/), shows why especially big industrial fans blowing polluted air in China on mining hardware can't be good even for short periods. I have 6 more scientific studies open here with quantification of pollution levels promoting corrosion, incl. on other gases like sulfur oxides, nitrogen oxides, etc. produced while burning coal and other manufacturing exhausts. I have as well studies on dust and humidity - ask me by PM if you want any of those. Even if the concentration of contaminants might not be high, the high air flow required to cool mining hardware is simply multiplying it a LOT.

I can provide you as well one data center architect at Baidu, the former chief data center architect at Tencent and a current data center manager at Tencent to back up my claim on air filtering in data centers in China - and those are modern data centers, NOT in industrial factory environments. Again, the amount of polluted air being pushed through is humongous. That's why they have to employ elaborate air filtering for a closed building. Now translate that to open air mining facilities.

TL;DR:  Everyone who fired up a soldering iron or stuck a PCB in a frying pan knows lead-free solder sucks ass.  Industrial electronics (of the type to fill your gigawatt mining containers) are not required to use PB-free solder.  Use normal solder, or don't build your mine inside of a rubber factory/pottery plant.  Problem solved.

Yeah, right. I hope you do know that the entire European Union has banned lead in ALL electronic products since 1 July 2006 - industrial electronics included: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1281311/

Especially China is very well aware that a lot of electronics trash is being shipped from all over the world (back) to China for 'recycling' by salvaging components, metals, etc. (or sometimes just dumping it on land fills). No matter whether really salvaging by reheating the solder or just being subject to rain in a land fill, lead can be set free into the air and ground water. Since they already can't export any lead-based electronics to Europe any more and a larger number of countries are adopting the same policy (less demand/revenue), the Chinese government is in discussion to reduce, if not ban lead in electronics as well.

NotLambchop:  Simple solutions to complex problems^TM

http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
Just to clarify: I don't say that you are subject to that, but without sources or where your experience comes from, there might be a correlation (feel free to prove otherwise). All what I have posted so far is based on working since the 90's in the IT field with hands-on data center experience, in one case just one reporting level away from the CEO of a multi-billion dollar listed global corporation with tens of thousands of employees, with CEO, SVP, VP, Senior Director references to prove. I have as well designed and built multiple air and open/closed loop water cooling systems for close to a decade. So I really do know what I'm talking about on air and water cooled systems and can provide a large HVAC contractor having built water cooling systems for high-rise hotels to confirm my experience.

I actually like simple solutions very much, since I'm a real certified Lean Six Sigma practitioner and have lead multi-million dollar offshoring projects to India and China to help the company save millions every year, but I know as well about reliability studies like corrosion and even more about overall cost efficiency, etc. since I've studied a related field in my MSc and used that at work. On top of that I have been involved in the design and construction of multiple large scale industrial crypto-currency mining clusters over years. I didn't see it as necessary to mention that from beginning and I don't care if 2112 calls me 'not even semi-literate', as long as he doesn't reveal his own experience to base his claims on.

There's still a lot I have to learn, but all I know is that my arguments are based on real actual academic and work experience in the field, plus reliable external sources. If you have ever written a thesis (MBA and MSc on my side), then you will know how critical referencing and backing up claims is. Unfortunately, I have seen mostly empty or unrelated claims from you so far and not either what experience you have to substantiate. I don't know you, and you could be highly intelligent and skilled. So if you want to continue discussing, please provide reliable external sources and personal experience. Otherwise it will be difficult to take your claims seriously. We should move it to another thread or PM as well, since this is going way OT and I could provide you with many research papers to contribute to the discussion, but which would be distracting in this IPO thread.

[IPO Magic]

Stop trying to make sciences with your brain--it's absolutely disheartening to watch.  You link to articles describing corrosion in one particular lead-free solder, the use of which was necessitated by the ROHS directive.  Yes, early lead-free solder sucked ass, as I have pointed out before.  Let's move on to modern-day scenarios:

1.  Stop harping on China--it is not bound by the ROHS directive, since ...wait for it... it's not a part of EU.
Anything made for local consumption, afaik, could be made out of lead served on a bed of led shavings topped with a dab of fresh molten led.  With a side order of led salad.

2.  While we have decades of data on air-cooled electronics, making studies such as the one you cited possible, there is scant data on electronics existing for even a year in Novec 7000-series engineering fluid.  The data we have is from enthusiasts, such as overclock.net, and is rife with anecdotal evidence of failure, from failing electrolytic caps (impregnated by Novec 7000) to broken solder joints (due to Novec trapped behind chips boiling->expanding vapor physically lifting the chip).
These experimenters are few, since even today Novec is sold for ~$350 a gallon for a full gallon.  The $350 number was quoted to me just today by a forum member.

3. To this day, exactly 0 cases of creeping corrosion have been reported by bitcoin miners.  Zero.  None.
Making it a safe bet that it's not a pressing concern.  Stop finding solutions to nonexistent problems.
ty

*Re. "If you have ever written a thesis (MBA and MSc on my side)":  R U 4realz?

[railzand]

http://www.dummies.com/how-to/content/lean-six-sigma-for-dummies-cheat-sheet.html

It's common sense, Jim, but not as we know it.

[phillipsjk]

But the much more important point is that if miners are selling you a finished case with PSUs, it will add on to lead time on their supply chain. Meaning that if a new mining chip has been developed, which looks really good on screen at the prevailing difficulty that time, it might only come into your hands and will be ready for mining several weeks or months later: How fast can boards be assembled into cases if they have to be supplied from elsewhere, shipping back to a logistics center after assembly, etc.? And as mentioned above on the NRE: Customizing previous cooling solution for the new mining board, testing it, starting to order fans/heatsinks/water cooling blocks in bulk with inevitable delivery time, before assembly can even start.

In comparison to immersion cooling, the mining board could be shipped out right away.

Reading this, I was thinking: the dark horse that wants to supply chips in exchange for hosting is Butterfly labs. That is the only way they can cool their 600W boards

Re the 300W heatsink thing:
Heatsinks are rated with their temperature rise per watt. So a "300W" heatsink would have a temperature rise of maybe 40°C/300W or 0.133°C/W. It is my understanding that getting numbers lower than 0.6°C/W can get expensive (for air cooling). Edit: probably using heat-pipes that use...state-change cooling.

[LeanSixSigma]

@IPO Magic

Yes, where did I claim that China is bound by ROHS? But did you know that China has its own RoHS? http://en.wikipedia.org/wiki/China_RoHS. Yes, they are not as strict yet, but that's what I've written already that the Chinese government is aware that they have to improve (unsatisfied people = more reason for uprising). Did you know that even California has implemented something similar to the EU RoHS - again, earlier pointing out that there is a clear development towards wider adoption of similar standards? http://en.wikipedia.org/wiki/California_Electronic_Waste_Recycling_Act. More to follow: http://www.dura-flex.com/rohsfaq.html#concern. But again, that's OT.

Yes, anecdotal evidence. Spontaneous miracle healing from serious illness - a rare freak event, usually not possible to be reproduced reliably, if at all. So even if you were to provide any reliable source (which you didn't), it's exactly called 'anecdotal' because it's statistically not significant.

So everything new is bad? Should we stop all clinical trials for new ground-breaking medicine, because we don't have enough numbers/experience? I'm not saying that air or water cooling are bad in general - there are valid applications like laptops, desktop computers, etc. But my point was that the higher the density (like Bitcoin mining hardware development), the more it becomes difficult to implement air and water cooling cost-efficiently. Do you know the Novec price in bulk purchase? Do you know whether Allied Control has any special deals in place with 3M because of the official 3M Technology Partner status? So why would you assume that this is the standard price?

How long have specifically those open air mining farms been operating? Do you know anyone personally (as in met face to face) - and even if you did: Would they reveal to you such issues or rather proudly gloss over it on a forum? There are lots of Bitcoin miners who do NOT build such open air facilities and/or operate in areas with much better air - never claimed that creep corrosion would be an issue everywhere. Yes, I can't prove any incidence of creep corrosion for open air Bitcoin miners (yet) and didn't claim that - only that it's a risk, since I have no source for that (yet). But you will equally have no source to prove your claim of zero incidences, only that you haven't heard/read of anything (not a proof). And this now stands again multiple studies on creep corrosion with high air flow and with high sulfur air, observed by a broad range of electronics made by DELL, HP, Alcatel Lucent, etc. Which one carries more weight? I know personally large scale Chinese and other Bitcoin miners who are despite those open air chicken farm mining facilities still seriously considering 2-phase immersion cooling.

Well, do you really want to see my transcripts? Question back to you: Are you for real? I highly respect many people without any degree - it's 'just' academic and in my opinion real work life experience counts much more. But that comment was meant specifically for the various claims made here without any reliable source, essential to anyone who went to college or even high school when quoting references. The USA will outlaw Bitcoins in 2014 - this wrong claim as ridiculous as that may sound currently, is easily written, but without source impossible to prove.

@Railzand

Dunning Kruger effect - similar to watching the World Cup in Brazil, Superbowl, or whatever and then discrediting the intensive years of training of those players as layman on the couch: What is your experience to make a qualified comment on Lean Six Sigma? It's heavy on statistics, with analysis like ANOVA etc. and did you know that many are using Minitab to crunch data, e.g. for reliability studies? There's quite a bit more to it with GE and Motorola heavily relying on it to save millions of dollars. Further, quoting 'for dummies' is already one thing. But then even only the summary or similar of the book cover for selling, yet another. Try doing the same for a physics book and you will get as well a generic summary - oh yeah, all that is common sense when they write multiple bullet point content headers without going into the details.

[NotLambchop]

...
Yes, I can't prove any incidence of creep corrosion for open air Bitcoin miners (yet) and didn't claim that - only that it's a risk, since I have no source for that (yet). But you will equally have no source to prove your claim of zero incidences, only that you haven't heard/read of anything (not a proof). And this now stands again multiple studies on creep corrosion with high air flow and with high sulfur air, observed by a broad range of electronics made by DELL, HP, Alcatel Lucent, etc. Which one carries more weight? I know personally large scale Chinese and other Bitcoin miners who are despite those open air chicken farm mining facilities still seriously considering 2-phase immersion cooling...

In other words, DataTank *might* offer a solution to a problem which *might* exist?  Miners are too proud and bashful to talk about creeping corrosion?  You make it sound like erectile disfunction
Other than the ridiculous costs, it might introduce more problems than it solves, no?  

...
2.  While we have decades of data on air-cooled electronics, making studies such as the one you cited possible, there is scant data on electronics existing for even a year in Novec 7000-series engineering fluid.  The data we have is from enthusiasts, such as overclock.net, and is rife with anecdotal evidence of failure, from failing electrolytic caps (impregnated by Novec 7000) to broken solder joints (due to Novec trapped behind chips boiling->expanding vapor physically lifting the chip).
These experimenters are few, since even today Novec is sold for ~$350 a gallon for a full gallon.  The $350 number was quoted to me just today by a forum member.
...

Let's explore some fun possible fails with Novec 7100 immersion fluid
Take the ubiquitous electrolytic cap, present in most miners and every power supply.  These little (or big, in case of power supplies) cans contain highly-corrosive electrolyte, and fail regularly enough that the failure is planned for by providing a vent/pressure release at the top (the stamped cross)...



...and, in some cases, a thin membrane at the bottom.



Sometimes, these caps fail catastrophically with a bang, filling the room with a distinct acrid smell.



More often, the "vents" function as planned, allowing the electrolyte to slowly seep out and quickly pointing the board-level tech to the failed part.



In air/water cooled gear, this is dealt with by removing the affected PCB, removing the blown cap, giving the PCB a nice bath, and replacing the capacitor.

What do we do if the same thing happens on a PCB basking in $300+/gal Novec, sharing its sauna with a hundred other PCBs?  Well, we first have to shut down everything, drain the contaminated liquid money Novec, repair the failed board, flush the tank and the remaining boards, and replace the liquid money Novec 7100.

~At this point, Lecturer makes a long, meaningful pause, fixing the audience with his hypnotic "Hows 'bout them apples, huh?" stare.  One could hear a pin drop.  Several do.
 The audience, visibly shaken, finally breaks into unabashed sobbing.~

*Stop the muppetry.  If the industry was interested in your 40-footer filled with unicorn tears, you wouldn't have to peddle it to rubes on Bitcointalk.

[Collider]

If this would be a significant problem, how do you think AM has managed to operate its facility until today?

Also, i think the much lower temperature in the liquid will probably help capacitators not to fail.

Using quality components should also help avoid most risks of component failure.


I haven´t seen any miner whos´rig didn´t work anymore due to such a failure.
Quite to the contrary, most miners seem to still work even if capacitors are broken off completely....

[NotLambchop]

How did the thousands of data centers and hundreds of mines manage to operate without a single report of "creeping corrosion" until today?

Also ur ignore button's broke:

Welcome, you are the very first noble person on my ignore list.

Re. your edits:  In other words, only gear that doesn't fail should be used in DataTanks?
Re. yet another edit:  Miners will work with a few filter caps missing, that's not the point.  They will stop working if exposed to corrosive electrolyte from failed caps.  Do you even electronics, bro?

[LeanSixSigma]

If this would be a significant problem, how do you think AM has managed to operate its facility until today?

Also, i think the much lower temperature in the liquid will probably help capacitators not to fail.

Using quality components should also help avoid most risks of component failure.


I haven´t seen any miner whos´rig didn´t work anymore due to such a failure.
Quite to the contrary, most miners seem to still work even if capacitors are broken off completely....

+1. Thanks for supporting me on this. Yes, some boards are actually over-engineered (in a good way, because sometimes they need to rush out hardware to be first on the market and make sure it works fine). Look at the first generation ASICMiner Boards. The first ones had a lot of caps still on them and later several of those got removed and the board was still working.

Let's explore some fun possible fails with Novec 7100 immersion fluid
Take the ubiquitous electrolytic cap, present in most miners and every power supply.  These little (or big, in case of power supplies) cans contain highly-corrosive electrolyte, and fail regularly enough that the failure is planned for by providing a vent/pressure release at the top (the stamped cross)...

Thanks for pointing this out - you're actually doing me a great favor. While I haven't searched for any reliability statistics, the very picture you've used out of Wiki mentions this as well: "Many of the capacitors had a life span specification (load life) of 2000 hours at 105 °C. With a lower average internal temperature of 45 °C on a printed circuit board and a ripple current within the data sheet specifications, these capacitors should have a life expectancy of about 18 years of continuous operation; a failure after 1.5 to 2 years is very premature."

Furthermore: http://www.electronicproducts.com/Passive_Components/Capacitors/Failure_modes_in_capacitors.aspx#.U7VeV_mSx8E
"Temperature is of great concern to any capacitor. On a circuit board, capacitors should not be mounted close to heat sources. This applies to most capacitors, but especially to aluminum. A radiation shield between the cap and the hot component prevents the hot component from accelerating failure mechanisms, which can be simply a shorter lifetime (or faster parameter drift), or the opening of the pressure relief vent in extreme cases. To avoid failures in high-temperature applications, the designer should use capacitors with lower losses, a larger size, or a higher temperature rating."

There are many more sources on that - PM me for more? Sure, temperature is not the only factor, but virtually every source points out heat as a key reason for the failures you've mentioned.

This means that with insufficient cooling like air and water in higher occurrence, the lifespan of capacitors is actually much shorter than 2-phase immersion cooling with more uniform cooling. You might be able to generate sufficient air flow for maybe the first row of mining shelves, but because of the already pre-heated air, the subsequent shelves might not be cooled sufficiently and exprience hot spots. Even for the first shelves you might have hot spots already (round fan, square shelves). The US Army Research Laboratory explains this in further detail and that 2-phase cooling provides uniform temperature, as well less thermal stress on components and PCB.

Water-cooling? Even worse since some omit fans completely for water cooling ASICs only - have you ever seen any water block cooling capacitors? As mentioned earlier, Bitcoin mining hardware manufacturers noticed that with increasing current, it's actually their power components which now emit a lot of heat. Even if caps may not radiate heat themselves that much, the radiating heat to adjacent components and via PCB would still cause higher temperatures. But with 2-phase immersion cooling, the temperature can be fixed to e.g. 49C or even 34C, depending on which liquid used. Excess energy is converted into vapor - similar like that you will never be able to grill a chicken and get crispy skin in boiling water since the water temperature will never exceed its boiling point.

Then on draining the liquid, etc. even if a cap should fail: How do you know that there are no constant cleaning mechanisms already included in the DataTank systems? Again, pure false conjecture without any sources.

How did the thousands of data centers and hundreds of mines manage to operate without a single report of "creeping corrosion" until today?

90% of all women are pregnant - until it's revealed that the observation was made in the maternity ward. Check my earlier posts. I have referenced a scientific study on data centers with creeping corrosion as problem and have personal contacts at Baidu and Tencent who can back this up. Bring me a counter study on Chinese data centers, until then that's again a false claim without proof.

Well, did you know that the US military has been using 2-phase cooling for electronics for more than 50 years already? They are using it for high reliabilty cooling of mission-critical electronics in vehicles, aircrafts and ships. Some flourocarbons are still under strategic export control, because the US military wants to restrict the use of it, e.g. excluding countries like Iran. Their exact applications are slightly different to Allied Control's, but it's still 2-phase cooling of electronics. You will just not be able to read a lot about that in public. Again, this doesn't mean that it hasn't been researched and used extensively for many decades already. But yes, cost was definitely a major factor with low density hardware why it didn't take off for the commercial market before. This is different now with increasingly high density hardware, since the fluid costs broken down per board comes down a lot.

This is an excerpt of a study from the US Army Research Laboratory:

"At the same time, however, increasing power density directly exacerbates the task of managing waste heat while staying within acceptable temperature limits for these components; new generation electronic systems are producing multiple kilowatts of waste heat with projected future device level heat fluxes upwards of 1000 W/cm² [...]

However, new power-dense electronic systems are further increasing waste heat and presenting great challenges to the capabilities of conventional air and single-phase liquid cooling systems. Considering strictly air cooling, the effect of higher heat flux electronics is larger, heavier, costlier heat sinks and fans to compensate for insufficient convective performance. The effect is equally dramatic with single-phase liquid cooling, with higher heat fluxes requiring larger coolant flow rates to sufficiently cool the system devices [...]

A two-phase system has several potential advantages over a standard single-phase liquid cooling system. First, a fluid’s latent heat of vaporization has a fundamental limit that can be two orders of magnitude larger than the specific sensible heat of single-phase liquid cooling. Thus, the boiling effect provides the possibility of increased heat absorption per unit volume of fluid and higher heat acquisition effectiveness (i.e., the amount of heat absorbed by a unit of flow relative to its maximum theoretical capacity)." - pretty much exactly what I've claimed in earlier posts. Why would the US military choose 2-phase immersion cooling for high powered mission critical electronics over air and water cooling where utmost reliability is required?

I can already foresee that you will feverishly try to come up with something new and either incorrect or unrelated, only waiting for it to be dispelled again like the many other wrong points already. I admire your persistence - seriously, do you get paid for that? I hope so, because with every single won argument which I backed up with sources and dispelled one of yours, you are actually promoting immersion cooling more and strengthening it for other readers than doing it harm. This thread would probably already have moved to page 2 until the next update or occasional post, but now it's always among the Top 10 or so. Thanks.

[bitcoiner49er]

Sixsigma - 3

Lambchop - 0

[NotLambchop]

[wall of text re. electrolytic cap MTBF is higher at low temps]

Well duh!  Oddly enough, this data is collected in respect to capacitors exposed to air, not Novec.  To be more explicit, it doesn't suggest that soaking capacitors in Novec or chilled donkey piss will extend MTBF.  With no published data re. electrolytic cap MTBF in Novec 7000-series fluids available, it's nonsensical to assume that a cap will live as long in cool Novec as it will in cool air, the environment for which it was designed.  For someone so fond of studies, you certainly have trouble interpreting them, and a lot to learn about controls

...
Then on draining the liquid, etc. even if a cap should fail: How do you know that there are no constant cleaning mechanisms already included in the DataTank systems? Again, pure false conjecture without any sources.

Does your DataTank system have the ability to filter out electrolytes from Novec?  Feel free to expand, but try to keep it terse...

90% of all women are pregnant - until it's revealed that the observation was made in the maternity ward. Check my earlier posts. I have referenced a scientific study on data centers with creeping corrosion as problem and have personal contacts at Baidu and Tencent who an back this up. Bring me a counter study on Chinese data centers, until then that's again a false claim without proof.

The maker of extraordinary claims is the one who must provide collaborating evidence.  You have provided none.  Personal accounts, "I talked to a d00d who said an alien anal probed him," doesn't constitute such evidence.  Lrn to science pl0x.

Well, did you know that the US military has been using 2-phase cooling for electronics since for more than 50 years already? They are using it for high reliabilty cooling of mission-critical electronics in vehicles, aircrafts and ships...

Electronics specifically designed for such cooling.  The US army has also purchased multi-thousand dollar toilet seats, and experimented with using bats as vectors for dispersing incendiaries over Japan...



...We're a wealthy nation, with a military not entirely averse to lulzy eccentricities.  We can afford it

[another wall of text]

Please try to be concise.  The shotgun approach you seem to favor is getting tiresome.
ty