Request for Discussion: proposal for standard modular rack miner

[Finksy]

Slightly OT but for ^^ above reasons am also a huge fan of liquid cooling. Not really worth the time/effort/expense for 1 miner at a time but -- start building a farm and whole 'nother story. Very easy to move as many kw as you want from the equipment to outside. Well worth the cost of plumbing/dry heat exchanger/fan and large pump. Aside from maybe in the Middle East, with the operating temps miners like don't even need refrigeration.

Amen to that! If only the C1 style would have picked up steam (No pun intended), it would be one of those situations where you buy once cry once. Water cooling would be the real beneficiary of standardized, modular designs like we're discussing, because you can size the heat exchanger/pump/fans as large as you need, Reuse it and the real savings would come from mass deployment (assuming water blocks are cheaper than heat sinks and high powered case fans in bulk). Even with 10kW of miners It would be worth it I think.

[NotFuzzyWarm]

heh heh... set up a peta farm somewhere and use the heat to setup a spa with heated pools.   Hell, I'd think that even a couple 100kw of otherwise wast heat would make that feasible. Hmm...
For the record -- if someone runs with that idea I'd love some of the proceeds... Wallet 14DdoPoEKiWQQj3WdLShbm6ppvm8349Hto You heard it here first!

[sidehack]

I am still of the opinion that a secondary (short-side) heatsink isn't really worth the trouble. I've mentioned half an inch between the face of one heatsink and the fins of an adjacent, which allows for PCB and some tall SMD components with clearance. This side of the PCB could have individual chipsinks glued on, but I don't think I'd worry about writing an actual full-size secondary heatsink into the spec since I'm guessing most every board implementation would have components near the ASIC which are taller than the ASIC.

For reference, see what they did with the S5+

[Witrebel]

Okay, fair enough.  What is the max main heatsink depth that would be compatible with the S1 chasis then?  Is that where 2inches came from?  Just looking for all the upper limits so I can play with layouts.

[NotFuzzyWarm]

I gotta agree the 2nd sinks don't seem to help much. I got an S5 with 1 card that has the Bitmain supplied secondary sinks and the board runs the same reported temp as all the others. Measured main sink and despite the additional chip cooling is same temp as others so... Doesn't seem to make much difference (on a S5).

[sidehack]

The actual Bitmain heatsink looks to be 1.5 inches.

If we have a 17.5 inch case and 3.5 inches is PSU, that leaves 14 inches. A practical upperbound is 2 inches per hashboard, meaning heatsinks and the clearance between them in which your PCBs will live. Mind you those are approximate, as this ignores metal thickness for the case and such. The actual heatsink will likely be less than 1.5 inches tall, unless we want to compress the 0.5" clearance between sinks. Reducing that clearance to increase heatsink size will increase power dissipation potential of the sinks slightly, but also can make PCB design more touchy and increases the risk of damaging a board during installation or removal. Everything's a tradeoff.

[NotFuzzyWarm]

Agreed. Anything that is SMD is very easy to dislodge with inadvertent handling. 'specially large SMD caps. popped a couple off shuffling boards around while modding s1's  

[Witrebel]

Gotcha, admittedly I'm still scheming for how to make 4U + internal psu + 8 cards + s1 compatibility work.  I think one of those variables has to change.  Safe to say you would prefer 7 boards 4U over 8 boards 5U?  Assuming both had internal power?  

One thing I am getting a little hung up on layout wise is the asymmetry.  In an S1/S3/S5 the boards are mirrored and symmetrical, so nominally they both lay in an identical flow path and identical cooling.  If we go with a single sided heatsink and simply pattern the cards 7 or 8 across, the flow path will be different for that first card that is up against the wall.  You could center everything and leave some space on either side so that although asymmetrical, you still have good flow all around.  But that would eliminates the possibility of PSU's on the side of the case, and makes having internal supplies at all quite tricky.  

Do you think that it matters at all?  That first card having its non sinked side up against the wall?  In reality it may be a non issue, it just looks wrong to me visually when compared to how the S1-5 is laid out.

[sidehack]

If you put the card nearest the PSUs with its non-sinked side up against the wall, it'd be up against the ducting and get good airflow. That might choke air to the heatsink on the card on the other end if it doesn't have good fan exposure.
Conversely, if you can maintain half an inch clearance between the heatsink face and the case wall, the gap won't be too much different than an S5 board inside its plastic side panel and the fan exposure it gets in the stock chassis.

I'd be more in favor of a 4U height with 7 blades, yes. That actually does power a bit better too I think, if we want to limit 300W per blade we have 2100W of mining and potentially another 100W in fans and controllers for 2200W total. 8 blades at 300W makes 2500W total, so your stock power dissipation is more than two 1200W PSUs are rated for. If you wanted 8 blades and reduce the power below 2400W (say, back down to 2200W, still 92% of rated PSU) you're at 262W per blade. That's right about the expected per-blade power dissipation of an S5 at stock settings, but not near what the machine can actually handle.

What would we rather have - 7 blades at 300W, or 8 blades at 260W? If the smaller heatsinks for 8 blades are actually still capable, with the proper fans, of safely handling over 300W each (which would be an overclock-at-your-own-risk level for sure, and our 30x BM1385 board would have an expected top-clock of about 320W, nominal clock more like 230W) I certainly wouldn't complain.

One thing that occurred to me is, if we're already building boards and sourcing heatsinks and fans, we're pretty much just endcap frames away from building an S1. There's no particular reason to require the heatsinks from the rack also work as standalone S1 replacements except that acquiring bulk quantities is already done.

[NotFuzzyWarm]

Ja
I was wondering about the S1/3/5 thing. Good general form factor to work with per-blade but -- since the blades are going in a case you should be more concerned about how they can be packaged in said case.

Now, certainly doesn't mean one can't slap on endplates and make smaller modules from the blades but doesn't have to fit the org S-series endplates. Hell, turn the endplates around and just call them fan mounts.

[NotFuzzyWarm]

Thinking about my AMT A1... Is a later Dragon clone and has 5 cards in it with room for 2 more. As the pics in my imgur link show the hash boards make a nice - package. Anywho, it is pulling 1,300w from the wall giving 250w/blade assuming reasonable efficient psu. Took out the 120mm turbines it came with and I put in 3 Cosair SP-20's. In a biz office is quiet and runs @ 55C inside.

Just a packaging reference point as you are shooting for moar power and ergo higher CFM/louder fans.

[sidehack]

I mentioned the S1 endcap thing because it opens up another product stream. Boards from this box could be sold in pairs as upgrade kits for S1, but if you punched out endcaps as well you could actually make whole units for sale instead of just as upgrade kits.

NotFuzzy, you got an opinion over 7x 300W-stock blades versus 8x 260W-stock blades?

[NotFuzzyWarm]

I say stick with 7x300w. Well within the range of using PCIe plugs and gives more space to fit PSu's and any air routing they will need.

Und ja, selling the smaller stand alones not a bad idea as like you said, end caps would be almost dirt cheap to have punched/bent. Now sizing the board to fit S1/3/5 sinks.... Shades of the S1/S3 upgrades I did 2 of them, interesting exercise but thass about it. Better to buy ready-made.

[sidehack]

The boards are already sized to fit S1/3/5 sinks. It's been my plan for months to build boards for that platform, and arguments to keep the shape and compatibility for a rack standard are convincing.

If we have 13.5 inches inside to fit eight boards with clearance, that's what 9.5 inches total heatsink thickness? That makes 1.2 inches. Is it feasible to make an extruded aluminum heatsink 10x4.5x1.2 inches comfortably capable of moving 260W of heat? Thinking about it, I do kinda like that underrating for power dissipation per board being as it's fitting with the stock power specs for S5 and that's the kind of heat the S1 chassis can be expected to comfortably clear.

That's really the question, I guess, in deciding between 7 boards and 8 in the same space.

[NotFuzzyWarm]

I'll measure what the blade size is in my AMT rig tomorrow. They marginally do it with a lot less air then you will be moving. I dismember the blades being pert near the same size as Ants, maybe a tad shorter and longer.

[AJRGale]

I countered my own argument, i was going to say "I'd say give more room for sinks, there will be someone with 150W BGA style cores they want to squeeze in there.."

but then i thought, "they can throw 2 maybe 3 of their cores and their own copper sink, that should suffice. Aluminium is good, but copper spread the heat quick enough from the hot spot.."

[Witrebel]

So I spent the last couple of hours playing with a 4U case 20in deep, seeing what can fit.   I opted to shoot for 8 cards and max out a single sided heat sink.  Obviously if you can fit 8, then 7 would be cake.  The first option came as a result of realizing that 2 DPS-1200 and 3 120mm fans do fit side by side in a .055in walled 17.5in wide case, with .016 in to spare.  This is just too tight to manufacture, but not too tight for some staggered fan action.  It's an oddball design, but I just wanted to throw it in the mix for consideration.





As you can see, this layout provides for 8 cards, with 1.438in for a heat sink and .5 spacing.  You can use the outside hashing cards and a small divider in front of those cards to effectively separate the PSU intakes from the hashing space.

There is less than a 0.1in overlap edge to edge of the fans, so you could just mount the center fan to the real panel, and use small strip of metal with clearance holes to attach the fans to each other.  The outside fans could then attach to the wall that is between them and the PSU.  I didn't take the time to render all the walls, but they are included in the spacing of all parts. So the fan is spaced off from the psu in this render by 0.055in + 2mm for clearance of the PSU. 

But alas, taking a page from aviation "If it looks like crap, it flies like crap" generally.  Personally I like the funky look, but I iterated on the idea and came up with something a bit more clean.  Rear mounted PSUs, they could be hotswap/backplaned I believe.  You still have the compartment you wanted for a controller.  And you get 3 120mm fans.  Only thing would be routing a bit of intake for the PSU's if they did get a back plane.  But I think thats a non issue.   You can route the intake in from where the controller section is up top. 





Again this runs 8 cards comfortably.  With full .5inch clearance board to wall and sink to board, you get an overall heat sink height of 1.625in.  And now you are back in the game of 8 cards 300 watts per, in a 4U setup.  Only catch is you are restricted to shorter PSU units. 

[sidehack]

Two PSU with staggered fans rules out the ability to run a third PSU, which isn't necessary but would be nice for redundancy and anyone wanting to get full speed out of 120V.

Short PSU requirement rules out the higher-efficiency Emersons and DPS1200TBA, which are both around a foot long. I reckon a 24" deep case would solve that problem, but it seems like an awful waste of space to have 10" hash cards in a 24" case. There'd be a lot of unused volume in there.

Them renders look pretty snazzy. Nice work.

[NotFuzzyWarm]

Ja +10
Something to consider is wire routing from the PSU's. Even direct soldering to the output lands needs some room, using 10ga main leads from my HP's is about 2" https://i.imgur.com/rbcz0pw.jpg. Split out as many 16ga leads right at the lands should be fine with an inch or so. Add a breakout card and it tacks on about 1.5" or so. There is pic of what I mean in that collection I linked to on Imgur.

[Witrebel]

I don't know that I'd say it actually "rules out" longer PSU's.   This isn't the cleanest looking solution, but I don't see any insurmountable obstacles with running longer PSU's that simply extend out past the back grill.  Securing them would be a trick, but I could cook something up for that.  And you could still go with interchangeable back planes/harnesses etc.  

For a staggered fan miner, you could either run two longer PSUS, or theoretically run a third horizontally above the fans.

For the stacked miner, you could run any 3 PSU's that fit in a 1Ux6U space, or any space you allocate on the shelf really.
If using backplanes and hotswap, you could ship the PSU's out of the bays so that it fits in a smaller box.

Now this gets me thinking, why can't we get the best of everything, truly modular, ultra compatible with S1, 4U, 8 cards, 3 PSU's, small case dimensions for shipping...  Why not use 4U modules.

The core module is 8 boards, and a shelf with the controller, lets call this a 4U module 12in deep.
You then offer another module that is basically a fan module.  
You could either make all the rear modules a standard 12 in for shipping and handling purposes, as well as chassis manufacturer, or you could offer various products to meet the needs of different setups.
Fans only,
Different PSU options,
Internal vs external

If all these miners are destined to sit on a shelf anyways, then you don't need to join the two modules in a terribly structural manner.  Just make sure they line up and mate repeatedly so you can ensure some kind of a standard when it comes to airflow.  

Now you can ship in a smaller box, waste less space, and have a more flexible product line.  Yes I think there will be some give and take with manufacturing costs.  Maybe you end up doing 12in across the board on all modules and you use a standardized hole pattern that can do all products.  Just some food for thought.