Request for Discussion: proposal for standard modular rack miner

[NotFuzzyWarm]

I'd say depends on where the miners are. Most conventional IT hosting facilities charge by rack space used (height) and power so yes hash density comes into play. But said facilities are by no means cheap (but do have other advantages such as security, often uninterruptible power, bodies on-hand for monitoring, etc.) .

Going the miner warehouse style like you are thinking of in Venezuela then it is rather a non-issue I'd think.

[1715195377]

1715195377

[1715195377]

1715195377

[1715195377]

1715195377

[1715195377]

1715195377

[NotFuzzyWarm]

Another thought - it looks like you are trying to stay somewhere around the depth of an s2/4. At least in telecom and industrial electronics case depths up to 28" are not uncommon (good example is Bitmain's 1600w PSU) sooo....

[Witrebel]

As I'm currently stuck on a boat for a few days waiting to get into port, I think I'll take a stab at this in Solidworks and see if I can't get some flow/thermal simulations running.

Edit: Here is what I have so far

120mm fans, plenty of spacing, I'm using .055in for all wall thicknesses.  C20 socket.  Separate flow path for the PSU space and for the hash space. 

For the PSU space I alotted 4inches, I think this is too much, but I wanted to play it conservative at first.  How much space wall to wall do you think this should be?

The other thing im still guessing on is the placement of the first hash card and the exact spacing.  Right now I have the first card a half inch off the wall, and then 1.75in spacing card to card.  This is totally a rough guess, and I believe you were all spec'ing 2" heatsinks.  Is this two inches overall? or .5inches on one side and 1.5inches on the other?  Should it include any chip standoff height?

Speaking of stand offs, what did you guys envision for vertical mounting?  I could see something like a 2 strut/board system with nylon standoffs to space the cards off the vertical struts. But I'd like to hear what you all wanted before I model that up.  I think it for the flow simulations that the board mounting will come into play so its worth getting that detail captured. 

I have to dig into the simulation stuff tonight and figure out how thats going to work, but safe to say I should run it at 300w per board?  I was thinking 200cfm per fan assuming no restriction?  So maybe like 100-150cfm per fan actual flow rate?  I may need an actual performance curve from a fan to get the simulation to mesh out correctly, so if anyone has a specific fan in mind I can try to find and incorporate that performance curve. 

Also, for heat-sinks, are you thinking custom jobs? Or re-purposed S1/S3 hardware?



I want to see what kind of numbers come out of a 3 fan pull only simulation.  But it would be pretty easy to rework the model and do a 6 fan push pull.  I would just mirror the duct shape from the back at the front and maybe stretch the whole unit by a touch.

I also could run this as 3 pull 2 push, which might make for an interesting angle on the "negative vs positve pressure" argument. 

 

[sidehack]

That mockup looks pretty much like what I'm thinking. I don't recall the exact numbers offhand, but I think a few posts back I gave some expected dimensions for heatsink height. The heatsink dimensions would be the same as an S1 chassis, but likely with a bit shorter fins. I think I figured for one half inch between the face of one heatsink and the fins of an adjacent heatsink, which allows for board thickness and some tall parts like tantalum caps and such before interfering with neighbors. Taller parts could be put at the top of the board above the heatsink, where clearance to the next board is roughly two inches. That might make pulling cards in and out a bit cumbersome, so it'd be nicer if tall parts were kept on the side with the heatsink, but that's probably going to be left up to manufacturer's discretion.

If you have seen a Dragon, I like how those are put together. Basically the PCBs are mounted to the heatsink, and the heatsink is stuck to the case. Screws through the base of the case run right up through tapped holes drilled into the fins. It's pretty solid, and there's no additional rail components like in the S2 - which increase manufacturing, and can get in the way of airflow, and shouldn't be relied upon for structure given how many of those arrived from the factory with cards flopping all over the place.

I'm not sure offhand what the board height on the S3 is, but S5 boards are a bit shorter than the S1 boards. They can be that much smaller because there's no power components due to string topology. I'd rather not design a flexible standard without allowing it room for components for a variety of topologies, so I'd rather make provision for something the full 5.9 inch height. The S1 heatsink I think is about 4.5 inches tall, so to fit inside a 5.25" case doesn't allow much extra room above that for anything requiring through-hole components. 3U would be nice, but I don't think it's possible without switching to a different board dimension.

[Witrebel]

I reread the thread and pulled out some of those dimensions. 
I mocked up a DPS-1200 type supply, I went with an overall dimension (including the power blade) of 8.25in x 3.5in x 1.6in.
The PSU channel is currently 11in up to that first bend.  If you opted to have the PSU section rectangular with no angled ducting, you could get away with 13 inches of bay space.  I would imagine you have some kind of backplane card for the 3 psus to plug into? What will this do to the PSU chamber airflow?  Is this worth consideration? 

Right now I have it with 0.4in between the first card and the side, and then 1.9 inches card to card.  This means that last card only has 1.5inches of space on the right side where it is up against the psu bay.  I could sink the first card up against the wall and get the full 1.9" per card for heatsink/space, but then you are restricted to single sided cooling only. 

I like the idea of mounting cards to heatsinks, and then heatsinks to chasis.  For the time being I modeled up a strut, giving a .1" clearance for a nylon washer or something between the strut and the card.  Also, im using 0.055in thickness for all the wall thicknesses.  I would imagine you could go thinner on the interior stuff and gain some fractions of an inch. 

Right now as it stands I have 0.075" clearance on the top and the bottom edge of each blade.  This affords you 0.75" of free space in the top shelf for cables and controller circuitry. Is that enough for you?  Theres also a good portion of the PSU bay that could do that same job.  I was thinking maybe even put the controller in the top of the PSU bay, and then use a blade style connector to interface PSU backplane.  This backplane could come in different flavors to accommodate different PSUs, and then your controller could use the signal quality lines etc. 

I'm going to start playing around with the simulation stuff soon and see how that pans out. Ideally id like to be able to run a full forced convection study, but I don't know if I have the right package for that.

[chiguireitor]

Better stick to 4U, that way you can have even taller boards. Also, on a 46U rack you would fit 11 of these rackable units, which would amount to 77 boards, not too shabby

[sidehack]

0.75" is a bit tight for power jacks, but you shouldn't have trouble plugging in a PCIe without kinking the wires.

I question your hashboard space width - a 17.5" case should have about 14" left over after 3.5" of power supply. But pretty much everything else - interchangeable PSU backplanes, controller location, all that - we're on the same page now.

[Witrebel]

If you can fit all of your controller circuitry in the PSU bay, then you really have more like .875 above the cards for cabling/connections.  I partitioned it in my model thinking you might want to stick a controller card up there.  But that could be worked out in a few different ways.

Hopefully this view answers most of the dimensional questions.  Let me know what you want tweaked and I can rework it.  The only thing I didn't remember to put a dimension on was the PCB thickness.  I assumed and went with the standard 0.062in.  If the S1 cards are thinner that might be part of why mine seems so tight.

[Meech]

Cool.  Not to offend the scam sites but those renders have more info than most of their sites.  NO PREORDERS HERE!    Nice work!  It does help to sketch it out and not just keep it in your head.

[AJRGale]

and here im playing with a HP server, 10 SAS HDDs, 2x Xeon CPUs, room for 6PCIex8/16 cards, and 2x 800W PSUs neatly in 2U, and people are having trouble understanding 7/8 cards +HS and even just 1PSU in 4U?

you can fit a full sized PC in 4U..
in a full length PC, you probably be able to have 2, maybe 3 cards length ways, but thats not the issue..

A S1 board is about 155 mm high, 4U rack unit is 177 mm high.
i cannot find the width of the heatsinks for the antminers so i'll use the dimensions of the 120mm fan, and the 2 sinks facing it, so each heatsink is approx 60mm, but the Rack unit is 465 mm in width so thats ~7 cards with sinks and breathing room.

Adding a single atx PSU, thats 90mm less space, so about 2 cards less.
these server cases are about 1000mm long each antminer unit is 300mm long, with fan.

so you can fit rows of boards in, one row will be 2 less for a single PSU, but thats 7 cards x 2 rows, plus another 5 more cards next to the PSU.
but are we looking for a backplane that sits at the base of the case and have multiple cards sitting side by side and layered?

my idea on how I would do a case'n'controller:

open face, with blank brackets thats 40mm long 170mm high (4U case) you could fit 11 cards right across the face, each card slot has nylon rails, backplane with power connection and usb. behind the backplane is HP style redundant PSUs. next to them, $50 atom itx motherboard, with whatever OS you want. Why ITX motherboard? im over trying to configure RPis or tl-wr702n's.

[sidehack]

What I would do is not have nylon rails because, as I've mentioned two or three times so far, they're pretty crappy structurally and a card with 2lb heatsink attached is going to get destroyed in shipping. It's better to secure the 2lb metal block to something structural and secure the 4oz fiberglass board to the metal block.

Another reason to do that is, if we stick with S1-style boards, the board and heatsink are the same length so there's no board edge for rails to grip anyway. The heatsinks on an S1 are less than 50mm wide. Take a look at Witrebel's case render, and you get pretty much exactly the geometry I'm envisioning for 7 300W blades and 3 redundant 1200W PSUs in a 4U case with built-in controller.

I think a good way to address PSU flexibility is to have basically a terminal block that all the 6-pin cabling to the cards comes back to. Someone could cook up PSU backplanes that link 2 or 3 supplies together load-balanced and route power up to the terminal block. If someone didn't want internal supplies, a $10 wiring harness with snap-in jacks at the rear of the case connects to the terminal block. That maintains modularity which reduces the overall cost and limits what has to be replaced when going from external to internal, or from one type of internal to another.

I also would not put rows of cards in a super-deep case because, well, anyone with Spondoolies gear can figure out why. The hot air from the front cards bakes the rear cards and they don't cool worth a hoot. Considering each board could be pushing out 300W, do you really want 2KW of heat blowing directly into another stack of things generating 2KW of heat?

I know there's been a lot of words said so far, but when a point that's been addressed twice already gets brought back up (with no new arguments in favor) it's a bit frustrating.

[QuintLeo]

and here im playing with a HP server, 10 SAS HDDs, 2x Xeon CPUs, room for 6PCIex8/16 cards, and 2x 800W PSUs neatly in 2U, and people are having trouble understanding 7/8 cards +HS and even just 1PSU in 4U?

these server cases are about 1000mm long each antminer unit is 300mm long, with fan.

 S1 size cards are quite a bit taller than PCI-E cards, though a few GPU cards have fans that stick up a fair bit above the top of the board.
 S1 cards are also longer than most PCI-E cards.
 You also have a case that is almost TRIPLE the length that sidehack is trying to fit everything into. Personally, I think he should go a little longer on the case, 15" is EXTREMELY SHORT for any rackmount case and even a couple of inches of extra length would help a lot on fitting stuff in and making the air flow smoother.


 I agree that 2 cards in a row = VERY BAD IDEA, especially if we're talking 300 watts PER CARD. Rear cards are gonna overheat BADLY unless you get very fancy with air ducts and such, and then you reduce the cooling to everything.

[kilo17]

If the internals are stacked closely together/high density then adequate cooling can be achieved with high static pressure fans instead of high flow (cfm) fans.... particularly in a push/pull configuration....I have used this type of setup on miners before and achieved better cooling than stock miners...I always swap out the fans on miners as it seems they always seem to choose poorly (or maybe it is to save money).

[sidehack]

The original spec is 15" between the fans, which puts the case at more like 18" length. This can be adjusted a bit since the blades are now only 9" long instead of about 12" but I wouldn't be surprised to see the final case come in longer than 15 inches, especially once dimensions get ironed out for PSU requirements.

[Witrebel]

The last render I posted had the S1 cards at 12", so you get alot of space freed up if they are really 9" cards. 

Furthermore, I *think* sidehacks issue with going deeper isn't as much about case dimensions as it is about how long the hashing cards are.  That is part of why I want to get some sort of simulation running that accounts for both the flowpath and the thermal loading. Alas my current cad seat does not have the right simulation package and I can't get anything new hooked up until I'm off this damn boat.  Simulations may be a bit premature at this point anyways, but I wanted to get a general objective idea of pro's and con's to some of these points that get argued like negative vs positive pressure, push-pull vs pull, flowpath length, etc. 

[NotFuzzyWarm]

So far looking good. For best airflow 2 things to keep in mine:
With 1-side fans the back of the case must be louvered vs just punched or preferably wide open and fan sized w/fan grills for if folks want to add outside fans in the back. Done to minimize dead metal in the way.

Preferably should have at least 1/2" between the fans and first edges of the heatsinks and/or use fans that have flow straightening built in like some of the nice thick Delta fans. This gives the air a chance to start straightening out and greatly reduces front edge turbulence. Also gives a significant reduction in noise by reducing the siren effect.

On longer case, I was more thinking about having the PSU bay possibly in front of the cards to fit in another card or 2 width wise vs cooking cards in a back row. Ja it adds heat to the hashboad airstream but works fine in an open air proof-of-concept. I do it in my farm to eliminate needing fans on some of my IBM 2kw supplies. https://i.imgur.com/XRlk5Tq.jpg btw that area often runs ambient up to 93F, and those psu's/miners still happy.

[chiguireitor]

On longer case, I was more thinking about having the PSU bay possibly in front of the cards to fit in another card or 2 width wise vs cooking cards in a back row. Ja it adds heat to the hashboad airstream but works fine in an open air proof-of-concept. I do it in my farm to eliminate needing fans on some of my IBM 2kw supplies. https://i.imgur.com/XRlk5Tq.jpg btw that area often runs ambient up to 93F, and those psu's/miners still happy.

One thousand sidehacks weeped with that pic

[NotFuzzyWarm]

heh heh heh
Works very well though and is same depth as the s4 under them. https://i.imgur.com/aTkVsGE.jpg
Was tossing that out there for consideration

[Witrebel]

Hopefully I can get a thermal simulation package running soon and put some of these ideas through the paces.  I'll play around with front mounting the PSU from a layout perspective just to see what can be accomplished. 

Does anyone have any experience with design for manufacture?  Specifically what are the cheapest construction methods?  I ask because if it doesn't add too much additional cost of manufacture, I think this may benefit from a deeper case, same size hash cards, and some dedicated flow straightening vanes/features.  Im thinking something like the struts, where they can just be simple stamped sheet metal pieces, tack welded or riveted to the case in strategic locations to optimize the flow path. 

[sidehack]

Thinkin' about it, the boards are more like 10 inches long. That's what I get for trying to remember stuff after being awake for 21 hours.

The original spec had space for nice 38mm-deep fans, and minimum one inch between fan and either leading or tailing edge of the PCB.

I also figured the rear panel for fans would best be done with the entire fan area knocked out and use a wire grille. If nothing else, those penetrations could also be used to route plumbing for waterblocks without having to make the entire back panel removable (which could cause problems for other things, like structural strength if machines are stacked).