Bitcoin Forum

So this first post will be somewhat incomplete. I have as of yet no diagrams drawn up except sketches on paper. Dimensions are currently in inches, but a fairly universal standard should probably be defined in metric since the majority of manufacturers and customers would be more familiar with that system.

But here's the score so far. I am interested in building miners that don't suck. To that point, I am interested in building miners with reusable parts, with standard interfaces and with full adjustability of chip-level operating points inherent to the design.

My intention to build a board which can fit on an AntMiner S1 chassis has already been discussed at length. I have no problem using that base hardware as a fairly standard small-format miner. I don't know of any terribly good large-format (say, rack-mountable) miners which are generic enough to do what I'd like. So Novak and I have discussed and would like to propose for comments a basic standard for rack-mountable bitcoin miner, which is designed to be inherently flexible and upgradeable.

I should have some layout diagrams generated in the next few days, but for now I'll give a bit of a text description.

The case

4U Rack-mountable, 7 inches high by 17.5 inches wide outside dimensions.
A total of six 38mm 4-wire PWM fans, three 140mm at the front and three 120mm at the back in push-pull configuration.
Two server-grade power supplies, 1U height, probably DPS-1200 or functional equivalent, mounted at the top rear on an internal shelf separating the hashboard volume from the upper chamber containing controls and cabling.

The hashboards

The span between front and rear fans (inside to inside) is 15 inches.
The hashboard PCB is five inches tall by twelve inches in length and bearing two heatsinks.
The main heatsink is four inches tall by ten inches in length with a total fin height of no more than 1.38 inches.
The secondary heatsink is four inches tall by ten inches in length with a total fin height of no more than 0.5 inches.
The secondary heatsink would be drilled for screws to run through, through the PCB and thread into the main heatsink. This is so that the secondary heatsink can, if the PCB design calls for it, be removed without hindering the mounting of the PCB to the main heatsink.
The main heatsink would be drilled and tapped through the fins such that it can be screwed to the bottom panel of the case.

The heatsinks will be aligned with the rear bottom corner of the hashboard, such that the topmost one inch and frontmost two inches of the hashboard have no heatsink contact on either side. This gives room for through-hole components, VRMs, connectors and other circuit parts.

There is a one-inch gap between the front of the hashboard and the inside of the front fan. There is a two-inch gap between the rear of the hashboard and the inside of the rear fan. Hashboards would be mounted perpendicular to the bottom of the case, with heatsink fins extending horizontally outward from the board.

Each board should have two 6-pin PCI connectors extending at a right angle from the top of the board, just past the end of the heatsink (the rearmost edge of the rearmost jack would be 10" from the rear of the board). To the front of the frontmost jack is the USB header. All connections would be accessible from holes punched in the separator shelf.

8 boards will fit widthwise across the inside of the case with a bit of room inbetween to ease vertical installation.

Boards should be limited to 300W maximum power dissipation each.

Controls

It's our intention that the controller can be fairly generic. Any small single-board-computer like the Raspberry Pi, Cubie or BeagleBoneBlack would be suitable provided it had both USB and Ethernet connections accessible, and one could compile cgminer for said device.
A space inside the upper chamber would be reserved for a "drawer" onto which the controller is mounted; dimensions for this are to be determined and a replacement controller should be provided with its own "drawer" tapped in the proper places for mounting said controller.

A board specific to the miner is to be assembled providing a modified USB interconnectivity on 8 ports. The connection would be a six-pin header, onto which would be socketed a cable with five wires and a shield. Four of the wires carry standard USB lines (5VDC, GND, D+, D-); the fifth wire carries a logic-level signal for a status LED. The sixth pin connects to the cable's shielding. An identical six-pin header would be present on the hashboard, which enumerates as a USB device and controls the status LED. The status LEDs would be visible through the front panel of the miner.
This board connects to the controller, which sees it as an 8-port USB hub.

The board would connect to thermocouples mounted in the airstream in front of the rearmost fans. All six fans would connect to this board, which would use the thermocouple feedback (and a calibration trimpot) to determine fan speeds, individually for each pair of fans.
Alternatively, if software control was desired, the controller could link on an unused USB line and interface with a cgminer driver tasked with determining fan speed from hashboard reports and pass commands to the controller on the board.



The goal of this endeavor is to define a standard box inside which any board designed for its form-factor could be run. By using a standardized dimension, heatsink and USB interconnect, boards from different manufacturers with different chips operating on different protocols could all be run in the same box off the same controller. Operating new boards might require updating cgminer on the controller to detect the new hardware. Piecewise upgrades would be readily possible as hashboards could be sold individually. As long as the basic dimensional standards are met, as well as the basic operating standards (12V 300W max, USB connection) and avoiding software conflicts, any number of small or large manufacturers could produce any number of boards which could all be run together in the one box. Replacement parts for the machine itself would be both standardized and fairly generic. The benefits in cost savings, maintainability and diversifying the market are substantial.
I know it's not a perfect household miner being as it would look to draw over 2.4KW power. The point is not to build a perfect household miner.


So, that's the gist of our idea. Anyone got any comments?

i would plan on using usb 3.0 for the data bus as it can move the data with less delay and you plan on making it have 8 hashing boards correct? with the amount of data on the bus require one controller per box? as in a box with lets say 12 boards on controller there links to the usb port on the main box with one, is this doable?

I will not use USB3.0 for the data bus because there are no good options for USB3.0 to microcontrollers or primitive protocols which hashing chips are likely to use. USB2.0 should have more than enough bandwidth for everything.

There would be no box with 12 boards on the controller. At most 8. One controller per box, ethernet out. The USB interconnect is strictly internal, tying the controller to the hashing boards.

Can you talk a little about why USB was chosen over saaayyy a PCI-E bus (a la Block Erupter Blade backplanes)? My thought would be that a USB driver is easier to work with but PCI-E is pretty cool and very modular...

okay sounds good so far. i would like more inter changeability of hashing boards no point re-buying the wheel each time.

The Block Erupter Blade backplane was power only. There was no signal interconnect, no centralized controller.

If we wanted a backplane it'd either be at the bottom (which can cause board security problems like the famous S2 shipping disasters) or at one end (which blocks your airflow), neither of which are terribly optimal. Small out-of-the-way cables cause no airflow restrictions to the main heatsinks, and without a baseboard you can secure your boards quite solidly with a few screws and not need additional runner slots in framework. Having a backplane board adds to the cost both in electronics (an additional large PCB and the cost of edge connectors on every hashboard) and in framework (as now you have to make steel runners as well). Additionally, you could test individual boards outside the box with a cheap USB cable and anything running cgminer.

I feel USB cabling is the best overall solution when cost, ease of manufacture and overall reliability are prioritized.

10 inches long contiguous aluminum heathsink?

The thermal expansion of such a slab of aluminum will be literally ripping the chips off of the PCB.

Either the heathsink or the PCB needs to be partitioned into sectors.

Ah makes perfect sense, thanks for the explanation!


It would be pretty cool to sell all in one miners (case, power, hasing boards and cooling) AND having a USB interface and 6pin power would allow users or GS to build out smaller miners using (roughly) the same equipment but in a PC mid tower with 3-4 hasing boards, or even a single board sitting on your bench using a 140mm fan just sitting on the heatsink and whatever PSU's you have laying around.

Point being standardizing the hardware (hasing boards controllers) and connectors provides (limitless) options for mounting and scalability -- I love it!


Isn't 10in roughly the same size slab as a GPU these days?

Have you ever looked under the GPU's heathsink? How many chips it touches and where are the chips in relation to the heathsink?

As in how it really only contacts VRMs and GPU die? yes...

The Bitmain S4 uses big ole aluminum heatsinks... they are mounted horizontally and not vertical but that's what comes to mind... this design is from what I read is several Bitmain Blades in a row with different connectors.  Very much reminding me of this...

https://ip.bitcointalk.org/?u=http%3A%2F%2Fe30.us%2F4sale%2F4u8s1%2FIMG_3871.JPG&t=555&c=bxIiCrGuw5dYEQ

I guess after re-reading the original post I missed the fact that the hashing chips may be in the supply-voltage-serial a.k.a. string configuration. Since each chip will have different ground potential then there is some sort of galvanic isolation provided between the chips and the heathsinks.

So the heathsinks may be able to slide over their isolation layer enough to accommodate thermal expansion.

I've dreamt about this for quite a while, I hope to see it come to fruition.  The real challenge IMO will be mass adoption in order to keep costs low enough to make it a worthwhile venture.  People don't mind paying extra for the "novelty" factor when it comes to USB sticks, but when you're purchasing a full blown dedicated miner, it's stritly about dollars and cents.  I know your stand on out-sourcing manufacturing, are you planning on sourcing all materials and labour in the US?  Also do you think it is a remote possibility that manufacturers would be willing to produce boards designed with this framework?  

From a challenges point of view, I think of Bitmain and their reluctance to even sell and market upgrade kits to their S1 at the time.  From what I gather part of it had to do with the extra time & labour required to install boards to test them, and then dis-assemble them for again for packaging and shipping. I just can't help but wonder if their decision not to provide kits themselves had to do with the actual production and re-using existing framework, or if it had more to do with not having the ability to run "kits" in their farms and instead sitting on the shelves.

We checked several other miners for heatsink size comparisons. The S1/3/5 heatsink is 9 inches long; the Avalon2 box and S2 have about ten inch heatsinks. I think the Dragon was more like 8 inches. None of them do quite 300W per heatsinks at stock power either though.

The heatsinks should not be adhered to any chips, unless your thermal paste hardens up I guess. I've never really liked not having some kind of isolation between heatsinks and any electricals. A top-cooler like the BM1384 wouldn't need isolation of its own, but if you had the backside (with its multiple ground planes) on the sink you'd definitely want an electrically insulating thermal pad.

The boards wouldn't have to be string. The 1" at the top can be used for logic level shifters in a string design (or if you drop the small heatsink you have the entire backside for a double-sided board like the S5) or it could be used for VRMs. We don't want the board dimensions to limit your topology choices.

Course, there's also no particular reason the heatsink has to be a single slab. It could be two five-inch chunks maybe with a slight gap between to allow for expansion. It's exactly for considerations like that that we wanted to open discussion - if it's going to be any kind of standard, it needs to be as good as it possibly can be.

I'd like to see the big guys adopt a particular standard, or at least utilize it. Being able to build boards for an existing case without having to build the case also opens up potential for independent manufacturers to get in the game. I mean if you want a computer you can call up Dell or HP or whatever and get a complete system, or you can dig out an ATX case and go fetch guts from NewEgg that all fit inside. You got choices. Some little two-man outfit like mine can build boards but doesn't have the tools to manufacture cases? So what? They can still put out a decent product because you can get a case from someone else - or maybe you already have one. Updating cgminer would be the only really tricky part. If something like PlanetCrypto's independent chip dev happens, or the main guys get back to selling chips to third-party manufacturers like some of 'em used to do pretty regularly, we can have mom-and-pops coming up with innovations and servicing the small-time miners again and doing some pretty good stuff.

2112 - have you talked to PlanetCrypto about chip dev at all? I figured something like what he's doing might interest you.

Testing boards doesn't have to be that tricky. Instead of screwing it onto a greased-up S1 chassis, plugging it in and then taking it all apart again, just rig up a clamp system on a free-standing cooler. Drop a board in, maybe have some pegs to keep alignment on screw holes, light pressure clamp presses it to the heatsink and you plug it up and go. Release the clamp, pull the board and put it in a bag.

I guess I'm overly cautious. I always worked on the devices with very strict quality assurance requirements: medical or industrial. Things like self-adhesive thermal pads were unacceptable, only mica leaves would do.

The lottery-ticket-printing devices that you are designing have much shorter lifespan, so a more relaxed design approach may be used. On the other hand, things like retail GPU cards have very carefully designed heathsinks: the die or at most 2 dies are in the center, everything else has a heat spreader or some other kind of interposer.

It is interesting, but I need to keep my nose very close to my own grindstone. I'm not in the position to really get involved in the new projects. I'm fine with openly sharing knowledge and commenting in public on the forum.

If you use good fans, push-pull configuration is a waste. The only effect it really has on a rack-mount case is to reduce back pressure, VERY little effect on actual flow.

 Based on measuring all of my existing rack mount cases, I'm not sure if you CAN fit 3x140mm - be nice if you can, but 3x120 is good if you can't.
 ALL of the 4u cases I own have appx. 16.5" of space between the handles (I have one 2U case model with about 17" thoiugh). Height isn't an issue.
 Might need a custom case design, or at worse have to use a specific case model that doesn't have handles.


 IMO don't use USB. USB connections to miners have ALWAYS been an unreliable PITA. Tolerable for small stuff on the block erupter scale as those are more "toys" than real miners anyway, NOT tolerable for a full-up rack mount large miner. PCI-E 1x or just PCI would be tons better from a RELIABILITY standpoint.


 Would be nice if an industry standard existed, but I doubt a custom design would be practical from a cost standpoint and I don't see the MANUFACTURERS ever getting together to create or work with a standard. 8-(

1) You want negative pressure displacement, likely you'll get better performance dropping the fronts to 3x120's or 2x140s and leaving a grill. Ramming in more air than is being released = high pressure internally = lowers airflow significantly.

2) There is also a problem with your sizing, 3x120mm don't actually fit at the back.
3x120mm = 360mm
Spacing = 15mm
PSUs    = 2x44mm = 88mm
Material thickness = 2x2mm = 4mm
= 467mm
Your specified back side is 438mm


3) 1U = 40mm fan = annoying as hell. (See SP10 and SP3X where the PSU fan was far louder than the actual miner)


4) Thermocouples are a shipping hazard as they can come loose and end up in fans and so addd another layer of inspection. They take too long to install and they're also superfluous as you should have already characterised the relationship between ambient intake and chip temps.


5) Remember that the distance you actually care about is between the outer two mounting holes, not the length of the entire heatsink. The material on the ends doesn't contribute to thermal expansion mounting stress.


6) You've not explicitly specified how many PCBs.

7) 4U should be smaller than 7" if you want stuff to actually fit, even if its shaving off a few mm.

1)  Or have it open across the back to let the air out.

2) No, the PSUs go over the top of the fans, not beside them.  Hence the 4U.

4) Not really any reason to use thermocouples over any other board mounted temp sensor with a cable alongside the USB.  I agree that a totally separate thermocouple is probably asking for trouble.  It's not correct to assume that you would know the input and output air temperature as the number of boards and the type of boards is explicitly undefined.

6) No more than 8 PCBs would fit given the heat sinks, but that many would not be required.

--
novak

 You want POSITIVE pressure in the case with filtered fans, anything else = more dust in the case = WORSE heat dissipation over time.
 Also, positive pressure DOES NOT REDUCE AIRFLOW. Dunno where that myth originates from.

 15 mm spacing between the fans is way excessive. 5 would be more than enough to allow for the differences in fan size tolerances.
 The specified "size" of a fan is the outside dimension of the fan housing, NOT the diameter of the blade (except for "no housing" type fans).


 AND relatively poor cooling. Personally I'd prefer provision for up to 4 long-case ATX power supplies, even if the case has to be longer to fit them.


 He was looking at overall length to figure physical mounting dimensions and limits, not for thermal stress calculations.
 Realistically, stress on the board isn't much of a factor with the relatively low heat density on the heatsinks at his specified max wattage per board vs. heat sink size.

Dogie, regarding your various points:

1) You want negative pressure displacement, likely you'll get better performance dropping the fronts to 3x120's or 2x140s and leaving a grill. Ramming in more air than is being released = high pressure internally = lowers airflow significantly.

With 3x140mm the entire front face is pretty well covered by fans. Looking at bladed miners like the S2, you can tell exactly which boards are between fans by looking at the per-board temps. I'd like to avoid poor-coverage zones like that if possible.
The rear fans were dropped to 120mm because of height requirements for fitting in PSUs. If that ends up actually restricting the airflow, we remove them and put in a grill instead. I'm not a fan expert and these points need to be evaluated by someone who is.



2. 3x120mm = 360mm
Spacing = 15mm
PSUs    = 2x44mm = 88mm "mounted at the top rear on an internal shelf"
Material thickness = 2x2mm = 4mm
= 467mm
Your specified back side is 438mm

The hashboard volume occupies the lower 3U (approximately) of the 4U case. The top 1U is separated from them with an internal panel above the hashboards (which mostly seals them off and keeps main cooling airflow restricted to between the heatsinks and boards) and is where the PSUs, controller and all cabling live.

3) 1U = 40mm fan = annoying as hell. (See SP10 and SP3X where the PSU fan was far louder than the actual miner)

Yes, 1U fans are really annoying. If I'm thinking right, the DPS1200 fan is quieter than the thing on the SP supplies but it's been a while since I fired one up. I prefer server supplies pretty much universally to ATX and would consider installing an ATX supply to be a waste of space, a waste of cost, and really asking for failure.


4) Thermocouples are a shipping hazard as they can come loose and end up in fans and so addd another layer of inspection. They take too long to install and they're also superfluous as you should have already characterised the relationship between ambient intake and chip temps.

Yeah, temp control hadn't been thoroughly discussed hence why I made brief suggestions for both hardware and software control. This point needs to be ironed out.
Regarding characterising ambient intake versus chip temps, how do you suggest we measure ambient intake if not by the same means as measuring exhaust? Additionally, as Novak pointed out, since the board design is deliberately unspecified there is no specific requirement (at present) for this to be true.


5) Remember that the distance you actually care about is between the outer two mounting holes, not the length of the entire heatsink. The material on the ends doesn't contribute to thermal expansion mounting stress.

There are no screw patterns specified yet, but it seems likely that the outer two mounting holes will be very close to the edges of the heatsink.
I'm no expert in thermal management so the heatsink point should be evaluated by an engineer knowledgeable in that field, make sure we can safely dissipate the heat spec in the volume provided with the expected mass flow of air.

6) You've not explicitly specified how many PCBs.

"8 boards will fit widthwise across the inside of the case with a bit of room inbetween"
The maximum number of boards you could fit is 8. The minimum is one. The machine must work properly with any number of boards between 1 and 8 inclusive.


7) 4U should be smaller than 7" if you want stuff to actually fit, even if its shaving off a few mm.

The hashboards are specified as 5" tall, which is about 6mm shorter than 3U. If the upper room (cable tray, whatever you want to call it) is at most 1U high, we should come in under this. If that's not good enough (which can be determined without a lot of trouble), I'd probably convert inch-measure specs to metric with a fixed 1" = 25mm conversion which gives another 2mm height reduction to the boards. If that's still not good enough I reckon we'll have to do some tweaking.



Regarding horizontal width - have you measured many rackable miners for width, or just servers? Most miners I've seen aren't too concerned with rails, which eats about half an inch off each side.
A custom case design is exactly what we'd need anyway, I think, so we should be able to specify our dimensions any way we want that doesn't violate rack requirements of maximum height and width.

Using PCI or PCIe requires using a backplane board with sockets. That can be a significant expense. It can also be significantly more fragile and cause board-mounting reliability concerns. It also, within this design, would require a lot more disassembly to disconnect one board. How many bladed miners have used an internal cable connection to the blade instead of a backplane connector, and how many of those cable connections have been unreliable?
We're not intending to use USB jacks (and especially not garbage like USB micro) which can be pretty flakey. The spec we propose is a pin header, which a decent pin header with a decent cable can sit there for years without losing connection.
Since USB hardware isn't used, the only real USB in the spec is the protocol. I assume not many off-the-shelf SBC (or whatever little guys like the Pi are considered) have PCIe built in, which means even if we want to use PCIe hardware for connection (which I don't, from a mechanical reliability and cost standpoint) we can't use PCIe protocol without moving to either a custom-designed or significantly more expensive controller. If, then, we want to use PCIe hardware we need to use a different protocol. Something everything supports. Something that is easy to adapt into the chip-level protocol. Like USB. I propose that USB protocol is common enough and flexible enough to do the job well, but that using USB hardware connections internally is a bad idea so something more resilient (and also cheaper, and also allowing for more mechanical reliability in the overall design) is suggested.

Dogie over in a Bitmain thread summed up my opinions of a PCI backplane pretty well in response to an S2 question: "It required a motherboard, slots that often broke, large areas of dead space for cooling and is probably the only Bitmain miner that was susceptible to shipping issues."

1) is worse than closed, as seen on the S4 testing. The initial cases had an open mesh next to the fans but it does performance because of this... *searches for picture*. You could actually feel this behavior with your hand and reduced the effectiveness of the rear fans significantly. Later models were

https://i.imgur.com/ArpAc6O.jpg

2) That's going to be a tight, tight squeeze along that side. Just drop a fan and mount the PSUs peacefully.
170-175mm external
= 166-171mm internal
Fan = 120mm
5-10mm spacing
= 36-41mm for an unmounted 44mm PSU. Putting the PSU above a huge fan like this is also a nono as it suffocates the PSU's fan which generates significantly lower static pressure. So you end up with barely any air movement there or sometimes backwards airflow. Its more of a problem when you use negative pressure flow but its still a design concern with positive pressure. Its the reason you see PSU "compartments" sectioning off the airflow - like in the S4, S4+ and SP3X.


4) What I was really trying to say is that the only temperatures you care about are your chip temps. If its 60C ambient and your chips are still fine there's no reason to stop mining.


1) Its not 15mm between fans, its 15mm total between case fan fan fan PSU PSU case. Yes fans are measured / rated across their linear dimension. 120mm fans are usually + / - 0.3mm, you do need a gap between all those components and all mounting holes will have a tolerance. You're not going to be comfortable with less than 12-15mm across all those things.


It was in reply to 2112 who was talking about thermal stress.

Think of the SPXX (all of them :P), they used the absolute perfect cooling layouts. You don't need to ram air in because it really doesn't do anything, as evidenced by swapping out SP120s on an S4's front fans not changing the temperatures pretty much at all. You then suck all the air out, in a long, smooth and continuous column. This column can be wedge shaped if your intake is larger than your exhaust and it works fine - air is a wonderful fluid.

Its all about them rear fans, and you certainly don't want grills on your exhaust (see above).


I think you're going to be super close (see above again) and it might fit on CAD but will probably be prohibitively tight to use IRL.


The fan is far, far far louder than the SP35 which can run 40% fans and its the thing you can hear through walls, floors and really gets to you. It was similar on the SP10 although that PSU was overloaded and the main fans were only 50mm. It was such a shame on the SP35 as the entire unit was quieter than an S5 if you underlocked it enough so that the PSU fans wouldn't spool up to max.


a) Put it on a test board in a test case, measure it, see what the rough delta is tracking. Use that in software. It might vary between chip type but you'll still be able to get the delta from your test boards.
b) Ignore ambient and just fan control off chip temp, which is really all you care about. Again as I said above you don't care if its a 60C ambient as long as the chips are happy.


Thermal expansion is a consideration and not an issue. Its countered by using clearance holes to mount the heatsink to the PCBs, so its sliding on the bolthead rather than loading your PCB.


In your current layout I don't think things will fit nicely (discussed under 2)), but I was actually referring to external being too large. U racks are explicit in that 4U = 4U, 1mm over and you're impinging on the next 4U of space and pretty soon screw holes don't line up. Which is why I suggested designing for external of <7" rather than =7".


Yeah, dimensions are in each individual guide. The majority are thinner than 19".

I don't plan on doing much with CAD. I prefer to actually build something, so if I test the dimensions it'll be a quick calculation on paper and then seeing how it works when I build a mockup. Hopefully I have time to do some of that in the coming weeks. I agree, it is probably going to be tight and will probably require some adjustments. You're right, maximum outer dimension height of seven inches (and not a millimeter more) must be maintained. Ideally we'd come in 1-2mm under that in case the units are shelved instead of stacked, since there probably won't be horizontal room for rolling rails.


If we want software fan speed control, which I'm not inherently opposed to, we'd need to see about writing a cgminer module that talks to a controller on the USB board. All driver code for the various boards made would have to talk to this module, which would unify all that data and send off to the controller with PWM commands. I'm not sure how hard or easy that'll be to integrate with the existing cgminer framework. Someone with more experience with the guts of it will have to weigh in.

I think the fan dimension point you're arguing with Novak is irrelevant. Removing a fan to fit in the PSUs does not apply, because the PSUs aren't in the same space as the hash boards. The hashboards take up the entire width of the bottom 3U, so you need that entire width with an unrestricted front-to-back airflow path. The PSUs are laid flat in the top 1U, which is almost entirely isolated from the bottom 3U by means of the horizontal separator panel forming a "ceiling" above the hashboards.

Additionally, the S4 reference picture is irrelevant. If we have rear pull fans, there would be no interstitial grilles. The rear would only be grille'd if the only fans were the front fans.

My office is right next to a room with about 30 various Spondoolies rigs in it (at least one of everything they made). I don't need to be told how annoying they can get.
Also, if you know me at all, you know I pretty much despise most of the engineering on SP's rack gear. This includes having a two-foot-long airpath which pretty well bakes the rearmost chips. I'm even a little worried ten inches will cause problems. But that's not really what you were talking about, I know; you mean the suction from rear fans instead of push from front fans.

If someone who knows the mechanics of air cooling can weigh in with a quantitative consideration of front push fans versus rear pull fans versus both would be nice.

I don't understand why not a single hardware manufacturer standardizes it's PCB's so the heatsinks and housings can be re-used...

Its easiest to show you. If you've got an S4 or S4+ around, remove one of the pair of fans and see, remove the other and see. The one with rear fans will do just about fine, the one with front fans will overheat. If you're really determined, you could try reversing the direction of an SP3X's fans and watch it overheat.

I do have access to an S4+ which could be set on fire and nobody would really complain. Maybe I'll play around with it in a few days when I'm not busy. All the Spondoolies gear is hosted.

The S4+ runs pretty close to its readline at 35-40C so you'll get better data whacking its frequency down first.

So, here's a thought.

Two boards from this machine could fit into an SP20-scale miner with a single 120mm fan and hook up to an external 650-750W PSU. Opinions?

Ok chiming in for the first time....

Sidehack, you know i love all the products you do, but in terms of heat dissipation we have our differences, so bear with me :)

It could be nice if the case could also fit water cooled blocks, as this is a need for most tropical countries, and would make heat management a lot better and let home miners mine without the noise they make (currently writing this with 6 S5 at my back).

Also, how about starting with the de-facto standard for boards that is the S1/S3/S5 hashboard? That would make adoption fairly easy, and the hole design on them allows for several individual boards screwed to a standard set of holes or having several different boards.... lemme explain myself with an image:

https://i.imgur.com/uw9AU4Z.png

The white (and yellow) outline is the S5 board dimension, the red circles are the holes preexistent on the boards, and the green lines are the "logical" separations on the boards.

You could have a case with vertical mounting spacers, which could host 4 Small boards, 2 Mid boards or 1 Large board. The boards themselves could have a nomenclature so you can know how much you can fit of them on the case (S, M, L) and have them mounted with atx spacers.

Also, have some cable managemente in the inside, so you can have them orderly without a mess.

I can make a model if you want (i know you're more of the hands-on type, but having a model doesn't hurts anyone).

Changing board sizes greatly changes cabling requirements. If you make quarter-sized boards now you have to provide for 32 internal data and power connections. The provision for having several different boards is already allowed in that there are eight separate heatsinks. Buying 16 or 32 small boards instead of 8 big boards is about like trying to make a large-scale miner out of USB sticks. You end up with a lot of needlessly repeated parts. Small boards also limits your topology choices, which limits your efficiency overhead options. I can give you more details about this point with reference to some of our internal board design decisions if you like.

I would prefer to stick to a design based around air cooling. If you can't fit a waterblock in the same space as a two-inch-tall heatsink there's problems with your waterblock. If the rear panel is removable could provide ready access for hoses and such.

Not sure what ATX spacers would be doing anywhere?

Indeed, the nice thing about it is that you could design a big board mix it up with a plethora of different architectures inside them.

A removable back plate would indeed solve most of the watercooling issues with a air-only solution (i understand your air only preference, but with >60% humidity points here on my country, high density only happens with water blocks).

Also, i was thinking about the thermal protection you proposed, it would be nice to have the thermal watchdog in a separate process NOT related to cgminer, as you can see what happened to the S5s with that option.

I was thinking that the boards should go with atx spacers to have them mounted like standard motherboards on the case (but in this case on the vertical mounting planes), with some separation of the PCB backside to let the heat flow thorugh the ground pads (not unlike the S5).

I really should get a 3D model because i don't seem to make sense with words today (had my birthday celebration yesterday and i'm a little verbally impaired atm :D )

Yeah, a standard miner shouldn't ignore watercooling requirements since a lot of folks would like that option. You'd still be limited by power and data connections, so density wouldn't be as good as some other options (like fitting twice the power from a C1 as from an S3). I think making the rear panel removable shouldn't be too difficult.

The vertical mounting planes are themselves the primary heatsinks, screwed to the bottom of the case. If you space the PCBs off things, you're removing their contact with heatsinks and then things catch on fire. The boards do not mount to the case at any point. It's like how boards are installed in a Dragon.
The separation between PCB and heatsink on the S5 is actually the chips themselves, since most of the heat in a BM1384 comes out the top. If the ground planes on that miner ever interacted with the heatsink they'd short out and break stuff - like the Prisma did.

I think what happens with S5 fan control is more the fault of whoever wrote the driver code than an inability of cgminer to do things right. A separate control software is an option, but that removes the single-point control which an end user might appreciate.

I like the 4U case, plenty of possibilities with cooling ala S2 and a controller like the S4 would be nice.  Massive heatsinks seem to be popular and cheaper but new cooling needs to be developed.  Companies should stick to a standard case and board setup, it saves them money and us to just ship new boards.  Saves us from ditching rigs altogether.

"rip the chips off" ? are you glueing on the HS? most heasinks have thermal gel, or thermal pads between the chip and sink, then its bolted down to a flex plate or secondary heatsink, or sometimes just machine screws holding onto the PCB.

I have an old heatsink that held onto 10 audio amps and is 30cm/12in long. it never warped due to the 80 odd degrees Celsius of thermal input.
the only way them heatsinks warp, is incorrect installation. Using the PCB to hold 1KG of heatsink whilst dangling it in-font of a fan board flexes off the sink. I'm thinking of the RK-Box that did this.

also still surprised no one wants to use the powerpeg style heatisnk..


the PCIe hardware is only designed for 175W. and I have a pet peeve of companies using "Standard" hardware with non-standard layouts. There will be a stupid person attempting to plug a video card in and go "LOL mah vidz card makes fire! I sue yooou!".. and also could lead to the PCISIG suing the company misusing their hardware.
 
now if there was data throughput via PCIe lanes, thats a complete different dev path again..
Hell, I'm not sure if it was just the mini-PCIe (laptop card slots) that only had USB lanes, or if the full sized PCIe does as-well.

If you're talking powerpeg like the Alpha 3085 or the Swiftech 370, those things were EXPEN$$IVE for a reason - high cost to MAKE that style of HS, though they worked well. They don't work better than fins for crossflow cooling though, they were intended for updraft/downdraft specifically.

 If not, you'll probably need to explain what you mean.



 PCI-E hardware can use a LOT more than 175 watts. Look at ANY of the Radiion "x2" cards, typically in the 400+ watt range, for examples. Just have to use enough power connectors.

 Standard PCI-E does NOT use USB in any way shape or form.

Out of curiosity, what constitutes as high density? in kW per volume.

I've been doing some custom watercooling stuff for fun, wondering how thin the cooling blocks should be. So far I've only gone down to (in total) 7 mm thick blocks and they have had problems with flow.

As for liquid cooling it should generally require little (or no) servicing, especially since miners aren't kept running for long.

SerialLain, have you done any... experiments... lately?

Our design philosopy is simple, durable, reliable. Given two options to provide the same function, we will always pick the one that does so more simply and more durably. PCI-type socket backplane is a nifty idea for yanking cards in and out quickly and easily, but:
- the backplane burns about an inch of vertical height, reducing hash density
- the backplane PCB is very large and built in low quantities (compared to PCBs) and is therefore comparatively expensive
- means of securing PCBs within the case is cumbersome and unreliable (especially if PCB is secured with a hanging heatsink, versus a secured heatsink with a hanging PCB)
- increases the number of breakable plastic parts
- requires edge-connector fingers on every PCB, which adds to cost
- if something breaks, zero modularity makes repair or replacement difficult or expensive

I'm heavily in favor of using USB 2.0 protocol, which keeps the board-level hardware interfacing very simple. The designer can leverage any number of protocol converters like CP2102, MCP2210 and so on, or tie into a USB-enabled microcontroller. This also simplifies coding at the controller end, as cgminer is already quite good at talking USB. This also simplifies hardware requirements for the controller, as it's trivial to find a decent minicomputer board with USB jacks.
Using a PCI-hardware backplane, even if the signal is still USB, really isn't any better than a PCB with a securely-mounted heatsink and a fifty-cent cable. By my consideration, it's substantially worse based on cost and longevity.


Also, PCIe standard (last I checked) provided for a total device power dissipation of 300W. There are nonstandard devices which exceed this, however, but they aren't labeled with the PCIe standard logo. The PCIe standard allows for one 8-pin jack (at 150W) and one 6-pin jack (at 75W) and 75W through the socket.

 I don't understand that statement, heat sinks don't care about humidity to cool, they care about temperature differential. They do NOT use evaporative cooling like humans do.

 Ignore "heat index", that's only an estimate of how hot a HUMAN feels due to humidity level reducing the ability of a human body to cool itself, NOT the same mechanics as for an item cooled by heatsink-to-air heat transfer.




 Cost more, definitely so when you include the cost of a backplane - though not as much as you think, passive PCI backplanes do exist and have been used for a long time in some hardware and aren't exactly rare. No need to reinvent the wheel there.

 Mount really isn't any more complex than mounting those heatsinks to the case or however you're planning to mount them.

 Longevity, IME PCI connections tend to last longer than the hardware they are being used by, Heck, I've got "ancient" ISA based gear that still connects reliably after 20+ YEARS of usage. I do NOT see a longevity advantage for the typical cheap connector used on any USB setup - though I doubt it would average much if any worse, BOTH will probably outlast Bitcoin mining.

 The size disadvantage I can see POSSIBLY being an issue, especially since you're trying to limit board length for a better balance on front-to-back cooling.


 Just had a thought - but I can see mounting issues getting "interesting". Make the hash boards horizontal, instead of vertical, then mount the power supplies to one sire of the case. Would probably need a subframe mounted inside the case, or spacers between the boards, to keep the boards from flexing too much. Would give fewer but larger boards, so would be a bit less "flexable" about incrimental upgrades. It WOULD make the cooling issues on the hash boards easier to manage.

 Second thought - why limit it to 4U? As I recall BitFury and Avalon both made rack-mount 6U miners, which would make space management a LOT easier inside the case.
 6x 120mm fans on the front would generally be more airflow per square inch than 3x140 too while using a LOT more common size of fan with a LOT more options available.

 Delta, for example, does not list ANY 140mm 12V fans on their website, but they have a TON of 120mm 12V options (the 140mm fans they DO list are 24V and UP).

I don't know about that, my HD7990s had 3x8-pins = 525W and they were official enough.

High Density varies a lot depending on the nature of your deployment, but it can go as high as 18Kw/m3 depending on the cooling you have.

However, i tend to go a little more low there, the densest deployment i have is about 2.7 Kw/m3.

Watercooling isn't only for density, though, it is also for ultra-high humidity deployments, like my country's usual >60% (even near 98% 4 months a year) relative humidity.

7mm block is too small, i'm designing mine with 2cm blocks at least.

Powerpeg: http://tem-products.com/index.php/thermal-connectors/power-peg.html
its basically a round bit of copper, that sits though a 2.5mm hole in the board, that the ground/thermal pad of them QFP can solder onto, and since its a full chunk of copper over via holes thats been plated with 2uM of copper, to pass heat though and you can screw a heatsink directly to it. its been around for a few years now, I even pulled it up in the hardware section here to ask why no one thats manufacturing miners with thermal vias are using it.


i would assume you are using power wires off the PSU directly.

Mini-PCIe in laptops do, many wifi cars use the usb protocol over the pcie bus. hell, used it for my old EEEPC701 mods, put 32GB of flash memory on it (via hub and 4x8GB drives).

I was wrong with the power throughput Via the bus only, its 75W, not 175W. about page 35 has the power requirements on the bus.
http://read.pudn.com/downloads166/ebook/758109/PCI_Express_CEM_1.1.pdf

Humid air increases static pressure on fans, lowering considerably the air pushed through the heatsink (that means CFMs go down by a whole lot).

Waterblocks OTOH don't rely on changing ambient conditions (if you're going closed loop).

There's also the possibility of open loop cooling with chillers and such, but i'm not a fan of that kind of cooling tech.

It also makes a difference if you want to use evaporative coolers to knock your 37C ambient down a bit.

The real point he's making is, for approximately equatorial locations air cooling is not easy to accomplish so some provision for waterblock installation is probably necessary.

If you have a string design with a dozen local ground planes all at different absolute potentials, you do not want a heatsink spanning them without isolation. If that's not the case, sure using powerpegs is probably great. I wouldn't design a standard heatsink to require them, but if you can use 'em affordably without catching stuff on fire I wouldn't rule it out.

If you're taking power in through cables instead of a backplane socket, that's not another argument in favor of using a backplane.

My thoughts,
1) Make the controller aware of power quality via the signal all psu's provide.
2) Since most server PSU's provide +5v control power, buffer that with a small supercap & use it to power the control board. Size the supercap to allow the controller to run for say 10-15sec and use that time to do a controlled shutdown and possibly a restart if the power come back fast.
3) Is it possible to just use a internal LAN via a multi port switch? Seems to me it would be a lot easier for addressing the boards & data xfr vs the common SPI bus from board to controller and there are chips galore out there made just for internal LAN coms.

I know copper is expensive now but aluminum retains heat for too long.  I'm sure you could reduce the heatsink size by using a utilizing a more efficient copper heatsink.  Copper may have the same properties but there must be something other than aluminum.  Could you have rear pull fans and  fans on the heatsink pulling of it's heat directly?  That would limit how many boards you could place in the case but it would be quieter and possibly allow for more dense placement of chips on each board.

yeah the modular design would solve so many issues.. something similar to the S2 backplane with interchangeable cards not depending on manufacturer

Copper has WAY better thermal conductivity than Aluminium, almost double, but the weight is a great deal too, so i doubt anyone is going to do big heatsinks with them.

So. The day's discussions.

Chiguireitor, we spent most of today figuring out how to do things well with S1-spec heatsinks. If we maintain the 4U case size, we can comfortably fit seven heatsinks of S1 dimension and screw pattern in with room alongside for power supplies. Boards would be too tall to allow for PSUs mounted at the top how we originally planned.

This would allow us (and others) to build a single standard S1-sized board which could act as an upgrade for S1 miners or fit into the rackable machine. It also has the side benefit of being able to build S1-formfactor standalone miners, since we'd already have boards and heatsinks. All that remains to acquire is fans, controllers and a bit of framework.

For GekkoScience specifically, it would mean basically merging the Spec1 and Spec2 designs into a single board. The intent for Spec1 being a quarter S1 instead of a half S1 was that the board could be run standalone as its own 50-150W machine. The Spec2 was almost from the beginning intended to be rack-buildable. If we divide the market into roughly three sectors using Bitmain products as examples, we have the U3, S1 and S2. Our Spec1 would have fit U3 and S1 sectors and the Spec2 fit S2. However, if we make this change, we'd have the Spec1 as a single 30-chip board meeting S1 and S2 sectors and design a different product for the U3 sector. I think we're okay with that plan.

Designing boards for the rack-standard as based on S1 standard also helps cement and maintain S1 as a standard for its own market sector, which I don't think anyone is going to really argue against. This also helps maintain driver compatibility, as a single driver per board design would work for both S1-refit and rack-miner installations.

Being as a plethora of waterblocks already exist for S1 standard, an S1-standard-derived rack machine could be refit with S1 waterblocks pretty readily.

The problems we're coming up with are geometric in nature. There's plenty of room widthwise to fit seven boards. We could probably do eight, but seven gives better power headroom and more efficient per-board cooling off 2400W of available power. The problem comes in when we want to fit supplies and fans together. The width of a pair of DPS1200 supplies plus three 120mm fans will not work in a 17.5" OD rack case. It could be done if we switched to something like the Emerson 1200 that Spondoolies uses, which are also fairly expensive and still pretty tight.
If we want to keep a DPS1200 (whether that exact supply, or something with its dimensions) we've got three choices, far as I can tell.
1. Put the supplies at the front. This makes the power cord readily available but also gets your PSU cooling air blowing out the front. At full power you could be venting over 200W per machine out into your cold-aisle space.
2. Recess the supplies inside the machine. This makes them inaccesible from the outside for replacement, which shouldn't matter to anyone except Spondoolies fanboys since nobody else has ever built a machine with ready-swappable PSUs. It does also, however, make plugging in the supplies more cumbersome - we either let you have to thread your cord carefully into the socket about 2" inside the machine, or we put on an external socket wired to an internal plug into your PSU. One option removes convenience and the other adds a fair amount of cost.
3. Use a matrix of 80mm fans across the back instead of 120mm fans. Putting in 2x4 80mm fans instead of 3x 120mm fans will give us an additional ~1.5" of horizontal room to play with, making space for PSUs and allowing some gap/play between fans. Small fans will have to spin faster and so will make more noise. Sourcing more fans will probably also increase cost.

Currently I'm in favor of using 80mm fans across the back, because it's the least cumbersome option and, though it results in more noise, the purpose of rack gear has never been "silent running".

On the psu's:
Or - dedicate a couple U of rack height and make/sell a PDU case for plugging in either the DPS1200 or better yet the IBM 2kw+ psu's either independently powered or wired 1+n to power rack miners above and below each PDU.... Please please please ;)

For a 2400w load 3 of the HP's wired for current share would be marvelous allowing true hotswap of a deunct supply (haven't had one yet though) while the other 2 take up the load. With all 3 in, gives a nice margin for the supplies right around the 80% load butter zone.

So, if i'm getting it right, the case would host things like these?

(Removed front and top face to make it clear what's hosted inside)

Front view
https://i.imgur.com/EicORp2.png

Rear view
https://i.imgur.com/JdDgjI5.png

Legend:

• Light grey: Case
• Green: Hashboards
• Violet: Heatsinks
• Blue: PSUs
• Yellow: Controller
• Black: 80mm Fans

As i see it, it only fits 6 S1 sized boards, so it could be tops 3.5 Kw of heat to be dissipated, with 80mm fans you could move enough air for them, but it has to be calculated as the case would be really tight.

Am i right on this one, or did i get it all wrong? I could do a fluid model to see if this thing would work.

I was thinking something like a 2U with pairs of DPS2K would be good for someone wanting to run dual-sided waterblocks. With BM1385 boards you could push about 30TH from 11U like that. I'd like to insist on internal supplies the default option but it shouldn't be difficult to circumvent them for a slightly more custom setup like that.

Chig, I can honestly say there's not a single thing about your 3D model that is correct. Either I suck at describing things or your English needs work - probably both.

The boards are mounted vertically. Their heatsinks are screwed to the bottom and stand up with their fins horizontal. This gives room for potentially 8 blades but seven is a bit better for power and space. These blades would take up approximately 14 inches of the 17.5 inch width of the inside of the case; the remaining space would have PSUs at the back and controller at the front. That's the current plan.

The original description from the first post had 8 blades spanning the full width of the case and occupying the bottom 5.5 inches. The top 1.5 inches had a pan across it with PSUs, controller and all cabling on this pan.

I think vertically you cannot fit 7, as the default heatsink of the S1 takes effin 3.5cm

Here's a model with vertical mounted blades

https://i.imgur.com/fWCkpqT.png

Also, i corrected the default heatsink width a bit, as i used 3cm off the top of my head, but just measured it and it is 3.5cm

Btw, i'm assuming 19in rack width and 4U height (which seems like the minimum for the S1 form factor vertical mounted blades).

IMHO a side-mounted-hack of the blades would be better spacewise for the case, however, it is yet to be seen if that much heat can be handled efficiently by the case.

You could also go the shorter way, and put the blades horizontally with centered PSUs with the sturdy Dell 750w models, and one RPi on top controlling the whole affair like this:

https://i.imgur.com/5XgS69A.png

But then, you might want to replace the 80mm fans with with two badass 140mm fans to end up with something like this:

https://i.imgur.com/aLlDJk1.png

The current width of the S1 heatsink doesn't matter, as we won't be using them.

The render with vertically-mounted blades is closer but still not right. Seven blades, heatsinks sized to fit, with about 3.5 inches to one side for PSUs and controller.

What width will have the heatsink? to render a model ;)

Here try this out:

http://i62.tinypic.com/152f0d4.png

Same legend as Chig.  Had to make it 18" width in order to fit 7 hash boards with the dual heat sinks as spec'd, in practicality you might be fine with 17.5".  All fans 120mm, I wouldn't personally bother with 140mm fans. Keep it simple, buy in bulk.  

The only way I'd practically see you with 8 hashing boards @ ~300W each would be to make it taller than 4U to accomodate a single 2880W PSU up top. Even with 2x DPS 1200 stood up with a breakout board/backplane at the rear of the PSU shell, you won't have enough room.  

7 Hashboards of BM 1385's @ 300W each will still be ~9.7 TH/s and around 2230W at the wall.

Edit: Don't mind the crappy render, it was my first time.

Omg it seems i was blind or something, just read it was 7-8 Blades, not 7-8 S1s

That gives a LOT better disposition... Will try a jab at it tomorrow

If we go exactly S1 dimensions, we probably aren't going to be able to double-side heatsinks. Individual boards could still have chipsinks on 'em (like the S5+) but with the limited room above the heatsink and no extra room past the end of the heatsink (the original 2" extra at the front) rules out a lot of what could be done with a single-sided board and still have room for full heatsinking on both sides.

Two DPS1200 stacked vertically are pretty much exactly 7" tall, so can't fit inside a case whose outer dimension is less than 7". The only real way to do it is to set them horizontally one on top of the other, or vertically side-by-side. This takes up between 3.25 and 3.5 horizontal inches depending on which option you take.

The outside width of the case should not exceed 17.5 inches to ensure it fits comfortably in and out of racks, where I believe the defined standard is 17.75 inch inside. 18AWG steel knocks off about fifty thousandths per layer. If we assume we get 3.25 inches for PSU and 5 layers of steel we have 14 inches. This gives us 2 inches of width per board and heatsink. I figure 1.5 inches for heatsink will give half an inch for PCB, components and gapping to get boards in and out.

Unfortunately, 3x 120mm fans is more than 14 inches. There's room across the front to do it since the PSUs don't go all the way up there. 4x 80mm PSUs wide covers 320mm or 12.6 inches, so there's room for play at the sides and inbetween. Two fans high is 6.3 inches, plenty of room left over in the 7-inch height (or, if we say 3 layers of steel and 2mm under 7 inches for shelf play in the rack, 6.75 inches inside) for fans to be comfortable. With that much fan on the backside you might not even need front fans to push.

Finksy's model is pretty close to what I'm thinking. Maybe tomorrow I'll run out a sketch and see if our scanner wants to work on my machine.

If we went to an S1-formfactor board, we'd probably shift the data cabling back to stock USB - no sixth wire to control link light. Sounds like software fan control is the best way to go, so the internal hub board would have 8 devices, where seven are boards and the eighth is a built-in controller that handles link lights and fans driven by cgminer integration. We'd probably design boards to have holes/pads for both USB jack and a pin header, the intention being to use the pin header for USB signals inside the rack case but populate the jack instead for boards intended for S1 use, with generic hubs and whatnot.

I have 2u "server" (no external drive option) cases sitting behind me with 4 80mm fans - about an inch on either side of the outside fans of clearence, quarter inch or so between each fan, and rack handles, so ballpark 3" of horizontal clearence that's not got fan in the way.

 I don't think it would be an issue to have 1" or so of one fan blowing on the PS though, don't see why 3x 120s would be a problem.

 Getting lots of airflow out of 80s is a VERY LOUD proposition - highest airflow I know of from an 80mm is the Delta 80CFM unit, which were COMMONLY known as "screamers" (6800 rpm or some such off the top of my head, those things make the 120s on a S5 sound QUIET) back when they were commonly mounted on Swiftech 370 or Alpha 8045 heatsinks on Athlon/Athlon XP CPUs.

 I don't have any of those, but I had a couple of the "next step down" 68CFM units - which were clearly audible THROUGH internal doors and the cases they were in.

 
 I'm wondering about specifying those DPS1200 - the specs I'm finding on those come out to 750-900 watts each at 110VAC depending on manufacturer, so you'd need 3 or you'd have to go 220v to power them (where they spec 1200 watts, but at that point might as well spec *1* of the IBM Bladecenter 2880 watt units instead). Is there a different DPS1200 than the ones I've been finding?

I'll say this a third time. The problem is physical dimensions. There's only two ways a DPS1200 would fit in a 4U case, and it takes up either 3.25 or 3.5 inches of horizontal space. This leaves less than 14 inches of width for fans, and 3x 120mm fans are more than 14 inches long. Therefore, impossible. It's not that 1" of fan would be blowing on the PSU. It's that 1" of fan would be coexisting in space with a portion of the PSU. If you want 3x120mm fans, you cannot mount your PSUs in a way that they're accessible from the outside - hence the three alternatives I posted.

110V on the DPS1200 gets you 900W, sure. That's still 3/4 power if you only have 110V available. But if you only have 110V available, why are you buying a 2400W rack-mount miner? I'm more than willing to ignore that small percentage of potential customers.

you have to realize the img below is true
then you just make that 2400 watt miner be able to down clock to 1700 watts on a low/110 volt setting

https://i.imgur.com/si98Q7g.jpg

Of course the problem is physical dimensions.  The amount of things you have listed originally do not fit in the physical dimensions you had planned for, therefore it's time to consider alternative options.  Either increase the height to accomodate upper-mounted PSU's, cut back the amount of blades to 7 and shrink the heatsinks by a small amount, or have the option to run an external PSU.  I'm a fan of individually sourcing external PSU's, as you can save a lot of money looking for used/refurb units as opposed to having to buy new over-priced PSU's from the manufacturer.  Look how much of a premium you pay for PSU's when buying SP1/3x or S2/4's, that's because brand new server PSU's are expensive and it is not feasible/practical to source used/refurb units for production volume.  When you shift that "problem" on to the customer, everyone wins IMO, and it's up to them to be as frugal/resourceful as they feel appropriate given their circumstances.

The PSU's should probably have their own partial enclosure, and I don't see having part of a 120mm fan blowing on part of that enclosure as much of a problem for airflow, especially given the potential distance between the fans and the enclosure (the DPS-1200 are only ~8" long + backplanes/cables, and there is 15" between the inside of the front and rear fans).  My render had allotted ~3.5" for two vertically oriented, side-by-side DPS1200's with clearance to mount them into a breakout board/backplane.

I think you can safely assume that 110V is on the way out even for any "home" miners, not just by your own standards.  It will be interesting to see how the S7 turns out, to see if it's still marketable to "home" miners, or if they start going larger form factors similar to S5+ for all their new models.  

Edit:  How many people really have spare 25A 110/120V circuits in their home anyways?  ???

When you say the amount of things I have listed originally, do you mean in the first post? Because I acknowledged then that it would require tweaking of dimensions to fit. I also already had an increased height for top-mounted PSUs. If we want to shift to S1-formfactor boards (which there are good reasons to do so) we'd have to either shift to a 5U case or put PSUs on one side.

The problem with your render is, now it's impossible to plug the PSUs in. You've opted to demonstrate my PSU location suggestion #2 - PSUs recessed inside the case and no longer accessible from the outside, which has its own problems as noted. It's either difficult/impossible to plug them in, or you need additional internal mains cabling to fill the gap to a pair of sockets mounted on the case wall.

I like the DPS1200 because they're not difficult to find for not expensive second-hand, and are nicely power-dense which allows more room for hashboards and such. Being as the innards would connect to PCIe for power, it wouldn't be hard to build a wiring harness with an external connection for running your own external PSU. I'd rather not design that as the defacto, because people seem to like being able to buy a box that needs no further toying-with to make it work. Included PSUs as a base default, no PSUs and wiring harness to external connections as an option.

As for PSU's being inaccessible, I forgot that the DPS1200's use C13 to connect, Im so used to the 2880w style with AC input as well as DC output all on the same side using blade connections.  Either way, if you ran internal cabling to a bulkhead AC plug on the case, you could use a single C20 outlet to power both PSU's, cleaning up the power cabling.

Yes, I was basing my render on the dimensions of your first post but with the PSU side-layout from last page. I'll leave future renderings to Chig, they're not my strength.

Side note, what about building the case tall enough to house a vertically oriented IBM 2880? They're No wider than 2 DPS 1200's stacked sideways, completely hot swappable with the rear connections and have beastly fans already built in. Plus you could power 8 boards @ 300w each.

I haven't done anything with CAD since about 2007, so don't feel too bad about not rendering.

Actually, a single C20 would be pretty handy if PSUs were internalized. Good idea.

With 3 internal DPS1200 PSU (or similar) stacked vertically and tied to a single external socket, one could get either higher power (overclocking and such, probably require cold ambient air or waterblocks) or full redundancy for stock loads as previously mentioned. This also gives an end user the option to circumvent the socket adapter and plug the PSUs in directly to 110V circuits (which would take a few minutes and a screwdriver, but not really a difficult process), still yielding well in excess of 2KW total (non-redundant) power. This reopens 110V home-use potential, and also by internalizing the PSUs provides ample room for using 3x 120mm pull fans which will surely be quieter than 8x 80mm fans.

If the PSU partition dimensions can be specified properly, it wouldn't be difficult to see making multiple interface board packs to fit the same space which opens up PSU options. I talk about the DPS1200 because it's what I have access to, but someone like pmorici has those Intel platinum PSUs (I think) which would also do the trick, or like the Emersons from SP3x gear. If the spec is done with enough consideration, it'd be fairly easy to present a variety of options that meet the same space, each with its set of pros and cons for any particular user.

As of right now I'm in favor of seven S1-sized blades with three push and three pull 120mm fans, with an internalized space for PSUs sized to fit three stacked DPS1200. If the same space can be made to accomodate a few other popular PSU models, I would like to see that done. Provision needs to be made for an externally-accessible 12VDC wiring harness but that's fairly trivial.

I believe I may have a few intel PSU in a box some where, if you need them for sizing etc. I will donate them to the cause.  I would love to see the PSU's modular as well with only having to change maybe the PSU interface board and of course the PSU.

Good to see a decent sized miner made for the masses.  :)

I may take you up on that. Probably got some other stuff to discuss as well, one of these days.

The community has been needing decent gear for the masses for entirely too long. I'm really hoping we can get enough support for basic standards and especially for PlanetCrypto's independent chip endeavor, which should give the tinker community bitcoin was founded on something to do for a while instead of just waiting for the tailings from various corporations to filter down to us.

Just for reference sake, IBM 2880W Platinum dimensions (with fan packs mounted): 17" x 8" x 2.5"

3x DPS-1200FBA (80+ Bronze?) minimum dimensions without clearance for board: 7.75" x 4.5" x 3.5"

The IBM 2880W would require a 5U case, but allow more room for the hash boards.    What dimensions are the platinum DPS 1200 TBA's? They would be promising, but more expensive. Considering how much we rely on ASIC technology to push the efficiency envelope, I'm amazed how many people disregard the ~5-10% efficiency that can be gained by switching lower quality PSU's out as well as moving to 220/240V.

I have a plat intel from pete running in my friends office  they use 110 or 220   dimensions in 1 minute or 2


40 mm by 73.5 mm by 265 mm

  1.58 inches by   2.90 inches by  10.44 inches

http://www.deltaww.com/Products/CategoryListT1.aspx?CID=010104&PID=ALL&hl=en-US

DPS-1200TB  next to bottom of the page

I'd rather not shift to a 5U if we don't have to. The S1 board is over an inch shorter than 4U, so pushing to 5U would just mean more wasted space at the top where there's nothing but cables.

I also have no problems at all about using more efficient PSUs, which is one of the main reasons we started building server supply interfaces to begin with.

Sent in a PM:


If the power spec is 2400W, three 1200W PSU is definitely redundant. It's only not redundant if you're running them off 110V, which is why I specifically said that'd be a non-redundant configuration but still actually possible. I assume there'd be an indicator for PSU functionality which would alert you to a downed supply. I think redundant supplies is probably not essential anyway, given that no miner yet made with an internal PSU had that as an option and nobody seems to complain.

There are a lot of good options for PSU, all weighing against availability, efficiency and price. I don't like the idea of resizing the case to fit one specific PSU that's not shaped like any other PSU ever, because if we want to make more of them than we can find that exact PSU in the secondhand market then we have to abandon the standard and go with something else. If it's designed in such a way that any of half a dozen existing PSUs which can be acquired new or secondhand will work without significant mechanical alteration, I'd rather see that option as the default.

Especially considering it won't be difficult to bypass any internal PSU and plug in whatever PSU you want as an external.

 Higher quality PS cost a lot more up front, unless you're willing to risk buying used units.

 220 isn't anywhere near as common in the US as 110. Most HOME miners have ONE 220 outlet available (for their dryer), many don't have any.
 You pretty much have to OWN your own home to be able to add 220 outlets, and most folks have no clue how to wire them up - at which point you're talking expen$$ive electricians to be ABLE to add any 220 outlets to your home.

 If you intend for this to be a HOME miner, ASSUME 110VAC not 220VAC or plan to lose most sales to the US (I believe Canada also defaults to 110, but not 100% certain).


 You seem to already be sizing the case around "one specific PSU".
 That would be the one advantage of specifying 2 standard ATX power supplies - even at the 1200+ watt level there are quite a few options - though they're a lot less convenient to make FIT into such a small case as you want to use.

If by "sizing around one specific PSU" you mean "any of half a dozen existing PSUs which can be acquired new or secondhand will work without significant mechanical alteration" - since the 1Ux2U server PSU is a very common dimension - then yeah, I'm sizing around one specific PSU.

I also don't really consider 4U rack to be a small case, given that it's a taller dimension than any decent rackable gear built in the last year.

Home miner is a secondary consideration for rack gear.

Buying used server PSU's is not much of a risk.  Out of the hundreds I've had a part in with either supplying or supplying boards for, there have been 2 failures that I have been made aware of, and I have received 2 DOA units. The real risk is buying an ATX PSU, new or otherwise, as they are complete rubbish compared to server PSU's and it's not a matter of if but when they will fail (after you have paid more for a PSU that provides less).


Actually 220/240V is the MOST common power supplied to homes in North America.  It's called split phase (single phase), and nearly all homes are powered with it in both Canada and US.  They split the single 220/240V phase into 2x 1/2 phases of 110/120V each at the panel, meaning anyone can have a 220/240V outlet made up by combining opposite phase 110/120V circuits.  Yes, I do recommend everyone get a certified electrician to do all the work, and all that other liability-ass-covering crap.  I believe that home miners should be looking to making the switch, because the few miners that will still be available to purchase will be moving to 220/240V anyways ala S4+ and larger.

Sidehack has already mentioned using the boards for a different form factor similar to S1/3/5.  For the less serious home miners, they would be best to wait for that and power it with 120V PSU's.  But to sacrifice a significant design element (large server PSU's) for a relatively small market would be silly.

I have never mined industrially and have never operated a data center/rack mounted setups, so pardon my potential ignorance here, but I have a question about the form factor.

Do mining operations really have an issue with overall hashing density per sq ft?  Is there a driving need for the entire unit to occupy a 4U space?  Or could other options be explored without reducing the value of the unit?

It seems like the layout is a key issue here, centering around balancing  variables: Number of boards, Number/size of fans, and arrangement of PSU's.  The ideal goal of the current discussion appears to be somehow fitting 8 properly cooled hashing boards alongside a robust PSU set inside of a 4U case. But the end goal of the discussion is a "standard modular rack miner".   It seems to me like a great number of the existing units on the market are powered externally.  And integrating the PSU with the hashing boards restricts modularity due to layout concerns. 

I envision something along the lines of a 3U hashing rack that has 8 boards and the 3 X 120mm fans layout. 
Alongside this would be offered "Power rack" that could be 1 or 2U.  I imagine this may add some cost in terms of additional cabling/chasis/wiring, but it does solve the modularity issue, and it gets you back in the 8 cards per rack with 120mm fans ballgame.  Just an idea, I haven't really looked at the actual unit dimensions so I could be missing something obvious here.

My opinion is, make the unit fully self-contained (having its own PSUs, controller, everything required to operate) but allow the option for external power. If you have no provision for internal power, it requires everyone (including folks who just want one) to source PSUs independently. That might not be a problem, but it can be cumbersome and it's pretty universally standard so far that rack gear has internal supplies. Sometimes those supplies suck and we wish external power was an option. With an internal supply the option for external power can still exist (with a simple wiring harness and external jacks, as previously discussed) - and now you have choices. If you dictate no internal power, it can never be an option.

The 4U height currently is required for the preferred board dimension. I think, and others have agreed, that using S1-size boards is good; even without considering cabling, an S1 board is about 3/4 inch taller than maximum allowable height for 3U. Blades could be laid flat (like in the S4) but that makes it difficult to access or maintain bottom-layer boards, and to fit more than about six boards you'd have to put them in rows which reduces air cooling efficiency (as hot air from front boards is trying to cool rear boards) and makes fitting waterblocks and plumbing very difficult in addition to greatly increasing the case depth. Dropping to 3U height is possible if we want to use a different size board, but the reasons for not doing so are fairly numerous.

I agree that a turnkey solution should be offered.  I am just suggesting that it come in the form of 2 physically separate units, both of which are produced and sold by you.  One would be the hashing unit at 3U, the other would be one or more forms of a "Power Rack" that is built specifically to run 1 or 2 of the 3U hashing units.  Anyone wanting turn key can simply purchase them both.  Anyone who has existing power or wants to use alternative power sources can buy the hashing rack without purchasing the power rack. 

Regarding dictating internal vs external power, I don't think there is much of a difference between a turnkey 4U unit with internal power or a turn key "3U hashing unit + (1U/2U) power unit".  At least not from a consumers perspective. 

The power rack could be build to allow hot swapping for spoondoolies fanboys. It could come in more or less redundant variations.  The point is that it would be a more flexible form factor/standard than an integrated system. 

Regarding fitting 8 cards in a 3U space, I understand that it is exceedingly tight.  Perhaps I am missing a key dimensions that makes it impossible, but skimming through I see the following numbers.

3U space: 5.25in Nominal, or 133.35 mm

Proposed S1 form factor card height: 5in nominal, or 127mm

Proposed chassis thickness: 2mm per, or 4mm overall.

Assuming nominal dimensions, you get a stacked height of 131mm assuming the hashing boards touch the chassis.  This affords you 2.35mm of margin for tolerance and making sure your equipment actually fits in a rack.  Admittedly this is tight, but I don't think that is impossible.

Can we shave a mm or two on the card height to raise the margin?

S1 card height is actually 5.9 inches. Ignore the proposed card dimensions from the first post as they're something else entirely. If S1 card height is to be the upperbound for allowable dimension, this will not fit in a 3U chassis with vertical cards and especially not when power cables are considered. As stated, the only way to get 3U is to either change card dimensions or not have vertically-mounted cards (which makes mechanics and thermal considerations more convoluted).

Regarding dictating internal versus external power, I think there's a substantial difference between a 4U unit with the option for either internal or external power and a "4U hashing unit + (1U/2U) power unit". Takes up more space and costs a lot more on account of now you have to build a second case and all the cabling required both internally and externally. As a manufacturer, as a miner and as a hoster of mining gear, it doesn't really make sense to me to do it that way.

With provision for internal PSUs, you are not mandated to use internal PSUs. A factory-option wiring harness (which can be sold separately for refitting machines later, hooray interchangeable parts) connecting to externally-accessible jacks would probably be cheaper than the set of internal PSUs and their backplane board (and would be required for your idea anyway) and it still gives you the ability to do what you propose with external power - but it doesn't remove the option to have a self-contained unit, which a lot of people appreciate. I argue for the option that incorporates - does not mandate, but at the same time does not exclude - what you'd like to see.

The internal power housing could, with small changes and replacing the backplane board, hold any number of PSU models - we've discussed 3 just in the last page. Or if you want no backplane board and the externalizing harness, you can wire it up to server supplies or ATX or whatever you want in whatever configuration. I'm fairly certain that's more flexible being as it includes all the flexibility of your proposal and still allows for a self-contained unit without requiring a separate box for power and with various options for internal supplies as well. The only thing it can't do is 8 blades in a chassis, which your 3U proposal can't really do either.

I still rather like the separate psu rack myself. Sell the miner and require that one of the 6-slot psu cases be bought (and preferably enough HP1200's to power it). Something like what is shown here from TRC http://www.trcelectronics.com/Meanwell/hot-swappable-rcp2000.shtml (http://www.trcelectronics.com/Meanwell/hot-swappable-rcp2000.shtml) Then when they get a 2nd miner they are all set, just get sockets/cables (and psu's) to fill the caase and hook up the 2nd miner. Seems pretty painless to me and frees up a lot room in the miner case.

Ahh, I knew I had to be missing some key dimension.  

The 3U suggestion was predicated on 5in cards that had "rear" facing connections, so they could be just ever so perfectly fit in a 3U case.  If a 5inch card is incompatible with the S1 form factor then throw my 3U suggestion out the window.

I hadn't really given a great deal of thought to backplane vs no backplane.  I was really just considering possible high level layouts.  It may not be feasible, but I envisioned that a "Power rack" could be built in a number of configurations.  Backplane if it was to be hot swappable, or wiring harness if it was to be cheaper and more simple.  

I think the separate PSU rack really depends on a 3U hashing unit.  At 4U I see the value in having the option to run it internally powered, and that probably offsets the loss of the 8th hashing card.  Is the S1 Form factor incompatible with a 5inch card? And even if it was, does 8 cards still not work in a 3U because of the controller card? The more I think about it, the more I realize that 8 hash boards in a 3U would be an impressively tight squeeze if it could even be done at all.  

This brings me back to my original question...

"Do mining operations really have an issue with overall hashing density per sq ft?  Is there a driving need for the entire unit to occupy a 4U space?  Or could other options be explored without reducing the value of the unit?"

I mean this appears to be aimed at some level of industry, vs the home mining market.  Do you guys that have hosting services and mining operations really feel the squeeze when it comes to space?  Is that a limiting factor in your operation?  At 4U you can fit I believe 10 units on a full rack, at 5U its 8. So for the same footprint, assuming you run a full rack, you have either 70 hashing cards at 4U, or you have 64 cards at 5U.   Thats less than 10% difference in the overall hashing density.  Is that significant to you guys?

I'd say depends on where the miners are. Most conventional IT hosting facilities charge by rack space used (height) and power so yes hash density comes into play. But said facilities are by no means cheap (but do have other advantages such as security, often uninterruptible power, bodies on-hand for monitoring, etc.) .

Going the miner warehouse style like you are thinking of in Venezuela then it is rather a non-issue I'd think.

Another thought - it looks like you are trying to stay somewhere around the depth of an s2/4. At least in telecom and industrial electronics case depths up to 28" are not uncommon (good example is Bitmain's 1600w PSU) sooo....

As I'm currently stuck on a boat for a few days waiting to get into port, I think I'll take a stab at this in Solidworks and see if I can't get some flow/thermal simulations running.

Edit: Here is what I have so far

120mm fans, plenty of spacing, I'm using .055in for all wall thicknesses.  C20 socket.  Separate flow path for the PSU space and for the hash space. 

For the PSU space I alotted 4inches, I think this is too much, but I wanted to play it conservative at first.  How much space wall to wall do you think this should be?

The other thing im still guessing on is the placement of the first hash card and the exact spacing.  Right now I have the first card a half inch off the wall, and then 1.75in spacing card to card.  This is totally a rough guess, and I believe you were all spec'ing 2" heatsinks.  Is this two inches overall? or .5inches on one side and 1.5inches on the other?  Should it include any chip standoff height?

Speaking of stand offs, what did you guys envision for vertical mounting?  I could see something like a 2 strut/board system with nylon standoffs to space the cards off the vertical struts. But I'd like to hear what you all wanted before I model that up.  I think it for the flow simulations that the board mounting will come into play so its worth getting that detail captured. 

I have to dig into the simulation stuff tonight and figure out how thats going to work, but safe to say I should run it at 300w per board?  I was thinking 200cfm per fan assuming no restriction?  So maybe like 100-150cfm per fan actual flow rate?  I may need an actual performance curve from a fan to get the simulation to mesh out correctly, so if anyone has a specific fan in mind I can try to find and incorporate that performance curve. 

Also, for heat-sinks, are you thinking custom jobs? Or re-purposed S1/S3 hardware?

https://i.imgur.com/Dgm7917.png

I want to see what kind of numbers come out of a 3 fan pull only simulation.  But it would be pretty easy to rework the model and do a 6 fan push pull.  I would just mirror the duct shape from the back at the front and maybe stretch the whole unit by a touch.

I also could run this as 3 pull 2 push, which might make for an interesting angle on the "negative vs positve pressure" argument. 

 

That mockup looks pretty much like what I'm thinking. I don't recall the exact numbers offhand, but I think a few posts back I gave some expected dimensions for heatsink height. The heatsink dimensions would be the same as an S1 chassis, but likely with a bit shorter fins. I think I figured for one half inch between the face of one heatsink and the fins of an adjacent heatsink, which allows for board thickness and some tall parts like tantalum caps and such before interfering with neighbors. Taller parts could be put at the top of the board above the heatsink, where clearance to the next board is roughly two inches. That might make pulling cards in and out a bit cumbersome, so it'd be nicer if tall parts were kept on the side with the heatsink, but that's probably going to be left up to manufacturer's discretion.

If you have seen a Dragon, I like how those are put together. Basically the PCBs are mounted to the heatsink, and the heatsink is stuck to the case. Screws through the base of the case run right up through tapped holes drilled into the fins. It's pretty solid, and there's no additional rail components like in the S2 - which increase manufacturing, and can get in the way of airflow, and shouldn't be relied upon for structure given how many of those arrived from the factory with cards flopping all over the place.

I'm not sure offhand what the board height on the S3 is, but S5 boards are a bit shorter than the S1 boards. They can be that much smaller because there's no power components due to string topology. I'd rather not design a flexible standard without allowing it room for components for a variety of topologies, so I'd rather make provision for something the full 5.9 inch height. The S1 heatsink I think is about 4.5 inches tall, so to fit inside a 5.25" case doesn't allow much extra room above that for anything requiring through-hole components. 3U would be nice, but I don't think it's possible without switching to a different board dimension.

I reread the thread and pulled out some of those dimensions. 
I mocked up a DPS-1200 type supply, I went with an overall dimension (including the power blade) of 8.25in x 3.5in x 1.6in.
The PSU channel is currently 11in up to that first bend.  If you opted to have the PSU section rectangular with no angled ducting, you could get away with 13 inches of bay space.  I would imagine you have some kind of backplane card for the 3 psus to plug into? What will this do to the PSU chamber airflow?  Is this worth consideration? 

Right now I have it with 0.4in between the first card and the side, and then 1.9 inches card to card.  This means that last card only has 1.5inches of space on the right side where it is up against the psu bay.  I could sink the first card up against the wall and get the full 1.9" per card for heatsink/space, but then you are restricted to single sided cooling only. 

I like the idea of mounting cards to heatsinks, and then heatsinks to chasis.  For the time being I modeled up a strut, giving a .1" clearance for a nylon washer or something between the strut and the card.  Also, im using 0.055in thickness for all the wall thicknesses.  I would imagine you could go thinner on the interior stuff and gain some fractions of an inch. 

Right now as it stands I have 0.075" clearance on the top and the bottom edge of each blade.  This affords you 0.75" of free space in the top shelf for cables and controller circuitry. Is that enough for you?  Theres also a good portion of the PSU bay that could do that same job.  I was thinking maybe even put the controller in the top of the PSU bay, and then use a blade style connector to interface PSU backplane.  This backplane could come in different flavors to accommodate different PSUs, and then your controller could use the signal quality lines etc. 

I'm going to start playing around with the simulation stuff soon and see how that pans out. Ideally id like to be able to run a full forced convection study, but I don't know if I have the right package for that.

https://i.imgur.com/Hrwqsb4.jpg

Better stick to 4U, that way you can have even taller boards. Also, on a 46U rack you would fit 11 of these rackable units, which would amount to 77 boards, not too shabby ;)

0.75" is a bit tight for power jacks, but you shouldn't have trouble plugging in a PCIe without kinking the wires.

I question your hashboard space width - a 17.5" case should have about 14" left over after 3.5" of power supply. But pretty much everything else - interchangeable PSU backplanes, controller location, all that - we're on the same page now.

If you can fit all of your controller circuitry in the PSU bay, then you really have more like .875 above the cards for cabling/connections.  I partitioned it in my model thinking you might want to stick a controller card up there.  But that could be worked out in a few different ways.

Hopefully this view answers most of the dimensional questions.  Let me know what you want tweaked and I can rework it.  The only thing I didn't remember to put a dimension on was the PCB thickness.  I assumed and went with the standard 0.062in.  If the S1 cards are thinner that might be part of why mine seems so tight.

https://i.imgur.com/fKy5E2t.jpg

Cool.  Not to offend the scam sites but those renders have more info than most of their sites.  NO PREORDERS HERE!  :D  Nice work!  It does help to sketch it out and not just keep it in your head.

and here im playing with a HP server, 10 SAS HDDs, 2x Xeon CPUs, room for 6PCIex8/16 cards, and 2x 800W PSUs neatly in 2U, and people are having trouble understanding 7/8 cards +HS and even just 1PSU in 4U?

you can fit a full sized PC in 4U..
in a full length PC, you probably be able to have 2, maybe 3 cards length ways, but thats not the issue..

A S1 board is about 155 mm high, 4U rack unit is 177 mm high.
i cannot find the width of the heatsinks for the antminers so i'll use the dimensions of the 120mm fan, and the 2 sinks facing it, so each heatsink is approx 60mm, but the Rack unit is 465 mm in width so thats ~7 cards with sinks and breathing room.

Adding a single atx PSU, thats 90mm less space, so about 2 cards less.
these server cases are about 1000mm long each antminer unit is 300mm long, with fan.

so you can fit rows of boards in, one row will be 2 less for a single PSU, but thats 7 cards x 2 rows, plus another 5 more cards next to the PSU.
but are we looking for a backplane that sits at the base of the case and have multiple cards sitting side by side and layered?

my idea on how I would do a case'n'controller:

open face, with blank brackets thats 40mm long 170mm high (4U case) you could fit 11 cards right across the face, each card slot has nylon rails, backplane with power connection and usb. behind the backplane is HP style redundant PSUs. next to them, $50 atom itx motherboard, with whatever OS you want. Why ITX motherboard? im over trying to configure RPis or tl-wr702n's.

What I would do is not have nylon rails because, as I've mentioned two or three times so far, they're pretty crappy structurally and a card with 2lb heatsink attached is going to get destroyed in shipping. It's better to secure the 2lb metal block to something structural and secure the 4oz fiberglass board to the metal block.

Another reason to do that is, if we stick with S1-style boards, the board and heatsink are the same length so there's no board edge for rails to grip anyway. The heatsinks on an S1 are less than 50mm wide. Take a look at Witrebel's case render, and you get pretty much exactly the geometry I'm envisioning for 7 300W blades and 3 redundant 1200W PSUs in a 4U case with built-in controller.

I think a good way to address PSU flexibility is to have basically a terminal block that all the 6-pin cabling to the cards comes back to. Someone could cook up PSU backplanes that link 2 or 3 supplies together load-balanced and route power up to the terminal block. If someone didn't want internal supplies, a $10 wiring harness with snap-in jacks at the rear of the case connects to the terminal block. That maintains modularity which reduces the overall cost and limits what has to be replaced when going from external to internal, or from one type of internal to another.

I also would not put rows of cards in a super-deep case because, well, anyone with Spondoolies gear can figure out why. The hot air from the front cards bakes the rear cards and they don't cool worth a hoot. Considering each board could be pushing out 300W, do you really want 2KW of heat blowing directly into another stack of things generating 2KW of heat?

I know there's been a lot of words said so far, but when a point that's been addressed twice already gets brought back up (with no new arguments in favor) it's a bit frustrating.

 S1 size cards are quite a bit taller than PCI-E cards, though a few GPU cards have fans that stick up a fair bit above the top of the board.
 S1 cards are also longer than most PCI-E cards.
 You also have a case that is almost TRIPLE the length that sidehack is trying to fit everything into. Personally, I think he should go a little longer on the case, 15" is EXTREMELY SHORT for any rackmount case and even a couple of inches of extra length would help a lot on fitting stuff in and making the air flow smoother.


 I agree that 2 cards in a row = VERY BAD IDEA, especially if we're talking 300 watts PER CARD. Rear cards are gonna overheat BADLY unless you get very fancy with air ducts and such, and then you reduce the cooling to everything.

If the internals are stacked closely together/high density then adequate cooling can be achieved with high static pressure fans instead of high flow (cfm) fans.... particularly in a push/pull configuration....I have used this type of setup on miners before and achieved better cooling than stock miners...I always swap out the fans on miners as it seems they always seem to choose poorly (or maybe it is to save money).

The original spec is 15" between the fans, which puts the case at more like 18" length. This can be adjusted a bit since the blades are now only 9" long instead of about 12" but I wouldn't be surprised to see the final case come in longer than 15 inches, especially once dimensions get ironed out for PSU requirements.

The last render I posted had the S1 cards at 12", so you get alot of space freed up if they are really 9" cards. 

Furthermore, I *think* sidehacks issue with going deeper isn't as much about case dimensions as it is about how long the hashing cards are.  That is part of why I want to get some sort of simulation running that accounts for both the flowpath and the thermal loading. Alas my current cad seat does not have the right simulation package and I can't get anything new hooked up until I'm off this damn boat.  Simulations may be a bit premature at this point anyways, but I wanted to get a general objective idea of pro's and con's to some of these points that get argued like negative vs positive pressure, push-pull vs pull, flowpath length, etc. 

So far looking good. For best airflow 2 things to keep in mine:
With 1-side fans the back of the case must be louvered vs just punched or preferably wide open and fan sized w/fan grills for if folks want to add outside fans in the back. Done to minimize dead metal in the way.

Preferably should have at least 1/2" between the fans and first edges of the heatsinks and/or use fans that have flow straightening built in like some of the nice thick Delta fans. This gives the air a chance to start straightening out and greatly reduces front edge turbulence. Also gives a significant reduction in noise by reducing the siren effect.

On longer case, I was more thinking about having the PSU bay possibly in front of the cards to fit in another card or 2 width wise vs cooking cards in a back row. Ja it adds heat to the hashboad airstream but works fine in an open air proof-of-concept. I do it in my farm to eliminate needing fans on some of my IBM 2kw supplies. https://i.imgur.com/XRlk5Tq.jpg (https://i.imgur.com/XRlk5Tq.jpg) btw that area often runs ambient up to 93F, and those psu's/miners still happy.

One thousand sidehacks weeped with that pic

heh heh heh ;D
Works very well though and is same depth as the s4 under them. https://i.imgur.com/aTkVsGE.jpg (https://i.imgur.com/aTkVsGE.jpg)
Was tossing that out there for consideration ;)

Hopefully I can get a thermal simulation package running soon and put some of these ideas through the paces.  I'll play around with front mounting the PSU from a layout perspective just to see what can be accomplished. 

Does anyone have any experience with design for manufacture?  Specifically what are the cheapest construction methods?  I ask because if it doesn't add too much additional cost of manufacture, I think this may benefit from a deeper case, same size hash cards, and some dedicated flow straightening vanes/features.  Im thinking something like the struts, where they can just be simple stamped sheet metal pieces, tack welded or riveted to the case in strategic locations to optimize the flow path. 

Thinkin' about it, the boards are more like 10 inches long. That's what I get for trying to remember stuff after being awake for 21 hours.

The original spec had space for nice 38mm-deep fans, and minimum one inch between fan and either leading or tailing edge of the PCB.

I also figured the rear panel for fans would best be done with the entire fan area knocked out and use a wire grille. If nothing else, those penetrations could also be used to route plumbing for waterblocks without having to make the entire back panel removable (which could cause problems for other things, like structural strength if machines are stacked).

^ ^ fan type and space between them and the sinks should be perfect  :)

Actually, the more space left between the rear of the board and the back panel, the easier it'll be to handle plumbing from waterblocks. 1" from board to fan and 1.5" fan makes 2.5 inches; is that enough room to be comfortable? Another depth consideration is PSU length, which the DPS1200 is about 8 inches but those Emersons are closer to a foot. If there's gonna be backplane for them to socket into, and also room for internal cabling, that's probably a minimum if 18" case depth.

I'd be a little leery of stacking PSUs in front of the hashboards. It might work for DPS2K (and that's a nifty setup; most of my big DPS2K installs here at the shop powered Tubes and Prismas which didn't really have the cooling overhead to add that kind of additional heat and airway restrictions) since air'd blow right through but any solid supply like is more common would block a lot of air. I mean there's probably a way to do it that isn't entirely stupid, maybe mounting the front fans internally and behind the PSUs, but then you need more gap for air to get around them and with more than one PSU on top of another you're really eating into your intake path.

It'd be really nice to fit an eighth blade in there, but I'm not sure if there's a better way to mount PSUs (without going full external or significantly changing case dimensions) that gives the heatsink space and also adequate airflow.

Yeah I realized today that depth wise I'm not really rendering to your spec.  I'll get that tweaked this evening probably.

regarding the fans/grills, I assume by rear panel you refer to is the panel that I currently show 3 fans on, and you are saying that instead of a panel with 3 holes, you envision an open wire mesh. It seemed like there was some concern that something like that ingesting hot air from the isle through the wire mesh due to the negative pressure in a strictly pull fan configuration.  If you want it like that then I think you may need the front fans after all.

As I have it laid out, you would intake through what I called the "front" which would be your cold isle, and exhaust through the rear to the hot isle.  Is this the "standard" flowpath that you guys run in your farms?  Just wanna make sure I have it straight it my head.

Also, for the sake of the discussion, are you okay with going deeper so long as it doesn't involve any more than the standard S1 ~10 card in terms of "hash cooling flowpath".

If we could go to say 24" deep max, I could probably get something pretty slick laid out.   The nice thing is that having the PSU's at the front of the miner reduces the bending moment on the rack mount ears that you would get if the weight was all the way in the back.  So it could probably be done with just a 2 post rack.  Watercooling might get heavy but thats up to the end user to figure out.

Having plumbing + Fans on the backplate would be amazing, as the S5 is known for its need of having fans anyways because a lot of heat is dissipated through the ground plane of the chips :(

Sidehack, do you think you can hand me the DPS1200 or the DPS2K measures so i can take Witrebel's design and rapidly iterate over it to find ways to fit them?

The only problem i see with the DPS2K is the lateral cooling they need, how would you handle that on this current design?

Only very light equipment is held only by the ears. Anything over a pound or 2 and deeper than a few inches MUST be supported by a sheet metal shelf and the ears are only to hold it in place. 1 shelf - 1 piece of gear. eg http://www.budind.com/view/Rack+Equipment/Server+Rack+Open+Rack+Assembly (http://www.budind.com/view/Rack+Equipment/Server+Rack+Open+Rack+Assembly) or http://www.budind.com/view/Rack+Equipment/Four+Post+Double+Rack (http://www.budind.com/view/Rack+Equipment/Four+Post+Double+Rack) with the shelves in it. While yes you could stack a couple miners per-shelf if soon becomes a pain when the one on the bottom has to be pulled...

If I were using DPS2K I'd wire them up externally, probably with dedicated fans.

DPS1200 measures are already in here somewhere, as well as I think the Emersons and DPS1200TBA (platinum-rated).

Witrebel, no you've got the back panel right. I meant a solid sheet with circles cut out for fans and then an external wire grille per fan. That should be structural enough for things to be comfortable stacked and if chig wanted to run waterblocks he could pull the rear fans and use the space to run his tubes out without having to take off the back panel (which would structurally weaken).
If we wanted airflow for something like S5 that really doesn't work well without both sides cooled, front fans would be necessary in this case. Yes, taking cold in the front and blowing hot out the back is the standard.

It might just be me but I'd like to limit case depth to about 20 inches. This wouldn't affect most rack installs (where things can be substantially deeper) but a bigger box would cost more to ship and be more cumbersome to handle - and for the home miners we're considering secondarily, bigger isn't always better.

Also, being as the box is probably going to weigh between 40 and 50 pounds, I'm not sure I'd recommend mounting it just by ears. That seems kinda dangerous. From the start we were figuring on shorting the case height by a couple millimeters to allow for rack shelving.

Chig - What package do you use for modeling? Do you want/need any source files from my model?

NWF/Sidehack - I didn't realize that shelf supports were standard.  I figured the tolerance on the height was just to play it safe.  Good to know as that frees things up a bit.

Regarding the depth I think I could make a 20 work, but it might not be optimal in terms of flowpath.  I just realized how much info I glossed over in your original post, I get to eager sometimes.  I have some actual work I need to do this afternoon but tonight I'm going to try and rework my first draft to really align with your first post and get the heatsinks/board mounting right. 

Actually, a lot of the information in the first post is no longer really relevant after changing board dimensions. Board and sink mounting still applies, but PSU placement and most everything else related to power is changed. I guess at some point I should write up a complete revision with updates from the last few days' discussions and link to it in the first post. Maybe I'll have time to draw some diagrams too.

Yeah I was mostly referring to the heatsink dimensions, and the double sided nature of the boards. 

I should have a couple of good visuals done up by tomorrow as well

Heatsink dimensions are out too, since the board size changed. And probably the provision for heatsinking both sides is gone.

Hmm,  I was planning on just chopping 2inches off the heatsink length but leaving the "Through hole/tall component keep out zone" dimensions the same.  And I don't see why double siding can't still be done.  With side mounted PSU's it might be a slightly smaller heatsink than you have speced but I wouldn't rule it out completely yet.

 

If you chop the heatsink down two inches, now you have two inches less board for putting chips on and you're no longer really using the board to its capabilities. The heatsink height and length should be the same as on the S1, with the fin height changed to work within our case. That way you can keep board specs fully compatible. Trying to keep an S5 board from catching on fire when the heatsink is two inches shorter will prove to be difficult.

The way we figured, having that inch at top and two inches on the end (in our original board spec proposal) would allow you to put any controllers and through-hole parts in that space, leaving the entire 4x10 chip field for placement of surface-mount components probably on one side of the board. This makes it pretty easy to double-side heatsink just the chipfield. Since we no longer have that much extra space (as we're now limited to the top approximately 1.4 inches only) for everything through-hole and such, it's more likely for other tall components (specifically not ASICs or low-profile local bypass caps) to be placed down in the chip field, so the majority of designs likely wouldn't be able to accomodate the backside heatsink. With this in mind, it's now up to the main heatsink to dissipate the entire power, even though we've just shrunk it in order to fit in a backside heatsink which will rarely be utilized. Way I figure, might as well just spec the one main heatsink to the fullest size possible and let it carry all the heat possible. A gap between the board and adjacent heatsink would still allow for low-profile per-chip heatsinks when necessary.

The board should be the same dimensions as an S1 board. The heatsink should be the same dimensions as an S1 heatsink, with the exception of fin height. Ignore everything said in the first post.

As long as we are on the heatsink topic, aside from being aluminum what kind are you thinking of? The thin almost shaved kind ala' Bitmain or sturdier extruded ones like in the Dragons?

If extruded then please spend the extra few cents and have the fins ridged - again like the Dragons used. https://i.imgur.com/lfAr2pN.jpg (https://i.imgur.com/lfAr2pN.jpg) Them tiny looking ridges double each fins exposed surface area making for damn efficient heat xfr.

That pic is from the AMT A1 1.1THs miner I have. One of the few actually delivered. Eventually. 7mo after advertised and emailed delivery date...

The shaved bitmain heatsinks are, in fact, better at dissipating heat than the extruded ones, contrary to what one would think.

I have both models (from the 7th batch) and the extruded heatsinks are considerably warmer than their "shaved" counterparts.

That's not really my field so I'm not sure what's best. I'd tend to lean toward something fairly sturdily extruded on account of the heatsink fins will be a mount point for things, and I doubt any screws tapped into the shaved sinks would be terribly reliable. If someone with more experience in heatsink design might chip in some opinions would be good.

added pic above ^^ https://i.imgur.com/lfAr2pN.jpg (https://i.imgur.com/lfAr2pN.jpg)
What the hell, all pics of AMT's 1.1 THs miner http://phluph.imgur.com/all/ (http://phluph.imgur.com/all/)
Built solid as a rock and has been running 24x7 since August 27 '14

Ja the shaved sinks are better but a lot more expensive to make. Main cost would be initial tolling charges as I assume they start with thick straight fins and multiple mill them thin then press through a curving die. Damn pricey to setup sizable production for but of course in the numbers Bitmain use I'd think is almost a 1-time for them. (Must keep a few shops damn busy in Shenzhen...).

No if's and's or but's the thicker sinks are also great mounts.

Ever look at Wakfields site? Ja can be pricey but still at least a good reference.
Link to full up custom design assuming you know some specifics http://www.wakefield-vette.com/company/contact/build-your-own-air-cooled-heat-sink.aspx (http://www.wakefield-vette.com/company/contact/build-your-own-air-cooled-heat-sink.aspx)
General extruded sinks catalog http://www.wakefield-vette.com/products/natural-convection/thermal-extrusions.aspx (http://www.wakefield-vette.com/products/natural-convection/thermal-extrusions.aspx)

No I don't work for them but for almost 40 years do design industrial laser systems and know a thing or 2 about handling multi-kw power and cooling ;)

Slightly OT but for ^^ above reasons am also a huge fan of liquid cooling using cold plates. Not really worth the time/effort/expense for 1 miner at a time but -- start building a farm and whole 'nother story. Very easy to move as many kw as you want from the equipment to outside. Well worth the cost of plumbing/dry heat exchanger/fan and large pump. Aside from maybe in the Middle East, with the operating temps miners like don't even need refrigeration.

Is 0.5in the absolute maximum height that a secondary heatsink can be to retain compatibility with all S1/S3/S5 chasis?  If not what is? 

Just measured one and is a from surface of board to the inside folded edge of an S5 endplate is just over 0.75" (prolly an even mm distance?)
I'd stick with the 1/2 inch for electrical safety and to allow some airflow outside of the sinks.

Amen to that! If only the C1 style would have picked up steam (No pun intended), it would be one of those situations where you buy once cry once. Water cooling would be the real beneficiary of standardized, modular designs like we're discussing, because you can size the heat exchanger/pump/fans as large as you need, Reuse it and the real savings would come from mass deployment (assuming water blocks are cheaper than heat sinks and high powered case fans in bulk). Even with 10kW of miners It would be worth it I think.

heh heh... set up a peta farm somewhere and use the heat to setup a spa with heated pools.  ;D Hell, I'd think that even a couple 100kw of otherwise wast heat would make that feasible. Hmm...
For the record -- if someone runs with that idea I'd love some of the proceeds... Wallet 14DdoPoEKiWQQj3WdLShbm6ppvm8349Hto You heard it here first!

I am still of the opinion that a secondary (short-side) heatsink isn't really worth the trouble. I've mentioned half an inch between the face of one heatsink and the fins of an adjacent, which allows for PCB and some tall SMD components with clearance. This side of the PCB could have individual chipsinks glued on, but I don't think I'd worry about writing an actual full-size secondary heatsink into the spec since I'm guessing most every board implementation would have components near the ASIC which are taller than the ASIC.

For reference, see what they did with the S5+

Okay, fair enough.  What is the max main heatsink depth that would be compatible with the S1 chasis then?  Is that where 2inches came from?  Just looking for all the upper limits so I can play with layouts.

I gotta agree the 2nd sinks don't seem to help much. I got an S5 with 1 card that has the Bitmain supplied secondary sinks and the board runs the same reported temp as all the others. Measured main sink and despite the additional chip cooling is same temp as others so... Doesn't seem to make much difference (on a S5).

The actual Bitmain heatsink looks to be 1.5 inches.

If we have a 17.5 inch case and 3.5 inches is PSU, that leaves 14 inches. A practical upperbound is 2 inches per hashboard, meaning heatsinks and the clearance between them in which your PCBs will live. Mind you those are approximate, as this ignores metal thickness for the case and such. The actual heatsink will likely be less than 1.5 inches tall, unless we want to compress the 0.5" clearance between sinks. Reducing that clearance to increase heatsink size will increase power dissipation potential of the sinks slightly, but also can make PCB design more touchy and increases the risk of damaging a board during installation or removal. Everything's a tradeoff.

Agreed. Anything that is SMD is very easy to dislodge with inadvertent handling. 'specially large SMD caps. popped a couple off shuffling boards around while modding s1's  ::)

Gotcha, admittedly I'm still scheming for how to make 4U + internal psu + 8 cards + s1 compatibility work.  I think one of those variables has to change.  Safe to say you would prefer 7 boards 4U over 8 boards 5U?  Assuming both had internal power?  

One thing I am getting a little hung up on layout wise is the asymmetry.  In an S1/S3/S5 the boards are mirrored and symmetrical, so nominally they both lay in an identical flow path and identical cooling.  If we go with a single sided heatsink and simply pattern the cards 7 or 8 across, the flow path will be different for that first card that is up against the wall.  You could center everything and leave some space on either side so that although asymmetrical, you still have good flow all around.  But that would eliminates the possibility of PSU's on the side of the case, and makes having internal supplies at all quite tricky.  

Do you think that it matters at all?  That first card having its non sinked side up against the wall?  In reality it may be a non issue, it just looks wrong to me visually when compared to how the S1-5 is laid out.

If you put the card nearest the PSUs with its non-sinked side up against the wall, it'd be up against the ducting and get good airflow. That might choke air to the heatsink on the card on the other end if it doesn't have good fan exposure.
Conversely, if you can maintain half an inch clearance between the heatsink face and the case wall, the gap won't be too much different than an S5 board inside its plastic side panel and the fan exposure it gets in the stock chassis.

I'd be more in favor of a 4U height with 7 blades, yes. That actually does power a bit better too I think, if we want to limit 300W per blade we have 2100W of mining and potentially another 100W in fans and controllers for 2200W total. 8 blades at 300W makes 2500W total, so your stock power dissipation is more than two 1200W PSUs are rated for. If you wanted 8 blades and reduce the power below 2400W (say, back down to 2200W, still 92% of rated PSU) you're at 262W per blade. That's right about the expected per-blade power dissipation of an S5 at stock settings, but not near what the machine can actually handle.

What would we rather have - 7 blades at 300W, or 8 blades at 260W? If the smaller heatsinks for 8 blades are actually still capable, with the proper fans, of safely handling over 300W each (which would be an overclock-at-your-own-risk level for sure, and our 30x BM1385 board would have an expected top-clock of about 320W, nominal clock more like 230W) I certainly wouldn't complain.

One thing that occurred to me is, if we're already building boards and sourcing heatsinks and fans, we're pretty much just endcap frames away from building an S1. There's no particular reason to require the heatsinks from the rack also work as standalone S1 replacements except that acquiring bulk quantities is already done.

Ja
I was wondering about the S1/3/5 thing. Good general form factor to work with per-blade but -- since the blades are going in a case you should be more concerned about how they can be packaged in said case.

Now, certainly doesn't mean one can't slap on endplates and make smaller modules from the blades but doesn't have to fit the org S-series endplates. Hell, turn the endplates around and just call them fan mounts.

Thinking about my AMT A1... Is a later Dragon clone and has 5 cards in it with room for 2 more. As the pics in my imgur link show the hash boards make a nice - package. Anywho, it is pulling 1,300w from the wall giving 250w/blade assuming reasonable efficient psu. Took out the 120mm turbines it came with and I put in 3 Cosair SP-20's. In a biz office is quiet and runs @ 55C inside.

Just a packaging reference point as you are shooting for moar power and ergo higher CFM/louder fans.

I mentioned the S1 endcap thing because it opens up another product stream. Boards from this box could be sold in pairs as upgrade kits for S1, but if you punched out endcaps as well you could actually make whole units for sale instead of just as upgrade kits.

NotFuzzy, you got an opinion over 7x 300W-stock blades versus 8x 260W-stock blades?

I say stick with 7x300w. Well within the range of using PCIe plugs and gives more space to fit PSu's and any air routing they will need.

Und ja, selling the smaller stand alones not a bad idea as like you said, end caps would be almost dirt cheap to have punched/bent. Now sizing the board to fit S1/3/5 sinks.... Shades of the S1/S3 upgrades :) I did 2 of them, interesting exercise but thass about it. Better to buy ready-made.

The boards are already sized to fit S1/3/5 sinks. It's been my plan for months to build boards for that platform, and arguments to keep the shape and compatibility for a rack standard are convincing.

If we have 13.5 inches inside to fit eight boards with clearance, that's what 9.5 inches total heatsink thickness? That makes 1.2 inches. Is it feasible to make an extruded aluminum heatsink 10x4.5x1.2 inches comfortably capable of moving 260W of heat? Thinking about it, I do kinda like that underrating for power dissipation per board being as it's fitting with the stock power specs for S5 and that's the kind of heat the S1 chassis can be expected to comfortably clear.

That's really the question, I guess, in deciding between 7 boards and 8 in the same space.

I'll measure what the blade size is in my AMT rig tomorrow. They marginally do it with a lot less air then you will be moving. I dismember the blades being pert near the same size as Ants, maybe a tad shorter and longer.

I countered my own argument, i was going to say "I'd say give more room for sinks, there will be someone with 150W BGA style cores they want to squeeze in there.."

but then i thought, "they can throw 2 maybe 3 of their cores and their own copper sink, that should suffice. Aluminium is good, but copper spread the heat quick enough from the hot spot.."

So I spent the last couple of hours playing with a 4U case 20in deep, seeing what can fit.   I opted to shoot for 8 cards and max out a single sided heat sink.  Obviously if you can fit 8, then 7 would be cake.  The first option came as a result of realizing that 2 DPS-1200 and 3 120mm fans do fit side by side in a .055in walled 17.5in wide case, with .016 in to spare.  This is just too tight to manufacture, but not too tight for some staggered fan action.  It's an oddball design, but I just wanted to throw it in the mix for consideration.

https://i.imgur.com/dUznbAA.jpg

https://i.imgur.com/Sqiefdz.jpg

As you can see, this layout provides for 8 cards, with 1.438in for a heat sink and .5 spacing.  You can use the outside hashing cards and a small divider in front of those cards to effectively separate the PSU intakes from the hashing space.

There is less than a 0.1in overlap edge to edge of the fans, so you could just mount the center fan to the real panel, and use small strip of metal with clearance holes to attach the fans to each other.  The outside fans could then attach to the wall that is between them and the PSU.  I didn't take the time to render all the walls, but they are included in the spacing of all parts. So the fan is spaced off from the psu in this render by 0.055in + 2mm for clearance of the PSU. 

But alas, taking a page from aviation "If it looks like crap, it flies like crap" generally.  Personally I like the funky look, but I iterated on the idea and came up with something a bit more clean.  Rear mounted PSUs, they could be hotswap/backplaned I believe.  You still have the compartment you wanted for a controller.  And you get 3 120mm fans.  Only thing would be routing a bit of intake for the PSU's if they did get a back plane.  But I think thats a non issue.   You can route the intake in from where the controller section is up top. 

https://i.imgur.com/PKx7b61.jpg
https://i.imgur.com/7FM61mE.jpg
https://i.imgur.com/ExyIugQ.jpg

Again this runs 8 cards comfortably.  With full .5inch clearance board to wall and sink to board, you get an overall heat sink height of 1.625in.  And now you are back in the game of 8 cards 300 watts per, in a 4U setup.  Only catch is you are restricted to shorter PSU units. 

Two PSU with staggered fans rules out the ability to run a third PSU, which isn't necessary but would be nice for redundancy and anyone wanting to get full speed out of 120V.

Short PSU requirement rules out the higher-efficiency Emersons and DPS1200TBA, which are both around a foot long. I reckon a 24" deep case would solve that problem, but it seems like an awful waste of space to have 10" hash cards in a 24" case. There'd be a lot of unused volume in there.

Them renders look pretty snazzy. Nice work.

Ja +10
Something to consider is wire routing from the PSU's. Even direct soldering to the output lands needs some room, using 10ga main leads from my HP's is about 2" https://i.imgur.com/rbcz0pw.jpg (https://i.imgur.com/rbcz0pw.jpg). Split out as many 16ga leads right at the lands should be fine with an inch or so. Add a breakout card and it tacks on about 1.5" or so. There is pic of what I mean in that collection I linked to on Imgur.

I don't know that I'd say it actually "rules out" longer PSU's.   This isn't the cleanest looking solution, but I don't see any insurmountable obstacles with running longer PSU's that simply extend out past the back grill.  Securing them would be a trick, but I could cook something up for that.  And you could still go with interchangeable back planes/harnesses etc.  

For a staggered fan miner, you could either run two longer PSUS, or theoretically run a third horizontally above the fans.

For the stacked miner, you could run any 3 PSU's that fit in a 1Ux6U space, or any space you allocate on the shelf really.
If using backplanes and hotswap, you could ship the PSU's out of the bays so that it fits in a smaller box.

Now this gets me thinking, why can't we get the best of everything, truly modular, ultra compatible with S1, 4U, 8 cards, 3 PSU's, small case dimensions for shipping...  Why not use 4U modules.

The core module is 8 boards, and a shelf with the controller, lets call this a 4U module 12in deep.
You then offer another module that is basically a fan module.  
You could either make all the rear modules a standard 12 in for shipping and handling purposes, as well as chassis manufacturer, or you could offer various products to meet the needs of different setups.
Fans only,
Different PSU options,
Internal vs external

If all these miners are destined to sit on a shelf anyways, then you don't need to join the two modules in a terribly structural manner.  Just make sure they line up and mate repeatedly so you can ensure some kind of a standard when it comes to airflow.  

Now you can ship in a smaller box, waste less space, and have a more flexible product line.  Yes I think there will be some give and take with manufacturing costs.  Maybe you end up doing 12in across the board on all modules and you use a standardized hole pattern that can do all products.  Just some food for thought.

I'd really prefer to keep a single box with fully modular innards than several boxes that stick together.


"For the stacked miner, you could run any 3 PSU's that fit in a 1Ux6U space, or any space you allocate on the shelf really.
If using backplanes and hotswap, you could ship the PSU's out of the bays so that it fits in a smaller box."

I honestly have no idea what that means.

If we determine that an S1-sized heatsink can cool 260W with 1.2" fin height instead of 1.5", I'm pretty sure we can then fit 8 S1-compatible cards with 3 PSU (up to 12" long) inside a single 4U case no more than 20 inches deep.

End of the day that's your call.  I'm just curious what leans you that way.  And for the record I don't see "many" boxes, so much as 2 identical 12" boxes that have modular innards with eachother. So you can could still source one box, one chassis, and build it up as you see fit for sale in various configurations, it just affords you alot of room to play with and keeps shipping size down.  But yeah just a fringe thought, one box is clean.

After a long few days at work I think I'm better at modeling than I am at articulating, I can see why that would be hard to follow.  

This is along the lines of what I was suggesting could be done if someone wanted to step outside of the DPS-1200 size envelope.  Basically just hang a PSU outside of the box.  
I thought you had referred to this size PSU as a 1Ux2U psu.  So I was saying that 1Ux6U across the top gives you the basic PSU bay envelop, and provided there is a backpane to match the model of PSU, you could really stick just about anything in there that will fit.  It's just going to look funny from the rear.  

https://i.imgur.com/NdBojkM.jpg

But yes, I can see how you could pull off 8 * 260w in a 20in case. 

I just feel like this places an unnecessarily low thermal bound on a 4U case. Granted 2kw isn't "low", but unless you water block it you are pretty much capped.  Maybe I'm biased because my particular situation doesn't really require efficiency.  So perhaps a 2kw thermal ceiling is pretty high all things considered. 

Some folks have trouble keeping 1.6KW cool in the 3U S4+, which translates to 2.13KW in 4U (yeah I know you can't directly scale like that, but for the sake of argument). 7 blades at 300W is the same heat as 8 blades at 260W; the first thing I'd do if I built an 8-blade is push it to about 320W per blade and see what breaks first.

Having PSUs hanging out the back looks goofy. But - BUT - if they're secure, maybe it's not so bad. I'd be worried about trying to slide the case forward and a cord hanging and pulling the PSU out of its slot. But if they can be secured in a non-clumsy way, that might be okay. Certainly worth considering. That would also remove the need for internal mains wiring - you now can no longer run it off a single C19 cord, but that's probably okay.

The only problem with encasing the entire front in 8U blades is where does the controller go? But of course you could partition off 1-2 inches along the side (basically what would be left over from 3 120mm fans comfortably laid out) for the electronics. And it wouldn't be difficult to build a snap-in panel to fill a drive bay or two with power jacks for external supply connections.

Could work.

 I didn't realise that DPS1200 was 1ux2u. Size objection overruled by additional fact input.

 15" deep is VERY SMALL for a 4u computer case. 20"+ is a lot more common. The only 4u cases I've seen that were under 18" were for audio gear, and ONE computer case designed for a microITX motherboard based server.

 If it's intended for commmercial specific, I'll point out that EVERY Bitfury rackmount miner appears to have been 6U and somewhat longer than you're aiming for.
 IIRC at least one of the Avalon rack mounts was also 6U.

 It's only common at the breaker box and in the main supplying the box, NOT in the home itself.
 NOT the same thing.

 Having does more than a little rewiring over the years, I was FULLY aware of how power commonly arrives at the breaker box - but you can't plug a miner into a breaker box much less a main supply.

 I have more than a few cases in the 10+ pound range that are held only by the ears - longest ones around 24" deep.

 The ears can be QUITE strong, and as short as the case Sidehack is proposing is, it could probably hit 15-20 pound range with zero issues.

 My Nikko Alpha 3 is 4u, over 30 pounds, about the same length he's talking, the power transformers are in the BACK with the heat sinks hanging OFF the back, and that rack mounts just fine (the front panal is bloody near a quarter inch thick though).

 Dunno where that "must use a shelf if more than a couple pounds" nonsence is from, but it has ZERO basis in reality.

And how many typical homes have rooms wired for a >24A load between 3 circuits using standard 110/120V outlets? You're talking about having a 2600+W miner that is power-able in the typical home on 15A 110V circuits without re-wiring or a spider nest of extensions cords, keep dreaming.  My point is that 240V can be had for those serious enough to want this miner in their home, otherwise wait for the smaller S1 formfactor miner and power it with ATX. Even the S4+ went 205+V for input, likely because they had too many PSU failures with the S4's on 110/120V, it is inferior in all ways for powering PSU's.

You can't say 240V is not common, because it is "at the breaker" as you said, it just means extra work and/or expenses to be able to utilize it from the panel, 3-phase on the other hand is not common in North American homes. There's a difference...

If i'm not mistaken this miner is being designed for rack-mounting, efficiency and power-density, I'm not sure that people who rent their home is the target market here. The fan noise alone on this thing will probably make it prohibitive to have in a typical home anyways.

You've never shipped racks loaded with equipment or made equipment that has seismic ratings have you? Our company does. Try just using the ears even with thick front panels (vs ears just bolted to the sides) and it's not pretty opening crates. ref https://www.youtube.com/watch?v=cJVr94drRIs (https://www.youtube.com/watch?v=cJVr94drRIs) (not ours but is a clear example of what the gear can go through) Point is shelves take up very minimal height and make life a lot easier when installing/removing gear.

240V is typical to homes, but is in very limited use inside homes where 120V circuits are substantially more common. Would it take a bit of work to get a sufficient 240V circuit for this machine in someone's house? Very likely yes. Is that impossible? Very likely no. Y'all are both saying the same thing but somehow disagreeing. How the information is packaged doesn't matter as much as the information itself.

I would not ask anyone with a 30lb 20-inch-deep case to ear-mount only.

I think 18" case is the shortest design that's really been considered so far. S2, S4, S4+, Avalon2 are all in that neighborhood.

I'd like to keep considering Witrebel's idea of having the PSUs short-socketed with part of the supply hanging out the back. That does offer a solution to using differing lengths of supply (so long as they can fit in a 1Ux2U hole), fits 8 blades with sufficiently large heatsinks, provides empty space in the airpath for turbulence to normalize (which can also be used for piping in a waterblock installation), allows for ready access to PSUs without requiring partial case disassembly, and doesn't somewhat arbitrarily increase case volume with deadspace (which would increase material and shipping costs). The layout is also more conducive to a snap-in solution for external power access. My only real concern would be making sure the supplies were secured in their holes, which will require some work since the latches are always at the back and supplies aren't always the same size.

Agreed. This point has always struck me along the lines of someone wanting a good size welder or ceramics kiln at home -- sure you can do it. After you put in the circuits to feed them! If you want something powered from 110v 15a then one must accept the limitations (smaller units) that come with that.

Before I mock up any specific mechanical solutions for attaching PSU's securely that protrude from the rear, I'd like to define the objective.

I don't know much about PSU interchangeability so if you could clear up a couple things it will help the brainstorming. 

Do PSU's have any standard for the output blade?  As in, if you spec the unit with a DPS-1200FBA and its internal and flush mounted, and someone wants to put in a DPS-1200TB, does the unit interface with the same socket on the backplane? Or would each PSU need an entirely separate backplane? 

I ask because it would help to know how you plan to offer the unit in its stock form. 
What upgrade/modification paths you want to design specifically for and support. 
And what upgrades/modifications you want to anticipate and allow for, but leaving actual excretion up to the end user.

This would shape the types of solutions that make sense for securing the PSU's IE:

Stock unit with DPS-1200FBA supplies, with optional adapter/guide that interfaces a different length PSU to the same internal fastener, this would mean same backplane.
Stock unit with DPS-1200FBA supplies, with optional PSU bay section that includes a new backplane and mechanical solution for fastening the PSU unique to some other model of PSU.
Stock unit with longer PSU supploes and a stock secure method for attaching those PSU's, with an optional method for attaching the shorter DPS-1200FBA. 

Basically I think that actually securing a PSU in a socket is relatively trivial.  What isn't trivial is picking the most flexible solution that is compatible with everyone's needs and not overly expensive to produce.

The backplane pretty much just needs to be sockets fit for the PSU (which Emersons and 1200FBA are quite incompatible, not sure on the 1200TBA) and a few signal lines for load balancing, turn-on and PGOOD flags.

As you mentioned, feasible options would be to either construct interchangeable backplanes, specific to the model of PSU but mounting up to the same space, or to spec a single standard backplane and make slot-in adapters to go from the stock PSU to something else.

I had previously assumed using a different backplane for different PSU, but adapters isn't a bad idea. That reduces the replacement cost if you want to use a different PSU. The problem is, it allows different models of PSU to run on the same bus - which depending on their means of load-balancing and stock voltage setpoints, could be fairly disastrous.

If we make a stock channel size for the PSU (based around the largest practical supply) and when you buy a kit for a different PSU it comes with the power socket adapter and maybe a spring insert that grips the supply and fits tight to the bay slot?

One thing I thought of with your model is, the PSUs are upside-down. That does make the case depth a bit shorter because now the backplane can ride in the space above the boards, but I'm not sure how safe that'll be. Good power connectors are through-hole parts, and unless a lot of care is taken to make sure things are durably insulated, it wouldn't be hard for something sitting on top to flex the case lid into those pins and short something out. That can also cause mounting problems and airflow restrictions to your supplies.

Good concern but not to hard to take care of. Use a strip of electrical grade fibre paper contact cemented to the case area above the bare points.

So now you don't short your pins, but the potential for mechanical damage is not decreased. You also have no means of getting any standing parts out of the direct airpath of your supplies, since the backside of the board (where you could put any tall parts that aren't the PSU connector itself) is up against your steel ceiling. Immediately below the potentially somewhat congested airspace directly in front of the PSUs, however, you have about five hundred cubic inches of empty.

It might make the case a bit longer to put connectors at the bottom, but a board could be built that doesn't interfere with hashboards and doesn't pose a risk of shorting or mechanical damage to the case lid.

Also... here's a question. If we allow for multiple PSUs in redundant configuration, is it possible to also isolate PSUs to particular subcircuits so different PSUs could be used in non-redundant (which is to say, not load-balancing) configuration without a substantial hardware change? That needs to be addressed if someone wants to use external ATX supplies or any supplies, internal or external, that can't be put on a common rail.

It might be simple as having three separate internal busses (3 board, 3 board, 2 board + controller + fans) that would be heavy-jumpered for a shared bus or remove those jumpers for separate rails. There's probably other ways to do it, but that requirement will partially dictate backplane and internal power design.

I hadn't thought of using slot adapters, not a bad trick. It sounds like the backplane having integrated load balancing circuitry is a foregone conclusion?  I ask because there may be some small number of people wanting to piece together systems using different PSU's, and this rules them out if you gang the output from the supplies to a single 12v channel supplying 8 cards in parallel.  Perhaps this is where a factory wire harness option comes into play.  

Lets explore the standard backplane with slot adapters.  Oversize the PSU bay slightly as you mentioned to accommodate all practical options.  The solution to mismatched PSU's can be in the form of a mechanical system. Something intrinsic to each flavor of power socket adapter that prevents two flavors of socket adapters from being inserted adjacent to each other.  Not entirely sure how this would work for stopping someone from putting PSA A in slot 1, no PSU in the slot 2, and PSU B in slot 3.  But at that point someone is intentionally trying to be stupid.  

Edit:

Am I missing something about PSU layout?  The fan on the PSU pulls air in through from where the edge connector is right?  And if the edge connector is closer to the top of the case than the bottom, their is a metric crapload of room below for the any parts that need to hang down farther.  These parts are also directly in the intake path of the PSU.  And the edge connector for a slot adapter can still be nice and high for mounting to a backplane that's up on top of hash cards.  The fibre mat protects the pins from shorting, and you could build in some struts between each PSU slot to give the case top resistance to being crushed.  I think I'm missing something here with the PSU orientation....

ref https://i.imgur.com/rbcz0pw.jpg (https://i.imgur.com/rbcz0pw.jpg) for a good look at the backs of the DSP1200's. 7.2kw worth  :P
Very much like the psu's sticking out. Frees up apparent case depth for the power adapters. Highly approve of forcing load sharing. 1 (in this case 2) +n _is_ what the supplies were designed for after all... Another advantage is that the AC end of them happen to be the hottest parts of the case so good that it's catching the outside airflow instead of making their contribution inside the case.

If the connector is at the top of the case, the fans are right below the connector. In order to access the crapton of room below, your parts hanging down are directly in the way of intake air to the PSU. If the connector is at the bottom, you can still have parts hanging down but now they're below the PSU's airway in all that open space otherwise unoccupied behind the hashboards. This puts the backplane at lower than the top of the hashboards, meaning now the entire backplane has to be behind the hashboards instead of allowing it to be above them. This makes the case a bit longer (potentially) but also means you don't have to remove your PSUs and power backplane and power backplane mounting framework before pulling a hashboard for servicing or replacement.

If we don't do common-rail we lose redundancy. If we're not concerned about being able to load-balance the supplies, then there's no real reason to do a common rail - which makes using different or external supplies relatively trivial. I would prefer if load-balanced redundancy were still an option, but not if it makes every other option more cumbersome to achieve. It's probably not worth the trouble.

One way it could be done is with an intermediate power block connecting the backplane to the internal cabling. Say you have three PSUs 1, 2 and 3 (and each PSU has three cables to the intermediate block) and you have three board groups A B and C. If you keep cabling between the supplies grouped, so 1 powers A, 2 powers B and 3 powers C, now you can use whatever supplies you want and they never interact. But if you take one cable from each supply and tie to each board group, now you have common rail by default because A is powered by 1, 2, and 3 - and so are B and C. Two minutes with a screwdriver and you can switch between common and independent rails. Maybe there's a jumper on the backplane which ties the current-share lines. This also further modularizes the separation between backplane power and the cabling to the boards themselves.

THB, I feel screw folks that would want to use ATX supplies at these power levels. Use them on the possible 2-blade s1 style modules. Load sharing makes things so much easier. The HP supplies and their ilk are still pretty damn easy to acquire in multi packs and cheap.

I don't mind screwing ATX folks. But you end up screwing anyone with mismatched server supplies as well. If I got a machine with a down 1200W PSU and I have a different 1200W PSU I could drop in while waiting for replacements to ship and arrive, but I can't do that and now I'm down 4TH because the manufacturer mandated single-rail redundancy but I don't really care about it, well, that kinda sucks. And the point of allowing external supplies (which folks were arguing very heavily in favor of a very short time ago) includes allowing mix-matching external supplies.

I agree with sidehack on not mandating either option but providing for both in an economical way.  I think you pretty much have the connectivity down pat with your A B C grouping situation.  This means each rack unit can have a common backplane and wiring harness that can be purchased in bulk. 

I am not sure on the cost differences between a wire harness, a slot adapter, or a terminal block on the backplane, but I think you could also solve the problem of people wanting to go with external supplies. 

You can either offer an entirely new wiring harness for external supplies.
Or you could make the existing wiring harness attatch to the backplane with a connector, such that an additional wiring harness can expose the internal harness to the back of the unit by simply connecting in place of the backplane.
Or you could offer slot adapters with terminal blocks that could be used to adapt any supply to the backplane, leaving the internals hardwired.  This would also allow you to expose P_GOOD and current sharing signals to the external harness. 
A slightly more labor intensive option would be to simply build ALL backplanes with terminal blocks for tying in external supplies.  Although that may impose an un-needed cost on all stock units with internal supplies.  Either way, I think there is alot of flexibility involved with the whole concept of grouped boards tied to a 3 PSU slotted backplane.

Regarding the layout and flowpath for PSU's,  it sounds like there is little need for cooling on components populating the slot adapter?  I'm hearing that you want the most unobstructed flow path into the PSU intake.  How critical is this? How much flow do we need? Can that number be quantified?  I assume it's something along the lines of "Enough airflow to cool 600-700w of PSU waste heat at up to ~35 deg ambient intake temps". 

I would need to work the numbers and run some simulations but intuitively I feel like if the PSU's generate negative pressure in the PSU cooling channel, the flowpath doesn't need to be absolutely perfectly straight. 

I reworked the model with the PSU's flipped.  I focused on trying to maximize the PSU intake tract, again, without knowing how much this really needs to breathe.  As such, I envisioned the power leads coming of from underneath the backplane, routing to the outside, then penetrating through the top shelf to be routed to the front of the hashing cards, where they plug in.  This provides balanced flow to all the PSU's, and it only means a little bit of cabling in the front of the PSU duct.   Tradeoff is you make two seperate areas for the controller, unless they end up living in the hot zone between fans and hash cards.  How hot do those exhaust temps tend to get?  Is it an issue that the backplane compenents are hanging down into that hot zone now? Would they get cooked?  My original thought was that you'd want absolute separation of the hotzone from all other components.  How critical is it that the components above the heatsinks on the hashing card see airflow?  Could they be in stagnant/low flow air? There is alot of space above the fans that could be sectioned off and made into a cooler space, but this would restrict airflow that comes in between the tops of the hashing cards and the heatsinks.

https://i.imgur.com/drXVpqz.jpg
https://i.imgur.com/ESOCYkn.jpg
https://i.imgur.com/BWgio71.jpg

Thoughts?

heh heh heh, ja I still like an external power bank but if they fit, well why not keep them(semi) internal. As for converting to using an external power rack, would be just as I do now -- run heavy gauge power feeds to split into short PCI cables near the cards. No use of the PSU adapters needed. Since the cards use PCIe for power in - is a non-issue to me at least.

As for mis-matched supplies, THAT is a problem but should not necessarily be yours... Drop-in replacement DSP1200 supplies abound with many sellers doing Amazon next-day Prime. Or, just have a few from a multi-pack buy  kept as spares. One has to draw a line somewhere.

IBM 2880W!!!!!!  1 power cord, 1 breakout board, 1 PSU, 80+ platinum, all for ~$50-70 with fan packs.  I realize I am biased because I sell boards for them, but I am biased for a reason.  They are the best...

Can't disagree with that. Aside from the redundancy aspect not sure why Sidehack is hell-bent on the DSP1200's but that is what is currently on the board so we follow his lead.

As it stands it looks like the PSU's will be sliding into a sleeve spot welded onto the case lid so it should be an easy change to use other supplies... And again, since the hashboards use PCIe power connectors, this should not be a sticking point.

After all, gotta leave _something_ for folks to tinker with ;)

If you want to take the machine apart and run your own cabling, taking cables in through a PSU slot directly to the boards would certainly be possible. If you didn't, a simple insert that plugs into the PSU socket on the backplane and has either screw terminals or PCIe jacks accessible sticking out the back of the case would do nicely.

I would rather see common-rail redundancy be not an option at all than see it be the only option. It'd be a good feature to have, and there are certainly situations when you'd see improved performance, but it leaves at least as many situations when split rail is desirable. With a bit of thought on flexible internal connections, any number of configurations of 1 to 3 PSUs in common or separate rails is possible from the same simple hardware.

The most recent render actually fairly closely resembles our original idea, except that PSUs are now behind the cards in a deeper case instead of right over top. Having that top chamber with a fixed separation does require more disassembly in order to get cards in and out. Hm...

What could be done with keeping about 2 inches of width at one end of the case wherein resides the controller and such? You could mount your cabling interconnects at the top of this, and your backplane terminates at that point. You could put any required tall parts on that end of the backplane board so there'd be no restrictions at all on hanging parts in the airpath of an inverted supply. Simple cabling interconnects between backplane output and main cabling input will give you the ability (as previously described) to combine or isolate rails as desired. Something like that could work.

I'm not hell-bend on DPS1200 so much as on a fairly standard 1Ux2U server PSU. Please note every time in the last five pages I've given consideration to a design change requiring accomodation for three different common models in that approximate dimension. I'm fairly set on 4U height, which takes the IBM 2880 out of consideration as an internal supply (and there are numerous other reasons already documented) but also note that every time in the last five pages I've given consideration to a design change I've also required accomodation for interfacing to external PSUs including the 2880. The first post mentions we're working around the DPS1200 because that's what we have, but from the outset it was intended that provision for other PSUs, internal and external, was essential.

Not if the chamber is part of the top of the case.

To me easiest way to hold the PSU's is in a sleeve that is part of the case and welded to the rear panel. That sleeve could also serve as a mounting point the power backplane. Thinking on it, having the sleeve as part of the top as I mentioned earlier means longer cables from the PSU's to the boards and not good...

I think the sleeve idea is very much what we are thinking, at least as you describe it and I picture it, I can't speak for sidehack.  

Regarding the 2 inches or whatever.  I think I see what you are getting at, I will play around with configurations like that next.

Again, it would be helpful to know how much flow you want the tops of the hashing cards to see, the part where there is no heatsink.  Does this need to be entirely open to to the hashing space airflow with little/no obstruction? There is ALOT of free space if you drop the hashing area flowpath to just about height of one fan and leave everything above it as fair game for circuits, cabling, or tall parts the don't need significant heat dissipation.

Now its probably a little late in the conversation for this, but I do want to ask.  What specifically is the reason for the S1 compliance?  As I understand it you would need to at a minimum replace the hashboards and controller.  Do you envision the actual S1/3/5 heatsinks being used on your upgrade hash boards?  If not, then you are basically building all dimensions of this rack unit around the ability to strap the same hash cards on an existing frame and fan unit.  Is that really worth whatever design sacrifices you make to achieve it?  Just playing devils advocate here, thinking that if we are trying to define a long standing form factor that will be applied to both 2 card standalone miners and larger rack mounted multi card miners, then we really ought to be certain that we define it right.  My understanding of the argument is basically backward compatibility = more adoption/sales? And perhaps the fear that existing waternblocks couldn't be designed into a new non S1 compliant frame standard.

Fuzzy, yeah a sleeve mounted to the back panel and used as structure to hold the backplane is exactly what I'm thinking. The top of the case should be an entirely independent panel, I think, for ease of digging into the works.

The top of the hashcard above the heatsink would probably be home to all the tall parts. Any through-hole or tall SMD caps, power jacks, interface or control chips, if you got VRMs in your design (that aren't super-large) they probably go up there. With a bottom-cooling chip your PCB is right up against the heatsink and you have clearance to adjacent for tall parts, but if they're at the top you have your ~2" clearance to the next board instead of the ~0.5" clearance to the next heatsink. I'm assuming you won't need a lot of airflow up there, and somewhere between 0% and 10% of your heat would be generated there.

Conversation about rolling with S1 dimensions is probably around page 2. Making a single board that works for both rack and small units means someone could design one product instead of two and fit both markets. There's already a lot of S1/3/5 chassis out there waiting to be messed with, and waterblocks for that formfactor are also pretty common.  The S1 design has proven itself pretty well, from almost silent running in the S3 to pushing pretty good power density with overclocked S5. I'm comfortable considering that board size and heatsink layout as a decent home-miner standard, and because of the opportunity for compatibility both from boards and waterblocks, I'm comfortable using that board size in a rack machine as well. The taller board certainly requires changes to case layout, as we've seen in the last several pages, but I think the long-term benefits of compatibility are worth the hassle now.

I took a stab at your 2in side channel idea.

It definitely has some layout advantages but you do sacrifice some heat sink to get it.  I ran three sets of numbers, all assuming a straight 2 inches of width wall to wall inside that side channel for controllers.

If you leave 0.50 inches between each circuit card/and the next heat sink or wall, you can have a maximum PCB - heat sink height of 1.29".
If you go with 0.475 inch clearance between cards, you can get 1.32in of heat sink
0.40 inch clearance gets you 1.4 inches of heat sink.

So you could definitely play around with the actual width of that side section, card clearance, and sink height, and get something workable.

This is what it looks like as I interpreted your idea.  Shown is the heat sinks at 1.32" tall with 0.475 card spacing, for what its worth.

https://i.imgur.com/Z4w0ocb.jpg
https://i.imgur.com/GiKZ741.jpg
https://i.imgur.com/lkoW7Vq.jpg

Fair enough on the S1 compatibility. 

 Common in kitchens. Not so common elsewhere. Also, I'll point out that 24A at 110 is easily handled by *2* circuits, most common US house outlets being on at least a 15A circuit. *2* circuits in a room are more common, even some of the MOBILE HOMES I've owned came with a seperate circuit on each side in at least the living room, going back into the 70s when I started paying attention to that sort of thing.


 Never disagreed with that point, just pointing out it wasn't common in homes without rewiring work.
 Kinda redundent debate at this point, sidehack having mentioned he has no interest in this being intended as a home miner.



 
 1u x 2u = 1.75" x 3.5" (actually a hair less due to case thickness). Figure if your "track" is 1" before you widen it, need 5.25" or so worth of width to feed each PS.




 Load balanced redundancy is a PITA to try to set up with PS that are not specifically designed for it. Takes a fair bit of additional circuitry to make it work reliably. I'd skip the whole idea.

Is the panel above the hashboards necessary? I can see how it'd be undesirable to suck in hot air from the hashboard exhaust into your PSU for "cooling". I ask because having a panel there will make accessing cards more difficult. The panel would have to be cut short or punched out for power and signal connector access, but now you have a lot more disassembly required in order to get to your hashboards for installation or maintenance. If it's necessary it's necessary (and given the kind of heat those cards will be generating, it's probably necessary) but that's going to make other things more difficult.

Could we make a kind of duct enclosing the backplane - maybe even open at the top for ready access but more-or-less sealed by the case lid - through which the PSUs can draw fresh air in through that two-inch side space instead of from the hashboard airflow? You could isolate the PSU intake from the hashboard heat without requiring a separator panel above the hashcards, which will keep access to them pretty easy. You wouldn't have to worry about getting your cabling out of the way before removing the separator to get to your hashboards, which might be handy if you need to do something to one board without shutting down the other seven. You also aren't limited to keeping your PCIe jacks in a predefined area of the board to be accessed through fixed punchouts.

That two-inch controller space can probably be shrunk without causing a lot of problems. If a small SBC were mounted vertically and the USB/interface board put above it (basically, both boards flat against the side wall) that space could probably be more like one inch. Some room for ventilation would need to be left open if this was providing cool air intake for the PSUs.


Load-balancing redundancy is indeed a pain for supplies not designed for it. If it's possible to allow load balancing for supplies with internal support for this, without making the option required (which means also allowing split-rail operation of non-balancing supplies) and also without requiring a lot more expense or complexity, I'd like to keep it possible. If it's too cumbersome to rig up a setup allowing for both configurations, I think split-rail should be kept instead of mandating a match set of load-balancing PSUs. However, probably some refinement on the two-point cabling idea presented earlier should allow the end user to decide between split and common rails without requiring complex circuitry.

Okay the ant miner s-7 dropped today.

Specs are unreal 4800 gh at 1200 watts

Box looks like 1/3 of an s-5+

I will suggest everyone look at it closely then come back to this thread.

This  s-7 is like the size of a s-5 and do more then 4x the hash. For 2x the power

Okay. Bitmain built another overpriced jet turbine and now there are even more perfectly good S[odd] chassis about to be retired.

My opinion is, 2x the power in S5 volume is asking for trouble - especially if it has the same firmware bug of excess heat generation on network dropout. Instead of melting a bit it'll probably go full Prisma.

What do you suggest we do differently in a rack machine because Bitmain built a 1.2KW "home use" miner with no innovation, dual 4200RPM fans and three hundred tiny glued-on heatsinks? Overall, I'm fairly disappointed.

Looking at the flowpath, I would advocate for at least the rear portion of the ducting, perhaps including the angled bit, and ending just at the back of the hashing cards, or maybe extending an inch or so over them.   I feel like this would help the flowpath significantly, but this is totally based on intuition and opinion, and I would strongly recommend we attempt some actual CFD simulations before settling on the exact flowpath layout.   High level though, I would say you could loose the panel over the cards.  I don't know how much those PSU's need to breathe, so its hard to say if they could pull all their intake through the side space, but it doesn't seem impossible. 

I also think it depends what sort of negative pressure the 3 120mm fans end up creating inside.  You wouldn't want that negative pressure fighting your PSU fans too much, but with the entire front face meshed and open, I doubt that would be a serious issue. 



As for the S7, the overpriced part is true, and while the jet turbine part is also true, I think it depends who you are building this miner for.  And lets not forget the fact that you aren't just designing a miner, you are designing a standard/form factor that is to be backward/forward compatible.

If you are designing for the home mining guys who want a few of these in the room or a room over from where they sleep, with wives and such, then yeah, jet turbine solutions don't cut it. 

But if you are designing this towards the economies of profitable bitcoin mining, then I think some allowances need to be made in regards to noise.  I am beginning to see the light about denisty, in terms of hash power per unit of volume. Chip efficiency held constant, the only way to increase this is increase the thermal capacity of the unit.  Which means either waterblocking, immersion cooling, or more airflow/better heat sinks.  Not saying that 2.2kw in a 4U case isn't dense enough.  But it certainly isn't as dense as the competition is trending towards. 

I tend to think of Spondoolies first when I think of power density, and I've never had one run nearly as well as advertised because they were so finicky about heat. Add the order of magnitude complexity increase because of their 100W BGA idea to insane internal temperatures and they're actually sorta terrible. There comes a point where density murders reliability, and reliability is one of my primary design requirements.

If we can safely put more heat into this machine (on pure aircooling) without losing reliability or becoming a fire hazard, I'm okay with that. Waterblocking is a secondary concern, but also one of the reasons to go with S1 board size - any C1 waterblock or aftermarket watercooler would drop right in and not require a new design there. That product already exists and is pretty good. Home mining is also a secondary concern so noise is a secondary concern - but it shouldn't be deafening either. 80dB fans is kinda stupid any way you look at it and pushing mechanical components to that extent probably sorely limits lifetime.

If the portion of panel over the rear of the cards is shorter than the gap between the cards and front fans, a board could be unscrewed, slid forward and lifted out without requiring anything else to be moved or disassembled. That's great. Keeping that rear portion in place (the angle and under the heatsinks) makes sense too as it keeps airflow pulling toward the fans without a lot of turbulence around the supplies.

How do you propose to power 3x PSU's evenly between 2 15A circuits?  2x PSU's on one circuit would make for 16A, and I'm not aware of a legal/safe way to balance 3x loads between 2x 110V circuits.  Not that this is relevant, but for discussion's sake.

You're right, it's not relevant. It's for the customer to figure out.

My take on the s7 and then 'nuff said about it:
Ya I will get at least 1 after they start delivering and we hear more about them. As to their pricing... well my first s2 was around $1,250 as I recall and the 1st s4's the same so, not too terrible for first release.

I will probably finally retire my last s2 that's on-line and few S3's to free up power for the s7(s). Also have 2 s4's that are starting to act up by dropping boards from time to time...All in all it is a very good balance on power/replaced hash rate but kills me to take ANY perfectly good miners offline :(

On the hot turbines -- that exhaust fan will need to to watched as most are NOT happy in the hot side and their bearing life tables reflect that. They should start looking at selling the s7 in pairs and use a dual-squirrel cage blower for them. I happen to have one in the shop and will do just that :P
We now return to our regular programming.

I don't believe that any of their problems are due to heat at all, if ambient is high the fans scale or the frequency / voltage scales down. The real problem is the custom controller's high failure rates out of warranty, the entire setup is crazy complicated.

They're so dense that it's difficult, if not impossible, to keep one running at top speed - which is to say, advertised rated speed. The overall complexity of their designs is definitely pretty bad. I mean I like that they give you plenty of information on what power is going where, but 200A 4-phase bucks into BGA chips with a custom protocol in an FPGA on the control board is not a recipe for reliability even if your stuff doesn't run 110C.

That's why this proposal intends to keep things relatively simple. Increasing complexity simultaneously increases both the odds of failure and the difficulty of maintenance - not to mention the initial cost.

I feel that a large part of that problem is their rating as well as the design.  Bitmain rates miners at a reliable level of clocking, which can be maintained more or less indefinitely.  SP-Tech -especially notable with their SP20- have ratings that for all intent and purpose were un-achievable, and completely high strung.

I do a bit of advising for a guy in a somewhat equatorial region, who bought several SP35 earlier this year. He set them up in a climate-controlled room with AC venting directly into their intakes and still can't sustain more than about 90% rated speed from the lot. Meanwhile it's not too difficult to get 20% over rated speed out of an S5 without any problem. Yeah I'd say there's something wrong with their rating system.

For the machine design being proposed here, I'd like to see at least 260W per blade stock rating sustainable in 30C ambient. It'd be nice if it could handle 320W per blade without a lot of danger. I know some of the danger is going to depend on the board design itself, low-resistance chip-to-sink connection and such, but a lot of it is also the design of the heatsink itself. I'm looking forward to seeing some of Witrebel's simulations. If we can get the basic layout hammered out, the next step is building a sturdy heatsink which can perform inside the specified dimensions.

Work has picked up a bit as I get ready to enter port and pack up all our gear. 

I will have to investigate what simulation packages I have access to through our license pool, but I have a feeling to do this right I might need to get some third party software like COMSOL or FLUENT and export our models geometry into one of those multiphysics packages.  I really hesitate to post anything I have from the "express" simulation packages I have on board as I doubt their completeness (I get outputs like average velocity  over the entire flow path, as opposed to seeing turbulent pockets etc). 

I just don't want to promise the world regarding simulations and then not deliver, as there is both the issue of gaining access to a reputable CFD/multiphysics package, as well as the learning curve associated with getting useful data out of it. 

I will attempt to get another model rendered up reflecting the latest ideas before I get into port tomorrow. But it will probably be a week at least until I have a better handle on what I can actually produce simulation wise. 

As for the density issue, I agree with you sidehack.  Reliability is key, and furthermore, I like 10 x 10watt chips alot better than 1 x 100watt chips.  On that note if a chip goes down in string topology, does that mean the entire string MUST go down?  Or are their any possible failure modes that don't result in loosing the whole string? 

One thing that could be implemented to maintain a string is I think something ASICMiner attempted on the Prisma - basically, FET bypass. You could put basically an analog voltage-detection circuit on each node that monitors the chips' core voltage per node and drives a bypass FET in parallel with the chips to maintain that core voltage. I'd probably use a simple op-amp circuit with a threshold reference of about 1.1* an externally-generated Vcore (basically, a fixed resistor divider across your total voltage giving you expected per-node voltage plus ten percent) driving the FET. If a chip drops out, the current through that node drops and the voltage spikes briefly - it'll be partially buffered by your node-level caps. If the op-amp detects the node-level voltage goes up to 110% of expected value, it'll kick on the FET which will start to draw excess current through it and buffer back down. The issue with this is your bypass FET is now sinking the entire power typically burned by an ASIC, meaning you're still running the same amount of power but doing 1 chip's-worth less work.

That system worked for ASICMiner because of how the comms worked on the BE200. Each chip was on a buffered parallel bus with individual addressing. One chip turning off didn't affect the operation of any other chip, at least directly. It wouldn't really work as well for BM1385, depending on the failure condition. BM1385 (if I understand correctly, and it was the case for BM1380, 82 and 84) relay comms, so the first chip talks to the second which talks to the third and so on. If one chip drops out in a way that it stops relaying comms, everything downstream from it will also turn off. A bypass FET to pick up the slack of a downed chip is okay if the chip is still talking but not hashing; in the event of a downed chip not talking, everything downstream will stop hasing and every node will be in full bypass mode - every FET will have to sink the power of all the chips on that node, and probably burst into flames pretty quickly.

That's one reason I'm hoping, if PlanetCrypto can get a chip dev project going, he'll go with parallel comms instead of relayed. It'll require a bit more work for the node-level comms shifting, but by fully parallelizing everything you increase overall reliability. If you were running an S1 (definitely not string) and you blew the first VRM, everything would stop hashing because the first 8 chips turned off and now you couldn't communicate with the last 24 which were still working perfectly. Compare that to an AM Tube, where you can smoke as many VRMs as you want and everything else keeps going.

You might ask PlanetCrypto what he has access to for simulation packages. Novak's last job centered around doing CFD for jet turbines so I know he knows a thing or two but I really don't know anything myself.

Interesting...

So help me get on the same page regarding the basic power distribution of these systems.  You have your PSU, which outputs some voltage, nominally 12v.  Some supplies may have the ability to tweak this voltage slightly?

On an S1 this 12v goes to a VRM (a buck converter?) that drops the 12v to your core voltage, which is then seen by 8 chips in parallel, so if any one goes down, the other 7 chips would still see power, unless the chip fails in such a way that it short out the supply, and then you loose all 8 chips.

In a string topology, I assume the chips are powered in series? And this is where your "total voltage" comes from, which is where you would use the voltage divider to get your 1.1*v_core_excpected?  Is the on resistance of a MOSFET significant at these levels and hence the power loss through the MOSFET?  I guess I have a hard time understand why you can't just use your voltage control to lower the total voltage across the string by 0.7v or whatever node voltage.  Then you bypass the chip and the rest of the string keeps working?  Please pardon my lack of knowledge here, I'm just trying to wrap my head around the whole thing. 

I guess the bypass FET only works if the chip fails open or stops hashing - conditions where current passing through the chip are substantially reduced. If the chip burns up and fails short, the voltage across that node is effectively zero.

Let's use S5 voltage levels as an example. It's easy. There are 15 pairs of chips and your total voltage is 12V, so each pair (the chips in each pair are in parallel with each other) sees 0.8V (12V/15). Each chip draws 12A at 0.8V to get 22GH.

Say one chip fries short; now instead of passing 12A at 0.8V alongside its partner (also passing 12A at 0.8V), the fried chip passes 24A at approximately zero volts. You now have effectively 14 pairs, so your per-node voltage increases from 12/15 to 12/14 or 0.86V. If you detect a condition like this, and you can control your total voltage, you can lower it to compensate. If your chips don't relay comms from one to the next, the rest of the chips might keep working without a hitch.
However, with Bitmain chips, if the chips drop out enough to no longer communicate properly, they can't relay work to other chips and they'll turn off. This upsets the current balance of the whole string and it stops working.

Say one chip fries open; now instead of passing 12A at 0.8V alongside its partner, the fried chip passes 0A at 0.8V. The circuit as a whole still needs to pass 24A for the other chip pairs to work, so the voltage at the damaged node will increase until the one functioning chip is passing 24A - either because now its core voltage is so high it's actually drawing that much current to operate, or (more likely) because it fries. If you have a bypass FET, when it sees the node voltage increase (because the apparent resistance suddenly doubled by taking out one chip, remember V=IR) the FET will kick on and start sinking 12A. This will bring the total current back up to 24A without breaking any more chips, and the node voltage will return to the 0.8V it's supposed to be at. Instead of two chips each hashing at 12A, you have one chip hashing at 12A and one as a dummy load at 12A. Again, if your chips don't relay comms the rest of the chips will keep working, but if your down chip can no longer relay, all the downstream chips it should be feeding work to will turn off and the string stops working.

The on resistance of the MOSFET is actually the part being actively controlled by the op-amp circuit. It'll be operating in the linear region (instead of switching between off and saturated) as a high-power variable resistor. The power loss through the FET will be the same as the power loss from a working ASIC. It has to be so, because for the rest of the system to work properly, there needs to be a chip there sinking 12A at 0.8V - either an ASIC, or a dummy load.

If you can actually reroute chained comms around bad nodes entirely, you could use a switched FET to draw all the current through a node and drop its node voltage to near zero (Inode*Rdson) which would basically turn off both chips whether one still worked or not. You'd then need to drop your total voltage by one node's worth.
For example, say you have six chips in three nodes: A+B, C+D, E+F. Comms go A->B->C->D->E->F. If chip D suddenly drops out, you can bypass the entire node by shorting it out (which turns off C+D) and redirecting comms so you now have A->B->E->F. It's definitely possible, but not really easy. You'd have to have a way of very rapidly determining which node wasn't functioning right (which I guess could be done with node voltage threshold measures - if it gets too high or too low, disable the node and latch it off), and then rig up multi-channel switches on every node. If bad-node voltage was almost zero, the level shifters from the last working node would still work for the next node up, but you'd need a set of dual-throw switches on every node. Possible, but cumbersome. Course now I'm interested in actually designing it and seeing how bad that would be. If it's only a matter of a couple bucks per board it might be worth putting in, since you'd be chopping a lot of the more probable board-failure conditions to reduced-capacity condition instead.

I guess the feasibility of the system is entirely dependent on the probability of chip failure which disables the entire board without the system. Economically, if the percent increase in the cost per board is greater than the probability of the failures it prevents, it's actually a net loss for the customer. On average.

So I assume the 2 chip per node layout is used to reduce part count on the VRM side?

I wonder if doing 1 chip per node and some fet protection circuitry on every chip would increase costs substantially?

Furthermore, it sounds like this is pretty much only relevant to a new chip design with parallel comms.  Unless of course you ran multiple strings on one board and called each string a separate device, again raising complexity and parts count. 

Any idea what the probability of failure actually looks like for these chips?

Lastly, not sure on the cost difference and board layout implications, but you could also offer this as a manual repair solution.  Simply place bypass jumper headers near each chip.  In your example, with 2 chips per node, failing closed would result in the auto voltage reduction at the source, but failing open would take down a string/board.  You could simply pull the board, adding a jumper to force the node closed.  Low tech but it gets you hashing again quickly.

Could work. If you're already doing board-level soldering, all I'd really need is jumper pads to reroute comms. It'd be best to pull the chips so you don't get comms interference, and once that's done you jumper from VCore to GND pads where the chip used to be. You could also lift the big node-level cap and jump across it. Pads are pretty much free, after all.
Actually, my two-chip "L board" for testing BM1384 is set up a lot like that. I have two power-input jacks, one for Vcore and one for 2*Vcore, and a set of five jumper pads to take comms from either a second chip at the same node (on Vcore) or a second chip at another node (at 2*Vcore). Heck, it's hooked up and running right now (http://eligius.st/~wizkid057/newstats/userstats.php/1BURGERAXHH6Yi6LRybRJK7ybEm5m5HwTr should be seeing 22GH, two chips in a string at 200MHz) so I know the concept is sound.

You'd have to remember to take your core voltage down by one node's worth or your remaining chips will run pretty hot.

On the S5, there are no VRMs at all. 2 chips per node gets them the desired power consumption and hashrate; it's a 30-chip board, just like the S1 had a 32-chip board. Using more chips per node increases your total current (which is one reason the S7 pulls around 400W per board versus the S5's 250W per board, because it has 3 chips wide instead of 2) but you also get better balance. If one chip is running a bit high but another on the same node runs a bit low, they kinda cancel each other out. It's easier to buffer out brief transients with a wider string because any one chip's ripples will be absorbed by the other two chips.
Wider nodes also reduces the number of level shifters required compared to the number of chips. Each node needs a level shifter to bring comm data up to its local ground reference (and other node-level parts for IO voltage and such). The S5 has 15 nodes and 30 chips, so 1 shifter per 2 chips. The S7 has 18 nodes and 54 chips, so 1 shifter per 3 chips. When considering the cost of parts that aren't directly increasing your hashrate, you want to maximize the ratio of ASICs to non-ASICs. This means more ASICs per node.

The optimal from that criteria would be to have all chips in one node, but then you have a VRM design and you start factoring in the relatively high cost of VRMs. The more chips you have per VRM the better, so things like the S1 were okay. The standard chip for VRMs has been the TPS 53355 which has a maximum current output of 30A, which is great for higher-voltage lower-current chips like the BM1380 on the S1, but not so great for the low-voltage high-current BM1384. At top clock, a '55 could power two chips. At midrange setting (say, 275MHz - 15GH/5.4W per chip) you could just barely run four but it'd probably catch on fire if your ambient was warm. The S5 would have needed 15 VRMs (at probably $5 each minimum) per board to run the same hashrate and those VRMs would have decreased the system efficiency by between 10 and 15 percent. By going string on the S5, Bitmain ended up saving $50 in VRMs by adding about $15 in additional node-level parts, and increased the board-level efficiency by at least 10%.

If you can keep the chips in recommended temperature and voltage range, I'd think the odds of failure would be pretty low. If an auto-rerouting system cost $10 for an otherwise $150 board, you'd want a probability of board failure greater than 10/150 or 6.7% for it to really be feasible in the long run. I highly doubt the odds of board failure are that high or we'd be seeing a lot more threads yelling at Bitmain. I'd be surprised if the odds of Prisma board failures was even that high, and those were famous for spontaneous (and often dramatic) death. I ran 44 boards for six months without any failures.

Initially I was thinking through hole style jumpers or zero ohm resistors with sockets or something physical that a non board level type person could manipulate, but why have a crapload of parts that you hope you never need to use. 

I mean realistically, you could just make a small "re work" section in the PCB art next to each asic that had comms lines readily available for an x-acto knife, and then just a couple of pads set up for a shorting Vcore to ground and rerouting the comms with a nice some nice crude solder blobs

Do the chips have any on board voltage regulation at all for the core voltage? Or do you need to drop your total input voltage, say from 12v to 11.2v?

Chips have zero voltage. It would be possible to set a "cut here" zone to disconnect local comms and solder pads to jump to the next node could be populated by either solder blobs or 0-ohm resistors. That's pretty much how my L-board works - except instead of "cut here" I just don't put a chip on the second pad in that node so there's nothing for it to talk to anyway. With a "cut here" you wouldn't have to worry about pulling chips, which reduces the "tools required" from a hot-air station to a $5 firestarter.
Leaving big fat pads to short around can get dangerous, because if you accidentally short them when you don't want to things could get screwed up ala Prisma. If you've got a soldering iron, knock off the node-level caps and short across their pads. Wouldn't take but a minute.
Total increase in PCB cost plus parts cost plus additional assembly, $0 - sounds like a win to me.

The chips have zero means of regulating voltage. The thing that keeps the node voltages at about 0.8V is that all the chips are operating at the same frequency and doing the same mount of work - they're all pulling the same amount of power, so the same amount of current is going through each node, so the voltage divides evenly just like it was a string of equal-value resistors. If one chip starts to misbehave, you can think of its resistance changing (either up or down depending on the error) which means the voltage across it changes and that affects how much voltage is left over to divide among the others in the string.

If you bypassed one node, you'd have to drop your board voltage by one node's worth. If you went from 15 nodes to 14 but didn't change your voltage, now your nodes get 0.860V instead of 0.800V and they start to cook a bit.

I am far from certain that the complexity of adjusting the supply sting voltage and bypassing the data string is worth cost? What I do think is worth doing would be to detect a chip that is not responding and to shut the board down to prevent further damage.

What would then be useful would be to identify which chip has failed. I do not know if the chips identify and are allocated addresses during power up, but assume this to be the case?

Then perhaps some pads  & cut areas that would enable comms to bypass the failed chip and to attach a suitable value of resistor for the power if chip replacement was not possible?

Rich

What you're talking about would be even more expensive. The material cost of a "cut here" bypass is zero dollars. The complexity of adjusting the supply voltage is already on the board; I refuse to build a fixed-voltage miner.

If you want to replace your chip with a fixed-value resistor, you need to replace that resistor every time you change your clock or voltage settings. It's not practical. Also, .05ohm 10W resistors probably aren't cheap.

Since now we're at board-level discussion, this should probably be moved to a board-level discussion thread. I think we should get back to the machine specs.

I have been wondering for a long time how the string setup worked reliably without any signs of a shunt regulator across them. So it is only because all chips are more or less doing the same operations at the same time that keeps the voltage divided evenly... Elegant solution to eliminating the need for VRM's provided all the chips have identical tested specs or apparently at least fairly close ones. Is also a dicey solution in that all must chips be doing the same operations so their loads are identical. I do hope that Bitmain is binning the chips to keep all in a chain reasonably matched!

I like your idea on adding mosfets to do just that for when a chip goes down. I assume that the bypass FET will be switched on hard for minimum losses? As you said only problem is that when a chip goes down hard ya lose coms from that point. In hardware mode, as you said - give the associated cap a push to pop it off and jumper across the pads. Perfect solution (until too many get tweaked).

Along those lines.... Looking at Bitmains spec for the S7 they call for 12vdc OR MORE. Up to 5% more.
With the s5 string setup I wonder how much neg margin we have vs stock clock speed? Bitmain had said the S5 could be fed as low as 9.7v when underclocked. On most of my rigs I usually read 12.1x at the supply and 11.92v at the PCIe connectors. The HP/IBM psu's do have a voltage trim/remote sense sooo.
Do you know how low the supplies can be set (for underclocking/volting) and how high when compensating for drop across power leads?
What pin terminal is the sense/trim input?

I've seen S5 node-level voltage variations of about +/-4% from nominal. The S5 init code ramps the chip work up slowly from zero to full-speed so there are no substantial differences in workload from one chip to another. The Prisma had node-level FET buffering but they got rid of it on BE300 test boards because they weren't having any of the balance issues they saw on Prisma. I don't know what chip-level differences there are between chips designed with string in mind versus parallel/VRM. I do know the 1384 appears to be internally shunted to about 1.2V, which might be a deliberate design choice to keep current at least close to balanced in a failing sring so you don't end up blowing caps and catching things on fire.

It is a certainty that process variations will result in chips that have slightly different electrical operating specs. How much different - dunna know but it can be substantial ergo my hope that they are testing/binning for best matches in a string or at least the chip pairs. Nice of them to think of the need to clamp Vcore. Passive shunt using 2 silicon diodes or zener?

A good discussion about DC power connectors and the process of selecting them. The course is 49min total http://www.techonline.com/asset/download/4437224/course (http://www.techonline.com/asset/download/4437224/course)
Some timestamps for it
A--Slide 1: Introduction and setting the stage 00:00
Part B--Slide 8: Misconceptions, examples, and basics 04:20
Part C--Slide 15: Starting the process 10:00
Part D--Slide 26: Selection process: theory & reality 18:20
Part E--Slide 41: What are other concerns?  33:20
Part F--Slide 46: Working with your supplier 39:10

And one from Intel on PCB design http://www.techonline.com/electrical-engineers/education-training/courses/4000356/Fundamentals-of-PCB-Design (http://www.techonline.com/electrical-engineers/education-training/courses/4000356/Fundamentals-of-PCB-Design)

You seen this about connecting hash boards via debug pins that are on them? https://bitcointalk.org/index.php?topic=889206.msg9902302#msg9902302 (https://bitcointalk.org/index.php?topic=889206.msg9902302#msg9902302)
It's a simple 3-wire serial port meaning that with a port-selection method of addressing the boards eg COM1, COM2, and such it could be translated with a multi-port rs232/USB switch(s).
Hmm. Easy way to get away from a shared SPI com bus?

Isn't that post talking about firmware on the controller, or am I reading wrong? Also I don't quite know how anything in your last couple posts has anything to do with anythign.

Especially if we're wanting to steer back to the actual topic of discussing mechanical design instead of board-level, which excepting the mechanical constraints of the PCB is completely outside this thread's intended scope.

Ja is rather OT from the mechanicals I guess. Figgered the connectors vid could apply to PSU backplane connections though, the debug maybe as alternate way to address chips  :P
And now back to regular scheduled programming.

Ah, okay. I see what you're saying on the power backplane.
Chip addressing shouldn't really be an issue how I see it, since each board would enumerate as its own device in cgminer if they were USB-connected. No one board should overlap with another. Or are you thinking a multiplexer on each board that talks to each node independently?

It would work if certain boards could be fabricated as one single form factor, with just a "parent" board with USB logic and the rest just plain old chip addressing. This would be just an option for a given manufacturer, so no need to enforce it.

a bit late in the game but - on packaging...
The rack mount format is great for aesthetics but -- most farms just place the miners on industrial shelving eg. https://bitcoinnewsmagazine.com/wp-content/uploads/2015/03/hashnest.png (https://bitcoinnewsmagazine.com/wp-content/uploads/2015/03/hashnest.png) Just tossing that reminder out there.

On bolting blades to the bottom of case: Will they be pan heads recessed in stamped dimples or tapered heads going into tapered holes? Can't have screw heads scrathing whatever is under the miner...

For a mfg standpoint I'd use stamped dimples. Many (most?) sheet metal fabricators have presses/laser cutters that can do the blanking/dimpling/louvers/cutting all on the 1 machine. Dimples will of course raise the blades higher by around .125" or so...

I had assumed a flathead screw in countersunk hole. If we go dimple recessing, there should still be plenty of clearance between the cards and the top of the case.

I know a lot of folks shelf miners, but there's not really a good reason to make it not rack-mountable just in case.

It would be nice if a blade had a mounting hole in each corner and that the total height of a blade was at least 13-15mm less than 4U (or whatever "U" you decide on), so that buyers could mount blades in the grooves of slotted extrusion, etc.  :D

-Rich

Full disclosure: I used to design and build open-air cases for most of the custom mining boards, and I miss it terribly.   :'(

EDIT:  What are the dimensions of a S1 board?

Quick update,  I am back stateside on dry land.  Haven't gotten a chance to model the latest version but to recap what I think I am still missing:

Remove the panel from over the hashing space, leaving only a small section towards the back to promote laminar flow.
Decrease the width of the sidepocket to 1" and rework the heatsink sizes to match
Add dimples and verify that the clearance is still good above cards, im thinking 0.125" dimples?

I was able to get access to a license of COMSOL with a good selection of modules, so now its just a matter of familiarizing myself with this enviorment and getting the models imported correctly etc.  I hope to have some results inside of two weeks, ideally much sooner.

Cool. I'm on the COSMOL and also Mentor Graphics email lists so am very familiar with their offerings but so far have not needed to actually use them. Common-sense and Industrial usage overkill which assumes worst-case to rather be the norm suffices :P

Schneider Electric is another with most focus on heat flow in the rooms (data center cooling) and power distribution. Witrebel, sent ya a pm re Schneider and yer Venezuela project.

Anywho, if you have access to a seat in any of them then most excellent.

Hehehe.... my country ;) Btw, Schenider electric has its offices near my city, if anyone needs directions, ask away.

Btw, i know the case is going to have a mainly fan cooling design, and it would be cool if it would be possible to also add mounting option for ducts, so one could direct the airflow outside the case more easily.

Yes, miss your work.  Gone are the day of open air cases, there's just too much heat generated by asic chips.  But keep thinking of
converting your talents to new products.

Yes, I understand that open-air cases no longer work with today's asics and I've expanded to building fully enclosed cases, for example:

http://www.spotswoodcomputercases.com/images/frames/5slotcase/5slotwCompletedTopLeftAngle-640.JPG (http://www.spotswoodcomputercases.com/images/frames/5slotcase/5slotwCompletedTopLeftAngle-640.JPG)