Request for Discussion: proposal for standard modular rack miner

[philipma1957]

I'll say this a third time. The problem is physical dimensions. There's only two ways a DPS1200 would fit in a 4U case, and it takes up either 3.25 or 3.5 inches of horizontal space. This leaves less than 14 inches of width for fans, and 3x 120mm fans are more than 14 inches long. Therefore, impossible. It's not that 1" of fan would be blowing on the PSU. It's that 1" of fan would be coexisting in space with a portion of the PSU. If you want 3x120mm fans, you cannot mount your PSUs in a way that they're accessible from the outside - hence the three alternatives I posted.

110V on the DPS1200 gets you 900W, sure. That's still 3/4 power if you only have 110V available. But if you only have 110V available, why are you buying a 2400W rack-mount miner? I'm more than willing to ignore that small percentage of potential customers.

you have to realize the img below is true
then you just make that 2400 watt miner be able to down clock to 1700 watts on a low/110 volt setting

[Finksy]

Of course the problem is physical dimensions.  The amount of things you have listed originally do not fit in the physical dimensions you had planned for, therefore it's time to consider alternative options.  Either increase the height to accomodate upper-mounted PSU's, cut back the amount of blades to 7 and shrink the heatsinks by a small amount, or have the option to run an external PSU.  I'm a fan of individually sourcing external PSU's, as you can save a lot of money looking for used/refurb units as opposed to having to buy new over-priced PSU's from the manufacturer.  Look how much of a premium you pay for PSU's when buying SP1/3x or S2/4's, that's because brand new server PSU's are expensive and it is not feasible/practical to source used/refurb units for production volume.  When you shift that "problem" on to the customer, everyone wins IMO, and it's up to them to be as frugal/resourceful as they feel appropriate given their circumstances.

The PSU's should probably have their own partial enclosure, and I don't see having part of a 120mm fan blowing on part of that enclosure as much of a problem for airflow, especially given the potential distance between the fans and the enclosure (the DPS-1200 are only ~8" long + backplanes/cables, and there is 15" between the inside of the front and rear fans).  My render had allotted ~3.5" for two vertically oriented, side-by-side DPS1200's with clearance to mount them into a breakout board/backplane.

I think you can safely assume that 110V is on the way out even for any "home" miners, not just by your own standards.  It will be interesting to see how the S7 turns out, to see if it's still marketable to "home" miners, or if they start going larger form factors similar to S5+ for all their new models.  

Edit:  How many people really have spare 25A 110/120V circuits in their home anyways? 

[sidehack]

When you say the amount of things I have listed originally, do you mean in the first post? Because I acknowledged then that it would require tweaking of dimensions to fit. I also already had an increased height for top-mounted PSUs. If we want to shift to S1-formfactor boards (which there are good reasons to do so) we'd have to either shift to a 5U case or put PSUs on one side.

The problem with your render is, now it's impossible to plug the PSUs in. You've opted to demonstrate my PSU location suggestion #2 - PSUs recessed inside the case and no longer accessible from the outside, which has its own problems as noted. It's either difficult/impossible to plug them in, or you need additional internal mains cabling to fill the gap to a pair of sockets mounted on the case wall.

I like the DPS1200 because they're not difficult to find for not expensive second-hand, and are nicely power-dense which allows more room for hashboards and such. Being as the innards would connect to PCIe for power, it wouldn't be hard to build a wiring harness with an external connection for running your own external PSU. I'd rather not design that as the defacto, because people seem to like being able to buy a box that needs no further toying-with to make it work. Included PSUs as a base default, no PSUs and wiring harness to external connections as an option.

[Finksy]

As for PSU's being inaccessible, I forgot that the DPS1200's use C13 to connect, Im so used to the 2880w style with AC input as well as DC output all on the same side using blade connections.  Either way, if you ran internal cabling to a bulkhead AC plug on the case, you could use a single C20 outlet to power both PSU's, cleaning up the power cabling.

Yes, I was basing my render on the dimensions of your first post but with the PSU side-layout from last page. I'll leave future renderings to Chig, they're not my strength.

Side note, what about building the case tall enough to house a vertically oriented IBM 2880? They're No wider than 2 DPS 1200's stacked sideways, completely hot swappable with the rear connections and have beastly fans already built in. Plus you could power 8 boards @ 300w each.

[sidehack]

I haven't done anything with CAD since about 2007, so don't feel too bad about not rendering.

Actually, a single C20 would be pretty handy if PSUs were internalized. Good idea.

With 3 internal DPS1200 PSU (or similar) stacked vertically and tied to a single external socket, one could get either higher power (overclocking and such, probably require cold ambient air or waterblocks) or full redundancy for stock loads as previously mentioned. This also gives an end user the option to circumvent the socket adapter and plug the PSUs in directly to 110V circuits (which would take a few minutes and a screwdriver, but not really a difficult process), still yielding well in excess of 2KW total (non-redundant) power. This reopens 110V home-use potential, and also by internalizing the PSUs provides ample room for using 3x 120mm pull fans which will surely be quieter than 8x 80mm fans.

If the PSU partition dimensions can be specified properly, it wouldn't be difficult to see making multiple interface board packs to fit the same space which opens up PSU options. I talk about the DPS1200 because it's what I have access to, but someone like pmorici has those Intel platinum PSUs (I think) which would also do the trick, or like the Emersons from SP3x gear. If the spec is done with enough consideration, it'd be fairly easy to present a variety of options that meet the same space, each with its set of pros and cons for any particular user.

As of right now I'm in favor of seven S1-sized blades with three push and three pull 120mm fans, with an internalized space for PSUs sized to fit three stacked DPS1200. If the same space can be made to accomodate a few other popular PSU models, I would like to see that done. Provision needs to be made for an externally-accessible 12VDC wiring harness but that's fairly trivial.

[MCHouston]

I believe I may have a few intel PSU in a box some where, if you need them for sizing etc. I will donate them to the cause.  I would love to see the PSU's modular as well with only having to change maybe the PSU interface board and of course the PSU.

Good to see a decent sized miner made for the masses. 

[sidehack]

I may take you up on that. Probably got some other stuff to discuss as well, one of these days.

The community has been needing decent gear for the masses for entirely too long. I'm really hoping we can get enough support for basic standards and especially for PlanetCrypto's independent chip endeavor, which should give the tinker community bitcoin was founded on something to do for a while instead of just waiting for the tailings from various corporations to filter down to us.

[Finksy]

Just for reference sake, IBM 2880W Platinum dimensions (with fan packs mounted): 17" x 8" x 2.5"

3x DPS-1200FBA (80+ Bronze?) minimum dimensions without clearance for board: 7.75" x 4.5" x 3.5"

The IBM 2880W would require a 5U case, but allow more room for the hash boards.    What dimensions are the platinum DPS 1200 TBA's? They would be promising, but more expensive. Considering how much we rely on ASIC technology to push the efficiency envelope, I'm amazed how many people disregard the ~5-10% efficiency that can be gained by switching lower quality PSU's out as well as moving to 220/240V.

[philipma1957]

Just for reference sake, IBM 2880W Platinum dimensions (with fan packs mounted): 17" x 8" x 2.5"

3x DPS-1200FBA (80+ Bronze?) minimum dimensions without clearance for board: 7.75" x 4.5" x 3.5"

The IBM 2880W would require a 5U case, but allow more room for the hash boards.    What dimensions are the platinum DPS 1200 TBA's? They would be promising, but more expensive. Considering how much we rely on ASIC technology to push the efficiency envelope, I'm amazed how many people disregard the ~5-10% efficiency that can be gained by switching lower quality PSU's out as well as moving to 220/240V.

I have a plat intel from pete running in my friends office  they use 110 or 220   dimensions in 1 minute or 2


40 mm by 73.5 mm by 265 mm

  1.58 inches by   2.90 inches by  10.44 inches

http://www.deltaww.com/Products/CategoryListT1.aspx?CID=010104&PID=ALL&hl=en-US

DPS-1200TB  next to bottom of the page

[sidehack]

I'd rather not shift to a 5U if we don't have to. The S1 board is over an inch shorter than 4U, so pushing to 5U would just mean more wasted space at the top where there's nothing but cables.

I also have no problems at all about using more efficient PSUs, which is one of the main reasons we started building server supply interfaces to begin with.

[sidehack]

Sent in a PM:

I'd rather not shift to a 5U if we don't have to. The S1 board is over an inch shorter than 4U, so pushing to 5U would just mean more wasted space at the top where there's nothing but cables.

I also have no problems at all about using more efficient PSUs, which is one of the main reasons we started building server supply interfaces to begin with.

Hey, I don't want to have any clutter from back and forth banter in your thread, I just want to re-iterate one last time my opinion, and leave it at that.  It's your project.

As you may know, cost-effectiveness is going to be the single most important factor to make your project a success (and in turn to provide any kind of real benefit to the community). Not sure if you're thinking about 2 or 3 DPS PSU's, but If you go with 3x DPS-1200FBA's, the cost of PSU's will be minimum $75, and I'm not sure I feel that having 3x PSU's in a miner make it any more reliable. I feel like it increases the likelihood of having a failure (which could easily go un-noticed, leading to further PSU failure due to over-loading the remaining). It's not redundancy if you need all 3 in order for it to run reliably.

Also, as I've mentioned, efficiency of PSU's should not be over-looked, and although running the DPS-1200FBA would be convenient due to availability, their lack of efficiency would not fit very well with your goal of creating an efficient miner worth re-using.  If you were to switch to the DPS-1200TBA Platinum supply, you're looking at nearly triple the price over the FBA.

On the other hand, the IBM 2880W is also proven reliable, Platinum rated, and can be had for $50-$70 each.  They are also much more common and available than the DPS 1200TBA.

 

If the power spec is 2400W, three 1200W PSU is definitely redundant. It's only not redundant if you're running them off 110V, which is why I specifically said that'd be a non-redundant configuration but still actually possible. I assume there'd be an indicator for PSU functionality which would alert you to a downed supply. I think redundant supplies is probably not essential anyway, given that no miner yet made with an internal PSU had that as an option and nobody seems to complain.

There are a lot of good options for PSU, all weighing against availability, efficiency and price. I don't like the idea of resizing the case to fit one specific PSU that's not shaped like any other PSU ever, because if we want to make more of them than we can find that exact PSU in the secondhand market then we have to abandon the standard and go with something else. If it's designed in such a way that any of half a dozen existing PSUs which can be acquired new or secondhand will work without significant mechanical alteration, I'd rather see that option as the default.

Especially considering it won't be difficult to bypass any internal PSU and plug in whatever PSU you want as an external.

[QuintLeo]

 I'm amazed how many people disregard the ~5-10% efficiency that can be gained by switching lower quality PSU's out as well as moving to 220/240V.

 Higher quality PS cost a lot more up front, unless you're willing to risk buying used units.

 220 isn't anywhere near as common in the US as 110. Most HOME miners have ONE 220 outlet available (for their dryer), many don't have any.
 You pretty much have to OWN your own home to be able to add 220 outlets, and most folks have no clue how to wire them up - at which point you're talking expen$$ive electricians to be ABLE to add any 220 outlets to your home.

 If you intend for this to be a HOME miner, ASSUME 110VAC not 220VAC or plan to lose most sales to the US (I believe Canada also defaults to 110, but not 100% certain).

I don't like the idea of resizing the case to fit one specific PSU

 You seem to already be sizing the case around "one specific PSU".
 That would be the one advantage of specifying 2 standard ATX power supplies - even at the 1200+ watt level there are quite a few options - though they're a lot less convenient to make FIT into such a small case as you want to use.

[sidehack]

If by "sizing around one specific PSU" you mean "any of half a dozen existing PSUs which can be acquired new or secondhand will work without significant mechanical alteration" - since the 1Ux2U server PSU is a very common dimension - then yeah, I'm sizing around one specific PSU.

I also don't really consider 4U rack to be a small case, given that it's a taller dimension than any decent rackable gear built in the last year.

Home miner is a secondary consideration for rack gear.

[Finksy]

Higher quality PS cost a lot more up front, unless you're willing to risk buying used units.

Buying used server PSU's is not much of a risk.  Out of the hundreds I've had a part in with either supplying or supplying boards for, there have been 2 failures that I have been made aware of, and I have received 2 DOA units. The real risk is buying an ATX PSU, new or otherwise, as they are complete rubbish compared to server PSU's and it's not a matter of if but when they will fail (after you have paid more for a PSU that provides less).

220 isn't anywhere near as common in the US as 110. Most HOME miners have ONE 220 outlet available (for their dryer), many don't have any.
 You pretty much have to OWN your own home to be able to add 220 outlets, and most folks have no clue how to wire them up - at which point you're talking expen$$ive electricians to be ABLE to add any 220 outlets to your home.

Actually 220/240V is the MOST common power supplied to homes in North America.  It's called split phase (single phase), and nearly all homes are powered with it in both Canada and US.  They split the single 220/240V phase into 2x 1/2 phases of 110/120V each at the panel, meaning anyone can have a 220/240V outlet made up by combining opposite phase 110/120V circuits.  Yes, I do recommend everyone get a certified electrician to do all the work, and all that other liability-ass-covering crap.  I believe that home miners should be looking to making the switch, because the few miners that will still be available to purchase will be moving to 220/240V anyways ala S4+ and larger.

Sidehack has already mentioned using the boards for a different form factor similar to S1/3/5.  For the less serious home miners, they would be best to wait for that and power it with 120V PSU's.  But to sacrifice a significant design element (large server PSU's) for a relatively small market would be silly.

[Witrebel]

I have never mined industrially and have never operated a data center/rack mounted setups, so pardon my potential ignorance here, but I have a question about the form factor.

Do mining operations really have an issue with overall hashing density per sq ft?  Is there a driving need for the entire unit to occupy a 4U space?  Or could other options be explored without reducing the value of the unit?

It seems like the layout is a key issue here, centering around balancing  variables: Number of boards, Number/size of fans, and arrangement of PSU's.  The ideal goal of the current discussion appears to be somehow fitting 8 properly cooled hashing boards alongside a robust PSU set inside of a 4U case. But the end goal of the discussion is a "standard modular rack miner".   It seems to me like a great number of the existing units on the market are powered externally.  And integrating the PSU with the hashing boards restricts modularity due to layout concerns. 

I envision something along the lines of a 3U hashing rack that has 8 boards and the 3 X 120mm fans layout. 
Alongside this would be offered "Power rack" that could be 1 or 2U.  I imagine this may add some cost in terms of additional cabling/chasis/wiring, but it does solve the modularity issue, and it gets you back in the 8 cards per rack with 120mm fans ballgame.  Just an idea, I haven't really looked at the actual unit dimensions so I could be missing something obvious here.

[sidehack]

My opinion is, make the unit fully self-contained (having its own PSUs, controller, everything required to operate) but allow the option for external power. If you have no provision for internal power, it requires everyone (including folks who just want one) to source PSUs independently. That might not be a problem, but it can be cumbersome and it's pretty universally standard so far that rack gear has internal supplies. Sometimes those supplies suck and we wish external power was an option. With an internal supply the option for external power can still exist (with a simple wiring harness and external jacks, as previously discussed) - and now you have choices. If you dictate no internal power, it can never be an option.

The 4U height currently is required for the preferred board dimension. I think, and others have agreed, that using S1-size boards is good; even without considering cabling, an S1 board is about 3/4 inch taller than maximum allowable height for 3U. Blades could be laid flat (like in the S4) but that makes it difficult to access or maintain bottom-layer boards, and to fit more than about six boards you'd have to put them in rows which reduces air cooling efficiency (as hot air from front boards is trying to cool rear boards) and makes fitting waterblocks and plumbing very difficult in addition to greatly increasing the case depth. Dropping to 3U height is possible if we want to use a different size board, but the reasons for not doing so are fairly numerous.

[Witrebel]

I agree that a turnkey solution should be offered.  I am just suggesting that it come in the form of 2 physically separate units, both of which are produced and sold by you.  One would be the hashing unit at 3U, the other would be one or more forms of a "Power Rack" that is built specifically to run 1 or 2 of the 3U hashing units.  Anyone wanting turn key can simply purchase them both.  Anyone who has existing power or wants to use alternative power sources can buy the hashing rack without purchasing the power rack. 

Regarding dictating internal vs external power, I don't think there is much of a difference between a turnkey 4U unit with internal power or a turn key "3U hashing unit + (1U/2U) power unit".  At least not from a consumers perspective. 

The power rack could be build to allow hot swapping for spoondoolies fanboys. It could come in more or less redundant variations.  The point is that it would be a more flexible form factor/standard than an integrated system. 

Regarding fitting 8 cards in a 3U space, I understand that it is exceedingly tight.  Perhaps I am missing a key dimensions that makes it impossible, but skimming through I see the following numbers.

3U space: 5.25in Nominal, or 133.35 mm

Proposed S1 form factor card height: 5in nominal, or 127mm

Proposed chassis thickness: 2mm per, or 4mm overall.

Assuming nominal dimensions, you get a stacked height of 131mm assuming the hashing boards touch the chassis.  This affords you 2.35mm of margin for tolerance and making sure your equipment actually fits in a rack.  Admittedly this is tight, but I don't think that is impossible.

Can we shave a mm or two on the card height to raise the margin?

[sidehack]

S1 card height is actually 5.9 inches. Ignore the proposed card dimensions from the first post as they're something else entirely. If S1 card height is to be the upperbound for allowable dimension, this will not fit in a 3U chassis with vertical cards and especially not when power cables are considered. As stated, the only way to get 3U is to either change card dimensions or not have vertically-mounted cards (which makes mechanics and thermal considerations more convoluted).

Regarding dictating internal versus external power, I think there's a substantial difference between a 4U unit with the option for either internal or external power and a "4U hashing unit + (1U/2U) power unit". Takes up more space and costs a lot more on account of now you have to build a second case and all the cabling required both internally and externally. As a manufacturer, as a miner and as a hoster of mining gear, it doesn't really make sense to me to do it that way.

With provision for internal PSUs, you are not mandated to use internal PSUs. A factory-option wiring harness (which can be sold separately for refitting machines later, hooray interchangeable parts) connecting to externally-accessible jacks would probably be cheaper than the set of internal PSUs and their backplane board (and would be required for your idea anyway) and it still gives you the ability to do what you propose with external power - but it doesn't remove the option to have a self-contained unit, which a lot of people appreciate. I argue for the option that incorporates - does not mandate, but at the same time does not exclude - what you'd like to see.

The internal power housing could, with small changes and replacing the backplane board, hold any number of PSU models - we've discussed 3 just in the last page. Or if you want no backplane board and the externalizing harness, you can wire it up to server supplies or ATX or whatever you want in whatever configuration. I'm fairly certain that's more flexible being as it includes all the flexibility of your proposal and still allows for a self-contained unit without requiring a separate box for power and with various options for internal supplies as well. The only thing it can't do is 8 blades in a chassis, which your 3U proposal can't really do either.

[NotFuzzyWarm]

I still rather like the separate psu rack myself. Sell the miner and require that one of the 6-slot psu cases be bought (and preferably enough HP1200's to power it). Something like what is shown here from TRC http://www.trcelectronics.com/Meanwell/hot-swappable-rcp2000.shtml Then when they get a 2nd miner they are all set, just get sockets/cables (and psu's) to fill the caase and hook up the 2nd miner. Seems pretty painless to me and frees up a lot room in the miner case.

[Witrebel]

Ahh, I knew I had to be missing some key dimension.  

The 3U suggestion was predicated on 5in cards that had "rear" facing connections, so they could be just ever so perfectly fit in a 3U case.  If a 5inch card is incompatible with the S1 form factor then throw my 3U suggestion out the window.

I hadn't really given a great deal of thought to backplane vs no backplane.  I was really just considering possible high level layouts.  It may not be feasible, but I envisioned that a "Power rack" could be built in a number of configurations.  Backplane if it was to be hot swappable, or wiring harness if it was to be cheaper and more simple.  

I think the separate PSU rack really depends on a 3U hashing unit.  At 4U I see the value in having the option to run it internally powered, and that probably offsets the loss of the 8th hashing card.  Is the S1 Form factor incompatible with a 5inch card? And even if it was, does 8 cards still not work in a 3U because of the controller card? The more I think about it, the more I realize that 8 hash boards in a 3U would be an impressively tight squeeze if it could even be done at all.  

This brings me back to my original question...

"Do mining operations really have an issue with overall hashing density per sq ft?  Is there a driving need for the entire unit to occupy a 4U space?  Or could other options be explored without reducing the value of the unit?"

I mean this appears to be aimed at some level of industry, vs the home mining market.  Do you guys that have hosting services and mining operations really feel the squeeze when it comes to space?  Is that a limiting factor in your operation?  At 4U you can fit I believe 10 units on a full rack, at 5U its 8. So for the same footprint, assuming you run a full rack, you have either 70 hashing cards at 4U, or you have 64 cards at 5U.   Thats less than 10% difference in the overall hashing density.  Is that significant to you guys?