Bitcoin Forum
November 14, 2024, 07:58:39 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 [3]  All
  Print  
Author Topic: How to get a 220-240 k-watt-meter in the usa  (Read 592 times)
senseless
Hero Member
*****
Offline Offline

Activity: 1118
Merit: 541



View Profile
February 03, 2018, 03:48:43 PM
 #41

To make life easy, I took some pictures for everyone - it's going to vary depending on what kinds of cables you've got, but you get the idea:


Jesus, that's a lot of work. Why not just get a switched PDU?



https://www.cyberpowersystems.com/products/pdu/switched/

Switched PDU's are too expensive and usually is not for huge loads, 10a, 20a and 30a for so much money is not worth. Sanoff for $10 for 16a is cost benefit.

I guess I'm looking at it more long term. Being able to auto reboot down miners, send notifications when necessary to physically inspect certain miners for issues, optimize your miner without needing to stand near it to do so, and monitor 3 phase balances.. Ya, mining hardware changes, but since i've been around in 2011, mining as been a thing, and it appears it will continue being a thing. Might as well build decent facilities and minimize manpower requirements.

MarkAz
Hero Member
*****
Offline Offline

Activity: 687
Merit: 511



View Profile
February 03, 2018, 07:01:21 PM
 #42

Jesus, that's a lot of work. Why not just get a switched PDU?

Actually, I've got a local friend who I helped setup a jig and a bunch of the tools who does them for me - I've been pushing him to offer them up on the forums for other people who are interested, so hopefully he will at some point.

The big advantage over a traditional PDU (and you can't see them, but we have Tripplite and APS ones in the background), is that it also gives you fast watt level per port monitoring.  Cost-wise it means we can skip buying an ethernet enabled port-metering PDU (which are expensive), and instead use a bunch of these and cheap distribution PDU's.  Plus with the port-metered ones I've worked with are generally low resolution (like .1 amp) or have slow update rates (like once a minute) - and they also tend to use SNMP instead of JSON/HTTP, which is really much easier for us to work with.

What we've made instead is like a single distributed PDU - so we can monitor per outlet live all the device, and define more advanced rules.  I also use the TH16 as well, which is their switch that you can add a temperature sensor onto, and scatter them around to give us a realtime view of the temperatures in the building, plus can use these to automate all of the cooling.  For sub $100 you can completely automate your evap system - one TH16 for the dump pump on the evap cistern, one TH16 on the recirculating pump, one TH16 on the fan.  Then I can do rules that you wouldn't typically be able to easily do - like I can program the system to dump water when it goes above a certain temperature, so really like a partial dump as the city water will then cool down the remainder in the cistern.  Really it's pretty amazing what you can cobble together using these things.
senseless
Hero Member
*****
Offline Offline

Activity: 1118
Merit: 541



View Profile
February 03, 2018, 07:13:19 PM
 #43

Jesus, that's a lot of work. Why not just get a switched PDU?

Actually, I've got a local friend who I helped setup a jig and a bunch of the tools who does them for me - I've been pushing him to offer them up on the forums for other people who are interested, so hopefully he will at some point.

The big advantage over a traditional PDU (and you can't see them, but we have Tripplite and APS ones in the background), is that it also gives you fast watt level per port monitoring.  Cost-wise it means we can skip buying an ethernet enabled port-metering PDU (which are expensive), and instead use a bunch of these and cheap distribution PDU's.  Plus with the port-metered ones I've worked with are generally low resolution (like .1 amp) or have slow update rates (like once a minute) - and they also tend to use SNMP instead of JSON/HTTP, which is really much easier for us to work with.

What we've made instead is like a single distributed PDU - so we can monitor per outlet live all the device, and define more advanced rules.  I also use the TH16 as well, which is their switch that you can add a temperature sensor onto, and scatter them around to give us a realtime view of the temperatures in the building, plus can use these to automate all of the cooling.  For sub $100 you can completely automate your evap system - one TH16 for the dump pump on the evap cistern, one TH16 on the recirculating pump, one TH16 on the fan.  Then I can do rules that you wouldn't typically be able to easily do - like I can program the system to dump water when it goes above a certain temperature, so really like a partial dump as the city water will then cool down the remainder in the cistern.  Really it's pretty amazing what you can cobble together using these things.

It seems like you have a pretty substantial setup..

Have you looked at 415/240? You can push the savings by not having step down transformers and getting transmission voltage to 415/240 service. Then you can get 24kw (usable) tripplite PDUs for around $43.75/kw. This give you switched per outlet monitoring and you don't need a step down to 208/etc (415 -> neutral = 240v). The only issue is the ISC, you need to use in-line fuses to reduce ISC at the PDU. Still, you're only talking another $50/kw for wiring, 480 distribution panels, fuses and a custom transformer at the multi-mw level.

Planning 415 for my next build.

MarkAz
Hero Member
*****
Offline Offline

Activity: 687
Merit: 511



View Profile
February 04, 2018, 12:02:35 AM
 #44

Have you looked at 415/240? You can push the savings by not having step down transformers and getting transmission voltage to 415/240 service. Then you can get 24kw (usable) tripplite PDUs for around $43.75/kw. This give you switched per outlet monitoring and you don't need a step down to 208/etc (415 -> neutral = 240v). The only issue is the ISC, you need to use in-line fuses to reduce ISC at the PDU. Still, you're only talking another $50/kw for wiring, 480 distribution panels, fuses and a custom transformer at the multi-mw level.

Planning 415 for my next build.

Hmm, I haven't heard of that service level, but maybe they're calling it something different here.  For all of our mines we've been running 208v, primarily because it's MUCH easier to find 208v/1600a buildings than anything larger, and so our building cost is much cheaper.  The issue we ran into the few times we've looked into bigger buildings is that usually the power hasn't been used a few years, and the allocation at the power company is gone, so they won't/can't actually deliver that anymore (no capacity left at the closest substation).  We've kind of setup most of our buildouts so as much of the infrastructure as possible can move with us, as we've moved from buildings before - so generally speaking we add a couple panels 400a panels, we use 30a breakers, and those are wired directly to L6-30R.  Then we have high quality extensions that we run out to the racks.  On the PDU side right now we have relatively expensive ones, because we've only been using the Sonoff's for a few months, but in that time we've been sold on them.  Right now our case per Sonoff is basically $30 (that's the unit, wired with C13/C14 extension, reflashed with our modded firmware), and each of our machines is about 1kw.  We also run TED units on the panels themselves, so we're monitoring both at the miner, and also at the panel, so we have a really solid grasp of how power is being used.  We also even go through everything with a thermal camera once a month to make sure that nothing is running abnormally hot, etc - super easy to do, and can really save your bacon.

I do know one other large miner who is running 480v, since each leg is 277v he's able to run directly off of those, as most of the server PSU's are tolerant up to that voltage.  Pretty slick and he's been doing it for years without issue.
puwaha
Sr. Member
****
Offline Offline

Activity: 700
Merit: 294


View Profile
February 04, 2018, 12:11:10 AM
 #45

To make life easy, I took some pictures for everyone - it's going to vary depending on what kinds of cables you've got, but you get the idea:


Jesus, that's a lot of work. Why not just get a switched PDU?



Ahem...   I wrote up a solution....  Puwaha's Poor Man's Networked PDU: Using Smart Plugs and Awesome Miner

https://bitcointalk.org/index.php?topic=2866608.0
markurian
Newbie
*
Offline Offline

Activity: 2
Merit: 0


View Profile
February 05, 2018, 02:03:56 AM
 #46

To make life easy, I took some pictures for everyone - it's going to vary depending on what kinds of cables you've got, but you get the idea:

http://www.analogx.com/images/gpu/img_1664s.jpg

And as a little bit of rig porn, here they are in production:

http://www.analogx.com/images/gpu/img_1663s.jpg

I mount them with nylon spacers on one of the PSU screws - works like a champ.

Hey Mark,

That's a damn fine setup you got there.
I'm wondering where you manage to source your D1800s and their respective cases? I'm wanting to migrate my three 6-card rigs from open-air to enclosed boxes for co-location.

Cheers,

Markurian
Pages: « 1 2 [3]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!