13 GPU rig. Lessons learned (the hard-way)

[inrockitrust]

Hello,

Been mining since nov 2017 with 3x 13 GPU rigs (39 GPU, mostly 1070, 1080 and 1080ti). I'm sharing my two cents for anyone interested:

a) Invest time in prep-work! what is the layout? which frames? how many PSUs? how many splitters? this was done, but I underestimated the prepwork. I ended up buying more PSUs with less wattage, were (in my case) fewer PSUs with more wattage would suit better the layout. The prep-work is extremely important and vastly underestimated, especially the layout, # of PSUs units, and # PCIe splitters (so everything fits nice and tidy).

b) if using windows, CPU and RAM DOES MATTER!!! try using afterburner in a 13 gpu with a celeron and 4gb of ram... it is a pain in the ass! switch to i3/i5 with 8gb and is a cool breeze. For example, when using celerons I was getting a delay in everything. Move a slider, 3 second delay, decrease voltage, 8 second delay. For mining this is not a problem, but for tweaking, it is. Imagine spending 4 hours tweaking your rig when you could have done so in 20 minutes (because of delays).  I have not use any other OS, so I'm sure this is a windows thing only.

c) tweaking =  free GPUs (in a sense): with 39GPUs a small increase in efficiency goes a long way. You teak a GPU for 2% gain and thats ok, tweak 39 with 2%-5% increase in each and you will find yourself with "1 or 2 free 1080ti gained by tweaking".

d) heat is an issue: oh yeah it is, specially if you live in the Caribbean... fortunately I found a well ventilated place where temps are usually at 27c (this is cold for my country). Good ventilation can reduce the temps by a long, long way.

e) 220/230V is mandatory: see point c, is the same concept. Higher efficiency in one PSU is nice, Higher efficiency in 10 PSUs is nicer .

f) chasing the dragon is bad idea: I spent months mining new coins trying to chase the next pump. Bad idea, the only good coin I mined that mooned was GIN coin and I sold it 10x cheaper than its ATH. It would have been better for me to just mine ETH with the pill or any other oldschool/popular coin (as equihash before the ASICs).

g) stability vs profitability (another chasing the dragon): I've also spent many a night tweaking my rigs just to increase a small  percentage, this turned fine until I went to work and my rigs were frozen, which means wait till the night to reset them, lower the OC and bring stability. In the long run, stability will always win (I can't use smart plugs or "resetters" for this as I have frequent blackouts in my country and smart plugs or "resetters" DOES NOT turn back on after a blackout, they need a manual input), if you know a solution for this, please let me know.

Anyway, just wanted to share this. I have a lot of more learned lessons, but for now this will suffice as I got to get back at work! hopefully it will be useful for new miners.

Cheers,

[1715111716]

1715111716

[1715111716]

1715111716

[1715111716]

1715111716

[1715111716]

1715111716

[1715111716]

1715111716

[1715111716]

1715111716

[revenant2017]

In the long run, stability will always win (I can't use smart plugs or "resetters" for this as I have frequent blackouts in my country and smart plugs or "resetters" DOES NOT turn back on after a blackout, they need a manual input), if you know a solution for this, please let me know.

You can enable the AC POWER RECOVERY/AC RESTORE AFTER POWER LOSS under Power management in BIOS. It will still depend on your motherboard if it's capable of doing this. Most newer version of Motherboard have this kind of option. I also highly advised using UPS when you turn this option on. As the Voltage is higher than usual after power outage. Could damage your components.

[remauto1187ma]

In the long run, stability will always win (I can't use smart plugs or "resetters" for this as I have frequent blackouts in my country and smart plugs or "resetters" DOES NOT turn back on after a blackout, they need a manual input), if you know a solution for this, please let me know.

I also highly advised using UPS when you turn this option on. As the Voltage is higher than usual after power outage. Could damage your components.

Your Power Supply (PSU) For your motherboard/GPU's doesnt care how high or low your incoming power to your house is as long as its within a rather large window.  The PSU is a regulated power supply.  It will keep DC at 12vdc going to your GPU very close to 12vdc even if your incoming house power is at 240vac, 218vac and even up around 255vac.  The owners manual will tell you what the exact spec for your particular PSU is but its a wide window.  The PSU will shut itself down if the incoming is too low or high outside of its design specs.  UPS is not needed but nice to prevent bumps to the motherboard but useless in the sense that your GPU's arent on a backup so they could hangup, freeze or just totally throw up once power is restored. (You gonna put your motherboard on its own dedicated PSU?) Otherwise the GPU's will be drawing power from the PSU (Ultimately the UPS) when the power goes out and it aint gonna last long.

[KriptoGuruTR]

a) The most important thing for me is prep work. Especially powering. After several rigs I have decided to only use server PSUs for 12+ GPU rigs. I do all the cabling and power distribution myself.

b) IMO Windows is simply not good for 8+ GPU rigs. Booting and stability problems all the time.

c) Tweaking is tricky with 12+ GPU rigs. I stay on the safe side with overclocking. IMO downtime vs tweaking, downtime hurts you alot so not worth going for %99.9 performance. %95 performance is perfectly fine for me.

d) Heat is a big problem for 12+ GPU rigs. Lots of chassis for such rigs do not have enough spacing between GPUs.

g) Totally agree with you. Stability always win. Downtime is the worst thing for mining. Stable rigs are less likely to have hardware failures. I go for low power, less heat.

I use UPS for expensive rigs (1080ti or RX580/Vega). Sometimes a voltage spike or a thunder can damage hardware. A UPS protects your rigs for such things. I stop all the mining, safely shutdown rigs under UPS power. It is not for downtime prevention. Not cost effective.

[gotminer]

UPS's for that many gpu's isn't cost effective.  That is a lot of wasted money in my opinion.  

If I have a power outage, the rigs come back on with the power with the bios setting mentioned above.

Agree with you on MS Windows.  That OS was a pain in the ass for me with Pentium processor's and 8gb ram.  Switching all of my rigs (with the exception of my Vega rig) to SMOS was the best decision that I ever made.  Makes it very easy to switch coins, power settings, fan settings, and overclock settings from anywhere you have an internet connection.  I do it with ease from my phone, if I need to.  Switched pdu's are a big help, but since I moved all but one rig to SMOS, I haven't even had the need to use my switched pdu to cycle power remotely.  

I'm mining with 31 gpu's and heat started to become a major concern for me when spring rolled around.  Each rig has a 20" box fan behind it.  I ended up installing a 3500cfm exhaust fan and it has been working very well.  We're currently in the hottest month of the summer here and gpu temps are in the low-mid 60's C.

Chasing very new speculative coins with all of your hashing power probably isn't the greatest idea, in my opinion.  There has been one that I started mining back in April that I'm glad I mined for four months with the majority of my hashing power.  Wish I would have started mining it a few months earlier when I first heard about it, but that's the game we play.  

All things considered, I'm happy with the experience that I've had in the 9-10 months that I've been mining.  It's obviously much different than I had originally expected back in November.  More of a roller coaster, with the current bear market.  However, if you can hold your coins and sell for btc with the pairing is up, pay your electricity without cashing out mined coins, and then just hold in btc for 3-5-7-10 years ... My best guess is that you're going to come out of it very well.  It's certainly possible that you could lose everything that you invested, but I'm playing the long game on this one.  We'll see what happens.

[adaseb]

I completely agree with your 2nd point. In Windows the CPU does make a huge difference.

I had a stack of old mobos and old CPUs during the Litecoin days. And I am talking about Celerons and AMD Sempron single core processors.

During the litecoin days using Windows XP or Windows 7, it was fine. However it took literally forever to get a Windows 10 installation to complete with the updates using a single core Sempron processor.

RAM isn't an issue because its dirt cheap and you can easily add a stick or two.

[EdgeOfSanity]

f) chasing the dragon is bad idea: I spent months mining new coins trying to chase the next pump. Bad idea, the only good coin I mined that mooned was GIN coin and I sold it 10x cheaper than its ATH. It would have been better for me to just mine ETH with the pill or any other oldschool/popular coin (as equihash before the ASICs).

disagree, mining new shitcoins made me a lot of money this past year and continues to do so. You have to know what you're doing otherwise you will earn less. I was earning 300-500% more than just mining ETH or ZEC late last year and into the first 4-5 months of this year.

[shater]

I am on the same page regarding doing the homework on PSU. I messed up once with a ASUS B250 board using 12 GPUs... instead of getting two PSUs, I found out I didn't have enough 8-pin plugs to power all the cards. I ended up with 4 PSUs. Massive fail.

[Metroid]

I set up a gpu mining for a friend last year with 13 gpus, after all done, g3900 celeron $35 never used more than 5% on eth, so like i said before, if you are mining eth then cpu does not matter, now if mining any other thing then yes.

[xxcsu]

b) IMO Windows is simply not good for 8+ GPU rigs. Booting and stability problems all the time.

b) built over 50x 12 or 12+ GPU rigs in the past few months for my customers, usually im testing them for a week for stability before giving them out from my hand, all windows based, never ever have any stability or booting problems. VGA Bioses is modded by me , i mean modded not just mem strip copied kinda mod )
Here is the last few build what i have under testing, ETH-11 is a 12 gpu rig running super stable in the past 228 hours:) actually all of my latest build not just the 12 gpu built

d) Heat is a big problem for 12+ GPU rigs. Lots of chassis for such rigs do not have enough spacing between GPUs.

d) with proper air flow, heat can not be a issue, the ambient temp ( actually filtered outside air) in my "home mining server room" is 34.4C right now, that is around 94F , all gpu's mining under 75C core temp, with 28-55% fan speed ... yeah i know.... i installed a new fan in ETH 07 , and the reading is incorrect on one gpu .. it says 99%

[kingcompu]

i have setup 10 eth mining system still pool mining is better or Pool mining..? plz suggest