Having bizzare issue(s) and need help.

[CanGuitarGuy]

Hey everyone,

So I'm going to go into a lot of detail here and hopefully can get my issues cleared up.

I've got 2 6 card 1080TI rigs that've been mining mostly Equihash based coins. I've got Windows 7 on one rig and Windows 10 on the other. Few nights ago my one rig was crashing on me. I had one particular card that Afterburner showed wasn't running well. I find most of my cards have a bit of variance from lets say 93-99% but usually around 99 but this particular card was diving down to 50 and upto 80 over and over again and eventually it'd crash out the rig. I tried swapping out the riser, nothing. PSU cable, nothing. USB cable to 1x and it worked. I found that odd so I swapped back to the other usb cable and it remained working so I just left it. Then again last night I noticed the same issue. Swapped to a different riser, usb cable and 1x and it seemed fine so I left it.

Then I noticed my other rig was doing the same thing but on 2 cards. This one is on Windows 10. The usage was up and down like a yoyo and pulling about 500SOL instead of my usual 700. I tried removing a card, same problem. I then removed the PSU and hooked up another just to 1 card and it was fine. Added the old PSU back in and it was fine. Plugged PCIE cable back in and didn't connect it to anything and the issue came back. Hrm. Set that cable aside and tried another to a card, seemed fine. Added a 3rd card, issue came back. Swapped the port the PCIE cable was connected into, it was fine again. Rebooted the rig, issue came back. Tried powering every riser off its own Molex cable and it seemed fine again. Had to reboot and it came back. At this point my start menu was randomly not working, I'd have to reboot it again. I moved one of the molex connnections to another port on the PSU and it started working again so I've left it for now but know it'll be back.

So I'm pretty perplexed what the issue could possibly be. I've had the same issue on 2 different rigs. One is using the G3930 and one is using the G4400 for processor. The things they've got in common are they're both using 2 1000W Corsair HX1000 PSUs. They're both using the Gigabyte Z270P-D3. Both have 8gb of DDR4 (different brands). Both have 128GB SSDs (different brands). I did try the special BIOS for mining from Gigabyte on the rig with the more significant issues but that was before issues ever began. This system has been up and running for almost a week without this problem.

I'm almost learning more towards the power supplies as being the issue(s). Seems the quickest way to straighten things out is to jump PCIE cable or molex cable over to a different port on the PSU. Worth nothing I've measured each unit I've got and they're all running at 690-710 watts so well within the 80% rule. Not using any adapters or nonsense. Just a direct connection from the 8+8pin cables they come with. Using 3/4 cables they come with. Generally I'd run 2 risers per molex cable but I only run 3 cards per PSU so one riser is always on its own.

I guess it could be the motherboard. I've seen lots of praise and lots of complaints for this particular board. I've not had many issues with it. One question I did also want to ask was usage of the cards. I find in my desktop PC with 2 cards directly into the motherboard's PCIE slots I get a solid 99% usage on both cards and its rock solid, that line doesn't move one bit. With my rigs I find usually they're at 99% but there is a certain variance. Just a few percent (excluding my larger issue here) but it's still noticeable compared to my desktop. Is this just what happens with 6 cards in a rig?

Last bit is my overclock settings are consistent across all rigs. 110 core and 500 mem. I tried mucking around with the settings and no difference.

[CanGuitarGuy]

Bump

[MarkAz]

Since most rigs are pretty unique it's hard to get specific advice - but it sounds like you're using a good process to identify potential problems.  One nice thing is that the problem you're having is for the most part repeatable, so you'll know when you nail things down.

My one general piece of advice is that when I used risers, if I had failures it was almost always due to the crap USB cables they include - I would just replace them out of the box with these:

http://amzn.to/2DU20DB

They might be a bit long for you, depending on your rig size, but spending a couple bucks extra to get quality USB cables almost always pays off.

Typically I like all our systems to be standardized, that way if something is failing (or seems to be failing), you can move it to another box and determine if it's continuing the same behavior.  With your systems, if you swap the card and the problem doesn't follow it, and you've swapped the USB cables, then I would agree with you that power would be the next most likely suspect.  I personally HATE dual consume PSU setups - I would only do that with server PSU's, and it may be worth your while to explore converting from those to a DSP-2000BB with breakout, it will give you more power than you have, and be much higher quality.

Beyond that, you could also play around with one of the Linux USB thumb OS's and see if it has the same issues on the same rig - the thumb ones can run without installing, so you don't screw up your Windows image, and if it works then you know it's something to do with your OS setup.

Hope that helps!

[CanGuitarGuy]

Appreciate the suggestions. I tried to rule out the PSU by swapping in a 750 watt and same issue happened. Swapped mobos, same issue. Tried SMOS and it would see the last 2 cards but not even mine on them. Swapped some USB cables and not seeing anything. Only thing I've not tried is messing with the cards on the main psu.

I agree, I hate chaining psus together. I've got an EVGA 1600W on the way soon hopefully and will test it on the rig. Will probably move my rigs over to those until it makes sense to run some 240 and go the server PSU route.

[CanGuitarGuy]

Figure I'd post the solution incase this ends up on Google.

The issue was I had no PCIE settings in the bios. Generation was set to auto. Not sure why but I suspect removing the PSU cable and plugging it again elsewhere fully turned the riser off for a second and it was auto detected as gen2, a reboot put it back to gen3. So just setting it to only be gen2 has massively helped. Slightly introduced another issue in that this Asus Prime board for some reason doesn't hash as high as my Gigabyte one did but I can overclock the cards a tad higher to make up for it where as the Gigabyte boards would crash at anything above 110mhz and I'm at 125.

[CryptoWatcher420]

Appreciate the suggestions. I tried to rule out the PSU by swapping in a 750 watt and same issue happened. Swapped mobos, same issue. Tried SMOS and it would see the last 2 cards but not even mine on them. Swapped some USB cables and not seeing anything. Only thing I've not tried is messing with the cards on the main psu.

I agree, I hate chaining psus together. I've got an EVGA 1600W on the way soon hopefully and will test it on the rig. Will probably move my rigs over to those until it makes sense to run some 240 and go the server PSU route.

its worth it, I went that route right off the bat, ran a 240v circuit, got a decent metered pdu, less cables to deal with though its a bit of a different setup cause you need a pico psu for the mobo and ssd but other than that its great, its the one thing ive standardized on using. server psu's are beefie, they do have there quirks like if you run them too high of a load there going to scream at you via the little fans that they have but other than that I love them, there way cheaper than an atx psu, I mean depending on the server psu I choose I could buy 2x server psu's and 2x breakout boards for the price of 1 atx psu, from a price point alone is more than worth it, you wont find a 1400w atx psu for 58 dollars then spend another 12 bucks for a good breakout board and your still way under, even buying the hp 1500w server psu and a breakout board is only gunna run you 200 bucks roughly compared to an atx 1500w psu that will cost you what double that roughly, then add the fact that server psu's were made to run in harsh environments which is something atx psu's are not made for as that's not what the average consumer wants or needs

id also like to note that I also ended up custom making my OWN sets of cables for a server psu only rig
one that powers both motherboard eps 12v and the pico via a pigtail w/ 2.5mm barrel plug(6pin to eps 12v 4+4pin w/ pigtail and 2.5mm barrel plug for pico psu)

also ended up making my own cables to power 1 gpu and 1 riser using one cable with minimal connection points, essentially ended up with something that combines both an 6 pin to 6+2 pin plus a second cable coming off the 6+2pin for the riser(6Pin)