5x GPU Rig Issues

[sundownz]

Hey guys,

Been working on my first setup of more than four GPUs over the last week.

We determined that six GPUs simply wouldn't run stable, although, it would run sometimes... so we dropped to five GPUs.

In any event... this is what we have for hardware & software:

ASRock AB350 PRO4 Motherboard
AMD Ryzen 1700X CPU
16 GB Corsair Vengeance DDR4
** Also set Virtual Memory to 128 GB
Western Digital 1 TB Hard Drive
Newest Version 009S Risers
EVGA 1600-watt Platinum PSU
Windows 10 Professional 64-bit
5x EVGA 1080Ti GPUs

So here is where I am at.

The systems will run perfectly for a while then result in something like "Cannot Validate on CPU" or "unspecified launch failure"

I can re-start the miner & it will run again for a while. Often times many hours but sometimes just a few minutes.

I have taken intensity down & no luck.

Any ideas of things I may want to check? I have been / still am checking quite a few things.

[1713963324]

1713963324

[Huseyinnn]

same errors on 1 rig out of  6. cant find why or when

[sundownz]

I am building these for a friend. I generally just use HP Z400 workstations and 3-4 cards and it works 100% of the time, haha... but he wanted higher density.

We are open to buying new motherboards if necessary to get six GPUs per rig working... but would PREFER if we could make this work with either 5 or 6 per system.

[CryptoWatcher420]

Hey guys,

Been working on my first setup of more than four GPUs over the last week.

We determined that six GPUs simply wouldn't run stable, although, it would run sometimes... so we dropped to five GPUs.

In any event... this is what we have for hardware & software:

ASRock AB350 PRO4 Motherboard
AMD Ryzen 1700X CPU
16 GB Corsair Vengeance DDR4
** Also set Virtual Memory to 128 GB
Western Digital 1 TB Hard Drive
Newest Version 009S Risers
EVGA 1600-watt Platinum PSU
Windows 10 Professional 64-bit
5x EVGA 1080Ti GPUs

So here is where I am at.

The systems will run perfectly for a while then result in something like "Cannot Validate on CPU" or "unspecified launch failure"

I can re-start the miner & it will run again for a while. Often times many hours but sometimes just a few minutes.

I have taken intensity down & no luck.

Any ideas of things I may want to check? I have been / still am checking quite a few things.

first off 128gb VM is just pointless and that's prolly contributing to why the system is unstable. should read up on what a page file does while on a hard drive or ssd, ssd's are affected more than a traditional hard drive. it could also be that your trying to use an amd gaming rig by the looks of it as a mining rig, you are better off just building a dedicated mining rig with an open air frame and a motherboard which was MENT for mining. stability prolly wouldn't be an issue then, not everything can do mining and that's something you need to keep in mind

[giveen1]

You really only need to set that pagefile to about 36GB.

[CryptoWatcher420]

You really only need to set that pagefile to about 36GB.

anything more than 16gb is beyond useless, especially if you've got 8gb or more ram. don't listen to newbies

[sundownz]

Hey guys,

Been working on my first setup of more than four GPUs over the last week.

We determined that six GPUs simply wouldn't run stable, although, it would run sometimes... so we dropped to five GPUs.

In any event... this is what we have for hardware & software:

ASRock AB350 PRO4 Motherboard
AMD Ryzen 1700X CPU
16 GB Corsair Vengeance DDR4
** Also set Virtual Memory to 128 GB
Western Digital 1 TB Hard Drive
Newest Version 009S Risers
EVGA 1600-watt Platinum PSU
Windows 10 Professional 64-bit
5x EVGA 1080Ti GPUs

So here is where I am at.

The systems will run perfectly for a while then result in something like "Cannot Validate on CPU" or "unspecified launch failure"

I can re-start the miner & it will run again for a while. Often times many hours but sometimes just a few minutes.

I have taken intensity down & no luck.

Any ideas of things I may want to check? I have been / still am checking quite a few things.

first off 128gb VM is just pointless and that's prolly contributing to why the system is unstable. should read up on what a page file does while on a hard drive or ssd, ssd's are affected more than a traditional hard drive. it could also be that your trying to use an amd gaming rig by the looks of it as a mining rig, you are better off just building a dedicated mining rig with an open air frame and a motherboard which was MENT for mining. stability prolly wouldn't be an issue then, not everything can do mining and that's something you need to keep in mind

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

[fanatic26]

Windows 10 Professional 64-bit

Found the problem. Run Linux and all will be well.

[sundownz]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

[CryptoWatcher420]

Windows 10 Professional 64-bit

Found the problem. Run Linux and all will be well.

people like you are the problem, running windows is just fine

[CryptoWatcher420]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

its very much possible that because you are using a gaming motherboard and not a mining motherboard that they just didn't update the bios for use with mining, that said you might wanna pick up a mining motherboard to switch that one out with, as I mentioned there are STILL motherboards out there even new ones that just WONT work with mining, it also could just be how you have things set up. me personally I wouldn't use AMD CPU's for mining as I'm sure that's part of the issue considering everyone uses what? intel cpus and intel motherboards....I mean why use something that isn't fully supported within the mining community amd mining mobo's are really new, there is one or two ive seen just for mining that are amd, but again theres lack of support within the mining community so why use it? you can use the intel cpu all the same for cpu mining just get a better cpu than normal

[sundownz]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

its very much possible that because you are using a gaming motherboard and not a mining motherboard that they just didn't update the bios for use with mining, that said you might wanna pick up a mining motherboard to switch that one out with, as I mentioned there are STILL motherboards out there even new ones that just WONT work with mining, it also could just be how you have things set up. me personally I wouldn't use AMD CPU's for mining as I'm sure that's part of the issue considering everyone uses what? intel cpus and intel motherboards....I mean why use something that isn't fully supported within the mining community amd mining mobo's are really new, there is one or two ive seen just for mining that are amd, but again theres lack of support within the mining community so why use it? you can use the intel cpu all the same for cpu mining just get a better cpu than normal

We are looking into swapping to i7-7700k CPUs and the GIGABYTE GA-H110-D3A motherboards. Thoughts on that board ? It appears to have specialized mining settings in the BIOS.

I personally don't use over four GPUs in my rigs... so I will buy the other parts from my friend and he can swap over for his six GPU rigs.

[sundownz]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

its very much possible that because you are using a gaming motherboard and not a mining motherboard that they just didn't update the bios for use with mining, that said you might wanna pick up a mining motherboard to switch that one out with, as I mentioned there are STILL motherboards out there even new ones that just WONT work with mining, it also could just be how you have things set up. me personally I wouldn't use AMD CPU's for mining as I'm sure that's part of the issue considering everyone uses what? intel cpus and intel motherboards....I mean why use something that isn't fully supported within the mining community amd mining mobo's are really new, there is one or two ive seen just for mining that are amd, but again theres lack of support within the mining community so why use it? you can use the intel cpu all the same for cpu mining just get a better cpu than normal

We are looking into swapping to i7-7700k CPUs and the GIGABYTE GA-H110-D3A motherboards. Thoughts on that board ? It appears to have specialized mining settings in the BIOS.

I personally don't use over four GPUs in my rigs... so I will buy the other parts from my friend and he can swap over for his six GPU rigs.

The Gigabyte board worked like a charm. We swapped one system over yesterday and it worked right off the bat no problem. So we are redoing the first six machines... And the next six we know to just start with the GB board.

Sadly the i7-7700k only does half as well on ITNS as the Ryzen but still is worth doing.

[VyprBTC]

FWIW I'm using this board for my Ryzen 1700x machines:

https://www.asrock.com/mb/AMD/X370%20Killer%20SLIac/index.asp#Specification


They are all working without any issues with 6 GPU's.

[bigjee]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

Darn that sucks. I thought the rig ran stable overnight with 5 cards or did I miss something.

[mobomofo]

I'll preface this by saying I don't have this exact board, but have dealt with this issue successfully.

Did you do a bios update and enable Tolud / 4g endcoding? IF not did you look for a bios from a similar board from AMI or Biostar (they both have AB350 bios). That usually solves it (read below).

Here is the issue.
1. You have a Ryzen which has 24 lanes.
2. Chipset dedicated 4 lanes for itself
3. out of the 20x lanes you have (if I'm reading the specs right) a 2- 16x (but one runs at 16x and the other at 4x unless you set differently in bios to 8x2?), 4 - 1x, and a M2 4x. That's 28 lanes (or 24 depending on PCIE setup) so its way over budget.

Now if you can get the bios to enable tolud / 4g encoding and disable the M2 port, this should fix this issue and grant access to all 6 GPUs.

But, you (in theory) can go bigger.

Ive had success (after doing the above settings), splitting the 16x slot using an 8 port board



And splitting the 4x slot using a 4 port splitter



Your mileage may vary, but in theory you should be able to squeeze 8+4+1+1+1+1 = 16 1x risers onto this board ( or if you can get the 16x to go 8x2 with the 2nd 16x slot then you can run 2 - 8 port splitters and the other 4 - 1x giving you all 20 lanes ie 20 - 1x GPUs).. Again I havnt worked with this board directly and if you cannot set the above 4g encoding or flash to a bios that allows it then there is no luck in getting above 5 GPUs. But the chipset architecture should support it.

Just My $0.02








    

[sundownz]

We started off with less VM so I simply increased it dramatically to rule that out. Either way had the same result.

These are open air rigs.

We are using the Ryzen CPUs to also mine.

Often you will get "cannot validate on CPU" if the CPU is extremely slow or if the CPU is already at high % load (like in your case if you are mining cryptonight at the same time).
First try without mining on the CPU, then if that is OK, try mine on 4 cores instead of 8.

--ypsi

I did try with no CPU mining at all -- same result.

After checking over a ton of things I am suspecting the motherboard simply won't allow more than four GPUs.

Darn that sucks. I thought the rig ran stable overnight with 5 cards or did I miss something.

Most of the machines have been working fairly well with 5 cards only needing occasional attention (one of them has been working flawlessly) -- but one in particular was extremely fickle. That is the one we swapped first.

Our goal was really six cards so we are doing a board swap on them all to the Gigabyte / Intel setup. The first one we did is perfectly stable =)

I am buying the Ryzen CPUs / Asrock boards from him for my farm -- as I generally do 4x GPUs max anyway.

[sundownz]

I'll preface this by saying I don't have this exact board, but have dealt with this issue successfully.

Did you do a bios update and enable Tolud / 4g endcoding? IF not did you look for a bios from a similar board from AMI or Biostar (they both have AB350 bios). That usually solves it (read below).

Here is the issue.
1. You have a Ryzen which has 24 lanes.
2. Chipset dedicated 4 lanes for itself
3. out of the 20x lanes you have (if I'm reading the specs right) a 2- 16x (but one runs at 16x and the other at 4x unless you set differently in bios to 8x2?), 4 - 1x, and a M2 4x. That's 28 lanes (or 24 depending on PCIE setup) so its way over budget.

Now if you can get the bios to enable tolud / 4g encoding and disable the M2 port, this should fix this issue and grant access to all 6 GPUs.

But, you (in theory) can go bigger.

Ive had success (after doing the above settings), splitting the 16x slot using an 8 port board

And splitting the 4x slot using a 4 port splitter

Your mileage may vary, but in theory you should be able to squeeze 8+4+1+1+1+1 = 16 1x risers onto this board ( or if you can get the 16x to go 8x2 with the 2nd 16x slot then you can run 2 - 8 port splitters and the other 4 - 1x giving you all 20 lanes ie 20 - 1x GPUs).. Again I havnt worked with this board directly and if you cannot set the above 4g encoding or flash to a bios that allows it then there is no luck in getting above 5 GPUs. But the chipset architecture should support it.

Thanks for the tips =)

[bigjee]

Most of the machines have been working fairly well with 5 cards only needing occasional attention (one of them has been working flawlessly) -- but one in particular was extremely fickle. That is the one we swapped first.

Our goal was really six cards so we are doing a board swap on them all to the Gigabyte / Intel setup. The first one we did is perfectly stable =)

I am buying the Ryzen CPUs / Asrock boards from him for my farm -- as I generally do 4x GPUs max anyway.

Thanks for the clarification. May I ask what boards you went with (assuming you're using the same case, fans, HDD drives etc.).
Also for power still using those self made adapters and connecting to ATX psu for the most part?

Cheers.

[MATTX]

can you confirm psu output that they are confirming to specifications. is your power good enough without any fluctuations or is there any other big power device connected to same line which trips over like ups.