Asus Prime Z270-P 8th GPU Issues

[sang]

Specs:
Asus Prime Z270-P Board
Celeron G3900
8gb RAM
EVGA Power supplies w/ 2psu connector
Windows Server 2012R2 (Windows 8.1 essentially)
Kingston 120gb SSD
Powered USB risers (Gen 1)
8x Zotac GTX1060 6gb Cards


So this board has 6 slots + 2 m.2 slots. I am using 2x m.2 adapters so I can have 8 PCIE slots to plug risers into, but...

I can't seem to get the 8th card to work no matter what I've tried. Spent about 4+hrs so far troubleshooting this.


Here's a list of what I have tried so far:

Adding cards 1 by 1 with a reboot each time
Setting all PCIE settings to Gen1, Gen2, and Auto (tried all 3 options no change)
Changing m.2 slot from Auto to PCIE and back
Disabling Fast Boot
Disabled the audio chipset
Reinstalling video drivers
Installing intel chipset drivers
Changing speed of m.2 slot from x2 to x4 and back
Swapping cards, risers, cables, and slots

The board works fine with 7 cards and mines stable. All 8 cards show up in device manager but 1 has the yellow caution icon. The issue does not appear to be solely the M.2 slots, as cards will mine no problem when plugged into them, and a 5 pcie + 2 m.2 configuration works great (for a total of 7 cards). The issue ONLY appears when I plug in the 8th card. My best guess at this point is some type of issue with one of the M.2 slots sharing PCIE lanes with a regular PCIE slot, however I have seen people make this board work with 8 cards (on youtube) so it has to be possible. I also find it odd that the BIOS only has an option to change ONE of the two m.2 slots to PCIE mode.

This isn't my first mining rig, so lets skip the "did you make sure power is plugged into the cards?" type responses, no offense. Any help will be greatly appreciated 

[1715416266]

1715416266

[1715416266]

1715416266

[1715416266]

1715416266

[poby]

It's definitely possible see https://www.nicehash.com/index.jsp?p=news&id=155

One of the M.2 is pcie only and the other can be pcie or sata is why only one has the option.  I don't know why the 8th card causes a problem.  It does sound like a pcie sharing problem so maybe more tinkering in the bios necessary.  I assume your total power supply power is at least 1500?

[sang]

It's definitely possible see https://www.nicehash.com/index.jsp?p=news&id=155

One of the M.2 is pcie only and the other can be pcie or sata is why only one has the option.  I don't know why the 8th card causes a problem.  It does sound like a pcie sharing problem so maybe more tinkering in the bios necessary.  I assume your total power supply power is at least 1500?

Rig is running two CX750M power supplies with a 2psu adapter. All the risers + mobo + cpu + ssd are plugged into main PSU, and all cards are plugged into 2nd PSU. At the wall with 7 cards going, I'm only using like 800watts.

Been doing more digging... maybe its an IRQ issue? When I disable one random card in Device Manager, the other problem card will enable and work fine. IRQs on each card are: 17, 16, 18 ,16, 16, 19, 16, 16, so maybe too much on IRQ 16?

[brether]

I'm having the same issue.

Which m.2 port have you got the 7th GPU working on? _1 or _2? Mine is running on m.2_2. Neither worked until I changed the pci-e express speed to gen1.

I'm not writing off the possibility of a faulty m.2 pci-e adaptor - I have some more on the way.

[Megaquake]

This guy in the video had the same board and same issue and did resolve it, it is a good watch

https://www.youtube.com/watch?v=rPU5jL36maw

[sang]

This guy in the video had the same board and same issue and did resolve it, it is a good watch

https://www.youtube.com/watch?v=rPU5jL36maw

Do you think you could tell me where to start at? It's a 3hr video man lol

[Vann]

BBT also has a setup walkthrough video for the Z270-A / AR. It should be similar to your board.

https://www.youtube.com/watch?v=DR-FkpU2KfI

[sang]

BBT also has a setup walkthrough video for the Z270-A / AR. It should be similar to your board.

https://www.youtube.com/watch?v=DR-FkpU2KfI

I have done all those steps, but thanks for the resource. The rig will work fine with 7 cards but not 8. And I just setup a new rig with the Z270-A board (instead of -P) and having the same problem. The m.2 slots work but not when I add 8 or 9 cards, only when 7 cards.

[poby]

Maybe try reconfiguring your power supply set up?  From what you have written, it doesn't seem to be any particular interface at fault, so I'm wondering if it might be a power issue, whereby it can handle 7 but not 8 cards.

[sang]

Update:

Switching to Windows 10 Pro from the previous Server 2012R2 has mostly fixed this. 8 gpus work no problem now on both my Asus Z270-A and Asus Z-270-P boards. 9 GPUs on the Z270-A (using two M.2 adapters) still refuses to work though, it will boot into windows and then instantly lock up completely (cant move mouse or interact).

[6ixnogood]

Any one try 8 GPUs on a Z170-A?
I can get 7 working fine, but using a M2 PCIE converter for the 8th card causes crashes after 10 min of mining.

[Vann]

Update:

Switching to Windows 10 Pro from the previous Server 2012R2 has mostly fixed this. 8 gpus work no problem now on both my Asus Z270-A and Asus Z-270-P boards. 9 GPUs on the Z270-A (using two M.2 adapters) still refuses to work though, it will boot into windows and then instantly lock up completely (cant move mouse or interact).

That's a limitation of Windows. In Windows 10 you can run up to eight cards using the same drivers, so you can eight AMD cards + eight Nvidia cards. Either add a different card for the 9th GPU or switch to ethOS, smOS which are based on Linux.

[brether]

Got my 8th GPU working by doing the following:
Replaced what I suspected was a faulty m.2 pci adaptor

Then I followed the below instructions:
https://bitcointalk.org/index.php?topic=1917749.0

After that, the 8th GPU was recognised in Xubuntu.