How to expand the number of PCIe devices on a Main Board?

[MisterJJ]

It appears that for newer MBs there is an MMIO memory limit that is usually 4 GB which can limit the number of PCIe cards (eg, GPUs).  There also appears that even if this limit is increased the system may not POST.  By disabling some unused PCIe devices the system POSTs.  This seems to me to be a UEFI limit on how many peripherals can be handled regardless of the MMIO space available.  Right now the system I am talking about is the AMD X399 Threadripper and several MB vendors (ASRock, ASUS, etc).  As you can tell I am not very knowledgeable here and any advice and information would help very much.  Thanks much and enjoy, John.

[1715228150]

1715228150

[adaseb]

It appears that for newer MBs there is an MMIO memory limit that is usually 4 GB which can limit the number of PCIe cards (eg, GPUs).  There also appears that even if this limit is increased the system may not POST.  By disabling some unused PCIe devices the system POSTs.  This seems to me to be a UEFI limit on how many peripherals can be handled regardless of the MMIO space available.  Right now the system I am talking about is the AMD X399 Threadripper and several MB vendors (ASRock, ASUS, etc).  As you can tell I am not very knowledgeable here and any advice and information would help very much.  Thanks much and enjoy, John.

Check your motherboards manual... on some motherboards the PCIe lanes can be shared with different slots.

So if you use one slot, another slot might become unavailable.

And also in your motherboard manual is a diagram which shows where each of the lanes are going to. Sometimes certain onboard devices can use some PCIe lanes, like SATA, sound, network card, etc.

[MisterJJ]

Thanks, adaseb, but I am looking at a mining type system with 10 to 15 GPUs.  This system is not being used for mining but other intensive computational work.  The UEFI will not POST if some limit of devices is exceeded.  I am looking for what that limit is and how to expand it.  I have tried to help others that were building mining rigs and ran into the same problem, apparently.  I hope some of you will be able to tell me how to circumvent this limit.  Thanks and enjoy, John.

[Raggie]

It appears that for newer MBs there is an MMIO memory limit that is usually 4 GB which can limit the number of PCIe cards (eg, GPUs).  There also appears that even if this limit is increased the system may not POST.  By disabling some unused PCIe devices the system POSTs.  This seems to me to be a UEFI limit on how many peripherals can be handled regardless of the MMIO space available.  Right now the system I am talking about is the AMD X399 Threadripper and several MB vendors (ASRock, ASUS, etc).  As you can tell I am not very knowledgeable here and any advice and information would help very much.  Thanks much and enjoy, John.

Hey friend, i see your question is more specific for mining. You can go here : https://bitcointalk.org/index.php?board=160.0
There are many peoples with knowledge of mining there because it is a mining forum section. i believe many will give you solutions. good luck

[MisterJJ]

Thanks, Raggie.  I could not find the sub-forum you recommended in the Move Thread pull down list, so I placed it here.  I do hope someone will help here.  Thanks and enjoy, John.

[remauto1187ma]

Thanks, Raggie.  I could not find the sub-forum you recommended in the Move Thread pull down list, so I placed it here.  I do hope someone will help here.  Thanks and enjoy, John.

ASUS B250 Mining Expert m/b will do 13 GPU's and another 6 P series cards for a total of 19.  They can be had for around $100 new and $50-$70 used on Ebay.

[MisterJJ]

Thanks, remauto1187ma.  Does the UEFI/BIOS give the user access to the "Above 4GB MMIO"?  Thanks and enjoy, John.

[remauto1187ma]

Thanks, remauto1187ma.  Does the UEFI/BIOS give the user access to the "Above 4GB MMIO"?  Thanks and enjoy, John.

https://www.youtube.com/watch?v=_fNpfOL7ZZA

[nitrobg]

Nearly all 4+ slot motherboards have Above 4G decoding available in the bios. It is a necessary perquisite, but it does not guarantee the proper function of the GPUs. It seems like there is a hard GPU limit of 13 cards, which can only be lifted by using mining specific cards.

Check my findings here:
https://bitcointalk.org/index.php?topic=2814284

[MisterJJ]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

[remauto1187ma]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

Check out Asus! They are coming out with the H370 Mining Master. 20 "slots", usb (No pci) connection.  https://www.asus.com/us/Motherboards/H370-MINING-MASTER/

[darkneorus]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

actually it has "above 4G decoding" setting.

[leowonderful]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

Check out Asus! They are coming out with the H370 Mining Master. 20 "slots", usb (No pci) connection.  https://www.asus.com/us/Motherboards/H370-MINING-MASTER/

The thing is, the H370 still requires you to have special 'Mining edition' cards to reach full capacity if you want to use more than 12-13 cards, and there's really not a lot of differences between it and the B250 board besides a gimmicky USB connection instead of PCI. Your current B250 board is just fine and the H370 will bring little to no improvements if you choose to buy it, and the B250s are likely going to be better in the future as well considering the likely higher pricing of the newer board.

[remauto1187ma]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

Check out Asus! They are coming out with the H370 Mining Master. 20 "slots", usb (No pci) connection.  https://www.asus.com/us/Motherboards/H370-MINING-MASTER/

The thing is, the H370 still requires you to have special 'Mining edition' cards to reach full capacity if you want to use more than 12-13 cards, and there's really not a lot of differences between it and the B250 board besides a gimmicky USB connection instead of PCI. Your current B250 board is just fine and the H370 will bring little to no improvements if you choose to buy it.

Youve never had that damn PCI slot to USB adapter shut down your whole rig because it got jostled or the wind blew the wrong way?  Id love for my Mining Expert board to have all USB connections instead of PCIe.

[leowonderful]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

Check out Asus! They are coming out with the H370 Mining Master. 20 "slots", usb (No pci) connection.  https://www.asus.com/us/Motherboards/H370-MINING-MASTER/

The thing is, the H370 still requires you to have special 'Mining edition' cards to reach full capacity if you want to use more than 12-13 cards, and there's really not a lot of differences between it and the B250 board besides a gimmicky USB connection instead of PCI. Your current B250 board is just fine and the H370 will bring little to no improvements if you choose to buy it.

Youve never had that damn PCI slot to USB adapter shut down your whole rig because it got jostled or the wind blew the wrong way?  Id love for my Mining Expert board to have all USB connections instead of PCIe.

It's happened on one or two occasions to me, but I keep my rigs securely placed and I personally think the potential savings are worth it on a used B250 Mining Expert- I guess we'll have to see the pricing on the H370 and some reviews of it in actual miners' hands to see if the board is as good as the articles I've seen about it claim. Hopefully a true 20-gaming-card board comes out in the future, though- those mining cards are still hard to find and expensive.

[remauto1187ma]

Absolutely, I havent heard anything concerning price.  Not even speculation as of yet.  I suspect this year at current coin prices there wont be an 1/8 of the demand that existed for the mining expert 8 months ago for the mining master motherboard.  That could work in our favor and maybe get them for pretty close to the cost of a mining expert.

[airdropstats.com]

Oh yeah. i need it. Where do to buy it?

[remauto1187ma]

Oh yeah. i need it. Where do to buy it?

Not released yet.

[nitrobg]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

actually it has "above 4G decoding" setting.

Unlikely, because he's using an AMD board and Above 4G is an Intel feature - I'll need to dig into my AMD rig's bios to see what the option was. I'm not sure what are AMD's limits because it could be utilizing the PCI-E lanes differently.
OP, please try running Linux.

[darkneorus]

Thanks, nitrobg.  By disabling as much PCIe stuff (USB Root controllers, NICs, etc) the ASUS board will post with 16 GPUs but only 12 are recognized by W10.  The MB does not have "Above 4GB" setting in UEFI.  Perhaps all 16 could be used if available.  Thanks all for the help.  I learned a lot.  Enjoy, John.

actually it has "above 4G decoding" setting.

Unlikely, because he's using an AMD board and Above 4G is an Intel feature - I'll need to dig into my AMD rig's bios to see what the option was. I'm not sure what are AMD's limits because it could be utilizing the PCI-E lanes differently.
OP, please try running Linux.

umm, what?
ASUS B250 Mining Expert is an Intel board, I have one and I'm pretty sure that it has the "above 4g decoding" option in UEFI.
12 GPUs are recognized because Adrenaline Driver for Windows can run up to 12 AMD GPUs max.