Title: [Guide] How to run up to 13 GPUs on most motherboards Post by: nitrobg on January 24, 2018, 06:14:18 AM After lots of researching, trial and error, I have managed to run 10 GPUs (https://i.imgur.com/7ajdBrw.png") on my ASRock X99 Extreme6 (http://www.asrock.com/mb/Intel/X99%20Extreme6/) using two PCI-E multipliers (http://us01.imgcdn.ymcart.com/25850/2017/09/08/0/e/0ec2442cc1ff7e57.jpg) for $12 each off eBay or AliExpress. It's likely that I'll be able to run up to 13 if I had more cards.
You need:
Potential issues:
More troubleshooting tips - try updating the BIOS to a newer version, disable unneeded peripherals (audio, additional SATA controllers, whatever), disable iGPU, set PCI-E to a different Gen, even reset the CMOS. HackFlags (https://support.microsoft.com/en-us/help/942959/error-message-when-you-attach-a-pci-express-expansion-chassis-to-a-com) registry trick might work but I've never tried it. YMMV, of course - please let me know if it worked for you. So far I've tried it on ASRock X99 Extreme6 (http://www.asrock.com/mb/Intel/X99%20Extreme6/) and Gigabyte GA-H110-D3A (https://www.gigabyte.com/Motherboard/GA-H110-D3A-rev-10) (>6 cards), MSI H110M PRO-VD (https://www.msi.com/Motherboard/H110M-PRO-VD.html) and several other mATX motherboards for 6 cards. Donations: BTC: 1P4oRbB76QGw3dzmhcxSFo1B9pxSswipfs BCH: 1M6TaEKzKYQEQdoBXRbPhzm3m23iYxFhyU ETH: 0x2211d972934ec7a66225883355e9d23f848f634a ETC: 0x64b30f6f1acd4ab9f1e5d6da7e7d9d22d506c8c4 Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: uskovartem on January 24, 2018, 07:37:16 AM What the reason to install GPT, but not MBR ?
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: nitrobg on January 24, 2018, 08:00:06 AM What the reason to install GPT, but not MBR ? AFAIK it is needed for a 64-bit address space on Windows, it is used to allocate memory for >6 GPUs, otherwise you'll get error code 12 or not boot at all.Can't find the right source but eGPU users encounter the same problem (https://egpu.io/forums/pc-setup/fix-dsdt-override-to-correct-error-12/) and have a similar workaround. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: amit_talkin on March 05, 2018, 06:14:38 PM Hi,
I recently bought PCIE 1 to 4 expansion cards from below. https://www.ebay.com/itm/USB-PCI-E-1X-Expansion-Kit-1-to-4-Ports-Switch-Multiplier-Hub-Riser-Card/122820612734?hash=item1c98add27e:g:TD4AAOSw9gRaFAFL However, on my old system with chipset X58 and also on new H97 chipset, only maximum 6 GPUs working. I tried every possible settings in BIOS but no luck getting more than 6 GPUs working on any of these chipset. I am now thinking to move towards newer chipset like B250. If I purchase lower end mobo. with 1x PCIE 16 and 1x PCIE1x slot , will it work? Do newer chipset support 8 GPUs? I am not talking about special mining motherboard, just a regular one. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: nitrobg on March 05, 2018, 08:09:49 PM X58 is too old and is likely limited to 5-6 GPUs.
H97 should work though. Does it even boot with more than 6 GPUs? If it does, your OS is likely the issue. You might be able to get 5-6 cards with the small boards, but YMMV. The chipset doesn't really matter I think, the multiplier combines the signal of all 4 cards into 1 lane. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: grosminer on March 05, 2018, 09:50:27 PM Great information. Thank you for sharing.
Didn't know about the partition mode Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: amit_talkin on March 06, 2018, 04:11:42 AM X58 is too old and is likely limited to 5-6 GPUs. H97 should work though. Does it even boot with more than 6 GPUs? If it does, your OS is likely the issue. You might be able to get 5-6 cards with the small boards, but YMMV. The chipset doesn't really matter I think, the multiplier combines the signal of all 4 cards into 1 lane. Yea I figured it lately that x58 can only handle up to 4 GPUs. I have a Gigabyte B85M-Gaming 3 motherboard, it works fine with 6 GPUs, but with 8 GPUs it just fails to even load into bios. That even with 4 GPUs plugged into PCIE1x slot via extender! so this means chipset is limiting its usage to 6 GPUs. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: hack4love on March 06, 2018, 05:50:21 AM What amd driver you using . What new in 1709 update I think the and driver not allowed more than 8 card not the windows
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: CarlOrff on March 06, 2018, 06:32:00 AM Nice search : thank you for sharing it.
Somone try this PCIe multiplier with the ASRock H81 Pro BTC ? With how many GPUs ? I'm curious about this solution :-). Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: amit_talkin on March 06, 2018, 08:04:17 AM What amd driver you using . What new in 1709 update I think the and driver not allowed more than 8 card not the windows I have 6x1070 1x1080 and 1x1080ti. OS is not related at all as it doesnt even pass bios screen. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: dominuspro on March 06, 2018, 09:52:33 AM Thank you for sharing... But the main problem is where to get 13 gpus for normal price in EU :)
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: amit_talkin on March 07, 2018, 07:26:54 PM Okey I got it working on my MSI Z170A Gaming M5 motherboard. Got all 8 GPUs detected and working on Windows 10 1709. Did bios update and I got "Above 4g Decoding" option. Set it to enabled, and set PCIE speed to gen2.
Popped 1to4 risers to each PCIEx16 slot, plugged 4 GPUS in each. You don't even need UEFI , my ssd had generic MBR partition and it worked fine. So, still confused if it will work on other chipset. Currently Z170 motherboard is costlier than current gen. Is there anyone who can confirm this working on entry level B250M motherboards? Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: amit_talkin on March 11, 2018, 01:59:44 PM Update : I just purchased ASUS Prime B250M-K motherboard. It has 1xPCIE16x and 2xPCIE1x slots. I disabled CSM, enabled 4G decoding, disabled every possible extra items in bios like serial port, IOAPI entries etc, it only allows 5 GPUs to work properly, rest 3 GPU gets detected but it ends up with Code12 error in device description ( Out of PCIE lanes ).
However here is a weird thing, when I uninstall a GPU in device manage, then it uninstalls all GPU from device manage as being in same class, now after uninstallation, all 8 GPUs shows up as "VGA Video Controller". Now I right click on one of these items, and select "update drivers" then it again installs all 8 GPUS but this time with 7 of them working!!! Now, if I reboot the PC goes back to OS again, it again shows only 5 of them working and 3 goes with CODE12. I am trying like for 18hours with diff. things. Any clue guys? p.s. I tried ethOS, it detects all 8 GPUS and all works without any issues. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: gsrcrxsi314 on March 12, 2018, 02:13:34 AM X58 is too old and is likely limited to 5-6 GPUs. H97 or any other lga1150 socket board won’t work. For more than like 7 GPUs. H97 should work though. Does it even boot with more than 6 GPUs? If it does, your OS is likely the issue. You might be able to get 5-6 cards with the small boards, but YMMV. The chipset doesn't really matter I think, the multiplier combines the signal of all 4 cards into 1 lane. I have 2 boards of this era, one is an Asrock H97 Anniversary, the other is a Gigabyte B85M board. Both of them will only work up to a max of 7 GPUs. If 8 GPUs are plugged in in any configuration, the board won’t even POST. You need a newer board that supports the Above 4G decoding setting. These old boards won’t do it. I have 8 running fine on a Z270 board, and I’m sure I can do more if I wanted. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: gfigg42 on March 12, 2018, 03:07:44 AM X58 is too old and is likely limited to 5-6 GPUs. H97 should work though. Does it even boot with more than 6 GPUs? If it does, your OS is likely the issue. You might be able to get 5-6 cards with the small boards, but YMMV. The chipset doesn't really matter I think, the multiplier combines the signal of all 4 cards into 1 lane. I've ran 7 on an old X58 board using a PCIe port multiplier card under Windows 7. Ran perfectly fine for weeks. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: remauto1187ma on March 12, 2018, 04:26:51 AM Anyone ever tried the pcie multiplier on a Asus B250 Mining Expert Motherboard to get more than the 13 out of it without using the 6 P106 cards it requires to do just that?
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: enhu on March 12, 2018, 04:39:57 AM How do you check whether all GPUs are recognized in a linux system, which command? I can see the image only shows windows where Nvidia series and one Radeon. I'm new to linux but prefer to use it because I don't want to keep reinstalling when a simply virus bothers the system and seem updating windows is unstoppable.
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: Bimmber on March 12, 2018, 07:18:10 AM type: nvidia-smi
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: nitrobg on March 13, 2018, 03:57:31 PM Update:
ASRock H81 Pro BTC R2.0 won't post with more than 7 cards. Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: bluecoinX on March 13, 2018, 04:28:16 PM Why does a motherboard use 13 CARDS? Is it just for fun or space? Why not buy a better graphics card?
Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: remauto1187ma on March 14, 2018, 01:34:01 AM Why does a motherboard use 13 CARDS? Is it just for fun or space? Why not buy a better graphics card? you cant be serious......and in a mining forum subsection.Title: Re: [Guide] How to run up to 13 GPUs on most motherboards Post by: dganm on April 28, 2018, 07:03:31 AM Update: ASRock H81 Pro BTC R2.0 won't post with more than 7 cards. Did you try the new beta bios with the above 4g decoding added option? |