7xGPU + Windows // modding AMD video driver is the answer

[lbr]

https://postimg.org/image/45s15sapz/
Any ideas?

Cool ; )

What's 80x25? Seems like it's not present in the manual.
Ok, that's text mode res ; ) So what does POST indicator say?
@70~77,D2 Late South Bridge Initialization
92~96,B5,D4 PCI bus Initialization@

Display should be attached to GPU in PCI_E2.
CPU PCI-E Lane Configuration -> dunno, but lowest possible, Gen1
PEGX - Gen X-> Gen1
PCI Latency Timer -> play with it
Initiate Graphic Adapter -> PEG
IGD Multi-Monitor -> Disabled
Integrated Graphics Share Memory -> dunno.. 32M
Integrated GPU should be disabled anyway, tho I can't see BIOS option to do it.
Intel(R) Rapid Start Technology -> Disabled
Internal GOP Configuration -> What's that?
Full Screen Logo Display -> Enabled
Boot Mode Select -> try both

Also try shorting PCI-e presence on all risers.
And maybe different BIOS versions.
RAM - try more/less/different.

https://bitcointalk.org/index.php?topic=1661923.0
https://bitcointalk.org/index.php?topic=1640050.0

[cc4fun]

https://postimg.org/image/45s15sapz/
Any ideas?

Cool ; )

What's 80x25? Seems like it's not present in the manual.
Ok, that's text mode res ; ) So what does POST indicator say?
@70~77,D2 Late South Bridge Initialization
92~96,B5,D4 PCI bus Initialization@

Display should be attached to GPU in PCI_E2.
CPU PCI-E Lane Configuration -> dunno, but lowest possible, Gen1
PEGX - Gen X-> Gen1
PCI Latency Timer -> play with it
Initiate Graphic Adapter -> PEG
[...]

I am so grateful for this hint. Thanks!!
Disabling on-board graphic adapter did the trick.

So now I have another issue. Both 6th and 7th GPU report Error Code 43 in Windows 8 PRO. Previously it was 6th GPU but I hoped that rootcausing 7th GPU start-up failure would help both for issues. Anyway what I already did after launching all 7 GPUs:
1 ) Remove all drivers in safe mode using DDU
2 ) Observe that actually some standard VGA Adapters report issues in Device Manager (code 10, 38, 38, 38 respectively):
https://s27.postimg.org/tx7itnez7/20161211_165713.jpg
3 ) Install 16.11.5 drivers
4 ) Observe that  GPUs on PCI 7, 8 and 6 show Error Code 43
5 ) Run 6xGPU MOD in admin mode
6 ) 1 GPU fixed, GPUs on PCI 7 and 8 show Error Code 43. Physically these are PCI slots: PCI_E1 and PCI_E6.
7 ) Try to revert to original BIOS (using micro-switch on the GPU) - doesn't help
8 ) Re-run 6xGPU MOD - doesn't help
9 ) Reduced RAM from 8GB to 4GB - doesn't change anything (2 GPUs with code 43 remain, so presumably if RAM was the issue more GPUs would cease to work)
10 ) Make sure that by swapping risers, cables, riser cards, GPUs - the issue doesn't change, these are always the same PCI slots that are affected.
11 ) Additionaly I noticed that graphic driver crashes randomly when there is at least one GPU with Error 43 (sometimes even in idle or right after start up).

I will keep on investigating and reading two threads you linked. But it seems in my case issue is related to some basics in Windows since even witout AMD driver there are Standard VGA adapter issues.

[batko]

Instal win 10, my 7xRX 480 works fine.

[ferol]

I would really love if someone could help me with another issue you might encounter in the future.

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

[cc4fun]

I would really love if someone could help me with another issue you might encounter in the future.

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

While I'm still working on my issue, maybe this could help:
- Does this always happen to the same GPU? If so, swap it with another one and see if the issue follows the GPU or the PCI/riser. If the issue follows the GPU try flashing it with the BIOS copied from a GPU that doesn't fail (backup first).
- Maybe this is miner's feature to slow the card down if it get's overheated - what's the temp it achieves prior to the failure? And the time you spend to reinstall drivers allows the GPU to cool down?

[lbr]

I would really love if someone could help me with another issue you might encounter in the future.

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

Raise vcore.

[ferol]

I would really love if someone could help me with another issue you might encounter in the future.

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

While I'm still working on my issue, maybe this could help:
- Does this always happen to the same GPU? If so, swap it with another one and see if the issue follows the GPU or the PCI/riser. If the issue follows the GPU try flashing it with the BIOS copied from a GPU that doesn't fail (backup first).
- Maybe this is miner's feature to slow the card down if it get's overheated - what's the temp it achieves prior to the failure? And the time you spend to reinstall drivers allows the GPU to cool down?

Hi! thanks for the quick response.

Nope, I realized it is always the 2nd GPU showed on Afterburner, I swapped every GPU between every PCIE slot, I also tested every card on the 16x port to see if every one of them could send the signal to the screen with no problems.

The low mining speed can also be seen on the ethereum miner.

Every GPU are working between 65 and 71 ºC.

I have taken between a day to 20 minutes, from the crash to the reinstall, I dont think this is a temp issue as the speed wont go back up even if I let it rest for a day.

BTW, I just updated MOBOs bios but I am still having the same issue, and I still cant even get the NiceHashs benchmark done. Might it be MOBOS fault?

I would really love if someone could help me with another issue you might encounter in the future.

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

Raise vcore.

Hi!, I just tried rising vcore by 6% and power limit by 10% and it might just have lasted a bit longer, but in the end, it ended up crashing again.

[lbr]

Hi!, I just tried rising vcore by 6% and power limit by 10% and it might just have lasted a bit longer, but in the end, it ended up crashing again.

Raise vcore until it stops crashing. Or decrease core clock.

[lbr]

Also, any interferance with the GPU except the miner can cause GPU to go idle/low power state.
F.e. - Screen saver/GPU-Z/MSI Afterburner/any soft which uses GPU.
Some software on launch/some rendering/profile apply/etc makes GPU to enter 3D mode, after it is finished GPU goes idle/low state.

Or it simply hangs in OpenCL due to unstable clocks and then resets itself.
Visually it can be seen on display as very quick flicker.

[ferol]

Also, any interferance with the GPU except the miner can cause GPU to go idle/low power state.
F.e. - Screen saver/GPU-Z/MSI Afterburner/any soft which uses GPU.
Some software on launch/some rendering/profile apply/etc makes GPU to enter 3D mode, after it is finished GPU goes idle/low state.

Or it simply hangs in OpenCL due to unstable clocks and then resets itself.
Visually it can be seen on display as very quick flicker.

I just installed windows 10 to see if that could solve the problem. Same Bios settings (8x 4x 4x, gen 2 PCIE ports, inboard audio disabled), amd drivers ver 16.9, afterburner, CDM ver 7.4, aand thats it.

Afterburner mods: +6% vcore, +10% power limit.

Results: 1 gpu mining at 4mh/s, 5 gpus mining at 25 mh/s. And a new problem: I dont see any speed improvement by modifying the clocks, its like afterburner didnt work. I havent overlocked my gpus since the release of ZCash but I remember getting speeds of up to 29 mh/s per gpu with my other rigs.

[lbr]

And a new problem: I dont see any speed improvement by modifying the clocks, its like afterburner didnt work.

Maybe it didn't - check core clock in GPU-Z and AF.

[cc4fun]

Instal win 10, my 7xRX 480 works fine.

This helped!
Fresh Windows 10 installation and all 7 GPUs able to run even without 6xGPU MOD! I can't believe how much time I lost trying to get 5+ GPUs to run on Windows 7...

Thank you batko and lbr for help!

Now I "just" need to resolve stability issue of 6th and 7th GPU, which crash always after 5-6 minutes of mining.

[ferol]

And a new problem: I dont see any speed improvement by modifying the clocks, its like afterburner didnt work.

Maybe it didn't - check core clock in GPU-Z and AF.

Hi, well, this did the trick:

I would really love if someone could help me with an annoying issue.

I really dont know if it is the miner causing it, windows, or maybe the GPU...

For some reason, after a couple of hours mining perfectly, one gpu drops its mem clock from whatever stock setting it is to 300, which causes that gpu to mine a 1/5 % of the speed, usually around 47 sols/s (amd 470 4 gb modded). so I am constantly having to reinstall drivers, patch, etc, to get it work normally again, since simple restarts do not solve the problem.

I am using 6x 470 4gb cards, z97 gaming 5 mobo, 8gb ram, windows 8.1

Do you have a screen plugged in (or dummy plug)? If it's the 'out card' it could be a restful state issue that your power management is forcing the card into.

HI! yes, i keep a screen plugged in all the time. I always use the same port to connect the pc to the scree, i think its the first 16x port, I think it is also the only one that allows me to enter the BIOS settings.

Double check, I had the same 300MHz problem when I first set up my rigs, it turns out I had to have a monitor plugged in to the primary slot - for me that was the 2nd PCI e slot from the top (not the first 1x slot).

I didnt remember I had to connect the screen to a different gpu during the installation. Everything seems to be working Almost fine.
My current configuration is:
Core volt +6
Power limit +10
Core clock 1300
Memory clock 1950

Windows 10 Professional

Hashing speed for CDM ver 7.4 are between 27.7 and 28.0
Temps go from 68 to 73.

For some reason it is mining a bit slower than the rest of my rigs, may be because of the Windows version.
I will let know if it remains stable.

THANKS A LOT FOR YOUR HELP!!!!

[ferol]

Instal win 10, my 7xRX 480 works fine.

This helped!
Fresh Windows 10 installation and all 7 GPUs able to run even without 6xGPU MOD! I can't believe how much time I lost trying to get 5+ GPUs to run on Windows 7...

Thank you batko and lbr for help!

Now I "just" need to resolve stability issue of 6th and 7th GPU, which crash always after 5-6 minutes of mining.

Did you try modifying the bios? set PCIE ports to gen 2, choose the 8x 4x 4x option, deactivate integrated audio port, update bios, and connect the screen to the first 16x port.

[batko]

Instal win 10, my 7xRX 480 works fine.

This helped!
Fresh Windows 10 installation and all 7 GPUs able to run even without 6xGPU MOD! I can't believe how much time I lost trying to get 5+ GPUs to run on Windows 7...

Thank you batko and lbr for help!

Now I "just" need to resolve stability issue of 6th and 7th GPU, which crash always after 5-6 minutes of mining.

try change extender. USe USB extender or?

[cc4fun]

While I'm fine-tuning my miner a couple of conclusions so far:

Hardware:
MSI Gaming 5 Z97
7x XFX Radeon 480 GTR Black
7x USB Risers (blue) v.005S, Windows 10
PSU: FSP Aurum Pro 1200W 80Plus GOLD
8GB DDR3 1600 low voltage
CPU Intel G3220T @2.60GHz, no mods

Conclusions:

• PEG Gen setting in BIOS - doesn't seem to impact stability, affects power drain slightly (average from 10 measurements taken each 10 sec after achieving a stable temp):
  - Auto/Auto/Auto - 928W
  - Gen2/Gen2/Gen2 - 927W
  - Gen1/Gen1/Gen1 - 919W - stayed with this one
  
• PCI Latency - seems to impact stability
  - Latency 128:  time to failure: 5min42s(BSoD), 21min31s(BSoD), 1min26s(GPU0 fail), 12min03s(BSoD)
  - Latency 32: 2 hours and keeps on running stable - stayed with this one
  
• Windows 7 - don't use it for 5+ GPUs, use Windows 10:)
  
• Other settings: onboard LAN adapter, audio and video disabled, PCIe set to 8x/4x/4x of course
  

Voltages and frequencies are of course main factors of performance/stability and now I'm playing around with best performance vs. power drain
At the moment I'm mining Zcash using Claymore's miner v. 9.1, -cvddc 975 -cclock 1225, intensity 8 and seems stable, Average total 1523H/s, power drain (from the wall) 903W, which gives 1.69H/s/W. Fans running quite at ~30%, GPU temps ~70C, ambient temp 24C, PSU fan also quiet. GPU's BIOS is slightly modded (mem timings to improve ETH mining, quiet fans and lowered voltages at top states)

[IojkinKot]

Hi lbr!
My hardware:
6x RX470 8G
Gigabyte z170 gaming k3
8G RAM
G4400 CPU

AMD drivers 16.11.5

In windows 7 x64 two GPUs have code 43 error. After i use your program and patch registry only 5 GPUs work fine and one - code 43 error. In Windows 10 work all 6 GPUs, but win10 unstabe, and telemetry, and updates...
Can you wrote new program, which can run all 6 GPUs on Win7? I'll donate you, if you can.

[Hotmetal]

Hi lbr!
My hardware:
6x RX470 8G
Gigabyte z170 gaming k3
8G RAM
G4400 CPU

AMD drivers 16.11.5

In windows 7 x64 two GPUs have code 43 error. After i use your program and patch registry only 5 GPUs work fine and one - code 43 error. In Windows 10 work all 6 GPUs, but win10 unstabe, and telemetry, and updates...
Can you wrote new program, which can run all 6 GPUs on Win7? I'll donate you, if you can.

You can not do it with RX cards at this point. Update to Windows 10.

[lbr]

Can you wrote new program, which can run all 6 GPUs on Win7? I'll donate you, if you can.

I've already tried and sadly was not able to make 6x470 work in Win7.
I do approximatly see a way - attach debugger to Win and find out what exactly is going on with the driver.
As in why it is throwing an exception(hence Code 43).
However it is somewhat complicated remotely. And I only have 6x290 systems on hands.
And since Win10 works..

Also you can try Server 2016 - same Win10 but without all the bullshit.
Telemetry is enabled tho, but can be disabled like in Win10 in services.

[lbr]

However, feel free to experiment yourself.
Just edit .inf file, reboot and check what happens ; )
I can show you what exactly and how to edit.
The problem is there are a lot of configuration parameters in the driver.
And n1 knows what all of them do and which ones affect this "Code 43" and even if driver config matters for 470.
Maybe driver has something hardcoded for 470 and ignores all the driver configuration or some part which matters..