New rig problem: direct pcie 16x connection is slower than 1x riser

[bobasaurus]

(also posted on reddit/r/litecoinmining, but maybe you guys have a solution for me)

Hello fellow miners.  I built my rig two days ago and noticed a persistent problem.  I have three R9 290 cards, two from powercolor, one from xfx.  The two powercolor cards are plugged directly into the pcie 16x slots, and the xfx is in a 1x riser.  For some reason, I get 80-100 kH/s higher rates on the card in the 1x riser.  I tried switching the xfx card and one of the powercolors, and I get the same results, so it's not from differences in the cards. 

I'm using cgminer 3.7.2 on Windows 7 x64 with Catalyst driver 13.11 Beta9.5 and no extra app sdk installed.  I've used many different configurations, all giving the same performance drops on the two direct pcie 16x cards.  Currently I'm using something similar to the rumorcity ["R9 290 and R9 290X – Sweet Spot"](http://rumorscity.com/2013/12/03/litecoin-gpu-mining-with-amd-r9-290-and-r9-290x-sweet-spot-for-1000khashsec/) article.  My launch batch also sets GPU_MAX_ALLOC_PERCENT 100 and GPU_USE_SYNC_OBJECTS 1 . 


Here is a screenshot showing the issue (GPU 2 is on the 1x riser):

https://i.imgur.com/nZTat60.png

My rig specs:

2x PowerColor AXR9 290 4GBD5-MDHG/OC Radeon R9 290 4G

1x XFX R9-290A-ENBC

mobo: MSI 970A-G46 AM3+ AMD 970 SATA 6Gb/s USB 3.0 ATX A

cpu: AMD Sempron 145 Sargas 2.8GHz Socket AM3 45W

ram: 8GB g.skill DDR3 1600 (4X 2GB)

psu: CORSAIR HX Series HX850 850W ATX12V 2.3 / EPS12V 2

hdd: some old seagate sata hard drive

I would appreciate any tips on how to get around this.  I'm kind of stumped as to why the direct pcie 16x ports are slowing down my hashrates... maybe the BIOS is clocking them down to save power?  I haven't seen any setting like this when looking though.  Is there some way that Windows 7 can slow the pcie bus? 

[Gazza1]

maybe a bandwidth issue.  I would try running 1x risers on all of them

also check gpu usage in afterburner and make sure it's constant and not throttling up and down.

[mikeybit]

You're comparing the PowerColor to the XFX cards.  They're not apples to apples.  Different brands run at different rates, have different memories, different BIOS's, etc.

[Flashman]

Blank the 16x 8x and 4x presence detect pins with tape on the cards and force them to 1x on the 16x and see if you get more hash that way.

[bobasaurus]

maybe a bandwidth issue.  I would try running 1x risers on all of them

also check gpu usage in afterburner and make sure it's constant and not throttling up and down.

I believe you are right.  I tried taking out one of the 16x slot cards, and the speed of the other 16x slot card rose to match the 1x riser card.  Something about running two cards in the 16x slots is dropping my hash rates by about 10%.  I'll order another riser today and give that a shot.  

You're comparing the PowerColor to the XFX cards.  They're not apples to apples.  Different brands run at different rates, have different memories, different BIOS's, etc.

Notice that I swapped around the cards and still get the same results.  Doesn't seem to depend on the make of the card.  

Blank the 16x 8x and 4x presence detect pins with tape on the cards and force them to 1x on the 16x and see if you get more hash that way.

This is an interesting suggestion... do you mean I should mask the pins on the card that detect 16x, 8x, and 4x presence?  I'll google it some more.  

[mikeybit]

Here's my launch command:
-I 20 -g 1 -w 512 --lookup-gap 2 --thread-concurrency 25600 --gpu-engine 1080 --gpu-memclock 1490 --gpu-powertune 10 --gpu-fan 40-100 --auto-fan .......


Getting 815-820 kh/s each out of the powercolor cards.

Make sure that you delete all of the .bin files before you run that.

Those have been the best settings I've found to work with my PowerColor cards.

[Flashman]

This is an interesting suggestion... do you mean I should mask the pins on the card that detect 16x, 8x, and 4x presence?  I'll google it some more.  

Yah, but nevermind if you're getting another riser, that will help with thermals also one would think.

[porscheman170]

You definitely don't have enough power for 3 290s. Trust me I have 8 of them each one draws ~300 watts and the rest of your components are an easy 50 watts. That could lead to inconsistent hash rates when running multiple cards. Also like somebody else said the cards are all different, you might be pushing the powercolors too hard/or not enough causing a smaller hashrate.

[bobasaurus]

You definitely don't have enough power for 3 290s. Trust me I have 8 of them each one draws ~300 watts and the rest of your components are an easy 50 watts. That could lead to inconsistent hash rates when running multiple cards. Also like somebody else said the cards are all different, you might be pushing the powercolors too hard/or not enough causing a smaller hashrate.

That's a possibility.  I assumed each draws about 250 W, but I have a power meter on order to actually test this.  I have a radeon HD 7970 that I plan to run in a separate junk computer, but maybe I could swap that in instead of a R9 290... assuming it's actually lower-power under load.  The Corsair supply is a good one, but I do doubt its ability to supply >900W continuously.  I've tried adjusting gpu-powertune to various levels without any change in the results.  

[bobasaurus]

You definitely don't have enough power for 3 290s. Trust me I have 8 of them each one draws ~300 watts and the rest of your components are an easy 50 watts. That could lead to inconsistent hash rates when running multiple cards. Also like somebody else said the cards are all different, you might be pushing the powercolors too hard/or not enough causing a smaller hashrate.

Holy crap, I just measured the rig at 1100 watts total with my new meter!  Each card is drawing roughly 320 watts, so you're right on the mark.  I've been pushing that corsair 850W supply pretty hard... I'm actually amazed it has worked this well.  Surprisingly, the supply feels cool to the touch and the fan is barely spinning.  Might be time for a higher-wattage supply or some other setup to split the load. 

[porscheman170]

Holy crap, I just measured the rig at 1100 watts total with my new meter!  Each card is drawing roughly 320 watts, so you're right on the mark.  I've been pushing that corsair 850W supply pretty hard... I'm actually amazed it has worked this well.  Surprisingly, the supply feels cool to the touch and the fan is barely spinning.  Might be time for a higher-wattage supply or some other setup to split the load. 

  Yeah these cards draw a surprising amount of current. I have 4 in each rig, 3 cards and the mobo hooked up to a 1300 Watt PSU and the 4th card hooked up to a 600 Watt. PSUs actually run more efficient when they are not pushed to the max.

[bobasaurus]

Holy crap, I just measured the rig at 1100 watts total with my new meter!  Each card is drawing roughly 320 watts, so you're right on the mark.  I've been pushing that corsair 850W supply pretty hard... I'm actually amazed it has worked this well.  Surprisingly, the supply feels cool to the touch and the fan is barely spinning.  Might be time for a higher-wattage supply or some other setup to split the load. 

  Yeah these cards draw a surprising amount of current. I have 4 in each rig, 3 cards and the mobo hooked up to a 1300 Watt PSU and the 4th card hooked up to a 600 Watt. PSUs actually run more efficient when they are not pushed to the max.

I ran a voltage monitor to test things out, and the PSU is actually doing a surprisingly good job even at 1100 watts:

https://i.imgur.com/B5dpY3O.png

The slight dip in the voltage chart is when I started the mining, but it's holding 11.928 V on the 12 V line under this crazy load (the vertical bar is the "cursor" where the voltage is displayed, not a giant outlier spike).  There is a slight ripple in the 3.3 V, maybe caused by pci-e bus use... maybe my second powered 1x riser will reduce this. 

[Flashman]

BTW before everyone goes buying 850W corsairs thinking they can put 1100W of load on them, what's happening is, it's supplying about 825W of load at about 75% efficiency. So at the wall sucks 1100W... 330W at the wall for a graphics card is probably 270ish watt actual depending where on the PSU efficiency curve you are when you measure it.

[porscheman170]

BTW before everyone goes buying 850W corsairs thinking they can put 1100W of load on them, what's happening is, it's supplying about 825W of load at about 75% efficiency. So at the wall sucks 1100W... 330W at the wall for a graphics card is probably 270ish watt actual depending where on the PSU efficiency curve you are when you measure it.

Yes exactly. Since you are pushing the PSU to it's upper limits the efficiency is dropping well below its 80+ rating. You really need to get either a bigger supply or a second smaller one. Continuing to run your setup like that will definitely cause something to fail.

[bobasaurus]

That's some good advise... didn't think about the efficiency dropping so much.  I'll look into getting another supply. 

[DirkDunkirk]

a little late with my reply, but i'm looking into 1x risers.

do you have crossfire enabled?

[Aalesund]

PowerColor are the slowest GPU for mining.
Their speed is less than the other brands, look here - https://litecoin.info/Mining_hardware_comparison
R9 290X, AXR9 290X 4GBD5-MDHG/OC = 720 Kh/s
So your 750-780 is not bad for this cards, should be happy that you reached 780
I also have PowerColor R9 290X and the maximum speed reached is that 740 Kh/s but sometimes speed drops below 700
PowerColor = bad cards.
Sapphire is the best for mining.
Example:
PowerColor R9 290X = 720-750 Kh/s
Sapphire R9 280X = 770-800 Kh/s
Sapphire 280X is faster than PowerColor 290X
the best GPU is Sapphire R9 290X Tri-X = ~1000 Kh/s

[Wipeout2097]

I would appreciate any tips on how to get around this.  I'm kind of stumped as to why the direct pcie 16x ports are slowing down my hashrates... maybe the BIOS is clocking them down to save power?  I haven't seen any setting like this when looking though.  Is there some way that Windows 7 can slow the pcie bus?  

Yes there is such a power option, called "PCI Express -> Link State Power Management".

There are also some settings like PCI-e latency, PCI-e bus frequency on the Bios. Once someone reported higher hashrate at 95 Mhz PCI-e than 100.

Did you disable sound, firewire, etc... on the BIOS and unnecessary drivers on Windows device manager?

Scrypt is voodoo anyway... 

[cryptmebro]

I'm having a similar issue and wondering if anyone can help.  As you can see, one card is under-performing significantly.  http://imgur.com/P4idOaT
Ignore GPU 0
GPU's 1-3 are R9 290 Tri-X
GPU 1 has a 1x riser plugged into primary pci 16 slot.
GPU 2 has a PCI 16x riser plugged into the 3rd x16 slot on the bottom.
GPU 3 has a 1x riser plugged into a 1x slot.

[phzi]

I'm having a similar issue and wondering if anyone can help.  As you can see, one card is under-performing significantly.  http://imgur.com/P4idOaT
Ignore GPU 0
GPU's 1-3 are R9 290 Tri-X
GPU 1 has a 1x riser plugged into primary pci 16 slot.
GPU 2 has a PCI 16x riser plugged into the 3rd x16 slot on the bottom.
GPU 3 has a 1x riser plugged into a 1x slot.

Try dropping Windows.  I always ran into annoying issues with single cards under performing when using Windows.

And please, in the future, do not bring threads this old back from the dead.  If nobody has posted in a thread in a few weeks, then you are better off making a new thread.