Why is a PCIE latency rise required for many GPUs???

[-ck]

I've seen it reported on numerous occasions that on some motherboards when you get to about 5 GPUs, they won't work unless you increase the PCIE latency from 64 to 96 in the bios. As it turns out I recently added a 4th 7970 to my GD70 motherboard and ran into precisely that issue. Xorg would crash every time I tried to start it with all 4 GPUs. So I went into the bios and tried upping the PCIE latency to 96 and then it worked fine.  Anyone know why that is?

[aqrulesms]

Although I don't know the technicality in this area, it would appear that the amount of data the GPUs are requesting is too much for the motherboard to handle, so naturally the higher the latency, the less bandwidth is being taken up.

[-ck]

Perhaps in my case because they're all running on 16x slots and they're 7970s I hit the limit earlier.

[aqrulesms]

Perhaps in my case because they're all running on 16x slots and they're 7970s I hit the limit earlier.

Yeah shouldn't be too big of an issue since it's working for you now. 

That's a pretty good thought.  Must be why some mobos reduce the speed as you add more cards.

[bulanula]

Perhaps in my case because they're all running on 16x slots and they're 7970s I hit the limit earlier.

Yeah shouldn't be too big of an issue since it's working for you now. 

That's a pretty good thought.  Must be why some mobos reduce the speed as you add more cards.

So the solution is to use only x1 risers to force x1 speeds and avoid x16 negotiations ?

[-ck]

Perhaps in my case because they're all running on 16x slots and they're 7970s I hit the limit earlier.

Yeah shouldn't be too big of an issue since it's working for you now. 

That's a pretty good thought.  Must be why some mobos reduce the speed as you add more cards.

So the solution is to use only x1 risers to force x1 speeds and avoid x16 negotiations ?

I'm pretty sure I've seen people suffer this with x1 as well, but usually when in combination with some higher speed ones, so perhaps with all of them 1x this is true... OR it could be the reason no one has successfully used more than 8 devices?

[Cablez]

I thought that was a driver limit and not a motherboard limit?

[malevolent]

I thought that was a driver limit and not a motherboard limit?

It can be a mobo limit, an OS limit or a driver limit.

[-ck]

The amd driver seems to have scope for 16 devices but no one has successfully booted more than 8.

[rjk]

I ran 6x 5870s an one board (ASRock 970 Extreme4) for a while, never had to do any BIOS tweaks like that. Maybe it was automatic.

[jjiimm_64]

I have a few rigs running 6 5870s.


still could not get the 7x series to do 6.  but I will try the 1x riser on all 6 and see if that works.