More than 2 gpu's in linux

[nandika]

Hi!
I've got 4 computers mining. Each has identical cards.
Currently I have 5x6770 of the same brand/type, 2x6970 and 2x5970. (more cards are coming...)
All my motherboards have at least 2X x16 slots (one has 3 of x16), and 2-3 PCI-e x1 slots. I've bought PCIe risers / extenders (converters) from eBay, which will convert the x1 slot to x16 connector.

If I use this converter even though if I short the specific pins on the x1 socket, the cards won't work.
Sometimes they get recognized in linux "lspci", sometimes not.
Ati driver won't work with them, shows get clocks failed.

Also, I have a motherboard with 3X x16 slots, and with that only 2 cards work.

The riser cables are good, because when I use them with x16 slots, with only 2 cards, they work without any problem.

Anyone had the same problem and was able to get it to work? Is there a trick to use more than 2 cards?

I'm using ubuntu 11.04 with ati sdk 2.1 and 2.4
I'll try this with linuxcoin, would that help?
But on 2 of my machines I need to use ubuntu, so I need to resolve this to be ready & working when the new cards  arrive:)

[Xmufa23X]

Dummy plug maybe? I thought Windows only had this problem

[-ck]

sudo aticonfig --adapter=all --initial -f

[Krakonos]

Hey, just for you to know, the aticonfig tool is broken. It never displays clock for me if more than one card is present.

Also, to your problem with only two cards working: be sure to run aticonfig after you added a card. Is the third card show in lspci?

I recently had problem with version 12.3 of atidrivers, that aticonfig --adapter=all  created xorg.conf for three cards on machine with only single card. I needed to delete the other two by hand. That leads me to an idea, that you could try to add your third card manually:

In /etc/X11/xorg.conf should be something like:

Code:

Section "Device"
        Identifier  "aticonfig-Device[0]-0"
        Driver      "fglrx"
        BusID       "PCI:1:0:0"
EndSection

Also, lspci says:

Code:

01:00.0 VGA compatible controller: ATI Technologies Inc Juniper [Radeon HD 5700 Series]

Notice, that the BusID is actually the same number what lspci says. Just create a third "Device" entry in you xorg.conf with third pciid and see what happens.

Note that this will only work if your problem is faulty aticonfig, as it was in my case. If your card is not even shown in lspci, the problem will be somewhere else. I've heard stories that some server boards have modified something that makes them not to accept graphics cards... Or simply there is something wrong with your third slot. (You may try 1x->16x rieser, even in your 16x; it could still help if there is short in some other pcie line)

[jake262144]

Hey, just for you to know, the aticonfig tool is broken. It never displays clock for me if more than one card is present.
...

Ummm... no, I really believe it's some user error on your part.

You REALLY want to supplement the aticonfig --initial --adapter=all command with the -f flag, otherwise aticonfig will only modify your current xorg.conf if it appears broken.
Naturally, adequate (root) privileges are required to modify the file.

[DeathAndTaxes]

Hey, just for you to know, the aticonfig tool is broken. It never displays clock for me if more than one card is present.
...

Ummm... no, I really believe it's some user error on your part.

You REALLY want to supplement the aticonfig --initial --adapter=all command with the -f flag, otherwise aticonfig will only modify your current xorg.conf if it appears broken.
Naturally, adequate (root) privileges are required to modify the file.

This.

If you don't need "-f" includin it won't hurt.  If you do need "-f" excluding it will cause all kinds of weird issues.

I always use:

Code:

aticonfig --adapter=all --initial -f

if not running as root use a sudo.

If using a USB drive a sync (or sudo sync if not running as root) is a good idea to make sure filesystem is up to date before rebooting.

Never had any aticonfig issue that a rebuild with --initial -f didn't fix.

[jake262144]

Death, when using USB drives add these two lines to your .bashrc file⁽¹⁾ and never worry about issuing the command by hand:
alias poweroff='sync ; poweroff'
alias reboot='sync ; reboot'

Notes:
(1) and to /root/.bashrc as well

[DeathAndTaxes]

Death, when using USB drives add these two lines to your .bashrc file⁽¹⁾ and never worry about issuing the command by hand:
alias poweroff='sync ; poweroff'
alias reboot='sync ; reboot'

Notes:
(1) and to /root/.bashrc as well

I am lazy.  Often with a hung or crashed rig I just flip the PSU power switch off and on (with BIOS set to always power on after AC loss).  So I will stick with a manual sync just to be sure.  Still I intend to move to a PXE solution eventually (too many things to do) which makes it moot.

For people who do it "right" that is a useful feature.  I didn't realize you could make an alias of an existing command so I learned something.  Makes me wonder why the developer of BMAT uses the non-intuitive commmand "coldreboot" (which IIRC does some housekeeping before rebooting) instead of just aliasing "reboot" and "poweroff"?  Maybe he was unaware of that too?

[Krakonos]

Ummm... no, I really believe it's some user error on your part.

Yeah, maybe it is. But it happens on multiple machines with different configurations, even on the same machine if I add/remove card. I have spent a lot of time researching why, but just gave up. I blame this to aticonfig. cgminer shows clock correctly. Aticonfig doesn't. In the same shell & environment. Maybe I should add --use-darker-magic-kind-of-switch, but I don't know it and don't really care anymore :-)

[DeathAndTaxes]

Yeah, maybe it is. But it happens on multiple machines with different configurations, even on the same machine if I add/remove card. I have spent a lot of time researching why, but just gave up. I blame this to aticonfig. cgminer shows clock correctly. Aticonfig doesn't. In the same shell & environment. Maybe I should add --use-darker-magic-kind-of-switch, but I don't know it and don't really care anymore :-)

No darker magic.  When you change cards you need to rebuild xorg config.  

The --initial param ONLY changes xorg for the sections which are invalid.  If they are valid but wrong (like when swapping cards in and out) it will not make ANY change to that section.

"-f" forces a change.  It makes aticonfig blow away the entire xorg config file and rebuild it regardless of it is valid or not on the next reboot.

Code:

aticonfig --adapter=all --initial -f

Yes if AMD had any brains they would remove the -f param and make --initial perform as --initial -f does now but that likely is never happening.