Building a rock solid multi-gpu linux mining rig with CEntOS 6.0

[Viceroy]

WARNING WARNING WARNING

THIS TUTORIAL WILL NOT WORK WITH CGMINER BECAUSE KANO AND ckolivas HAVE DECIDED TO ABANDON SUPPORT FOR THEIR PRODUCT INSTEAD ONLY CHOOSING TO SUPPORT ASIC BITCOIN MINERS.  THEY HAVE ENDED ALL SUPPORT FOR GPU MINING AND ABANDONED THEIR COMMUNITY.  DO NOT USE CGMINER AS THE AUTHORS ARE TRAITORS AND JACKASSES.

WARNING WARNING WARNING

NEW PREFERRED MINER IS BFGMINER


DO NOT FOLLOW THIS TUTORIAL.




Current single GPU stable working version combination is SDK 12.7 and Catalyst Driver 12.3.
Current multi GPU stable with 6 GPU's working under Centos 6.0 w/SDK 12.4 & Driver 11.4  

Code:

--------------------------------------------------------------------------------
 [P]ool management [G]PU management [S]ettings [D]isplay options [Q]uit
 GPU  0:                | 310.2M/349.0Mh/s | A:3 R:0 HW:0 U:16.72/m I:12
 GPU  1:                | 309.0M/374.0Mh/s | A:2 R:0 HW:0 U:11.15/m I:12
 GPU  2:                | 277.9M/349.0Mh/s | A:0 R:0 HW:0 U: 0.00/m I:12
 GPU  3:                | 294.6M/374.0Mh/s | A:0 R:0 HW:0 U: 0.00/m I:12
 GPU  4:                | 271.4M/349.0Mh/s | A:2 R:0 HW:0 U:11.15/m I:12
 GPU  5:                | 311.8M/374.0Mh/s | A:2 R:0 HW:0 U:11.15/m I:12
--------------------------------------------------------------------------------

Why would you want to build a CEntOS based miner?

CEntOS is the "Community ENTerprise Operating System".  What happened about 15 years ago is that many many people using Linux were using RedHat based distributions like Fedora (free desktop version) and the "commercial" version called RedHat Linux.  Then, all of a sudden, RedHat decided they wanted to charge $1000 plus per machine for the commercial version and it made people mad.  So the mad people went off and created a mirror of the RedHat Linux system and they call it CEntOS.  

The beauty of this OS over other choices is that it is designed to operate in a production environment.  You can even buy commercial software to run on these servers from suppliers like Oracle and Adobe.  Because of the planned cycles the software is stable, unlike most other flavors of linux who are trying to constantly do the "latest" thing people who run enterprise software on their systems are much more interested in reliability.  These distributions (Centos, Fedora, Redhat, Scientific Linux) are the best choice for multiple-gpu cards and multiple-gpu systems, especially when there are only windows drivers available for the video card you own.

Here's the instruction sheet.  Let me know what can be improved.  

Code:

#############################################################
####
####  Installing mining Software on CEntOS or Fedora based distributions
####  
####  Currently 2 GPU's working under Centos 6.4 w/SDK 12.7 & Driver 12.3
####  Currently 4 GPU's working under Centos 6.0 w/SDK 12.4 & Driver 11.4
####
####  testing to increase GPU's per motherboard
####  
####  by:  Viceroy who accepts tips at   144RtpxYKbigosiqTyXNwzwtH6Z7s96xUx
####
#############################################################
####
#############################################################
####  Section 0 - Table of Contents
#############################################################
####
####  Section 0		Table of Contents
####  Section 1		Disclaimers
####  Section 2		Setting up CentOS
####  Section 3		Setting up Base Packages
####  Section 4	        Installing ATI Development Kit (SDK)  
####  Section 5         Installing ATI Catalyst Drivers
####  Section 6         Testing ATI Catalyst Drivers
####
#############################################################
####  Section 1 - Disclaimers
#############################################################
####
####  I am not responsible for you using these instructions.  
####
####  If you choose to follow anything you read here that is
####  your problem, not mine.  I have written these instructions
####  for my personal use for my particular machine in an effort 
####  to help me mine bitcoin.  If you follow these instructions 
####  you may or may not have success, neither of which are any 
####  concern of mine because you are the fool who followed me 
####  off the bridge.  I did not push you.
####
####  I am proposing here to build a TOTALLY INSECURE HIGH
####  POWERED PASSWORD CRACKING MACHINE which MAY OR MAY NOT
####  BE LEGAL TO POSESS WHERE YOU LIVE.  If the policeman
####  puts you in jail because you followed what I wrote here
####  that is your fault, not mine.  You need to be aware of 
####  local, national and international laws before you decide
####  to build a high powered password cracker.  If this gets
####  into the wrong hands and it starts a nuclear war and 
####  and some hostile government decides to end the world
####  guess what... that's not my fault either.  
####
####  Be responsible, do no let Kim Jong Un get this machine
####  http://en.wikipedia.org/wiki/Kim_Jong-un
####
#############################################################
####  Section 2 - Setting up Centos
#############################################################
####
####  While at your windows desktop machine you need to 
####  download the ISO from http://vault.centos.org/6.0/isos/
####
####  (I will be using the 64 bit OS from here on out)
####
####  The ISO can be burned to CD, DVD or USB
####
####  (I will burn the ISO to a 2GB+ USB flash drive)
####
####  Download the ISO from:
####  http://vault.centos.org/6.0/isos/x86_64/CentOS-6.0-x86_64-LiveCD.iso
####
####  Download and install Pendrivelinux Windows Software:
####  http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/
####
####  0. Plug in your flash drive and wait for it to register
####
####  1. start pendrivelinux universal USB installer
####  2. selects CentOS from "other distros alphabetically"
####  3. select the CentOS LIVE iso you downloaded
####     (CentOS-6.0-x86_64-LiveCD.iso)
####  4. select your drive letter and format the drive
####  5. safely remove flash drive and plug into new machine
####  6. boot up and install CentOS with the menus
####  7. after intallation log in as root (or su) and:
####	7a. set up network so IP is fixed so you can log in remotely
####	7b. turn on the SSH daemon so you can log in (service sshd start)
####	7c. make sshd starts each time you boot (chkconfig sshd on)
####  8. now I like to go back to my nice desk and chair
####    8a. dowload and install putty so you can talk to your box
####    http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
####    using a new ssh connection, connect to your machine as the
####    user you set up during installation.  (it's a bad idea to log
####    in as root from a remote location).
####   9. while you are sitting there at your desk download and install
####    xming windows xwindow emulator from here:
####    http://sourceforge.net/project/downloading.php?group_id=156984&filename=Xming-6-9-0-31-setup.exe
####  10. modify yum
####
####  cut and paste one command at a time
####

####  become root user
su -

#### change directory to yum repo directory
cd /etc/yum.repos.d

#### move original file to backup
mv -f CentOS-Base.repo CentOS-Base.repo.old

# COPY the following into your buffer 
## (in windows you use ctrl-c)
##
## start copy ---->>

# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the #mirrorlist= does not work for you, as a fall back you can try the
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
baseurl=http://vault.centos.org/6.0/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#released updates
[updates]
name=CentOS-$releasever - Updates
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
baseurl=http://vault.centos.org/6.0/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
baseurl=http://vault.centos.org/6.0/extras/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
baseurl=http://vault.centos.org/6.0/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
baseurl=http://vault.centos.org/6.0/contrib/$basearch/
gpgcheck=1
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

## <<----  end copy
##
## then using the built in world processor in linux type

vi CentOS-Base.repo

# while in Vi hit [insert key]
# then paste the contents of your buffer with the right mouse button
# then hit the escape key
# hit :
# then hit x 
# then hit the [enter key]
# this will save the file and exit

# test out your changes

yum check-update



#############################################################
####  Section 3	- Setting up Base Packages
#############################################################

####  become root user
su -

yum -y groupinstall x11

#### if you see an error about yum being locked be patient it will unlock in a minute
yum -y install gcc rpm-build git wget make kernel-devel python python-libs numpy openssl openssl-devel ncurses ncurses-devel ncurses-libs autoconf automake m4 pkgconfig libcurl-devel libtool libudev libusb glx-utils libX11 libstdc++ compat-libstdc++*	

####  create new user or place all items in personal home directory
useradd miner

####  become prefered user (no password needed)
su -l miner

####  create a directory to hold all the packages
cd ~
mkdir miningpacks
cd miningpacks

####  download all driver package as user "miner" 
####
####       wget http://download2-developer.amd.com/amd/APPSDK/AMD-APP-SDK-v2.4-lnx64.tgz
####       wget http://www2.ati.com/drivers/linux/amd-driver-installer-12-3-x86.x86_64.run
####
####  goin old-school - working version is best ... 
####  note:  modern versions may break other things
####
wget http://www2.ati.com/drivers/linux/ati-driver-installer-11-4-x86.x86_64.run

####chmod a+x amd-driver-*
chmod a+x ati-driver-*


exit #go back to being root user

#############################################################
####  Section 4	- Installing ATI Development Kit (SDK) 
#############################################################
####
####
####  Install this BEFORE you install the drivers
####
#############################################################
####
####
####  preparing the machine for a new SDK install
####
####
####  First Seek and Destroy ALL Existing ATI Drivers:
####      only needed if you are rebuilding.  If fresh
####      install this command will fail, no directory.
####
cd /usr/share/ati
./amd-uninstall.sh --force

#### more cleaning to be sure 
####
#rm -Rf /etc/ati
#rm -Rf /usr/lib/fglrx
#rm -Rf /usr/share/ati
#rm -Rf /opt/AMD*
#reboot  #HIGHLY RECOMMENDED IF YOU REMOVED FILES

####
####  installing the new SDK install
####

####  become root user
su -  

####  become prefered user (no password needed)
su -l miner

####  move to directory to where you are holding the packages
cd ~
cd miningpacks

####  download SDK
####     
####     wget http://download2-developer.amd.com/amd/APPSDK/AMD-APP-SDK-v2.4-lnx32.tgz
####     wget http://download2-developer.amd.com/amd/APPSDK/AMD-APP-SDK-v2.6-lnx32.tgz
####     wget http://developer.amd.com/wordpress/media/2012/11/AMD-APP-SDK-v2.8-lnx64.tgz
####
####  again, oldschool because working beats not-working:
##
wget http://download2-developer.amd.com/amd/APPSDK/AMD-APP-SDK-v2.4-lnx64.tgz


exit #go back to being root user

#### newer style SDK install
####
#### cd /home/miner/miningpacks/
#### mkdir APP-SDK-2point7-temp
#### cd APP-SDK-2point7-temp
#### cp ../AMD-APP-SDK-v2.7-lnx64.tgz .
#### tar xzf AMD-APP-SDK-v2.7-lnx64.tgz
#### ./Install-AMD-APP.sh
#### cd .. 
#### rm -rf APP-SDK-2point7-temp
#### #reboot

### old school style SDK install
cd /opt
tar -xf /home/miner/miningpacks/AMD-APP-SDK*
cd /
tar xf /opt/AMD-APP-SDK-v2.4-lnx64/icd-registration.tgz
ln -sf /opt/AMD-APP-SDK-v2.4-lnx64/include/CL /usr/include
ln -sf /opt/AMD-APP-SDK-v2.4-lnx64/lib/x86_64/* /usr/lib/
ldconfig
reboot

#### echo AMDAPPSDKROOT=/root/AMD-APP-SDK-v2.4-lnx64 >> .bashrc
#### echo AMDAPPSDKSAMPLESROOT=/root/AMD-APP-SDK-v2.4-lnx64 >> .bashrc
#### echo LD_LIBRARY_PATH=/root/AMD-APP-SDK-v2.4-lnx64/lib/x86_64 >> .bashrc


#############################################################
####  Section 5	- Installing ATI Catalyst Drivers for linux
#############################################################
####
####
####  Install this AFTER you install the SDK
####
#############################################################

####  become root user
su -

##
## important note on installing AMD Drivers under linux:
##
## be certain you have the kernel-devel for YOUR kernel
yum list installed | grep kernel-*

##
## output should look like this
#dracut-kernel.noarch                004-32.el6                         installed
#kernel.x86_64                       2.6.32-71.el6                      installed
#kernel-devel.x86_64                 2.6.32-71.29.1.el6                 @updates
#kernel-firmware.noarch              2.6.32-71.el6                      installed
#kernel-headers.x86_64               2.6.32-71.29.1.el6                 @updates
#[root@localhost ~]#
#
#  notice that I DO NOT have a matched kernel and kernel-devel version.  
#  they must match EXACTLY or you will not be successful in the
#  next steps.  I have:
#     kernel.x86_64                       2.6.32-71.el6                      installed
#       and
#     kernel-devel.x86_64                 2.6.32-71.29.1.el6                 @updates
#
#  the difference is so slight you may have missed it
#     my subversion  32-71.29.1.el6 does NOT match 32-71.el6

##
## check your distribution
uname -r

##
## you can have multiple kernel-devel's that is not a problem
## if you are lucky this command works
yum install kernel-devel-$(uname -r)

#### if this was successful move on to the next section
####     proceed to package generation


####******#*#*#*#*#*##*#*####******#*#*#*#*#*##*#*
####
####  optional section if kernels do not match
####
####
#### You will need to find another "kernel" with 
###  a matching "kernel-devel" package
####
#### yum --showduplicates list kernel-devel
####
#### to download a specific package just add 
#### the version to the name, for example if
#### I want kernel-devel 2.6.32-358.el6 I 
#### just add a hyphen 
####
#### yum install kernel-devel-2.6.32-358.el6
####
####
####******#*#*#*#*#*##*#*####******#*#*#*#*#*##*#*


#############################################################
####  Generate a Distribution Specific Package of fglrx 
#############################################################

# as root

####  Move to package directory
####    using menus generate RPM package for your OS 
####    (generate centos 64 packages)
####
cd /home/miner/miningpacks/

####  chmod u+x *-driver-installer*  #step taken earlier
####  //alternative:// sh amd-driver-installer  
sh ati-driver-installer-*

#### (file was downloaded above)
####
####
####  Install generated package
cd /home/miner/miningpacks/
yum -y install fglrx64_p_i_c-* --nogpgcheck 
aticonfig --adapter=all --initial --force 
reboot


#############################################################
####  Section 6 - Testing ATI Catalyst Drivers 
#############################################################


#### check installation by reading output from
more /usr/share/ati/fglrx-install.log

#### further testing SHOULD be done at this time against this page:
####
#### http://wiki.cchtml.com/index.php/Frequently_Asked_Questions#How_do_I_know_fglrx_is_installed_correctly.3F


#### one possible error
####
####[Message] Kernel Module : Precompiled kernel module version mismatched.
####[Error] Kernel Module : Kernel module build environment not found - please consult readme.
####
####
####  This error indicated you do not have the correct kernel-devel libs to match your kernel
####     (I told you it wouldn't work).
####
####  with xming installed on my pc when I use putty I don't even need to 
####  export my display.  I can just start playing with opencl and opengl
####  libraries and samples.  But only if I did everything correctly
####  and there were no hardware issues along the way.
####
####  my machines 'hello world' test passed with flying colors
####
aticonfig --lsa

## output looks like this:
#* 0. 0e:00.0 AMD Radeon HD 6800 Series
#  1. 0f:00.0 AMD Radeon HD 6800 Series
#  2. 07:00.0 AMD Radeon HD 6800 Series
#  3. 06:00.0 AMD Radeon HD 6800 Series
#
#* - Default adapter
#
# which is EXACTLY what I want!!!

#### if you are not seeing all your GPU's at this stage something is wrong.
#### first check lspci
lspci -nn | grep VGA

#### if that does not show all your cards you need to hard-boot your computer.
#### it make take several attempts and some plugging and unpluggging monitor
#### cables.  There seems to be little method to the madness but make sure
#### you can see all your GPU's now or you can't mine with them.  Even though
#### centos was designed to handle multiple GPU's off the bat I still find the
#### best way to 'light up' a port is to have a monitor plugged in during hard
#### boot.  I'm sure the resistor based dongle approach would work as well.
####
glxgears

## Segmentation fault
##
## uh-oh I broke something!
## oh yea, I remember
##
## I forgot to log into the linux box.
## so here's what I need to make this work
## I go to the centos box and log in as my
## regular user in the graphical interface.
## then I pull down the menu and open a 
## terminal window which must be 'turning 
## x on for my local user'.  Once I open
## that terminal (xterm) I can walk away
## and do everything else remotely.  So
## here we go, let's have some fun...
##
glxgears

## Segmentation fault
## oh yea... duh
export DISPLAY=:0
glxgears

##40014 frames in 5.0 seconds = 8002.695 FPS
##40291 frames in 5.0 seconds = 8058.047 FPS
##40288 frames in 5.0 seconds = 8057.494 FPS
##40287 frames in 5.0 seconds = 8057.313 FPS
## ^C
##
## don't be hatin on me now...


#############################################################
####  Install ATI Display Library Development Kit (ADL SDK): 
#############################################################

###wget http://developer.amd.com/wordpress/media/2012/11/AMD-APP-SDK-v2.8-lnx64.tgz
### cannot find APP without going through website license
### pain in the ass requires a browser download
###

####  become root user
su -

####  become prefered user (no password needed)
su -l miner

####  move to directory to where you are holding the packages
cd ~
cd miningpacks

####  place ADL zip in current directory

wget http://download2-developer.amd.com/amd/GPU/zip/ADL_SDK_3.0.zip

#### unpack it and install it into the correct directory

mkdir ADL-SDK-3point0-temp
cd ADL-SDK-3point0-temp
cp ../ADL_SDK_3.0.zip .
unzip ADL_SDK_3.0.zip

cp /home/miner/miningpacks/ADL-SDK-3point0-temp/include/*.h /home/miner/cgminer/ADL_SDK
rm -rf /home/miner/ADL-SDK-3point0-temp



#############################################################
####  Install CGMiner
#############################################################

####  become root user
su -

####  become prefered user (no password needed)
su -l miner

####  move to home directory
cd ~

####  download CGMiner from author
git clone https://github.com/ckolivas/cgminer


####  copy ADL libraries to correct location
cd /home/miner/cgminer/ADL_SDK
cp /home/miner/miningpacks/ADL-SDK-5point0-temp/include/*.h .
rm -rf /home/miner/miningpacks/ADL-SDK-5point0-temp/
cd ..

./autogen.sh

make

exit #go back to being root user

cd /home/miner/cgminer/
make install

reboot #if you have not after changes to libraries

#if you did everything as I have above you are finished
#and can run cgminer.  test with  ./cgminer -n

####
#### Now test out your new miner for 24 hours and if you like all the time and effort I
#### put into this guide I encourage you to think about testing your new miner here:

./cgminer -o stratum+tcp://mining.eligius.st:3334 -u 1NQc6tCtRmT8zBeLKNpboFaUBMLcxDwUm6 -p x0 -I 9 --per-device-stats 2>logfile.txt

####
#### After 24hrs of burn in read the logfile.  Make sure nothing strange happened.  
#### Here's how you read the log:

more logfile.txt 

#### If you don't see any errors set your miner to your favorite pool and off you go.
#### Then set that file aside so you can refer to it later.

cp logfile.txt burninlog.txt

#### in the future, unless you are having problems you can remove the --per-device-stats and the logging
#### from the command line, so it looks like this:  (don't forget to change the address!!)

./cgminer -o stratum+tcp://mining.eligius.st:3334 -u 1NQc6tCtRmT8zBeLKNpboFaUBMLcxDwUm6 -p x0 -I 9



#### Best of luck, greenhorn!

#############################################################
####  The End
#############################################################

Any comments in this thread should be related to this script and improvements.  Spam and
trolls (like kokjo) posts designed to do anything other than help people build and improve on
this script will be reported to admins.  Please be civil and contribute related info only. thx.

ps. Here's a cool command, in my case 4:
env DISPLAY=:0.0 aticonfig --pplib-cmd "set fanspeed 0 100"
env DISPLAY=:0.1 aticonfig --pplib-cmd "set fanspeed 0 100"
env DISPLAY=:0.2 aticonfig --pplib-cmd "set fanspeed 0 100"
env DISPLAY=:0.3 aticonfig --pplib-cmd "set fanspeed 0 100"
[/s]

[glendall]

Thanks for posting

Haven't built a Linux miner yet but this time is rapidly approaching 8D 

[c789]

Good for you man...thanks for posting. I've been considering a distro from that side of the house for a few weeks but haven't seen anything as comprehensive as this. I and several others have had problems with 2 of the same cards giving the full hash rate on Xubuntu. I was fortunate in that plugging in a monitor solved my problem, but for others, they're still looking for a solution. Maybe this will help.

[Grumlin]

so, I have 2*7870, what I need to change? Could you change it for me please, just I bad understanding in it
2.6.32.12-0.7.1.xs6.0.2.553.170674xen #1 SMP Thu Jul 12 07:41:58 EDT 2012 i686 i686 i386 GNU/Linux

[Grumlin]

so, I have 2*7870, what I need to change? Could you change it for me please, just I bad understanding in it
2.6.32.12-0.7.1.xs6.0.2.553.170674xen #1 SMP Thu Jul 12 07:41:58 EDT 2012 i686 i686 i386 GNU/Linux

You are not running CEntOS. 

The way I install it is I download it onto a flash drive.  I like this one: 
http://www.pendrivelinux.com/universal-usb-installer-easy-as-1-2-3/

I am testing different kernels right now.  You can probably use the latest kernel.  At the time of this posting the last LIVECD is CentOS-6.3-x86_64-LiveCD.iso which you can download here: http://mirror.liberty.edu/pub/CentOS/6.3/isos/x86_64/

so, it's very bad, because I have a XenServer 6.0.2 and I can't reinstall OS. Maybe I have a variant?

[Viceroy]

you have this: http://www.citrix.com/products/xenserver/overview.html

you can put centos on it, yes.  I do not know how.  start here maybe:
http://forums.citrix.com/thread.jspa?threadID=302677

[Grumlin]

you have this: http://www.citrix.com/products/xenserver/overview.html

you can put centos on it, yes.  I do not know how.  start here maybe:
http://forums.citrix.com/thread.jspa?threadID=302677

well, but I don't need to X server. your link show it.
Or you mean that Driver of ATI don't stay on console OS without X???

[Viceroy]

well, but I don't need to X server. your link show it.
Or you mean that Driver of ATI don't stay on console OS without X???

You cannot mine without X Server.  Must 'startx' to operate, yes.  ATI makes own X driver is called fglrx appears when you compile.  (####  "GENERATE DISTRIBUTION SPECIFIC PACKAGE" )

[btcminer021]

For those that care, CentOS is binary-equivalent to RedHat. The only difference is that the RH logos are not included in the CentOS distro

Thank you Open Source!

Oh, and no technical support for CentOS (unless you consider Google and forums support)

[gnarcoinguy]

Unfortunately, even with CentOs I get the same results as I get with Mint and Ubuntu regarding dual 6950s.  I can run one just fine, and the second GPU is visible in lspci, but as soon as I enable both GPUs in the xorg.conf (aticonfig --adapter=all --initial --force ) I can no longer boot into Gnome/Cinammon/X.  On Debian variants the system hangs entirely, and I end up having to live boot off a USB drive to mount the HD and restore the old xorg.conf.  At least on CentOS I can still SSH into the system and restore my old xorg.conf, Debian hasn't been so kind. 

If anyone has any ideas I'd be much obliged.  I just haven't been able to get dual GPUs to mine in Linux, no matter what Distro.  It seems that they must both be enabled in Xorg.conf in order to have the option of being addressable and usable, but by enabling both I always encounter failures to load the fglrx module of some sort or another---and I've tried about 10 different driver versions at this point (and multple SDKs).

If I install Doze temporarily I can mine like a champ, but doze sucks and I need a headless solution.  I'm running out of ideas. 

[Viceroy]

Yep.  I'm testing different versions Centos kernels now.   I've used it for a long time and as far as I can tell it looks like something changes around centos 6.3 where the cards are no longer visable.  Are you saying you can actually see all your cards with lspci on Centos?  What distro? 

The problem I've had running dual dual-gpu cards is that I cannot see the second card in lspci.  If I can see it there I can mine with it.

Are you following this guide from the beginning?

[gnarcoinguy]

Yeah I've literally tried a dozen ways.  I followed your guide to a T (with SDK 2.7 and AMD 12.3) and still the same issue.  In all the linux variants I've tried though I can get both cards to show with lspci, however I can only mine on one of them (the active one).

So yeah with Centos6.4 I can see both cards, here is the output:

lspci sees both cards:

# lspci | grep -i VGA
01:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Cayman PRO [Radeon HD 6950]
02:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI Cayman PRO [Radeon HD 6950]

uname -ar
Linux tdavis-miner 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Notice the clinfo does NOT see two usable GPUs:

# ./clinfo | grep CL_DEVICE
  Device Type:                CL_DEVICE_TYPE_GPU
  Device Type:                CL_DEVICE_TYPE_CPU

aticonfig sees them though:

# aticonfig --list-adapters
* 0. 01:00.0 AMD Radeon HD 6900 Series
  1. 02:00.0 AMD Radeon HD 6900 Series

* - Default adapter

My belief is both GPUs aren't available for OpenCL mining is becuase only one is active.  Hence the need to 'activate' both via the Xorg.conf (like others have done).  Unfortunately that always leads to a dead screen upon reboot as mentioned earlier.

I've even tried creating a dummy DVI dongle to trick the damn thing into thinking everything is a-ok.  But still no luck......

[gnarcoinguy]

The problem I've had running dual dual-gpu cards is that I cannot see the second card in lspci.  If I can see it there I can mine with it.

How do you get it to mine from there?  I've had no luck no matter what distro.  What does ./clinfo show you in the SDK?

[Viceroy]

The problem I've had running dual dual-gpu cards is that I cannot see the second card in lspci.  If I can see it there I can mine with it.

How do you get it to mine from there?  I've had no luck no matter what distro.  What does ./clinfo show you in the SDK?

Your output looks fine.  

Code:

$ ./clinfo | grep CL_DEVICE
  Device Type:                                   CL_DEVICE_TYPE_GPU
  Device Type:                                   CL_DEVICE_TYPE_GPU
  Device Type:          

$ aticonfig --lsa  #this is a dual gpu card but looks the same as having two graphics cards
* 0. 06:00.0 AMD Radeon HD 6800 Series
  1. 07:00.0 AMD Radeon HD 6800 Series

* - Default adapter

first check, I'm sure you did this...  
aticonfig --force --adapter=all --initial

I have never had to edit my Xorg, I just let the system generate it's own.  I presume you went to the ATI Linux Wiki for further testing.  I, honestly, don't see anything wrong with your output.

After compiling cgminer what happens when you run
./cgminer -n

Code:

$ cgminer -n
 [2013-04-11 09:33:14] CL Platform 0 vendor: Advanced Micro Devices, Inc.
 [2013-04-11 09:33:14] CL Platform 0 name: AMD Accelerated Parallel Processing
 [2013-04-11 09:33:14] CL Platform 0 version: OpenCL 1.2 AMD-APP (923.1)
 [2013-04-11 09:33:14] Platform 0 devices: 2
 [2013-04-11 09:33:14]  0       Barts
 [2013-04-11 09:33:14]  1       Barts
 [2013-04-11 09:33:14] GPU 0 AMD Radeon HD 6800 Series  hardware monitoring enabled
 [2013-04-11 09:33:14] GPU 1 AMD Radeon HD 6800 Series  hardware monitoring enabled
 [2013-04-11 09:33:14] 2 GPU devices max detected

The above allows me to mine on two GPU's with ./cgminer using a command like this:
./cgminer -o stratum+tcp://mining.eligius.st:3334 -u 1NQc6tCtRmT8zBeLKNpboFaUBMLcxDwUm6 -p x0 -I 12

#substitute your own address, obviously, unless you want to make ME coin  

>My belief is both GPUs aren't available for OpenCL mining is becuase only
>one is active.  Hence the need to 'activate' both via the Xorg.conf (like others have done). 
>Unfortunately that always leads to a dead screen upon reboot as mentioned earlier.

My experience has been that when " lspci -nn | grep VGA "  sees all your cards you should
be able to use them all during mining.

[gnarcoinguy]

Except if you look closely my output only shows one GPU with clinfo.  The other is the AMD CPU.  So it doesn't actually see the extra GPU available for use. 

# ./clinfo | grep CL_DEVICE
  Device Type:                CL_DEVICE_TYPE_GPU
  Device Type:                CL_DEVICE_TYPE_CPU

Any chance you could post your xorg.conf?  I'm curious how it looks. Whenever I create one with aticonfig --force --adapter=all --initial I can no longer boot into X and end up having to fix things in single user mode.  This is with two monitors even hooked up for testing, dummy plugs, only one monitor, etc. 

After compiling cgminer it only sees one usable card as well, which matches up with the clinfo output. I would love to see your Xorg.conf..... And do you have a second monitor hooked up?

[Viceroy]

I've never edited this file directly.  When X doesn't start and I'm stuck at login I will often rerun aticonfig --intial ...

I have two monitors, sometimes three.  I screw around with them furiously plugging them into various dvi ports until some unknown magic allows lspci to see multiple cards.  Then I never hard boot again.  The hard boot seems to effect lspic but soft boots do not.  I am suspicious that the problem is related to later versions of the OS.  Something "bad" happened recently when they removed the old Xorg and replaced it with a newer version.  I'm thinking Centos 5 is what we need here....  Per my message at the top... I wish I had my old working system and never began the update process to begin with UGH!

Code:

more xorg.conf
Section "ServerLayout"
        Identifier     "aticonfig Layout"
        Screen      0  "aticonfig-Screen[0]-0" 0 0
        Screen         "aticonfig-Screen[1]-0" RightOf "aticonfig-Screen[0]-0"
EndSection

Section "Module"
EndSection

Section "Monitor"
        Identifier   "aticonfig-Monitor[0]-0"
        Option      "VendorName" "ATI Proprietary Driver"
        Option      "ModelName" "Generic Autodetecting Monitor"
        Option      "DPMS" "true"
EndSection

Section "Monitor"
        Identifier   "aticonfig-Monitor[1]-0"
        Option      "VendorName" "ATI Proprietary Driver"
        Option      "ModelName" "Generic Autodetecting Monitor"
        Option      "DPMS" "true"
EndSection

Section "Device"
        Identifier  "aticonfig-Device[0]-0"
        Driver      "fglrx"
        BusID       "PCI:6:0:0"
EndSection

Section "Device"
        Identifier  "aticonfig-Device[1]-0"
        Driver      "fglrx"
        BusID       "PCI:7:0:0"
EndSection

Section "Screen"
        Identifier "aticonfig-Screen[0]-0"
        Device     "aticonfig-Device[0]-0"
        Monitor    "aticonfig-Monitor[0]-0"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24
        EndSubSection
EndSection

Section "Screen"
        Identifier "aticonfig-Screen[1]-0"
        Device     "aticonfig-Device[1]-0"
        Monitor    "aticonfig-Monitor[1]-0"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24
        EndSubSection
EndSection

[gnarcoinguy]

Yeah aticonfig sets up virtually the same xorg.conf for me as well, but even if I have second monitor plugged in it still boots to a black screen and I can't do anything.  Can't even CTRL-ALT-F1 to a prompt.  The only difference between ours is the BUSID line is different and corresponds to the lspci output on my machine (as it should). 

I'm out of ideas here.  So you said Centos 5 did work?

[Viceroy]

I'm out of ideas here.  So you said Centos 5 did work?

It did, though I can't recall the subversion.  Centos 6 was working too, before the upgrade.  Whatever the upgrade was it was within the last 12 months.  What broke my system was doing a yum upgrade.  So I HIGHLY recommend starting with a CD and NOT contacting the latest repositories because that's what broke mine.  I'm trying to set up yum.comf so it only goes to the vault, not the new mirrors.  As soon as I yum upgraded I could no longer see more than 2 gpu's on my pci bus...   I've got the Centos 6.0 live iso.  I'm gonna try that WITHOUT upgrading beyond the vault.

[gnarcoinguy]

Cool, I'm downloading 5.9 now, we will see if it works.  Let me know how 6 goes. Also you can put an exclude line in your yum.conf to keep yum from trying to upgrade any important packages---of course this would require you to know what upgrade broke it previously. 

Example exclude line that would prevent mysql, bind, cups, gdm, and heartbeat from updating.

exclude=mysql* MySQL* bind* cups* gdm heartbeat*

[Viceroy]

Cool, I'm downloading 5.9 now, we will see if it works.  Let me know how 6 goes. Also you can put an exclude line in your yum.conf to keep yum from trying to upgrade any important packages---of course this would require you to know what upgrade broke it previously. 

Example exclude line that would prevent mysql, bind, cups, gdm, and heartbeat from updating.

exclude=mysql* MySQL* bind* cups* gdm heartbeat*

STOP  5.9 won't work as well.. it doesnt' have a single LIVE cd.  You need to do 5.8 or 6.0.  Good news on 6.0 reinstall... I'm seeing both my cards and I'm predicting if I can lock the yum to base distro's I can get multi-gpu mining working again...

* Viceroy  exuding confidence