7/8 GPUs per Motherboard

[Madazam]

Thx man, this really helps in configuration and increasing my hashrate. My prob is that, one GPU core simply isn't there in device manager no yellow nothing. I am using win7 32 bit, 11.12 driver and and i think 2.5 sdk.

I know i can work it out with 64 bit and win8 does support more gpu than 5 but dont wanna change the whole system for this......if this can be solved with some other way......

Do you get another yellow icon in device manager for PCI-e controller (with error message that there is no enough system resources)? if so try to rearrange cards to different PCI slots - it did help me. In my experience: if GPU doesn't show up in device manager with yellow icon - no driver will help.

Thx mate. May be I will try Bamt. Have u used it ?

[kslavik]

Didn't try yet - I might try it in the future to avoid windows all together.

[Madazam]

Thanks for the help mate. I solved it with a hint from one of your post above.

The missing GPU core was not yellow, bit it was installed as other devices as a video controller. I updated the driver from 11.12 driver directory. After reboot it crashed once, when i quit the cgminer but after that running stable with 6 cores showing of "AMD ATI Radeon 5900" in device manager.

Now just have to patch the kernel to increase the limit of Ram from 4 GB to 8gb ... ...need that for scrypt...

[kslavik]

Thanks for the help mate. I solved it with a hint from one of your post above.

The missing GPU core was not yellow, bit it was installed as other devices as a video controller. I updated the driver from 11.12 driver directory. After reboot it crashed once, when i quit the cgminer but after that running stable with 6 cores showing of "AMD ATI Radeon 5900" in device manager.

Now just have to patch the kernel to increase the limit of Ram from 4 GB to 8gb ... ...need that for scrypt...

I'm mining scrypt wth 3x5970 and 4gb - CGMiner doesn't use system memory - it uses GPU memory instead.

I didn't know it was possible to patch Windows 7 Kernel to increase memory to 8 GB. Let me know how if you are successfull

[Madazam]

Thanks for the help mate. I solved it with a hint from one of your post above.

The missing GPU core was not yellow, bit it was installed as other devices as a video controller. I updated the driver from 11.12 driver directory. After reboot it crashed once, when i quit the cgminer but after that running stable with 6 cores showing of "AMD ATI Radeon 5900" in device manager.

Now just have to patch the kernel to increase the limit of Ram from 4 GB to 8gb ... ...need that for scrypt...

I'm mining scrypt wth 3x5970 and 4gb - CGMiner doesn't use system memory - it uses GPU memeory instead.

I didn't know it was possible to patch Windows 7 Kernel to increase memory to 8 GB. Let me know how if you are successfull

I am trying to do the same thing (3x5970), how much KHs are you getting ? if possible can you share your settings?Coz when i run with my 3x5970 the system hangs. If i dont need that much ram i would not patch the kernel....

I am trying to patch with this ...

http://www.evga.com/forums/tm.aspx?m=960087&mpage=1

http://prasys.info/2012/05/enabling-more-than-4gb-ram-on-windows-7-32-bit/

http://www.unawave.de/windows-7-tipps/32-bit-ram-barrier.html?lang=EN

[kslavik]

I'm getting 394Khs per GPU core, so 394x6=2364Khs total

do you see all your GPUs inside Device Manager without yellow icons. if so, you will be ready to run your 5970 to mine scrypt.

Here is what I have inside cgminer.conf:
CGMiner id 3.3.1

...
"intensity" : "18,18,18,18,18,18",
"vectors" : "1,1,1,1,1,1",
"worksize" : "256,256,256,256,256,256",
"kernel" : "scrypt,scrypt,scrypt,scrypt,scrypt,scrypt",
"lookup-gap" : "0,0,0,0,0,0",
"thread-concurrency" : "6400,6400,6400,6400,6400,6400",
"gpu-engine" : "840,840-840,840,840,840,840",
"gpu-fan" : "30-85,50-85,50-85,50-90,50-90,50-92",
"gpu-memclock" : "1180,1180,1180,1180,1180,1180"
...

All GPU cores are set to 1099 voltage (do it before you run cgminer)

When you start cgminer, do it for 1 card first and then enable other GPUs one by one to make sure you have enough power to run it:

cgminer -d 0

Make sure that your GPUs are correctly mapped (if not, you will have to remap them manually, it is always the case for me. You would see that they are not mapped correctly if you have 1 gpu enabled but activity would show up for another gpu, let me know if it is the case for you, I will tell you how to map them correctly)

If your machine doesn't have enough power in PSU, the computer will just reboot or hang when you enable enough power draw. You need at least 1250W PSU to run 3x5970.

I recomend enabling 1 GPUs inside each card initially, wait for couple of minutes for the temperature and fan to reach about 80 degree and 4000 rpm, only then start enabling second GPU inside each card.

Let me know how it goes

[Madazam]

I'm getting 394Khs per GPU core, so 394x6=2364Khs total

do you see all your GPUs inside Device Manager without yellow icons. if so, you will be ready to run your 5970 to mine scrypt.

Here is what I have inside cgminer.conf:
CGMiner id 3.3.1

...
"intensity" : "18,18,18,18,18,18",
"vectors" : "1,1,1,1,1,1",
"worksize" : "256,256,256,256,256,256",
"kernel" : "scrypt,scrypt,scrypt,scrypt,scrypt,scrypt",
"lookup-gap" : "0,0,0,0,0,0",
"thread-concurrency" : "6400,6400,6400,6400,6400,6400",
"gpu-engine" : "840,840-840,840,840,840,840",
"gpu-fan" : "30-85,50-85,50-85,50-90,50-90,50-92",
"gpu-memclock" : "1180,1180,1180,1180,1180,1180"
...

All GPU cores are set to 1099 voltage (do it before you run cgminer)

When you start cgminer, do it for 1 card first and then enable other GPUs one by one to make sure you have enough power to run it:

cgminer -d 0

Make sure that your GPUs are correctly mapped (if not, you will have to remap them manually, it is always the case for me. You would see that they are not mapped correctly if you have 1 gpu enabled but activity would show up for another gpu, let me know if it is the case for you, I will tell you how to map them correctly)

If your machine doesn't have enough power in PSU, the computer will just reboot or hang when you enable enough power draw. You need at least 1250W PSU to run 3x5970.

I recomend enabling 1 GPUs inside each card initially, wait for couple of minutes for the temperature and fan to reach about 80 degree and 4000 rpm, only then start enabling second GPU inside each card.

Let me know how it goes

Thx for the detailed guide.

One question ? how to set voltage ? i thought cgminer does that when we state the voltage in cgminer.config?

I have a LEPA 1600 gold, do you think it will be enough?

[kslavik]

I use program called TRIXX (sapphire tweak utility), works great for 5970, Somehow cgminer always fails in setting the voltage for 5970, it works for other cards, just not for 5970.

1600 PSU should be enough to run 3x5970.

[ssateneth]

I use program called TRIXX (sapphire tweak utility), works great for 5970, Somehow cgminer always fails in setting the voltage for 5970, it works for other cards, just not for 5970.

1600 PSU should be enough to run 3x5970.

CGMiner can only set to voltages that are programmed into the programmable VRM. On a stock 5970, these voltages are 0.95, 1.0, 1.0375, and 1.05. Trixx and afterburner uses other methods to get the voltage set to different steps.

[Madazam]

I use program called TRIXX (sapphire tweak utility), works great for 5970, Somehow cgminer always fails in setting the voltage for 5970, it works for other cards, just not for 5970.

1600 PSU should be enough to run 3x5970.

CGMiner can only set to voltages that are programmed into the programmable VRM. On a stock 5970, these voltages are 0.95, 1.0, 1.0375, and 1.05. Trixx and afterburner uses other methods to get the voltage set to different steps.

thx man. As always valuable insight from a seasoned miner. Appreciate it

[Madazam]

I'm getting 394Khs per GPU core, so 394x6=2364Khs total

do you see all your GPUs inside Device Manager without yellow icons. if so, you will be ready to run your 5970 to mine scrypt.

Here is what I have inside cgminer.conf:
CGMiner id 3.3.1

...
"intensity" : "18,18,18,18,18,18",
"vectors" : "1,1,1,1,1,1",
"worksize" : "256,256,256,256,256,256",
"kernel" : "scrypt,scrypt,scrypt,scrypt,scrypt,scrypt",
"lookup-gap" : "0,0,0,0,0,0",
"thread-concurrency" : "6400,6400,6400,6400,6400,6400",
"gpu-engine" : "840,840-840,840,840,840,840",
"gpu-fan" : "30-85,50-85,50-85,50-90,50-90,50-92",
"gpu-memclock" : "1180,1180,1180,1180,1180,1180"
...

All GPU cores are set to 1099 voltage (do it before you run cgminer)

When you start cgminer, do it for 1 card first and then enable other GPUs one by one to make sure you have enough power to run it:

cgminer -d 0

Make sure that your GPUs are correctly mapped (if not, you will have to remap them manually, it is always the case for me. You would see that they are not mapped correctly if you have 1 gpu enabled but activity would show up for another gpu, let me know if it is the case for you, I will tell you how to map them correctly)

If your machine doesn't have enough power in PSU, the computer will just reboot or hang when you enable enough power draw. You need at least 1250W PSU to run 3x5970.

I recomend enabling 1 GPUs inside each card initially, wait for couple of minutes for the temperature and fan to reach about 80 degree and 4000 rpm, only then start enabling second GPU inside each card.

Let me know how it goes

Bless you man...got my rig working on scrypt. Now only 2 MH but thats because the damn XFX gets overheated and i have to run it on stock. But really appreciate it man, i had more or less given up on scrypt. And with difficulty rising, i think it is the way to go. Once again thanks 

will donate to you once i am able to mine some coins...

One question through? on my MSI Z77A-G45, rig gets hanged sometime, and than the 5970 dont work/get recognize on the same slot (whether PCI 1x or 16x)...after i change the card to some other slot they start work, have you ever faced anything like it ?Also you said regarding mapping 5970, how does that work?

I have found a crude of maping by changing cards in different slots etc. would be interested to know your way/thoughts?

Once again thx  .....

[kslavik]

Bless you man...got my rig working on scrypt. Now only 2 MH but thats because the damn XFX gets overheated and i have to run it on stock. But really appreciate it man, i had more or less given up on scrypt. And with difficulty rising, i think it is the way to go. Once again thanks  

will donate to you once i am able to mine some coins...

One question through? on my MSI Z77A-G45, rig gets hanged sometime, and than the 5970 dont work/get recognize on the same slot (whether PCI 1x or 16x)...after i change the card to some other slot they start work, have you ever faced anything like it ?Also you said regarding mapping 5970, how does that work?

I have found a crude of maping by changing cards in different slots etc. would be interested to know your way/thoughts?

Once again thx  .....

I don't know how to solve your hangs.
Several points:
- Disable Sleep option on your computer and disable "monitor switch off" after some period of time.
- Make sure you monitor is always connected to the same card and to the same port - if you change it, your cards will be mapped differently on reboot.
- Set intensity lower on the card/slot where the monitor is connected.
- Set resolution to a lower level of 1024x768 and only enable one monitor.
- if you are only using 5970 cards, you can just replace couple of files inside c:\windows\system32 (OpenCL.dll and amdocl.dll
) to get to downgrade to 2.5 SDK which is the best for 5970 cards.

Here is how to correctly map GPUs:
- Launch cgminer with one GPU enabled (0 for example), press g inside cgminer and see which GPU has activity inside it, if it is GPU 2, then your first mapping would be 0:2
- Disable GPU 0 and enable GPU 1, see the activity, if you see activity inside GPU 3, then your second mapping is 1:3
- Disable GPU 1 and enable GPU 2, see the activity, if you see activity inside GPU 0, then your third mapping 1:4
and so on

once you done for all the GPUs, put the following into cgminer.conf:
"gpu-map": "0:2,1:3,2:4....",

To combat heat, you should map GPUs correctly first, so auto-fan and auto-gpu would actually regulate the fan speed and the GPU speed of a right GPU. For the overheating card, enable auto-gpu  so it would auto adjust engine speed based on temperature and set fan range to 30-90, then set max GPU speed to 825 and memory speed to 1150, it would allow you to lower the voltage to 1050-1074 for this card only and would still hash at about 383MH.

Please consider my tool to always mine the best coin: https://bitcointalk.org/index.php?topic=260940, it will switch your cgminer to the most profitable coin based on current difficulty and exchange rate. (if you like it, you may click donate button to get even better results   )

[gnix72]

@ssateneth, several questions about your 8 GPU MSI Z77A-G45 setup.

1.  You mention that you have to use a processor with an Ivy Bridge core due to IRQ conflicts that disable the 8th GPU, there are number of Ivy Bridge cores of widely varying capability, and I assume you're using these for Scrypt mining, so does it matter which processor you get as long as it's Ivy Bridge?  Would Scrypt mining benefit from the increased processor cache from the i3 and i5 CPUs?

Ivy Bridge CPUs:
Celeron various G440 (1.6 GHz) through G1620 (2.7 GHz)
Core i3-3240, 3245, 3250
Core i5-3570, 3570K, 3770, 3770K
Full List at the bottom of this page: http://www.cpu-world.com/CPUs/Celeron_Dual-Core/Intel-Celeron%20G1620.html

2.  The MSI Z77A-G45 only has 7 PCI-e slots, so I'm assuming that you are using at least one 5970 to get up to 8 GPUs with 7 slots, does it matter which slot the dual GPU goes in and/or have you noticed problems using the dual GPU in any specific slots?  I've heard from other forums that sometimes using a dual GPU will disable adjacent PCI-e slots depending on the mobo config (with or without jumpers).

3.  Do you have to use the PCI-e 1x jumper procedure on the MSI Z77A-G45 mobo?

4.  If you do scrypt mine, how much system memory is required for your 8 GPU setup on the MSI mobo?  I've heard sometimes 1GB per GPU and other times as much GB as the video cards have, but everyone seems to agree that memory is important for scrypt mining.

Much appreciated on your informative posts here, packing as many GPUs per mobo as possible really helps to cut overall costs.

[kslavik]

I was finally able to run 8 GPU under Windows 7 Home 32bit:

7 GPUs are mining Scrypt and 1 Card 7970 is mining SHA256 (not enough memory to run it without hardware errors)

So, 3x5970
1x5830
1x7970

[superresistant]

I was finally able to run 8 GPU under Windows 7 Home 32bit:

7 GPUs are mining Scrypt and 1 Card 7970 is mining SHA256 (not enough memory to run it without hardware errors)

So, 3x5970
1x5830
1x7970

I don't get it. Which GPU count double ?

[BR0KK]

I was finally able to run 8 GPU under Windows 7 Home 32bit:

7 GPUs are mining Scrypt and 1 Card 7970 is mining SHA256 (not enough memory to run it without hardware errors)

So, 3x5970
1x5830
1x7970

I don't get it. Which GPU count double ?

Red = 2 GPU Cores on on card

[InSpades]

I have one of these boards in my setup.  The board has no problem recognizing 6 physical cards.  There seems to be a quirk with this board I have found that prevents it from recognizing card #7.  Seems that the board disables one of the 1x slots when the the adjacent 1x slot is occupied.  Found a couple of YouTube vids confirming this issue.

Under that premise, I see no reason why you couldn't load this board up with 12 GPUs.  I do not have any way of testing this though since all of my cards are single GPU.  I'd love to see pictures of something like that.

I settled with 5 x 7950 cards on mine.

Edited for clarification.

[kslavik]

I have one of these boards in my setup.  The board has no problem recognizing 6 physical cards.  There seems to be a quirk with this board I have found that prevents it from recognizing card #7.  Seems that the board disables one of the 1x slots when the the adjacent 1x slot is occupied.  Found a couple of YouTube vids confirming this issue.

Under that premise, I see no reason why you couldn't load this board up with 12 GPUs.  I do not have any way of testing this though since all of my cards are single GPU.  I'd love to see pictures of something like that.

I settled with 5 x 7950 cards on mine.

Edited for clarification.

I have heard that 8 GPU limit is imposed by the PCI express design or operating system design. I never seen anybody who run more than 8 GPUs on the same motherboard. Please post a link if you have seen otherwise.

[InSpades]

I have been told that this is a limitation of the OS.

[superresistant]

Just for your information : It is possible to run 2 motherboard with one PSU. If you can easily find cheap motherboard/gpu/ram that accept 4+ GPU (using pcie1x) then run 8+ GPU with one PSU.

I know that's not exactly what you aim for but it can be an alternative.