CCminer(SP-MOD) Modded GPU kernels.

[joblo]

I had to disable SATA and USB3.0 otherwise card #6 sometimes doesn't show up.

Isn't this the problem exactly?

Edit: And it probably is related to PCIe lanes because those devices sometimes use them.

[1714629991]

1714629991

[1714629991]

1714629991

[1714629991]

1714629991

[exemplaar]

Hi crysx, guys.

I have two rigs, with the exact same motherboard and processor you have, with 6 x 750Ti working perfectly. (plus 1 rig with G3250)

16x Slot has a 16x Riser, the rest 1x powered risers.

I had to disable SATA and USB3.0 otherwise card #6 sometimes doesn't show up.

so i don't think it is an issue with lanes count in this case, it got to be driver, or it would make no sense why it works with 6 x 750Ti but not more than 5 , with newer cards.

that processor is crappy as hell, had to play with intensities sometimes to make it work, no problem with G3250 can handle anything

Why are you using x16 riser on x16 slot. PCI-E bandwidth practically does not matter at all when mining.  x1 riser is better cause it puts lower strain on chipset/cpu.

6xGTX950/960/970 also works on H81PRO-BTC. I do not know for 6x980. Seems only 6x980Ti is a problem right now.

[AzzAz]

Hi crysx, guys.

I have two rigs, with the exact same motherboard and processor you have, with 6 x 750Ti working perfectly. (plus 1 rig with G3250)

16x Slot has a 16x Riser, the rest 1x powered risers.

I had to disable SATA and USB3.0 otherwise card #6 sometimes doesn't show up.

so i don't think it is an issue with lanes count in this case, it got to be driver, or it would make no sense why it works with 6 x 750Ti but not more than 5 , with newer cards.

that processor is crappy as hell, had to play with intensities sometimes to make it work, no problem with G3250 can handle anything

Why are you using x16 riser on x16 slot. PCI-E bandwidth practically does not matter at all when mining.  x1 riser is better cause it puts lower strain on chipset/cpu.

6xGTX950/960/970 also works on H81PRO-BTC. I do not know for 6x980. Seems only 6x980Ti is a problem right now.

well 5x 980 + 1 x 960 works too. G1820, 8gb ram... Only difference is 980Ti has 6GB mem and is bit more sensitive power-wise

[induktor]

I had to disable SATA and USB3.0 otherwise card #6 sometimes doesn't show up.

Isn't this the problem exactly?

Edit: And it probably is related to PCIe lanes because those devices sometimes use them.

You are right, so it could be a lane issue if crysx didn't disable it, crysx can you check that?, would be interesting to know.

[induktor]

Why are you using x16 riser on x16 slot. PCI-E bandwidth practically does not matter at all when mining.  x1 riser is better cause it puts lower strain on chipset/cpu.

6xGTX950/960/970 also works on H81PRO-BTC. I do not know for 6x980. Seems only 6x980Ti is a problem right now.

Because i have a LOT of 16x risers , and less 1x risers, but yes, 1x risers would be better, will use less lanes indeed.

[bensam1231]

Hi crysx, guys.

I have two rigs, with the exact same motherboard and processor you have, with 6 x 750Ti working perfectly. (plus 1 rig with G3250)

16x Slot has a 16x Riser, the rest 1x powered risers.

I had to disable SATA and USB3.0 otherwise card #6 sometimes doesn't show up.

so i don't think it is an issue with lanes count in this case, it got to be driver, or it would make no sense why it works with 6 x 750Ti but not more than 5 , with newer cards.

that processor is crappy as hell, had to play with intensities sometimes to make it work, no problem with G3250 can handle anything

Why are you using x16 riser on x16 slot. PCI-E bandwidth practically does not matter at all when mining.  x1 riser is better cause it puts lower strain on chipset/cpu.

6xGTX950/960/970 also works on H81PRO-BTC. I do not know for 6x980. Seems only 6x980Ti is a problem right now.

Amount of lanes doesn't matter, there is no such thing as 'putting less strain on the CPU by using less lanes'. The chipset chooses where to put lanes and when. If you had 7 slots that are all x16 slots, they all don't get allocated x16 lanes even if they have a physical connection for it. Using more or less lanes wont increase or decrease the amount of work your chipset or CPU is doing.

x1 risers are just easier to maneuver, otherwise there is no real difference between them unless you need the bandwidth.

[tbearhere]

it IS a dual core celeron g1840 intel chip ... 5 cards will work without issue ... 6 wont ... h81btcpro motherboard ...

the ram is kingston hyperx 1600 16gb ( 2 x 8gb ) and is the most ram you can put in these systems apparently ...

there are TWO psu - corsair ax1200i ... more than enough to power the whole system ...

#crysx

FWIW: That CPU maxes out at 16x PCIe 3.0 lanes.
http://ark.intel.com/products/80800/Intel-Celeron-Processor-G1840-2M-Cache-2_80-GHz

Depending on the motherboard implementation (specific lane design/splitting/switching), you'll run out of PCIe lanes to feed the last GPU. Any features that you disable on the BIOS, might help, especially if it has SATA, M.2, USB3.0, or any setting to force PCIe v2.0/1.0 mode. I didn't look into the motherboard specifics at all, so I'm just generally speaking in this case.
A quick enough test, would be to plug in a more capable CPU, if you happen to have one at hand... Good luck with that! (seems like nothing on the regular consumer line would do)

There you go... good job myagui.
In my case I ran out of lanes and my psu maybe bad. That's why I am having so many problems. But it does work with 6 low end gpu's = 750ti's.. but not with the highend but then again I have psu issues.
So my extra 970gtx can't be used in that rig. I should try taking out the 750ti and put the 970gtx in its place to see if that will work. Not sure if the psu will handle it. It cuts off sometimes at 900 watts and it is rated 1300 continuous.
That is my second psu replacement from  ANTEC.  I will never buy ANTEC again. To send it back for a second rma would cost me more BTC then buying a new different psu.
Again the rig that has issues has 2 980ti  1  970gtx  1 750ti.

[Amph]

it IS a dual core celeron g1840 intel chip ... 5 cards will work without issue ... 6 wont ... h81btcpro motherboard ...

the ram is kingston hyperx 1600 16gb ( 2 x 8gb ) and is the most ram you can put in these systems apparently ...

there are TWO psu - corsair ax1200i ... more than enough to power the whole system ...

#crysx

FWIW: That CPU maxes out at 16x PCIe 3.0 lanes.
http://ark.intel.com/products/80800/Intel-Celeron-Processor-G1840-2M-Cache-2_80-GHz

Depending on the motherboard implementation (specific lane design/splitting/switching), you'll run out of PCIe lanes to feed the last GPU. Any features that you disable on the BIOS, might help, especially if it has SATA, M.2, USB3.0, or any setting to force PCIe v2.0/1.0 mode. I didn't look into the motherboard specifics at all, so I'm just generally speaking in this case.
A quick enough test, would be to plug in a more capable CPU, if you happen to have one at hand... Good luck with that! (seems like nothing on the regular consumer line would do)

There you go... good job myagui.
In my case I ran out of lanes and my psu maybe bad. That's why I am having so many problems. But it does work with 6 low end gpu's = 750ti's.. but not with the highend but then again I have psu issues.
So my extra 970gtx can't be used in that rig. I should try taking out the 750ti and put the 970gtx in its place to see if that will work. Not sure if the psu will handle it. It cuts off sometimes at 900 watts and it is rated 1300 continuous.
That is my second psu replacement from  ANTEC.  I will never buy ANTEC again. To send it back for a second rma would cost me more BTC then buying a new different psu.
Again the rig that has issues has 2 980ti  1  970gtx  1 750ti.

SP is running 6x970 on one rig with a small cpu so it definitely possible

[myagui]

For any of you guys that actually have the 6 GPUs working (or even someone maxed out at 5), see if you can check the actual PCIe configuration that your cards are running. For instance, I only have 2 GPUs. At most, my motherboard/cpu can only handle x16 PCIe 3.0 lanes, and my GPUs connect with 8x PCIe 3.0 each, as reported by Nvidia Inspector (no risers here).

Something like:
GPU Slot 1 - GTX980 - PCIe 3.0 x8
GPU Slot 2 - GTX980 - PCIe 3.0 x8

On Windows, one can use Nvidia Inspector to see what the actual connection rate is. The speed that a GPU gets from the PCIe slot, results of a startup handshake between the card itself and the chipset that is feeding the slot. The card will ask for the most that it supports (usually PCIe 3.0 x16), and the chipset will provide the most that it has available overall.

Different cards (and definitely different BIOS settings/features) might produce different results. While obviously slots that run in PCIe v2.0 mode are providing less bandwidth, I'm not sure that the actual number of lanes being used is any lesser in 2.0 vs 3.0.
Fun stuff though, wish I was troubleshooting a 6x 980 TI rig here myself  

[tbearhere]

it IS a dual core celeron g1840 intel chip ... 5 cards will work without issue ... 6 wont ... h81btcpro motherboard ...

the ram is kingston hyperx 1600 16gb ( 2 x 8gb ) and is the most ram you can put in these systems apparently ...

there are TWO psu - corsair ax1200i ... more than enough to power the whole system ...

#crysx

FWIW: That CPU maxes out at 16x PCIe 3.0 lanes.
http://ark.intel.com/products/80800/Intel-Celeron-Processor-G1840-2M-Cache-2_80-GHz

Depending on the motherboard implementation (specific lane design/splitting/switching), you'll run out of PCIe lanes to feed the last GPU. Any features that you disable on the BIOS, might help, especially if it has SATA, M.2, USB3.0, or any setting to force PCIe v2.0/1.0 mode. I didn't look into the motherboard specifics at all, so I'm just generally speaking in this case.
A quick enough test, would be to plug in a more capable CPU, if you happen to have one at hand... Good luck with that! (seems like nothing on the regular consumer line would do)

There you go... good job myagui.
In my case I ran out of lanes and my psu maybe bad. That's why I am having so many problems. But it does work with 6 low end gpu's = 750ti's.. but not with the highend but then again I have psu issues.
So my extra 970gtx can't be used in that rig. I should try taking out the 750ti and put the 970gtx in its place to see if that will work. Not sure if the psu will handle it. It cuts off sometimes at 900 watts and it is rated 1300 continuous.
That is my second psu replacement from  ANTEC.  I will never buy ANTEC again. To send it back for a second rma would cost me more BTC then buying a new different psu.
Again the rig that has issues has 2 980ti  1  970gtx  1 750ti.

SP is running 6x970 on one rig with a small cpu so it definitely possible

But it's a 4 core sp has. But the psu is probably bad.

[crysx]

For any of you guys that actually have the 6 GPUs working (or even someone maxed out at 5), see if you can check the actual PCIe configuration that your cards are running. For instance, I only have 2 GPUs. At most, my motherboard/cpu can only handle x16 PCIe 3.0 lanes, and my GPUs connect with 8x PCIe 3.0 each, as reported by Nvidia Inspector (no risers here).

Something like:
GPU Slot 1 - GTX980 - PCIe 3.0 x8
GPU Slot 2 - GTX980 - PCIe 3.0 x8

On Windows, one can use Nvidia Inspector to see what the actual connection rate is. The speed that a GPU gets from the PCIe slot, results of a startup handshake between the card itself and the chipset that is feeding the slot. The card will ask for the most that it supports (usually PCIe 3.0 x16), and the chipset will provide the most that it has available overall.

Different cards (and definitely different BIOS settings/features) might produce different results. While obviously slots that run in PCIe v2.0 mode are providing less bandwidth, I'm not sure that the actual number of lanes being used is any lesser in 2.0 vs 3.0.
Fun stuff though, wish I was troubleshooting a 6x 980 TI rig here myself  

i think the easiest way to test this theory IS to buy a more powerful cpu and replace the celeron thats in there ...

this will squash any theories about it being a cpu issue ...

this machine has no issue with 6 x gigabyte 750ti oc lp cards ... none whatsoever ...

6 x gigabyte 980ti g1 though - maxes out at 5 ...

i may take pics of the temporary shelf its sitting on at the moment - as ive pulled it from the custom aluminium frame i built for it earlier ( which will allow me to improve the frame further while the components are out ) ... the machine is not going back to its owner for quite a while now - as he will be overseas for some time with work now ( initially it was just a holiday ) ... so lucky me - that i get to 'play' with this machine more ...

had we been neighbours myagui - there would be no issue with you messing with this system ...

i wont be able to test much though - as ive seen the new warehouse and applied for it - it seems there maybe a few hiccups with it - so i applied for another place that is much smaller - but will allow thefarm to start again in the next couple of weeks ... looks like this second one is going through - so ill be very busy with the preparation over the next few days ... ill know tomorrow after the paperwork is finalized and the money is paid for the initial deposit ...

which is a little frustrating as it is happy - as this issue will have to wait until thefarm is back up in the new premises ...

i cant wait till the 1080 takes root into the nvidia space - as im expecting a great many 980ti g1 / extremes to be dumped on the market at half the price - or thereabouts ...

apart from that - the new umbrella is slowly taking place - and development is continuing for the improvement AND growth of thegrid and thefarm ... as well as the development of the coins and miners involved ...

if all goes well though - the relocation and new home of thefarm will start next week ... which means ill have my hands full for at least a week after all is delivered ...

ill be setting a new thread soon with the new name and details ... as well as theforum - and continue to update there also ...

tanx for all your help everyone ... ill get back to rebuilding this beats when i get the chance to after the relocation ...

too much to do now ...

#crysx

[myagui]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

Edit: One last note on this mental exercise: I'm with joblo in suspecting that it could just be that the problem is when you cross the 32GB boundary in memory attached to PCIe devices. If this is the case, you'll find that the same rig will max out at 4 * GTX1080's 

[crysx]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

may be ...

ill look into it when the rest of the 'bigger fish to fry' are actually fried and done ... ...

internet / electricity / cabling / heat dissipation ( which ive been redeveloping into a more productive animal - and its working - even to the extent that it will work more efficiently WITHOUT its fans ) / etc ...

when the frame is set also - will have them sent to a metal fabrication / engineering place to optimize space and heat flow also ...

meanwhile - this issue IS an issue - and no so much a problem ... yet ...

when density starts to play its hand here - then i have a problem and will HAVE to find a solution shortly after ...

all that can be switched off in the bios - is ... so it will boil down to whether the theory of the cpu power / lane is actually the issue ... i highly doubt it - but it is moments like this that i am really eager to be wrong here - and will have the utmost elation if in fact i am completely wrong ...

i just want all 6 x gb 980ti g1 / extreme working ...

btw - i can be found more on #digitaldzyne irc channel ( the new umbrella name - but will post a new thread about it later ) and will keep that channel uptodate as to what is happening ... as well as the new thread when i get that done also ... IF you want to chat more about this - of course ...

tanx again ...

#crysx

[antonio8]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

Edit: One last note on this mental exercise: I'm with joblo in suspecting that it could just be that the problem is when you cross the 32GB boundary in memory attached to PCIe devices. If this is the case, you'll find that the same rig will max out at 4 * GTX1080's  

I have a 6 card 750ti setup and 5 cards are in 3.0 @ 2.0x1 and 1 card is 3.0 @ 2.0x4

[tbearhere]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

Edit: One last note on this mental exercise: I'm with joblo in suspecting that it could just be that the problem is when you cross the 32GB boundary in memory attached to PCIe devices. If this is the case, you'll find that the same rig will max out at 4 * GTX1080's 

Thx myagui
I'm going to look now. One thing, the asrock will reset to defaults on power interrupt..so I have to go and check things out. I do many power interrupts.

[crysx]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

Edit: One last note on this mental exercise: I'm with joblo in suspecting that it could just be that the problem is when you cross the 32GB boundary in memory attached to PCIe devices. If this is the case, you'll find that the same rig will max out at 4 * GTX1080's 

Thx myagui
I'm going to look now. One thing, the asrock will reset to defaults on power interrupt..so I have to go and check things out. I do many power interrupts.

not all of them do tbear ...

but then again - its hard to tell with this one - as ive just updated the bios yesterday - so its prone to do anything ...

#crysx

[tbearhere]

http://ark.intel.com/products/80798/Intel-Celeron-Processor-G1850-2M-Cache-2_90-GHz?q=G1850

My cpu on 2 rigs............... this computer and my little vista are 4 core.

[myagui]

I have a 6 card 750ti setup and 5 cards are in 3.0 @ 2.0x1 and 1 card is 3.0 @ 2.0x4

Cool antonio8, that is right about what I would expect from a working setup.
My napkin math tells me that the one card in the faster slot (which you have running at x4), might be doing x8 on the systems that are maxing out at 5 GPUs. If that faster slot runs at x8, lanes will be pretty tight, depending on how the rest of the onboard devices are wired up.

If you have a chance, what is on your BIOS for that PCIe setting I posted before? If it is set to auto, please post what other options are available for that setting (but don't change it ofc, as you have a working system)

@crysx, the damn fish won't fry itself buddy, get moving 

[crysx]

I have a 6 card 750ti setup and 5 cards are in 3.0 @ 2.0x1 and 1 card is 3.0 @ 2.0x4

Cool antonio8, that is right about what I would expect from a working setup.
My napkin math tells me that the one card in the faster slot (which you have running at x4), might be doing x8 on the systems that are maxing out at 5 GPUs. If that faster slot runs at x8, lanes will be pretty tight, depending on how the rest of the onboard devices are wired up.

If you have a chance, what is on your BIOS for that PCIe setting I posted before? If it is set to auto, please post what other options are available for that setting (but don't change it ofc, as you have a working system)

@crysx, the damn fish won't fry itself buddy, get moving 

hehehe ...

full steam ahead and all systems go ...

i just need to get organized now ...

this is NOT going to be easy - as its just me doing the physical work now ...

fish-a-fryin soon ...

#crysx

[tbearhere]

Adding to my previous post...
Had a quick look at the AsRock BTC Pro manual, and couple of things sparked my interest:
- Onboard NIC uses x1 PCIe lane!
- 1 Slot can do PCIe v2.0 x16
- 5 Slots are PCIe v2.0 x1

I don't know what other Onboard devices might be using up lanes as those are not clearly stated (though I posted about the usual suspects already). It would be wise to disable every single motherboard/chipset feature that is not critical to the rig operation.

@crysx & others, what setting do you have on the BIOS for "PCIE2 Link Speed"? Maybe worth fiddling with the possible options there.

Edit: One last note on this mental exercise: I'm with joblo in suspecting that it could just be that the problem is when you cross the 32GB boundary in memory attached to PCIe devices. If this is the case, you'll find that the same rig will max out at 4 * GTX1080's  

Thx myagui
I'm going to look now. One thing, the asrock will reset to defaults on power interrupt..so I have to go and check things out. I do many power interrupts.

Ok PCIE2 Link  set in Bios.... doing some test runs. And I added the other 970gtx.