Bitcoin Forum

It's a boring Sunday afternoon so this topic popped into my head. There' been a lot of discussion about
rig building with some nice pictures, very interesting stuff.

But how big is too big? What is the limiting factor? Power limits of household circuits comes to mind
but that can be overcome with commercial power. Hard limits in software? With a well matched CPU and
lots of RAM just how many cards could a single rig support? Most big rigs seem to have 6 cards,
are there any bigger?

My fork supports up to 16 gpu's in a single rig. But most bioses have problems with more than 8. This might change with the nvlink interface.

Pci-e slots on the mainboard seem to be the primary factor.

I believe there are extenders to create more slots. PCIe bandwidth isn't an issue because you
can run a miner with one lane, so 1 16 lane slot could drive 16 GPUs mining theoretically.

OS and driver limitations could be an issue since neither Windows nor Nvidia are opensource.
Linux and open source AMD drivers (do they work?) might be a possibility since anyone with
the skills could mod the code to raise the limit.

i think it's not wise to go with many gpu on a single rig, you need anyway to pay for additional psu, nothing will change for the wattage, and you need to buy splitter that cost a lot, like the other part of another rig, which are cheap anyway

Well my 4th (kinda) Titan is going up under my 100 amp service (no issues but I have boiler heat and no central A/C) no issues

14 cubes 3 controllers 4 power supplies (3 corsair 1200i's and 1 seasonic 860w all platinum)

So if you flip your question to how big is too big...probably already too big (so addicted i am to home mining) :)

it is 'nerd city' in my basement however

lostgonzo.imgur.com (http://lostgonzo.imgur.com) (10 cubes shown no pics of last 4 cube rig yet)

Also note the 8 slot sub panel off my 32 sized slot 100 amp panel ...check out YouTube videos on such ..easy to do  if you need more slots.

(addicted I am to home mining ..the horror) :)

I tend to agree. Once you start needing multiple PSUs and port multipliers it's probably better to build a
seperate rig.

stick at 5 or 6 cards per rig.  


make a second rig at that point.


reasons are simple enough.

1)  if one rig fails  you are still mining on the other rig.

2)  cooling may be easier with two rigs   ie use a  separate room

3)  you can do one rig at each pool

4) cost for the asrock board is under 100

5) cost for windows 7 is under 50

6) cost for some ram is under 40

7) cheap ssd is under 60    

8 ) so the net extra is  under 250     you get a lot for the 250

9) if a second coin arises  besides Eth  one rig on eth on rig on new coin

Well put. A few comments:

2) limited cable length would increase the card density and heat concentration.

3) hadn't thought of that, a little easier than running 2 instances on one rig.

5) linux is free and has better remote management.

10) No risk rig will overload electric circuit.

well windows 7 is actually free, you know what i mean, you can argue that we can be infected because of torrent and stuff like that

but i have my own 100% secure copy(plenty of btc transaction using this copy, no infection in years) that i'll use for all my rig

Windows 10 is OFFICIALLY free (in at least one version) for a short while longer - if you don't mind issues with buggy driver versions etc.

 Seems like the only time Microsoft does a SERIOUS beta test of any of their OSs is right after they release a serious turkey (anyone remember MS-DOS 5 being the FIRST Microsoft OS with a wide beta release - after MS-DOS 4.0's issues with Smartdrive)?

IMO the hassles of coordinating multiple power supplies isn't worth doing - so however many cards you can run from one PS would be where I set the limit.

windows 10 is not that good if you have nvidia and want to mine etheruem, even with the drivers fix, you still will not reach the same hash like in windows 7

If you have more than 6 cards,you need powered risers . A 750ti draws 50-60 watt depending on what you mine. So a 16 cards rig will only draw around 1000W. The bios might need to be cracked to support 16 cards, and more than 8 cards in windows is a pain.  Have anyone tried 5 7990's in one single rig ? (10 gpu's)
You probobly need linux. Expensive PCIE splitters etc.

You'll be limited in the end by the OS card limits AND the make of card you're using (AMD or NVidia).

With Windows, it's usually 6-8 AMD cards (see lbr's AMD driver mods (https://bitcointalk.org/index.php?topic=712228) - tip him, plx) 

I have seen higher limits with *nix OS kernels, but that's with some serious hackery.

NVidia cards i am not sure about given my lack of experience with them and mining.

powered riser are useless if you have asrock h81 with two molex, i always used ribbon standard risers and they work fine for every card out there, also cost less than powered riser

That is right. When I use the H61 or H81 pro btc motherboard, I do not use powered risers. It is just a waste.

I like 6 cards per rig, it's my happy medium.

I have to disagree. Although the mobo can supply the power cheap risers sometimes can't. At least with powered risers
you're not trying to push 75W through a ribbon wire. It also reduces the overall power going through the mobo. The cost
difference is trivial, no more than .05 BTC with risers on all 6 cards.

If using two PSUs (not my thing) I believe you need powered risers to isolate them.

average wattage per card today, especially with new nvidia is very low, they consume 150w on average, very low as i said they will use the 8 pin if not enough current come from the x1 slot, i think you don't need to provide 75w fromt he slot, lol future gpu with 10nm or less, will consume so low that not even a 6 or 8 pin would be required, only the riser

Even 6 cards gives a lot of troubles sometimes - rig hangs\miner crashes. In my experience each additional card increases efforts to achieve stability exponentially. 5-6 cards per rig is good.

good thing about powered risers are you can put gpus far from each other for cooling. I have 5 gpus in a rig, if I change the distance just a few cms between them, i can see temperature change in a minute.

i cant imagine powering 16 Gpus. would be a nightmare. would a regular household put out that much?

Bad advice @"if you have more than 6 cards, you need powered risers".

Always use a powered riser on any card that takes more than what the x1 pci-e slot/ribbons (75w) + its pins are rated to provide.

Manufacturers expect these cards to be used in full x16 slots (150w) and as such place the appropriate x4/x6/x8 pins on them.

That extra 75w has to be made up elsewhere, and if you're using anything more powerful than a R9 270/GTX 950, there's a good chance that extra power is gonna come through the pins / pcie slot.  Expect fire.  And sparks.  Lots of sparks.  Powered risers are usually not even more expensive than unpowered risers most of the time.  Be safe - this is your investment(s) at stake here.

An unrelated FYI: the one thing you do want to make sure of is that you do not overload any of your 5v lines if you hook your risers up to them (molex / sata connector etc).

A big rig that is too big is one that is 1YHs (1 yotahashes).
The network difficulty is lower than that so it would push it up too high, though you would get most of the new bitcoin technically!
I'd assume that a domestic rig for hobby use should be no larger than 1000W.

Agree completely with powered risers but I think you're a little confused about power from the bus. The bus power
is supplied by the part of the PCIe slot that is common regardless of the number of lanes. A x16  connector
can't carry any more power than x1.

What is different with x1 slots is they are intended only for 25W (x16 is 75W). Unless the mobo is designed for
high power to the x1 slots (ie H81) you're asking for trouble without powered risers in those slots. Either way the maximum
power from the slot connector is 75W.

If the GPU needs more it will have 6 or 8 pin auxiliary connectors to supply an additional 75 or 150W respectively.
If your PSU doesn't have enough connectors you can combine 2 Molex or 2 SATA power connectors to one 6 pin
75W connector for the GPU. If your GPU takes 8 pin or 2x6pin you will need 4 Molex or 4 SATA power connectors.

6 Card rigs. It's the way to go, bail on windows and run Ubuntu. Stable as a rock are the rigs that I sell. No lockups, no overheating. Simple really.

those 75w will be drawed from the cables no problem as long as the gpu get its wattage that it need, it does not matter from where it come from

if you use a 8 pin and the gpu has a tdp of 180w, you are already good enough and you won't burn anything

@joblo, thanks for the corrections on the pcie x1 vs. x16 power delivery specs.  Correct information is what we want out there.

@Amph, you're right.  That's why i mentioned that using powered risers is really a must on "any card that takes more than what the x1 pci-e slot/ribbons (75w) + its pins are rated to provide."  If the rig is running a bunch of GTX 950s or R9 270s then fine.  OTOH, a bunch of 280Xs, 290s, 290Xs - that's playing with fire, literally.

Powered risers seem like a pain to hook up/use, and possibly even negative in light of the ROI question - but why take the risk of stuff burning up?

I don't mean to derail the thread here, but i feel this question of unpowered vs. powered risers is pertinent as a tertiary topic here!

i don't know about powered being better, i've always used the standard riser and nothing is even burned on my side

just be sure to power what the card need and be sure that this condition apply to the motherboard as well which is more tricky

some motherboard can not sustain all the power needed for many gpu without an additional molex, well we have the h81 pro btc so no problem there

no gpu rig in there - but really nice and clean setup in your basement.

For me, I will use the unpowered risers if I use the Asrock H61/81 Pro BTC cards. I usually undervolt and underclock cards. So I will not burn the motherboards.

In another thread there is discussion about getting 6 980ti's working. It seems only 5 work.
Has anyone gotten a 6x 980ti rig working or is it too big? The speculation is it's the amount
of memory on all cards, 6 X 6GB = 36 GB.

i also think so, and with new nvidia this will be even worse, since they come with 12 gb for the 1080ti and 24gb for the titan pascal

but then you have that amd work fine even now with 8gb for each 390(you can put six easily in a rig) and 6x8 is 48gb, so it must be somethign different

The memory is not the issue. The cryptomining blog had an article a year ago where they run my fork successfully with 8 titanx cards in one rig.

http://cryptomining-blog.com/5263-mining-with-a-8x-gpu-geforce-gtx-titan-x-system/

They had to do a small  registry modification to make it work.

do you need this modification for something smaller like 970, 980?

High end graphics will still need 2x8pin to consume total of 375W as they are more powerful cards.

To my knowledge 2x8 pin is not standard, and, IMO, won't be needed as power draw is dropping even as performance
increases. Top end GPUs will be engineered to require no more than 300W (slot+6+8).

For the AMD cards, the sapphire 280x uses 2x8pin, the 7990 uses 2x8pin, the R9 Fury also use 2x8pin.

the trend is to consume less for future gpu, so they will end up with only one 8 pin and then 6 pin eventually it's only a matter of time

in few generation you will have zero 6 or 8 pin needed for mainstream gpu, not for high end like 980ti or 390x

the trend is to consume less for future gpu, so they will end up with only one 8 pin and then 6 pin eventually it's onyl a matter of time

in few generation you will have zero 6 or 8 pin needed for mainstream gpu, not for high end like 980ti or 390x
[/quote]

Agree mostly, but I think top end GPUs will continue to push performance within the power ceiling until one GPU can
do 4K@120 or more. But the days of triple slot and dual GPU cards that can heat a small house are over.

The 750ti set a new standard with no extra power connector. the 730, although Kepler, is also a trend setter as
a half length, half height card. The R9 Nano is another card that packs a lot of performance in a small energy
efficient pasckage. Pascal will set a new standard. AMD, well I hope they can at least keep up to keep Nvidia honest.

People are greedy, they want high performance cards, that will use more power. Maybe the limit is around 400W.

it must be said that at some point titan x and 1080ti would become the new meanstream if the consumption level will be very low for that, so it may be possible to assist to a new rank for gpu

But you look at AMD, they are using the water cooling for the 295x2 and Fury XT. That is happening.

In the future, maybe just the Dual GPU cards will consume 375W. All the single GPU cards will consume less than 200W.

I've never found any affordable motherboard with more than 6 GPU PCI-E, so since 6 GPUs would be my answer.

The MSI Z77A GD 55 or GD 65 have 7 PCIE slots and you can use all the slots. I used to own GD55. I bought it now $120, sold it second hand $150.

These two boards are very robust. They both have onboard power switch and reset switch, so very convenient.

It is better to use the 6 PCIE boards. They are more available. If any thing is wrong with one board, you can find a replacement easily.

I used to use the Asrock H61 pro btc. They are all working well for over two years. I am building new rigs with the H81.