[SOLVED] 5+ Cards in a rig & MINING GUIDE

[YipYip]

After going backwards and forwards with this stuff for the last 6 months I have finally got all the info together and 4m hash template rig

I wanted to share so you dont have to go through all of my pain

Lets start with 5 cards as it is simpler and also there is a lot more choice of equipment

POWER:
Each 6 and 7 seried card depending on undervolt settings is going to be using about 190-220+ watts

Any 1200+ 80plus  Gold Rated (XFX 1250 is my currrent choice) if u are going for 6+ SIlverstone 1500 Strider is the one i use (only down side is it is Silver rarted)
You can get things like http://www.add2psu.com/

But the idea of having 2 x 1000 PSU connected sounds rather scary to me ...Also in playing with running 2 x PSU setups i have blown one 900watt unit and a rather large spark about 1 foot long ..so really fuck that shit !!! Use 1 big unit

OS:

This will save you hours of pain and suffering plain and simple Either run
Linux  or Windows 8  (WIndows 7 Can go suck IT!)....people have reported 12.8 working 5 cards win 7 but the cards i use need 13.1 and above so just for the record WIN 7 can SUCK IT AGAIN ..lol

MOBO:

This will tell u whether u need to do the a1 b17 short to get teh 1 x slots working ..also if u are running a 1 x riser in a 16x slot you will usually still need the a1 b17 short

https://bitcointalk.org/index.php?topic=36061.0

Also here is a list of 5+ boards

https://bitcointalk.org/index.php?topic=186877.20

I have 10 MSI 990fxa gd 80 v2 boards running 5+ cards

Personally my new favorite is teh MSI Z77a gd65 with an intel sandy bridge i3 chip...this has been confirmed with 6x7970's (thats a lot of hash for 1 rig ) ALso ssateth has a screen shot with 8 GPU's ...2 x dual and 4 singles on this board mining BTC

RISERS:

To Power or not to power that is the question ...lol

If u are mining BTC/SHA then u dont need them if u are LTC\SCRYPT u MUST have them if u go above 3 cards ...I have many reports of 30 rigs running without them  for BTC but also I have 3 PSU's and 1 mobo with burnt out connectors for mining LTC in 4 card setups without powered risers

So long story short use powered risers on setups with 4 cards and up ..also I find that the cheap risers from china ( i cant find anywhere is to get them from anywhere else except cablesaurus) start to die after about 4 weeks ...so turn them into powered and they are ok

Build your own

https://bitcointalk.org/index.php?topic=76121.0

Just use some left over sata to molex connectors that ususally come with the MOBO

On a rig with 5 cards u will need 2 with powered risers ...also the 1 x slots MUST have a powered riser as there is not enough juice getting to the card from this slot ..they are only rated at 25 watts which is not enough...if u are getting dead/sick cards after about a day or so ..it is 80% of the time tehy are not getting enough power ...so u are ok upto 3 cards without powered risers above that the mobo cant supply enough power and u are heading to burnt out lane !!!

MINING S\W:

Plain and simple CGMINER its the best ...learn it and u will be happy in the long run ..is awsome s/w ckolivas should be thanked
https://github.com/ckolivas/cgminer

Also it will get about 10-15% more than reaper and others in most situations in my xp (this is my xp so dont jump on me about others..or there was no support or with my tech knowledge i was not able to get it to work ..so go figure etc )

COOLING & TEMPS:

Heat heat heat and some more friken heat ...welcome to mining...long story short get your cards to run at =< 75 c any higher and u are killing a card slowly ... 85 c is the limit of absolute max u want to be running your GPU's above this and you can here the card dying ...also above 85c they will not stabilize and will usually creep very slowly to thermal cut off

If u can get them to run at 65c then thats great .... but also try not to have the fans at 100% if possible ... a good comprimise is auto fan ...temp target 73 in cgminer .

Depending on the card the older units had an air intake and the rear and the exhaust out the front (i.e the faceplate is the front)

So any fans should be at the rear of the rig blowing to the front ...(the new cards mostly just vent upwards but this config is still the best)

I live in OZ and our summer had 100+ F for 10 days straight (thanks global warming) it was a nightmare having said that you can still mining without AC

In a small room with 8 rigs we where using
2 x http://www.hyalite.com.au/J1114/EZI-AIR-FAN-315mm-230watt-545-L_or_sec/pd.php

For Exhaust and

2 x http://www.hyalite.com.au/J1301/EZI-AIR-200mm-IN-LINE-FAN-BALL-BEARING/pd.php

Air Intake

If u have these high temps u will need to direct the air onto the rear/fans of the GPU's via 100mm duct ..if u get this setup right u can mine while it is 100+ F outside ...also have the air intakes come from under the house/shaeded (if possible) as it will always be at least
5 c cooler there

Have the intake for the exhaust located at the top or cieling of the room and the air comming in to the bottom if not directed directly onto the GPU's via duct as stated..this creates a natural heat rises flow dynamic so you get a current or cycle of cold going from the bottom going to the top and hot air at the top exhausted (sounds easy in theory ...lol )

This takes a lot of frigging around but to just AC 8 rigs u are looking at 2 k/w (in those temps) instead of 500watts or use a combination of the 2 etc

The best soltion is water blocks but thats another story and also resale and time and effort is a really big step IMHO ....so try this first

REMOTE MANAGEMENT :

I now have 2 locations with a 3rd coming online soon or even when u are at work or getting drunk in a bar like me (... not really but i could be take away wife and kids ....lol )

Have a Non-rig pc that will be your management unit ..it may even be your personal pc but try and get some real low spec unit with onboard video type of deal ... (i.e 150 watts )

Leave this on 24/7 and install logmein (this will provide the VPN ish link to your rigs )

Now get a IP KVM switch something like this unit
http://www.ebay.com.au/itm/290878446981?ssPageName=STRK:MEWNX:IT&_trksid=p3984.m1497.l2649

These are awsome it is 100x better than trying to remote (RDP) onto a machine as a RDP remote session uses huge amount of resources and will

a) not work
b) kill the current session

The last thing u want to be doing is breaking a running rig ... these IP KVM switches connect u to the rig and save on cabling and everything else...i cant tell u how much value u get from the $300 u spend on this type of solution ...also that particular unit was $4500 new so its old top shelf shit !!!!..its like an older BMW its still a bmw

Each rig set S5 to wake on lan event or PCI to edge -PCI event something of this nature you want the unit to wake up from POWER off from a WOL or magic packet

ALso set the bios to POWER ON from power cycle event ...so when the unit gets turned off and on u dont need to press the digital power button on the mobo etc

Check this link out http://windows7-issues.blogspot.com.au/2011/03/wake-on-lan-wol-for-windows-7-made-easy.html

Logmein has a wake on lan thing but use a 3rd party app running on the management PC you have setup previously its more reliable

..this has u able to turn on an off pc and also to manage a running PC ..the final peace is BSOD and hard freezes

I use these cards http://www.berkprod.com/Product_Web_Pages/usb_v1_pc_watchdog.aspx

They are $80 and not super cheap but are industrial grade

I just saw somebody using a rasphberri PI and a relay that looks awsome

http://www.ebay.co.uk/itm/New-5V-8-Channel-Relay-Module-Board-for-Arduino-PIC-AVR-MCU-DSP-ARM-Electronic-/170745298865?pt=UK_BOI_Electrical_Components_Supplies_ET&hash=item27c1367fb1

And WEBLOGICIO for the PI ..now that would be super cheap but require a bit more skill than isntalling a USB and running a heatbeat client on each rig

Ok Thats about it for now

MORE TO COME

[1715052777]

1715052777

[zumbador]

 Doing my due diligence before building my first miner. This post is extremely helpful. Good stuff!  Thanks Yip.

[YipYip]

Doing my due diligence before building my first miner. This post is extremely helpful. Good stuff!  Thanks Yip.

The forums are not very clear on what u just want to know at the beginning

[3ham3]

Question for you, YipYip.

How do you find those Berkshire Products - USB PC Watchdog Timer Cards?

Looking over the product details, I want to put one in each of my rigs, will save so much downtime.

What is your feed back on them? Do they work as designed?


Cheers!

[shacky]

Hi,

I don't know about USB Watchdog, until today.

I use in my setup a solution from Ubiquiti, called mFI.

The power strips connect to my wireless network, to the controller, where I apply rules (turn on or off the power) if any conditions or all conditions are meet. Like If current on power strip one is below 6A, turn off, and wait 10 seconds, and turn it on again.

In my lab, I tested a lot of rigs, and the amount of power can tell to me if the rig is mining OK, or not.

Thanks for your information,

[YipYip]

Question for you, YipYip.

How do you find those Berkshire Products - USB PC Watchdog Timer Cards?

Looking over the product details, I want to put one in each of my rigs, will save so much downtime.

What is your feed back on them? Do they work as designed?


Cheers!

Click on the link and order them direct...i just got 2 sample units and they work a treat as designed

They run off USB and connect directly to the reset +/- pins on the mobo

U run a client heartbeat app on the rig and if and when it locks\freezes\BSOD it will wait a preset time i.e 1-5  mins and hard reset the RIG as if u pressed the reset switch ....also will wait a preset time to allow the rig to boot and run up ...just put the heartbeat app shortcut in startup and u are done

There is full API if u want to get under the hood vb.net and c code libs as well which is nice

$83  1 off and $72 10+ buy price is pretty cool ... real plug and play type of stuff ... they are in coke machines and all other remote kiosk type devices that have a brain..

[YipYip]

Hi,

I don't know about USB Watchdog, until today.

I use in my setup a solution from Ubiquiti, called mFI.

The power strips connect to my wireless network, to the controller, where I apply rules (turn on or off the power) if any conditions or all conditions are meet. Like If current on power strip one is below 6A, turn off, and wait 10 seconds, and turn it on again.

In my lab, I tested a lot of rigs, and the amount of power can tell to me if the rig is mining OK, or not.

Thanks for your information,

They look pretty cool .. whats the cost on those units ??

[YipYip]

Jama gave me some great info on using a Rasberry PI to control all of this setup

Here is the detail

I just grabbed a one of these bad boys
http://www.auseparts.com.au/index.php?route=product/product&product_id=118

Also the relay board u had was using arduino so i thought i needed one of these interface I/O boards
http://www.ebay.com.au/itm/330838954789?ssPageName=STRK:MEWNX:IT&_trksid=p3984.m1497.l2649

And of course i grabbed
http://www.ebay.com.au/itm/390568255094?ssPageName=STRK:MEWNX:IT&_trksid=p3984.m1497.l2649

You dont need to get the $120 version of the PI or the Seperate ARDUINO I/O board but i did because I am a nerd ...lol

Here are some links and advice

That looks like a good Haul of stuff.

The IO board is not necessary but it is better to have it (I don't)
As the Pi's pins are only 3.3v logic where that relay board uses 5.0v so that way is better for the pi
You are on your own getting that working i'm afraid.

What I can help you with is:
Getting your pi accessible through the internet.
Read these 2 tutorials to get a dynamic address that auto updates.

http://www.stuffaboutcode.com/2012/06/raspberry-pi-access-from-internet-using.html

http://www.stuffaboutcode.com/2012/06/raspberry-pi-run-program-at-start-up.html

As for getting webiopi up and running this was what I used

http://www.youtube.com/watch?v=wGahWkjettw

I will draw you a diagram on how to connect the relays and get that over to you tomorrow.

A warning in advance, you will be splicing the mains wire 110 or 240v so know what you are doing and take appropriate precautions. Also it does not comply with any safety regs.

And also

I have had a quick look into that board and it appears you can use it just as an IO buffer, which is what you want.
Meaning that if you connect it all up. when you toggle port 1 for example on the pi, one port should go high on the board.

I have drawn a very rough diagram for 4 channels.
The numbers I have used are just for example, and I have also assumed you are US or somewhere non-uk so have used (hot, neutral and ground).

I have just drawn one power cable as the others are just repeats.

If you want me to clarify or draw a different way I will.

Here is the diagram: http://imgur.com/sZTAzNK

You can have all of this setup to remote control 16 rigs for about $45 on the cheap ...now u cant beat that

Thanks JAMA

[shacky]

Hi,

I don't know about USB Watchdog, until today.

I use in my setup a solution from Ubiquiti, called mFI.

The power strips connect to my wireless network, to the controller, where I apply rules (turn on or off the power) if any conditions or all conditions are meet. Like If current on power strip one is below 6A, turn off, and wait 10 seconds, and turn it on again.

In my lab, I tested a lot of rigs, and the amount of power can tell to me if the rig is mining OK, or not.

Thanks for your information,

They look pretty cool .. whats the cost on those units ??

The mFI mPower strip with 3 outlets cost 59$ (max load 15a, one machine). They have in production a mPower with only one outlet (Will cost 39$) that will fit better in my setup, but they don't have ETA for that product.

[3ham3]

Question for you, YipYip.

How do you find those Berkshire Products - USB PC Watchdog Timer Cards?

Looking over the product details, I want to put one in each of my rigs, will save so much downtime.

What is your feed back on them? Do they work as designed?


Cheers!

Click on the link and order them direct...i just got 2 sample units and they work a treat as designed

They run off USB and connect directly to the reset +/- pins on the mobo

U run a client heartbeat app on the rig and if and when it locks\freezes\BSOD it will wait a preset time i.e 1-5  mins and hard reset the RIG as if u pressed the reset switch ....also will wait a preset time to allow the rig to boot and run up ...just put the heartbeat app shortcut in startup and u are done

There is full API if u want to get under the hood vb.net and c code libs as well which is nice

$83  1 off and $72 10+ buy price is pretty cool ... real plug and play type of stuff ... they are in coke machines and all other remote kiosk type devices that have a brain..

Thx for that, I had looked over all the info and understood how it all works, it's a nice clean simple solution for each rig.

Thanks for the feedback and info, Will be ordering a couple of samples before I grab 43 more.

Cheers!

[YipYip]

Question for you, YipYip.

How do you find those Berkshire Products - USB PC Watchdog Timer Cards?

Looking over the product details, I want to put one in each of my rigs, will save so much downtime.

What is your feed back on them? Do they work as designed?


Cheers!

Click on the link and order them direct...i just got 2 sample units and they work a treat as designed

They run off USB and connect directly to the reset +/- pins on the mobo

U run a client heartbeat app on the rig and if and when it locks\freezes\BSOD it will wait a preset time i.e 1-5  mins and hard reset the RIG as if u pressed the reset switch ....also will wait a preset time to allow the rig to boot and run up ...just put the heartbeat app shortcut in startup and u are done

There is full API if u want to get under the hood vb.net and c code libs as well which is nice

$83  1 off and $72 10+ buy price is pretty cool ... real plug and play type of stuff ... they are in coke machines and all other remote kiosk type devices that have a brain..

Thx for that, I had looked over all the info and understood how it all works, it's a nice clean simple solution for each rig.

Thanks for the feedback and info, Will be ordering a couple of samples before I grab 43 more.

Cheers!

Also found these guys similar to the Ubiquiti unit

http://www.pwrusb.com/powerUSB-watchdog.html

Are around $89 a pop...have UK at this point only and waiting on a reply as to either US or Australian solutions etc

[papaminer]

I remembered my English teacher from high school...

whenever she came in... me and my classmates will start counting how many "ANYWAY" she says...

anyway.. on this thread.. you gotta count how many "LONG STORY SHORT"..



good thread.. and good info... long story short.. +1 on this thread...

[TObject]

Some motherboards have integrated watchdog functionality. These are usually server/workstation grade boards. I have not done the math if this makes sense financially; just thinking out loud.

Also these boards often have built-in KVM-over-LAN IPMI functionality for out-of-band management.

[YipYip]

Some motherboards have integrated watchdog functionality. These are usually server/workstation grade boards. I have not done the math if this makes sense financially; just thinking out loud.

Also these boards often have built-in KVM-over-LAN IPMI functionality for out-of-band management.

Links Please of units with all said functions and place to buy....

There is definately a price point cut off for this funtions but considering that my current default mobo Z77a GD65 is ~ $200 in Australia an extra 100 or so would still be worthwhile

[shacky]

Hi yipyip,

About this:

http://www.pwrusb.com/powerUSB-watchdog.html

I like the Ubiquiti mFI because it's wireless, and don't need a USB port to monitor it.

[TObject]

Some motherboards have integrated watchdog functionality. These are usually server/workstation grade boards. I have not done the math if this makes sense financially; just thinking out loud.

Also these boards often have built-in KVM-over-LAN IPMI functionality for out-of-band management.

Links Please of units with all said functions and place to buy....

There is definately a price point cut off for this funtions but considering that my current default mobo Z77a GD65 is ~ $200 in Australia an extra 100 or so would still be worthwhile

Here is one with six PCI-Express slots, Watch Dog, and IPMI 2.0 KVM:
http://www.supermicro.com/products/motherboard/Xeon3000/3400/X8SIA.cfm?IPMI=Y

Financially, you really have to calculate if it is worth it though. Sometimes you can get piles of used servers for really cheap, when a company goes out of businesses or upgrades.

Places to buy: https://www.google.com/#q=X8SIA-F&source=lnms&tbm=shop&sa=X&ei=R36eUfC8DcXRigLhhoGQDQ&ved=0CAkQ_AUoAw&bav=on.2,or.r_qf.&bvm=bv.47008514,d.cGE&fp=139f6c2fdb58a26c&biw=1920&bih=1084

[YipYip]

Some motherboards have integrated watchdog functionality. These are usually server/workstation grade boards. I have not done the math if this makes sense financially; just thinking out loud.

Also these boards often have built-in KVM-over-LAN IPMI functionality for out-of-band management.

Links Please of units with all said functions and place to buy....

There is definately a price point cut off for this funtions but considering that my current default mobo Z77a GD65 is ~ $200 in Australia an extra 100 or so would still be worthwhile

Here is one with six PCI-Express slots, Watch Dog, and IPMI 2.0 KVM:
http://www.supermicro.com/products/motherboard/Xeon3000/3400/X8SIA.cfm?IPMI=Y

Financially, you really have to calculate if it is worth it though. Sometimes you can get piles of used servers for really cheap, when a company goes out of businesses or upgrades.

Places to buy: https://www.google.com/#q=X8SIA-F&source=lnms&tbm=shop&sa=X&ei=R36eUfC8DcXRigLhhoGQDQ&ved=0CAkQ_AUoAw&bav=on.2,or.r_qf.&bvm=bv.47008514,d.cGE&fp=139f6c2fdb58a26c&biw=1920&bih=1084

Thats not such a bad board ...5 usable slots and at $248 on amazon...I will give it a go on next MOBO purchase in 2 weeks and give feedback

[shacky]

That mobo sounds good, If you try it, please let us know!

Thanks

[3ham3]

Hi all.

Just received my USB PC Watchdog card from Berkshire Products, great little tool and worth every cent.

I found that because my rig is under such load and very laggy the "heart beat " program running on the rig would not respond within the correct time frame, multiple tests confirmed this over a 12 hour period.

The solution was very simple, the program can turn a . exe program into the "Heart Beat" (.exe to monitor) , one is able set cgminer.exe as the "Heart Beat" and because cgminer.exe is blazing away (always on) it keeps the USB PC Watchdog card timer from resetting the rig, as soon as cgminer.exe hangs/crashes or locks up in anyway the Card then Resets the Rig.

Will be implementing these across all my rigs.

Just my two bitcents worth.

[shacky]

Hi YipYip,

I have a question about your intake/Exhaust system.

I live in Venezuela, and the temps outside are 95F.

For 12 rigs with 4 x 7970 each one, I use 60.000 BTU of air conditioner (without exhaust.) only with fans.

I read that you use 2 intake with ducts, and 2 exhaust to run 8 mining rigs with external temp over 100F.

Can you show me some pictures of your setup.

Im thinking of opening a new datacenter and want to reduce the air conditioner.

Thanks for your help