Bitcoin Forum

Other => Beginners & Help => Topic started by: elrodvoss on February 09, 2012, 05:42:31 AM



Title: Advice for a start for a Mining Farm
Post by: elrodvoss on February 09, 2012, 05:42:31 AM
Evening everyone,

I wanted to get some input on the first of a mining farm and see if there is any suggestions that you all could give me.  I have dug through and tried to find the best hardware for my money limitations.

Case:  $50  (custom frame case from forum posting)
Motherboard:  $167 (MSI 890FXA-GD70 AM3+)
CPU:  $40 (AMD Sempron 145 Sargas 2.8GHz Socket AM3)
Ram:  $24 (Crucial 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1066 (PC3 8500) Dual Channel Kit)
Drive:  $15 (Verbatim 8GB STORE N GO FLASH DRIVE USB 2.0 NANO)
PSU:  $270 (XFX ProSeries P1-1250-BEFX 1250W ATX12V V2.2 & ESP12V V2.91 SLI Ready CrossFire Ready 80 PLUS GOLD Certified Modular)

Extras:  $12 x 4 (PCIe x16 Extender Cable)
            $20 (Box Fan)

Total base cost: $634 (prices pulled from newegg, cablesaurus, and walmart as of this posting)

I think that is about everything.  Of course the only thing missing is what video card I can use.

Well so far, I have been using 2 x 5850s in my current setup.  Im getting 350 each and on ebay they are selling around $150 each.
So with 4 video cards, it would cost me $1234 and give me about 1.4Ghash/s

So of my reasons behind my choice in hardware is:

Motherboard cause it supports 5 PCI-e slots, CPU and Ram is the cheapest I can find, Drive I plan on using Unix so 8 GB USB drive should be plenty.  The PSU has 8 6+2 pins and 1250W, so should have enough overhead to support 4 cards and computer for 24/7 mining.  Also I am trying to keep the power under 1400 for each system since I don't have the proper setup to have multiple rigs in the same room.

So you have any suggestions for tweaking this setup?  Thanks in advance.

-elrodvoss




Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on February 09, 2012, 05:53:02 AM
Well you get a momma GPU and a poppa GPU.  Leave them in the barn and give them plenty of privacy ...

Oh you want serious answers?
MB:  The 890FXA-GD70 is a monster.  I own 7.  Very solid.  A quick heads up BAMT won't boot unless you have latest bios flash and whatever the setting "protection bios upgrade" when I accidentally enabled it caused a freeze before it even finished posting.

RAM: 2GB is more than enough.  1GB works fine.  Save a couple bucks.  One rig doesn't matter but if you ever end up with 2, 3, 4 it starts adding up

Drive: I would go w/ 4GB.  Some MB/BIOS have difficulty booting USB sticks >4GB.  For dedicated rigs you don't need more than 4GB.  Hell you really don't need more than 2GB but lets splurge a little.

PSU:  That PSU is a nice one but it is a rebrand of Seasonic 1250W 80Plus-Gold.  Just get the real thing.  Seasonic customer support and service is better anyways.  The Seasonic is same price, has 5 yr warranty and a better fan.



Title: Re: Advice for a start for a Mining Farm
Post by: Wandering Albatross on February 09, 2012, 06:00:09 AM
have a look at FPGAs, cablesaurus selling them. Are you okay (with that rig you spec'd) with getting half as many BTC before end of year?
There's some speculation that it may happen this year and then mining becomes less worthwhile unless you can get power costs way down.


Title: Re: Advice for a start for a Mining Farm
Post by: check_status on February 09, 2012, 08:14:25 AM
A kilowatt meter to measure your draw at the wall.
Design your system around 6 cards working and the PSU at 70% load.
A Dr. Power II is good to keep around to measure the PSU's fitness when hardware issues creep in. It'll help to isolate motherboard issues from PSU issues.


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on February 09, 2012, 09:16:44 AM
have a look at FPGAs, cablesaurus selling them. Are you okay (with that rig you spec'd) with getting half as many BTC before end of year?
There's some speculation that it may happen this year and then mining becomes less worthwhile unless you can get power costs way down.

I'm replying on phone so parden grammer and typos.

I have a unique situation, that I don't pay for power.  I do own my own place, but without getting into details I have no power cost. 

I love the concept of fpga, one computer, space and power and heat savings.  Sadly, at this stage, its more bang for buck with video cards, but I want to go fpga in the future.  I dreamed up mounting 10 fpgas to acrylic and submerge them into oil, but that is an upgrade step after ROI.  I would like the rigs to pay for their own upgrades down the road.

So far thanks for tips.  I have a huge step to learn lynix enough to run bitcoin on them and get them wireless (due to power restrictions placement).  But at least i know I'm heading in the right direction and won't waste money on poor setups with all of everyones help.

Thanks again, if u got more advice, keep it comming.

-elrodvoss


Title: Re: Advice for a start for a Mining Farm
Post by: check_status on February 09, 2012, 06:24:41 PM
Well you get a momma GPU and a poppa GPU.  Leave them in the barn and give them plenty of privacy ...
Is the barn made out of VPN lumber with ntpd windows shut tight?  ;D

On Newegg you sometimes come across an open box of the board saving you a little more.
I'm partial to the GA-990fxa-UD3 or UD5 myself. I read on BCT someone having difficulty running more than 4 cards on MSI. DeathandTaxes probably made posts in that thread.


Title: Re: Advice for a start for a Mining Farm
Post by: sunbreak on February 09, 2012, 08:47:21 PM
I would save some money on the psu and get an Antec TPQ-1200 ... the OC edition has even more power connectors for video cards ...

Either way ... the TPQ-1200 has served me well and they run about $220 ... sometimes you can get them for $150-$180 used.

You can also save yourself some money getting 16x->16x extenders from 9mart.com. You will need 16x->16x unless you want to diddle with the pci-e lane detect pins. The 16x->16x just works, of course if you want to use the 1x slot, then you need a 1x->16x as well.

No need for powered extenders a the GD70 has successfully run 3 6990's without them, stable for almost a year now. The machine uses 1170 watts at the wall to do 2520 mhash/sec. Remember the 1200 watt rating is on the DC side, not the AC side at the wall. So my PSU is actually at about 1000-1050 watts load.

Also on the memory, on my machines with 4 video cards the resident running memory requirement is just over 1 gb ... so .. 2 gb would be fine. I actually have 4 gb in each machine which is handy if you want to netboot as 1 gb will get eaten up holding the iso image. Netboot is highly recommended because the USB is so slow and also ... extra cost you don't need to spend.


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on February 09, 2012, 10:26:06 PM
Netboot is highly recommended because the USB is so slow and also ... extra cost you don't need to spend.

Wait huh?  iso over network is faster than USB drive read time?


Title: Re: Advice for a start for a Mining Farm
Post by: chrisp on February 09, 2012, 10:43:19 PM
Read times for USB flash drives are typically 10-30Mbyte/second.  Mostly secondary to higher protocol overhead.

With giga-ethernet you can get 90-110Mb/sec.  Under similar conditions.

Doesn't account for USB 3.0 though, haven't used it personally.


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on February 09, 2012, 11:26:30 PM
Read times for USB flash drives are typically 10-30Mbyte/second.  Mostly secondary to higher protocol overhead.

With giga-ethernet you can get 90-110Mb/sec.  Under similar conditions.

Doesn't account for USB 3.0 though, haven't used it personally.

Maybe in theory but as practical reality you aren't going to get 90 MB/s+ usable throughput unless it is RAM to RAM.  Still lets say it is 90MB vs 20MB in a day my rig might have about 100MB of disk activity.  So it saves 1-2 seconds per day?     Lots of good reasons to use netboot but "slow" USB speeds is hardly one of them.   The distros are very small and boot times are very fast.


Title: Re: Advice for a start for a Mining Farm
Post by: chrisp on February 09, 2012, 11:38:32 PM
Agreed!  Netbooting has a certain wow factor though :)


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on February 10, 2012, 12:04:41 AM
Well not sure if I will need to use the netboot.  One of the biggest problems I will have is that since the wiring in this house is on the fair side.

I could not support more then one computer on each circuit breaker and each room has its own breaker.  Ive looked at my breaker box and there are no free breakers, so I would have to upgrade the breaker box and then wire up some outlets into a close room.

So the hurdle that I would have to deal with is I would need to set these rigs up wireless, which will be another challenge for me.  I don't foresee any problems, but of course most everyone else is running their multi-rigs via routers and switches over hardline.  Don't think I have seen a single wireless bitcoin rig, but have not looked either.

So I hope I don't have to resort to having to have cat 5 cables running between rooms or worse extension cables.






Title: Re: Advice for a start for a Mining Farm
Post by: Wandering Albatross on February 10, 2012, 12:46:02 AM
Quote from: elrodvoss
I don't pay for power

If you don't pay for power than you just have to recoup HW costs. So might as well buy a bunch of cheap HW.


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on February 10, 2012, 06:11:42 PM
If you don't pay for power than you just have to recoup HW costs. So might as well buy a bunch of cheap HW.

Well then the problem with that is I don't have enough circuit breakers to have all these systems.

Each circuit can only support like 1450 watts before the breaker trips.  My house is set up to have one breaker per room.  So with my above set up, I would be pulling "roughly" 800-1000 watts on each circuit (not including anything else in the room).

So unless I either:

  • Put a computer in each room with wireless adapters.
or
  • Have a bunch of extension cables and cat5 cables running through the house.
or
  • Drop $$$ to have my breaker box upgraded and have several breakers installed and wired to an adjoining room as a "rig room".

While the first option is easy, it would be weird to have a stand along rig just sitting in the corner with a box fan on it running (living room, family room, laundry room)  Of course not an option in the kitchen due to the oven and microwave.  Turn them on and pop a breaker.

In the end the final option will need to be done, but I also plan on moving to FPGAs in the future (1-2 years) as well.



Title: Re: Advice for a start for a Mining Farm
Post by: sunbreak on February 10, 2012, 06:16:15 PM
If you don't pay for power than you just have to recoup HW costs. So might as well buy a bunch of cheap HW.

Well then the problem with that is I don't have enough circuit breakers to have all these systems.

Each circuit can only support like 1450 watts before the breaker trips.  My house is set up to have one breaker per room.  So with my above set up, I would be pulling "roughly" 800-1000 watts on each circuit (not including anything else in the room).

So unless I either:

  • Put a computer in each room with wireless adapters.
or
  • Have a bunch of extension cables and cat5 cables running through the house.
or
  • Drop $$$ to have my breaker box upgraded and have several breakers installed and wired to an adjoining room as a "rig room".

While the first option is easy, it would be weird to have a stand along rig just sitting in the corner with a box fan on it running (living room, family room, laundry room)  Of course not an option in the kitchen due to the oven and microwave.  Turn them on and pop a breaker.

In the end the final option will need to be done, but I also plan on moving to FPGAs in the future (1-2 years) as well.



I vote for the rig room and running a subpanel from your main box, connecting all your miners at 240 volts and having a cold air intake and a hot air exhaust from the room.

For example; My rig room is in the pic in this forum posting

https://bitcointalk.org/index.php?topic=63163


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on February 10, 2012, 06:58:32 PM
installing a new panel is big $$$ but installing a sub panel is a lot cheaper.

If there is literally no spare breaker slots the electrician will dewire 2 of them.
drop a breaker in the main panel w/ sufficient amperage for the subpanel.
Install new panel, wire to new breakers.
wire the dewired circuits into new panel.
drop in all your new breakers for new circuits.

Since it doesn't require disconnecting mains from the residence it can be done by qualified electrician in a couple hours.


Title: Re: Advice for a start for a Mining Farm
Post by: CA Coins on February 11, 2012, 02:45:54 AM
I tried out BAMT (linux that runs off a USB, prepackaged for mining) for the OS when I tried GPU mining.  It worked pretty well so you might want to check them out. 
Also, send me a PM if you decide to go with the PCIe extenders.  I have some new extenders from Cablesaurus that I can give to you for half what they are selling.  I gave up building a GPU cluster and moved to FPGAs because my electric cost is outrageous.


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on February 11, 2012, 11:11:11 PM
Again,

I would like to thank everyone for the advice and support.  It does have a lot of meaning to me when you have a community that is so tied together and freely helps out others.

Reminds me of my City of Heroes MMO days.  The game was very good, but it was how tight the community was and how the DEVs would interact with the users on a social level rather then play the company role and give cookie cutter responses.

I would like to personally thank DeathAndTaxes along with a few other that I have seen pop up in my various postings and help me out with advice and direction.  It sounds simple, but its people like this that will ensure that Bitcoin continues and grow to everything that it can be. 

Many thanks to all here that is willing to take there time to help those that are learning their way.

-elrodvoss


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on March 06, 2012, 01:11:52 AM
Evening everyone,

     Well I started to get all my hardware together.  I got the Mobo, CPU, Ram, USB Drives, and PSU in about 2 weeks ago.  Ive been slowly picking up 5850s as they come up on ebay cheap.

I got 2 in now, third comming in.  So I have enough to get the first of three rigs started at least.


So now the next question.  I DLed BAMT, but I am 100% stupid when it comes to lynux.  So I have a new rig, and bare as a new baby and a empty 4 gig usb drive.

Can someone direct me to a fairly simple "lynux and BAMT for dummies" somewhere that will tell me step by step to set this up?

Thanks again.



Title: Re: Advice for a start for a Mining Farm
Post by: boconniff40 on March 11, 2012, 11:29:13 PM
have a look at FPGAs, cablesaurus selling them. Are you okay (with that rig you spec'd) with getting half as many BTC before end of year?
There's some speculation that it may happen this year and then mining becomes less worthwhile unless you can get power costs way down.

I'm replying on phone so parden grammer and typos.

I have a unique situation, that I don't pay for power.  I do own my own place, but without getting into details I have no power cost. 

I love the concept of fpga, one computer, space and power and heat savings.  Sadly, at this stage, its more bang for buck with video cards, but I want to go fpga in the future.  I dreamed up mounting 10 fpgas to acrylic and submerge them into oil, but that is an upgrade step after ROI.  I would like the rigs to pay for their own upgrades down the road.

So far thanks for tips.  I have a huge step to learn lynix enough to run bitcoin on them and get them wireless (due to power restrictions placement).  But at least i know I'm heading in the right direction and won't waste money on poor setups with all of everyones help.

Thanks again, if u got more advice, keep it comming.

-elrodvoss

What in the eff do you mean with this? Why would you submerge "10 fpgas to acrylic and submerge them into oil" ?


Title: Re: Advice for a start for a Mining Farm
Post by: chsados on March 12, 2012, 07:10:44 AM

What in the eff do you mean with this? Why would you submerge "10 fpgas to acrylic and submerge them into oil" ?

http://youtu.be/PtufuXLvOok


Title: Re: Advice for a start for a Mining Farm
Post by: boconniff40 on March 12, 2012, 12:38:12 PM
lmfao that's lame... just stick to liquid cooling


Title: Re: Advice for a start for a Mining Farm
Post by: John (John K.) on March 12, 2012, 12:45:38 PM
lmfao that's lame... just stick to liquid cooling
You would need to have waterblocks for the fpga's custom made then.


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on March 12, 2012, 01:39:16 PM
lmfao that's lame... just stick to liquid cooling
You would need to have waterblocks for the fpga's custom made then.

They do make universal waterblocks.  If BFL would just put some damn holes in the board.  They make water blocks which can adjust to everything from 28mm to 80mm hole spacing.  BFL just punch two holes in the PCB and people could use any custom cooling they wanted.  It you can't/won't for this board at least consider it for the next board.  Having 2 push-pin holes put into the board would add about $0.02 to PCB production cost.  Everything from third party CPU coolers to waterblocks to phase change and TECs would be possible if they had 2 pennies worth of holes punched into the board.


Title: Re: Advice for a start for a Mining Farm
Post by: John (John K.) on March 12, 2012, 01:43:12 PM
lmfao that's lame... just stick to liquid cooling
You would need to have waterblocks for the fpga's custom made then.

They do make universal waterblocks.  If BFL would just put some damn holes in the board.  They make water blocks which can adjust to everything from 28mm to 80mm hole spacing.  BFL just punch two holes in the PCB and people could use any custom cooling they wanted.  It you can't/won't for this board at least consider it for the next board.  Having 2 push-pin holes put into the board would add about $0.02 to PCB production cost.  Everything from third party CPU coolers to waterblocks to phase change and TECs would be possible if they had 2 pennies worth of holes punched into the board.

Now that's news to me. Can't we drill the holes in, aftermarket style? Or is it the PCB's so damn packed till we can't even drill em ourselves?


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on March 12, 2012, 01:53:44 PM
lmfao that's lame... just stick to liquid cooling
You would need to have waterblocks for the fpga's custom made then.

They do make universal waterblocks.  If BFL would just put some damn holes in the board.  They make water blocks which can adjust to everything from 28mm to 80mm hole spacing.  BFL just punch two holes in the PCB and people could use any custom cooling they wanted.  It you can't/won't for this board at least consider it for the next board.  Having 2 push-pin holes put into the board would add about $0.02 to PCB production cost.  Everything from third party CPU coolers to waterblocks to phase change and TECs would be possible if they had 2 pennies worth of holes punched into the board.

Now that's news to me. Can't we drill the holes in, aftermarket style? Or is it the PCB's so damn packed till we can't even drill em ourselves?

The board is very tight.  Generally you would want the board designed for holes in particular.  Also a PCB shop will have very high speed drill presses with tapered bits to ensure no slipped bit = destroyed $600 board.

The other issue is it looks like BFL used a thermal expoxy for mounting the heatsink.  You aren't getting it off.

However if BFL offered a board with
a) holes in PCB for mounting heatsink
b) no heatsink, fans, case, or powersupply
c) offered it for $30 less
you could make a monster watercooled cluster.

:)

Send them an email asking them for that.  They likely won't but if they get enough requests maybe they will consider it for next revision of the board.


Title: Re: Advice for a start for a Mining Farm
Post by: John (John K.) on March 12, 2012, 02:08:00 PM
Hey, that's a good idea! I'll do this too, maybe to let them know there's interest in this getting done. I need liquid cooling here to run one as I'm living in the tropics right now.


Title: Re: Advice for a start for a Mining Farm
Post by: DeathAndTaxes on March 12, 2012, 02:14:10 PM
As an example here is a universal block which can mount to hole spacing between 53.5mm to 61.5mm.

http://www.frozencpu.com/images/products/main/ex-blc-1070.jpg


Title: Re: Advice for a start for a Mining Farm
Post by: twoglovedanny on March 14, 2012, 01:47:56 AM
Where are you located?


Title: Re: Advice for a start for a Mining Farm
Post by: elrodvoss on March 14, 2012, 02:57:52 PM
Where are you located?

Im south of chicago, IL