Bitcoin Forum
October 23, 2017, 09:06:57 PM *
News: Latest stable version of Bitcoin Core: 0.15.0.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 [36] 37 38 39 40 41 »
  Print  
Author Topic: Data Center Mining Garage and Man Mining Cave ( PART 2 !!!!!!!!)  (Read 87562 times)
This is a self-moderated topic. If you do not want to be moderated by the person who started this topic, create a new topic.
uray
Hero Member
*****
Offline Offline

Activity: 560


View Profile
May 24, 2017, 08:18:36 AM
 #701

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

<snipped>

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!


hey uray -- any primers on how get the PXE boot solution and HDD-less rigs up and running? thanks

you can just follow this guide https://www.gadgetdaily.xyz/boot-linux-from-an-nfs-server/

you need to run all rigs on linux, a dhcp server, and a hdd attached to that dhcp server and its done
1508792817
Hero Member
*
Offline Offline

Posts: 1508792817

View Profile Personal Message (Offline)

Ignore
1508792817
Reply with quote  #2

1508792817
Report to moderator
1508792817
Hero Member
*
Offline Offline

Posts: 1508792817

View Profile Personal Message (Offline)

Ignore
1508792817
Reply with quote  #2

1508792817
Report to moderator
1508792817
Hero Member
*
Offline Offline

Posts: 1508792817

View Profile Personal Message (Offline)

Ignore
1508792817
Reply with quote  #2

1508792817
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
CryptoMensch
Member
**
Offline Offline

Activity: 84


View Profile WWW
May 24, 2017, 08:30:26 AM
 #702

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

<snipped>

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!


hey uray -- any primers on how get the PXE boot solution and HDD-less rigs up and running? thanks

you can just follow this guide https://www.gadgetdaily.xyz/boot-linux-from-an-nfs-server/

you need to run all rigs on linux, a dhcp server, and a hdd attached to that dhcp server and its done

Looking forward for the update #3 after this!

BANCOR
citronick
Hero Member
*****
Offline Offline

Activity: 770


winter*juvia


View Profile
May 24, 2017, 10:10:32 AM
 #703

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

<snipped>

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!


hey uray -- any primers on how get the PXE boot solution and HDD-less rigs up and running? thanks

you can just follow this guide https://www.gadgetdaily.xyz/boot-linux-from-an-nfs-server/

you need to run all rigs on linux, a dhcp server, and a hdd attached to that dhcp server and its done

thanks

your farm is awesome - esp those hanging on the wall - it wont win any sexy rigs competition but truly a beast of a GPU farm.

pls share too on how do you deal with heat and solutions to get the farm cool

Citronick's Beer Fund : 32oo5tk1RfTbndrUjWvHJ9yT3bb2EyQYkq
stef_stef
Member
**
Offline Offline

Activity: 89


View Profile
May 24, 2017, 03:26:29 PM
 #704

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!




Wow, that's an impressive build Smiley

Can you please reveal more info on the network boot? Also, what OS are you running on the rigs?

Thanks
uray
Hero Member
*****
Offline Offline

Activity: 560


View Profile
May 24, 2017, 04:00:54 PM
 #705

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

<snipped>

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!


hey uray -- any primers on how get the PXE boot solution and HDD-less rigs up and running? thanks

you can just follow this guide https://www.gadgetdaily.xyz/boot-linux-from-an-nfs-server/

you need to run all rigs on linux, a dhcp server, and a hdd attached to that dhcp server and its done

thanks

...
pls share too on how do you deal with heat and solutions to get the farm cool

my initial plan was for each rig, I will use dual 120mm fan to blow air out of frame


I need to use best airflow and air pressure fan available, since i can't put huge fan on rear side and on front side, since there are simply no space to put those fans, so after some research I see this Nidec VA450DC



it can even hover on itself, lift up and fly !
https://www.youtube.com/watch?v=FXFnQsIoB5A

I can get these fans just 8$ each from china, for 30 rigs i need to have 60 of these, which 480$ for these 60 fans is good deal, so I ordered one just to test it, but the noise was... so awful, its so annoying and my wife yelling at me, so instead of using nidec, I use regular pc desktop 120mm fans which doing nothing for the heat LOL

for temporary solution I just put 3x 18" fan on the table, and break my window open


room temperature at midday from 46C+ drop to 43C still not good enough, and i am still working on it
uray
Hero Member
*****
Offline Offline

Activity: 560


View Profile
May 24, 2017, 04:04:52 PM
 #706

I am trying to fit 150 GPUs into room of 3 meters x 2 meters x 2 meters as fast and as cheap as I can

  • 80GB SSD for network disk (all 30 rigs running without hdd, its O/S are all loaded from this tiny cheap ssd via PXE Boot)
  • PoE switch for my IP Camera to monitor this room
  • ARM Based board to run network disk and boot server, it use only 10W via USB powered
  • Mikrotik Router for my main internet via Fiber network, and also attached USB dongle for LTE modem as backup internet
  • All Glued to the wall using 3M double tape!




Wow, that's an impressive build Smiley

Can you please reveal more info on the network boot? Also, what OS are you running on the rigs?

Thanks

I use ubuntu server 16.04.02, I choose "server" edition one, because it has no graphical desktop, and disk usage is under 2GB
then just install AMDGPU-Pro Driver for linux, for network boot setup, I follow this guide https://www.gadgetdaily.xyz/boot-linux-from-an-nfs-server/
smaxz
Sr. Member
****
Offline Offline

Activity: 402


VeganAcademy


View Profile
May 25, 2017, 05:06:26 PM
 #707

are you pool mining all these gpu's?

- NGdTwHRSdnThdi1drQuHGT3khAHRtZ1HMq -
uray
Hero Member
*****
Offline Offline

Activity: 560


View Profile
May 25, 2017, 07:45:37 PM
 #708

are you pool mining all these gpu's?

yup i don't have time to self manage and update eth wallet, downloading blockchain, etc
Elder III
Sr. Member
****
Online Online

Activity: 336


Hobby Miner - Twitter for Crypto Info - @elderiii


View Profile
May 26, 2017, 12:47:31 AM
 #709

This may be a dumb question, but this is the first time I have dealt with PDUs of any kind so I figured I should ask just to be safe.... Is there anything wrong with having a 2400 watt 240v server PSU powering the GPUs + risers (plugged into a 240v 40 amp PDU) and a 120v ATX PSU powering the motherboard, CPU, SSD, etc, plugged into a simple 120v surge protector?  Basically, is it ok to have a PSU running off 120v paired with one running off 240v on the same rig?


BitSend ◢◤Clients | Source
www.bitsend.info
█▄
█████▄
████████▄
███████████▄
██████████████
███████████▀
████████▀
█████▀
█▀












Your Digital Network | 10MB Blocks
Algo: XEVAN | DK3 | Masternodes
Bitcore - BTX/BTC -Project












BSD -USDT | Bittrex | C.Gather | S.Exchange
Cryptopia | NovaExchange | Livecoin
CoinPayments | Faucet | Bitsend Airdrop













████
 ████
  ████
   ████
    ████
     ████
      ████
       ████
        ████
       ████
      ████
     ████
    ████
   ████
  ████
 ████
████

████
 ████
  ████
   ████
    ████
     ████
      ████
       ████
        ████
       ████
      ████
     ████
    ████
   ████
  ████
 ████
████
jwarren81
Full Member
***
Offline Offline

Activity: 204


View Profile
May 26, 2017, 12:56:32 AM
 #710

No, it shouldn't matter.  Each power supply is taking the input voltage and stepping it down anyways.  As long as the GPU and Risers are on first I don't see any issues with that.  Although, most ATX power supplies these days take up to 240V input as well so plugging it into the PDU would actually be beneficial and allow it to run more efficiently.
VyprBTC
Member
**
Offline Offline

Activity: 112


View Profile
May 26, 2017, 01:02:51 AM
 #711

holy mama I want my room to look like that! Amazing job wow thanks for the inspiring pics!!!
Elder III
Sr. Member
****
Online Online

Activity: 336


Hobby Miner - Twitter for Crypto Info - @elderiii


View Profile
May 26, 2017, 01:16:34 AM
 #712

No, it shouldn't matter.  Each power supply is taking the input voltage and stepping it down anyways.  As long as the GPU and Risers are on first I don't see any issues with that.  Although, most ATX power supplies these days take up to 240V input as well so plugging it into the PDU would actually be beneficial and allow it to run more efficiently.

I did want to connect the ATX PSUs to the PDU as well, but it only has 6 plugs, 4 of which will be used by server PSUs, so that's not an option at this point. We will have other equipment using 120v in the same area, so it won't be a big deal to have ATX PSUs running the motherboard plugged in with them.


BitSend ◢◤Clients | Source
www.bitsend.info
█▄
█████▄
████████▄
███████████▄
██████████████
███████████▀
████████▀
█████▀
█▀












Your Digital Network | 10MB Blocks
Algo: XEVAN | DK3 | Masternodes
Bitcore - BTX/BTC -Project












BSD -USDT | Bittrex | C.Gather | S.Exchange
Cryptopia | NovaExchange | Livecoin
CoinPayments | Faucet | Bitsend Airdrop













████
 ████
  ████
   ████
    ████
     ████
      ████
       ████
        ████
       ████
      ████
     ████
    ████
   ████
  ████
 ████
████

████
 ████
  ████
   ████
    ████
     ████
      ████
       ████
        ████
       ████
      ████
     ████
    ████
   ████
  ████
 ████
████
chohav
Full Member
***
Offline Offline

Activity: 230



View Profile
May 26, 2017, 11:50:34 AM
 #713

WOW uray, nice job man! Sexy in its own way. What is the outside temperature in your area, is it gonna become hotter next months and what are your plans for additional cooling solutions? Are you gonna install AC or stick with more blowers? Cheers
50ShadesOfDamon
Newbie
*
Offline Offline

Activity: 20


View Profile
May 26, 2017, 12:34:12 PM
 #714

*********DRAFTING!!!!!!!!!!!!!!!!!!!!!!

My original thread was getting over 22K Views in less than a month but I forgot 1 important thing.  I need to make myself the MODERATOR.   So I'm redoing / continuing the thread with this extension so I can make any edits / changes where it will be Easier for new Readers to follow as well as for original followers to be able to continue where we left off.

Link to Original Thread.
https://bitcointalk.org/index.php?topic=1761303.0

MY FARM SETUP

I have been mining for a very long time.  Most of you would know me as Nhan Do from my old Nick that was Hacked, hence why I'm using this new Nick.  I have been thru the many ups and downs of Crypto as well as hundreds of designs and layouts.  What I learned from my experience / ]mistakes, is that it's always best to keep things SIMPLE, EFFICIENT and use Quality components and only buy / support reputable vendors / brands.  I want to have this thread so that many other readers can hopefully learn from my many mistakes and have a better start / experience then I did when I started.


My GARAGE Data Center Mining Design

* 7 x 42U RACKs that can power up to 490 GPUs    
 (Can expand to add up to 3 more racks or 210 GPUs)  
* Intake - 8 WindeeVents   ( Soon to be 12 )      
* Exhaust Fans - Dual 36 " x 4600 CFM Fans.   (Soon to be 3)
* Network - Cisco Catalyst 48 port and NetGear Prosafe 24 port
* GPU - 100 % AMD (470 / 480 / R9 Furies and soon VEGAS!).
* POWER
         - 550 AMP via 3 Panels ( 200AMP, 200AMP and 150 AMP)
         - PSU used are all EVGA 1300 or EVGA Platinum P1200 for the server cases
* PDU - APC Switched PDUs 7911 or Dell metered / managed PDUs
* Surge Protection - Eaton Ultra whole house surge protection  x 3 ( 1 for Each Panel)
* 17 x 30 AMP L6-30R outlets
* Monitoring Software - Awesome Miner or Simplemining.net

http://i.imgur.com/K9R4pSr.jpg?1

http://i.imgur.com/OlNoLki.jpg

http://i.imgur.com/4EhP1X5.jpg?1

http://i.imgur.com/UXp5uvw.jpg?1

http://i.imgur.com/ghsrXYF.jpg?1




Shed

* Can support up to 32 Rigs or over 200 GPUs but I'm going to stay below 25 Rigs for safety reasons
* Designed as Play zone and to support my Legacy Racks vs Server Rack for Garage.
* 5 Exhaust Fans with expansion space prebuilt
* Intake via floor with Bug Nets and Metal rails for Rodent protections
* Network = HP Switch 24 port x 2
* POWER
         - 550 AMP availability but only 120 AMP currently patched in
          - 8 x 30 L6-30R Outlets sharing 4 x 30 AMP GFI Breakers
         - PSU are mostly 4KW Server PSU via FINKSY with 700W EVGA baby PSU (Soon to be PICOs).  
         - The Chinese white rigs are powered by EVGA 1300

http://i.imgur.com/iSTgLDk.jpg

http://i.imgur.com/XzPXasb.jpg

http://i.imgur.com/MnyJVFV.jpg

http://i.imgur.com/rUBPHqY.jpg

 Quick Links

1) Motherboard Reviews
Links to Follow

2) Simple Rig Resetters
Links to Follow

3) Pico Reviews
Links to Follow

4) SEEK / FLIR - Thermal cameras
Links to Follow

5) PDU Reviews

6) PSU Reviews

7) Awesome Miner Review

Cool SEXIEST Mining Gears

I do lots of Product reviews in my thread and I get ZERO $ kickbacks from those Vendors or resellers.  However, no one understands other miners better than miners themselves, so support our community.

For Risers /  Fans - I would recommend reaching out to Hawkfish007
For Server PSU / Fans - I would recommend reaching out to Finksy
For the Most kick ass Mining OS and Rig resetting Automation - Reach out to Tytanick and www.simplemining.net










Had to go clean my self up. Smiley Beautiful setup my friend! happy mining
Techman34
Newbie
*
Offline Offline

Activity: 2


View Profile
May 26, 2017, 02:23:27 PM
 #715

*********DRAFTING!!!!!!!!!!!!!!!!!!!!!!

My original thread was getting over 22K Views in less than a month but I forgot 1 important thing.  I need to make myself the MODERATOR.   So I'm redoing / continuing the thread with this extension so I can make any edits / changes where it will be Easier for new Readers to follow as well as for original followers to be able to continue where we left off.

Link to Original Thread.
https://bitcointalk.org/index.php?topic=1761303.0

MY FARM SETUP

I have been mining for a very long time.  Most of you would know me as Nhan Do from my old Nick that was Hacked, hence why I'm using this new Nick.  I have been thru the many ups and downs of Crypto as well as hundreds of designs and layouts.  What I learned from my experience / ]mistakes, is that it's always best to keep things SIMPLE, EFFICIENT and use Quality components and only buy / support reputable vendors / brands.  I want to have this thread so that many other readers can hopefully learn from my many mistakes and have a better start / experience then I did when I started.


My GARAGE Data Center Mining Design

* 7 x 42U RACKs that can power up to 490 GPUs    
 (Can expand to add up to 3 more racks or 210 GPUs)  
* Intake - 8 WindeeVents   ( Soon to be 12 )      
* Exhaust Fans - Dual 36 " x 4600 CFM Fans.   (Soon to be 3)
* Network - Cisco Catalyst 48 port and NetGear Prosafe 24 port
* GPU - 100 % AMD (470 / 480 / R9 Furies and soon VEGAS!).
* POWER
         - 550 AMP via 3 Panels ( 200AMP, 200AMP and 150 AMP)
         - PSU used are all EVGA 1300 or EVGA Platinum P1200 for the server cases
* PDU - APC Switched PDUs 7911 or Dell metered / managed PDUs
* Surge Protection - Eaton Ultra whole house surge protection  x 3 ( 1 for Each Panel)
* 17 x 30 AMP L6-30R outlets
* Monitoring Software - Awesome Miner or Simplemining.net

http://i.imgur.com/K9R4pSr.jpg?1

http://i.imgur.com/OlNoLki.jpg

http://i.imgur.com/4EhP1X5.jpg?1

http://i.imgur.com/UXp5uvw.jpg?1

http://i.imgur.com/ghsrXYF.jpg?1




Shed

* Can support up to 32 Rigs or over 200 GPUs but I'm going to stay below 25 Rigs for safety reasons
* Designed as Play zone and to support my Legacy Racks vs Server Rack for Garage.
* 5 Exhaust Fans with expansion space prebuilt
* Intake via floor with Bug Nets and Metal rails for Rodent protections
* Network = HP Switch 24 port x 2
* POWER
         - 550 AMP availability but only 120 AMP currently patched in
          - 8 x 30 L6-30R Outlets sharing 4 x 30 AMP GFI Breakers
         - PSU are mostly 4KW Server PSU via FINKSY with 700W EVGA baby PSU (Soon to be PICOs).  
         - The Chinese white rigs are powered by EVGA 1300

http://i.imgur.com/iSTgLDk.jpg

http://i.imgur.com/XzPXasb.jpg

http://i.imgur.com/MnyJVFV.jpg

http://i.imgur.com/rUBPHqY.jpg

 Quick Links

1) Motherboard Reviews
Links to Follow

2) Simple Rig Resetters
Links to Follow

3) Pico Reviews
Links to Follow

4) SEEK / FLIR - Thermal cameras
Links to Follow

5) PDU Reviews

6) PSU Reviews

7) Awesome Miner Review

Cool SEXIEST Mining Gears

I do lots of Product reviews in my thread and I get ZERO $ kickbacks from those Vendors or resellers.  However, no one understands other miners better than miners themselves, so support our community.

For Risers /  Fans - I would recommend reaching out to Hawkfish007
For Server PSU / Fans - I would recommend reaching out to Finksy
For the Most kick ass Mining OS and Rig resetting Automation - Reach out to Tytanick and www.simplemining.net




Hey Yun / Nhan Do... Must give you a Big shout out as I have appreciated your tips helping me and pushing me to continue building out!  My shed is being delivered today and I will be getting it built out over the coming weeks.  Will post some progress.  Keep being an inspiration!  Cheers
soulmachine
Newbie
*
Offline Offline

Activity: 24


View Profile
May 28, 2017, 11:25:38 PM
 #716

@yun9999 what kind of 42u server rack cabinet are you using ? Is it this one  StarTech.com Open Frame Server Rack Cabinet 4POSTRACK42 https://www.amazon.com/dp/B00HVKOPBW/ ?

https://images-na.ssl-images-amazon.com/images/I/31BbT53i7AL.jpg
yun9999
Sr. Member
****
Offline Offline

Activity: 336


View Profile
June 01, 2017, 03:33:16 AM
 #717

@yun9999 what kind of 42u server rack cabinet are you using ? Is it this one  StarTech.com Open Frame Server Rack Cabinet 4POSTRACK42 https://www.amazon.com/dp/B00HVKOPBW/ ?



Any standard 42U Server Rack will work.  Not AV Rack, server racks.  I have both Dell and HP Racks.  The rack you're showing seem very weak with little support on the side, most likely best used for lighter AV equipment then servers.
cavaliersrus
Hero Member
*****
Offline Offline

Activity: 740

★YoBit.Net★ 350+ Coins Exchange & Dice


View Profile
June 01, 2017, 03:48:17 AM
 #718

yun i love reading in here and seeing you help people out and also looking at the progress this has been doing one of these days i will feel a 100th of the power you guys are proully getting

luvien
Newbie
*
Offline Offline

Activity: 5


View Profile
June 06, 2017, 06:27:27 PM
 #719

Hello Yun,

First, huge thanks for these 2 part.  I read a lot, bought a 42U, 2x Rosewill case.

I read that you use fixed rails instead of Rosewill rails. That make sense.  I can't find fixed rails in Canada. Is it only some brackets? Can you give me an example? Also, I understand that you leave 1U of space between them.

Finally, I found someone selling the GPU rack for the Rosewill case.  Any other option? I will contact him, but the cost of shipping to Canada will be a show stopper I guess.

Huge thanks!

LJ
iaoorin
Newbie
*
Offline Offline

Activity: 13


View Profile
June 06, 2017, 11:21:33 PM
 #720

Do you know where can I buy risers in Europe, even better southern Europe? And also does @Finksy send his PSU to this part of Europe and how much will that be? Or is there someone similar to him who sells similar PSU's?
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 [36] 37 38 39 40 41 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!