Building Cheap Miners : My "Secret"

[sundownz]

So I am learning Linux in preparation for my Dell PowerEdge R815 servers to get here... ended up grabbing 36 of them total (initially I bought 9... but I couldn't help myself).

These things are cheap to work on too... 2GB RAM sticks are $5-7 each... NEW hard drives are $38.99 each (300GB)... so I'm liking that as a value guy (hence my love for the HP Z400s).

Can't wait to fire them all up and rack up some ITNS coin!

Today I installed Ubuntu on my Gigabyte BRIX mini system and my single HP DL580 G7 system. I had been using ETHOS on the DL580 but I couldn't get a lot of stuff to work... so switched over. On the BRIX I tried PIMP-OS first but it didn't like the integrated graphics.... looks like I'm gonna be a solid Ubuntu guy on all the Dells.

Still messing around with the Dell R815 systems  -- getting a bit more creative since the whole market is crap right now.

I had taken them off ITNS and switched to Cryptonight on NiceHash for a bit as alts fell hard... now that isn't really worth it so my next stage...

YescryptR16 on ZergPool seems like my best bet at this  moment... 4  kH per system... assuming 0.100 mBTC/kH/Day that makes $3.32 / Day at $8315 Bitcoin... but I figure on average ~$2.50 / Day compared with $1.86 on NiceHash right now for CryptoNight.

When I add my hash power, though, I will be tripling (or more) the hashrate of YescryptR16 on Zergpool so I hope for a bit more / towards the higher end.

[1715533365]

1715533365

[1715533365]

1715533365

[PharmEcis]

You know we could be working together on things like this, I don't have the power you have, but I'm pulling over 13kwh now so that should gives you a rough idea of what I can bring to the table.

I'm still not sure why you choose to ignore me.  I'm a team player!

[MinersRus]

For those who have picked up Z400's this may be another option.

I picked up four of the following servers on eBay

SuperMicro Storage Server 24 Bay 2026TT-DLRF 2x X8DTT-HEF+ CSE-217
https://www.ebay.com/itm/192395848816?ViewItem=&item=192395848816

I paid $274 total for all four or $68.50 each.

These do not come with processors, memory, heat sinks or power supplies.

These systems have two independent dual processor servers in one case. Each of those independent servers can take dual Xeon 5600 series processors. So each SuperMicro Storage Server 2026TT-DLRF will have four processors.

I purchased a set of six heat sinks for $48.71 or $34.47 for set of four for one of the SuperMicro Storage Servers.
https://www.ebay.com/itm/312081078444

That raised the price for one server to $102.97.

I am picking up locally four other complete "SuperMicro Storage Server 24 Bay 2026TT-DLRF" that have dual 1400 watt power supplies. Since they will run with only one installed (the 2nd is for redundancy) I will use the spare ones to power the four above servers.

---------------------

The following items are zero cost to me as they were leftovers from me buying and selling parts stripped from computer workstations and servers. I made a profit on selling those parts:

a bunch of PC3-10600R 4GB memory
four Xeon E5645 hex core processors

---------------------

The current price is $8.30 for two Windows 10 Pro digital licenses.

So my total price for one of these servers is $111.27.

These servers also come with the "Dual Slot PCIe x8 Riser Card PN:RSC-R2UT-2E8R" so I can install four single slot (or two dual slot) GPUs for additional hashing in each system.

A feature on Supermicro servers is that you can set the fans to economy mode and they are much quieter than traditional servers.

Over time I will purchase faster processors and more heat sinks to fill out the remaining systems.

I will post results when I get these systems running.

[jayntguru]

Interesting build.

The current price is $8.30 for two Windows 10 Pro digital licenses.

Where are you finding this price?

[MinersRus]

Interesting build.

The current price is $8.30 for two Windows 10 Pro digital licenses.

Where are you finding this price?

https://www.ebay.com/sch/i.html?_from=R40&_sacat=0&_nkw=GENUINE+WINDOWS+10+PRO+PROFESSIONAL&_sop=15

[PharmEcis]

Sundownz,

Are you running the R815's with HT Assist turned on or off in the bios?  With it on, the cpu's only report 12mb of L3 and with it off they report 16mb.  I've seen better speeds with the lower l3.  How about you?

[jayntguru]

Interesting build.

The current price is $8.30 for two Windows 10 Pro digital licenses.

Where are you finding this price?

https://www.ebay.com/sch/i.html?_from=R40&_sacat=0&_nkw=GENUINE+WINDOWS+10+PRO+PROFESSIONAL&_sop=15

Thank you much. I didn't even know this was a thing!

[MinersRus]

I just purchased a HP DL 580 G7 system with quad Xeon X7560 processors. I will be swapping out the four X7560 Xeons with four E7-4830 Xeons that have the AES instruction.

However I need help with a suggested Linux install and with getting XMR-Stak (for both CPU and GPU) to run on Linux.

All of my miners to date have been Windows 7 & 10 installs.

PharmEcis and Sundownz I see that you have Dell 815's mining on quad processors any help or advice is appreciated.

[MinersRus]

On the picture only 2 1070?! You also mining with cpu, you know how much wattage with cpu only mining? Or the complete system?

I take picture when setting thing up, its not done yet. I tried cpu mining but miner(Claymore CryptoNote CPU Miner v3.9) report that the CPU(Xeon W3565) does not support AES-NI - slower mining! Mining IntenseCoin speed only 65 h/s for 8 core CPU  , so I mining GPU only.

Not all Xeon's support the AES-NI instruction set.  You will need to upgrade that CPU for mining.

https://ark.intel.com/products/39721/Intel-Xeon-Processor-W3565-8M-Cache-3_20-GHz-4_80-GTs-Intel-QPI

Scroll to bottom, you will see it says AES NO.

Ark should be your first stop whenever looking for intel processors for mining. 

I like to use CPU World (http://www.cpu-world.com) to check on both Intel and AMD processors specs and features.

I like the family major features. The following link clearly shows that the E7-4800 family has the AES instruction and it also has links to all of the Production microprocessors which you can filter by values like number of cores:

http://www.cpu-world.com/CPUs/Xeon/TYPE-Xeon%20E7-4800.html

The Detailed side-by-side comparison feature is also cool:

http://www.cpu-world.com/Compare/935/Intel_Xeon_E7-4830_vs_Intel_Xeon_X7560.html

[sundownz]

Sundownz,

Are you running the R815's with HT Assist turned on or off in the bios?  With it on, the cpu's only report 12mb of L3 and with it off they report 16mb.  I've seen better speeds with the lower l3.  How about you?

I have not checked that setting directly but I do believe my CPUs are reporting 12 MB.

[PharmEcis]

Sundownz,

Are you running the R815's with HT Assist turned on or off in the bios?  With it on, the cpu's only report 12mb of L3 and with it off they report 16mb.  I've seen better speeds with the lower l3.  How about you?

I have not checked that setting directly but I do believe my CPUs are reporting 12 MB.

Default BIOS setting is with HT Assist on, which explains your 12MB L3.

Are you still using 64T?  At 12MB, my boxes are running 56T with some at 2x memory.  64T was significantly slower.

I've gutted all my boxes to 16gb of udimms.  This has saved 50w at a performance penalty of about 30-40H/s.  It also allowed me to sell the bigger more expensive memory.

Minerus:  Protip:  Best CPU's you can get for the 580 series is the 8837.  I had a set of the E7-4870s and they performed worse than the 8837's and I tried EVERYTHING.  You won't be happy w/ the 4830's, I think you'll top out around 1000H/s.  I hope you are going to be adding GPUs to the box or it will just barely pay for it's own electricity.  The X series of CPUs aren't even worth trying to resell, they have become targets for practice at 100y.  

Download Ubuntu 16.04.  Install, it's a no brainer.  Then just search xmr-stak install ubuntu and you'll find several guides that walk you through it.  By now I have it memorized and should have just used a docker...

I racked out for another 20 R815's but I'm going to HODL at the 8 I've got going for now.  Their ROI has taken a big shit ATM and they cost me $1.25 each a day to run basically even w/ my energy costs at .08/kwh.

I'm sure Sundownz has done something similar to what I did and bought an X3 or in his case, ten...

[PharmEcis]

Also, MinersRus, you might toss those CPUs in and the box won't boot.  If that's the case, you need to go snag a torrent or two.  All HP's BIOS stuff is locked behind a paywall but the firmware DVD and SPP DVD's are available via torrent.

[MinersRus]

Sundownz,

Are you running the R815's with HT Assist turned on or off in the bios?  With it on, the cpu's only report 12mb of L3 and with it off they report 16mb.  I've seen better speeds with the lower l3.  How about you?

I have not checked that setting directly but I do believe my CPUs are reporting 12 MB.

Minerus:  Protip:  Best CPU's you can get for the 580 series is the 8837.  I had a set of the E7-4870s and they performed worse than the 8837's and I tried EVERYTHING.  You won't be happy w/ the 4830's, I think you'll top out around 1000H/s.  I hope you are going to be adding GPUs to the box or it will just barely pay for it's own electricity.  The X series of CPUs aren't even worth trying to resell, they have become targets for practice at 100y.  

Download Ubuntu 16.04.  Install, it's a no brainer.  Then just search xmr-stak install ubuntu and you'll find several guides that walk you through it.  By now I have it memorized and should have just used a docker...

I see that the DL580 has two internal USB slots were you able to install Ubuntu 16.04 to an internal USB stick?

Linux newbie I am - "should have just used a docker..." - what does this mean?

I looked up the specs for the E7-8837 vs the E7-4830 and the only difference I see is the clock speed. The 8837 runs all cores at 2.8 Ghz and the 4830 runs at 2.27 GHz.

http://www.cpu-world.com/Compare/24/Intel_Xeon_E7-4830_vs_Intel_Xeon_E7-8837.html

Both the 8837 and the 4830 contain: 8 real cores, 24 MB L3 Cache, AES Instruction

Hopefully the DL580 will allow me to disable HT so that only the real cores are exposed.

Could you post the best XMR-Stak CPU Thread config file you used to the 8837 as I will be able to use that for my 4830's.

The Hash rates for the 4830's should be 19% lower than the 8837's because of the clock speed difference.

If it turns out that the DL580 is not worth it for mining I plan to resell it on eBay. I only paid $360 total for it including the E7-4830 processor upgrade.

[MinersRus]

Also, MinersRus, you might toss those CPUs in and the box won't boot.  If that's the case, you need to go snag a torrent or two.  All HP's BIOS stuff is locked behind a paywall but the firmware DVD and SPP DVD's are available via torrent.

I'm not sure if I got lucky or if HP changed their access but I was able to download the latest BIOS (SP99256) for the DL580.

SOFTPAQ NUMBER:  SP99256
PART NUMBER: N/A
FILE NAME:  SP99256.EXE
TITLE:  System Firmware Upgrade for HP ProLiant DL580 G7 Servers (For USB Key-Media).

VERSION:  2018.02.22A
LANGUAGE:  English
ROM FAMILY: P65
REVISION: A

The only change for this BIOS update is microcode for the Intel processors. Maybe they made this available to all because of Specter and Meltdown.

[PharmEcis]

I wrote a lengthy article about my journey into the E7 world of xeon's here:

http://www.cointainer.life/2018/03/10/say-l3-cache-king/

[PharmEcis]

Someone just bought a set of 10 8837 CPUs.  I bet they read this thread. 

[MinersRus]

Someone just bought a set of 10 8837 CPUs.  I bet they read this thread.  

That was me. It also was two sets of 10.

The seller just relisted another set of 10 if anyone else is interested:
https://www.ebay.com/itm/LOT-OF-10-INTEL-E7-8837-2-66GHZ-24MB-LGA1567-SLC3N-8-Core-CPU/332596230484

I currently have an offer for another five HP DL580 G7 from the seller I bought from. Hopefully it will be accepted before 12 pm tonight because eBay has a current 15% in Bonus eBay Bucks on purchases of $150 or more. Than would earn $200+ eBay Bucks that I would be able to spend early April.

Edit: Opps, looks like I killed the golden goose in asking. Even though I purchased other servers from this seller with a Best Offer this seller decided that the auctions they were doing were not getting what they wanted so the killed the current auction and just left the BIN at $450. They stated that the parts inside were more valuable than what these were selling for at auction.

Oh well because of the higher price I only bought two more of these servers for $974 including tax (free local pickup) and they did qualify for $100 in Bonus eBay Bucks. I forgot that eBay capped the 15% promotion at $100 per purchase.

[MinersRus]

I picked up my HP DL580 G7 late yesterday. It is a beast.

It came with 64GB (16x 4GB) PC3-10600R 1333 MHz memory installed in eight memory cartridges two dimms per cartridge.

HPE ProLiant DL580 G7 Server - Configuring Memory
https://support.hpe.com/hpsc/doc/public/display?docId=c02283239

For mining I don't need all that memory nor that speed as this server the max memory speed is PC3-8500R or 1066 MHz.

I plan to remove four memory cartridges as only four are needed and resell them on eBay.

I am also looking to get 2GB PC3-8500R or PC3-10600R memory to replace the 4GB PC3-10600R installed memory so that I can resell the 64GB PC3-10600R memory.

Selling the above will help in reducing my purchase cost.

The server has the 591205-001 HP PCI-E I/O Expansion Board installed and four 1200 Watt Power Supplies. These power supplies are really 900 watt at 110 VAC. I like that I can set in the Bios how many are active and the number of spares.

I have yet to install Linux as I am still learning how to correctly do memory configuration on the DL580 G7.

In the Bios there are settings for DCU PreFetcher, Hemisphere Mode, System Locality Information Table. It looks like the optimal settings are to Enable DCU PreFetcher, Disable Hemisphere Mode and Enable System Locality Information Table (NUMA).

http://www.linux-mag.com/id/6868

[MinersRus]

PharmEcis I have a few questions on the HP DL580 G7

I see that there is an Internal solid state drive expansion bay option on pages 54 & 55 in the HP ProLiant DL580 G7 Server User Guide. Inside my server I see the place where a 2.5" SSD can be placed and the area on the system board where the SSD Power/Data cable plugs into. I do not have the cable but would like to use a SSD for OS installation and save power by not using any hard drives. Do you have any information on that cable. It appears to be a single connector cable with both power and data. A part number or where I might be able to obtain it is appreciated.

On your Ubuntu install what did you install to: Internal SSD, Internal USB Stick, Internal SD Flash Card or the hard drives on the Embedded HP Smart Array P410i Controller?
Any gotcha's I need to be aware of?

I have a lot of Nvidia GPU's that only take their power from the PCI-E slot. The DL580 G7 with the PCI-E expansion board has lots of PCI-E slots. I have not seen any documentation on what power can be supplied by which slot. I have Quadro 600's that need 40 watts and GTX 750's that need 70 watts. On Dell workstations they clearly state which slots can supply 75 watts and which can only supply 25 watts. Can you point me to any documentation.

--------------------

Edit: I have come across the "Graphics card power cable kit Installation Instructions for For HP ProLiant DL580 G7, DL585 G7, and DL980 G7 servers" and in it it does state IMPORTANT: The PCIe slots provide 75W of power. The remaining wattage is supplied by the power cable to obtain the total wattage required.

Since that statement doesn't have any qualifiers I interpret it to mean ALL of the PCIe slots can supply 75 watts.

Examining my DL580 I see X16 physical slots at slot #2, 3, 5, 6 on the PCIe expansion board and slot 9 and 11 on the main board. Slots 9 and 11 have the card plastic release lever whereas the slots on the PCIe expansion board do not.

Another question is how many GPUs can the DL580 G7 have installed and still boot. I have come across some google links that claim a max of four and if a fifth is installed the system will not boot.

--------------------

Lastly the 1200 watt power supplies are only rated at 900 watts with 110 AC. The 1200 watts is only available if they are powered by 220 VAC. I am also concerned about load balancing in my home if I were to have six of these running. My question is can I put 220 VAC directly to the inputs of these power supplies?

Thanks

[vorpalspear]

I wrote a lengthy article about my journey into the E7 world of xeon's here:

http://www.cointainer.life/2018/03/10/say-l3-cache-king/

thx for the blog, good info
I hit a wall at 1550 H/s with 4x 8870 v1
but i've moved on to yescrypt/koto so i don't care now!
i have 2x 8837 with 2 more on order, i wonder if they can do better than the 8870s on yescrypt (17.3kH with -t 40, 18.5kH with -t 44)
weirdly(?), disabling hyperthreading in the bios (of my R810) didn't help the yescrypt hashrate like it seems to do on i5/i7 in windows