Antminer S5 - Underclock - Undervolt - Best J/GH

[VirosaGITS]

There must be a table somewhere in amongst the files with all of all the frequencies you can select from 100MHz to 500MHz? Not at all up on the firmware side of things so need to know where to look. I would hope that you can then add say 75MHz to the table and then select it from cgminer? I suspect there will be a reg value for the divider frequency that can be worked out and also a delay variable? Just need to know where the table is?

Not expecting to find any voltage control, just frequency?


Rich

The only list i have would not be of use to us;
http://pastebin.com/bPpbWbEb

But if later tonight no one gave us the answer, i will try to just figure it out. It can't be that complicated.

[RichBC]

Yes that table looks correct for the frequencies, same values as in the BM1384 Data Sheet. 75Mhz is probably going to be 0583 and will need a delay something like 52ms. Just need to know where the table is?

Rich

[VirosaGITS]

Yes that table looks correct for the frequencies, same values as in the BM1384 Data Sheet. 75Mhz is probably going to be 0583 and will need a delay something like 52ms. Just need to know where the table is?

Rich

I'm not sure how to set the delay or where it comes into play. I'm glad you figured it out because i thought 75 was 4B in hex. So clearly i am missing something.
What do you mean "where the table is"? Do you mean where to plug the value? If so then that is strait in the config json in /config.

[RichBC]

As I said the the software is not at all my thing, but I assume there is a file that you can SSH into and edit on the controller board that will have the frequency divider & delay values? Same as there is for the S3?

Rich

[VirosaGITS]

As I said the the software is not at all my thing, but I assume there is a file that you can SSH into and edit on the controller board that will have the frequency divider & delay values? Same as there is for the S3?

Rich

Yes. I use WinSCP myself, pretty nice as you can edit everything from your windows computer more normally. Anyhow the json config file is /config/cgminer.conf. Open it and you will see the normal JSON style config.

Relevant line is;
"bitmain-freq" : ""

Example of proper value "bitmain-freq" : "3:393.75:1f06".

[RichBC]

So let's take stock on where we are on the project? Underclocking and Undervolting has proved possible on two S5 Miners with V1.91 Hash boards. There are differences in performance between the 4 boards tested which needs to be understood at some point.

Improvement in J/GH is possible at all Frequencies when you have the ability to adjust the voltage. This includes the Standard Frequency of 350Mhz where the voltage can be reduced from 12V to 11.1V reducing the Watts at the Wall from 568W to 510W and improving the efficiency from 0.492 J/GH to 0.442J/GH

Best efficiency with reliable starting and operation was at 150MHz,  9.4V supply (Core Voltage 0.63V) consuming 166W at the Wall giving 495GH with an efficiency of 0.335J/GH.

Best efficiency seen by winding down the voltage after hashing had started, but not sustainable was at 100MHz, 8.4V supply (Core Voltage 0.56V) consuming 97W at the wall giving 330GH with an efficiency of 0.294J/GH.

I feel that I have not yet found the best solution to an efficient, cost effective variable voltage PSU. Best that I have is 2 x 5V Meanwell PSU's in series. These give the full adjustment range needed for undervolting, but are a little low on efficiency at 82% and also a little low on power as they can only be used up to 300Mhz.

The DPS 600 Server PSU allowed adjustment of the output voltage from 10.6V to 12.25V but was disappointing, as when the voltage was reduced so did the power available.

I have also tested the DC-DC Buck Converter that Phil identified, on the output of a 12V Server PSU. The efficiency was extremely good at 96% however they were only good for 10 Amps as after that the load regulation was poor. However the voltage adjustment range was good and they are an option for lower power on a single S5 Hash board. (More detail available if people are interested?

So what next? I think the fun is now over and I need to knuckle down and work out why the V1.91 can be undervolted and the earlier versions can not? Once that is understood I want to explore if it's possible to improve on the results I have and if there is a way of enabling the earlier versions of the Hash board to be undervolted.

When I have worked up the detail I will describe the differences that I have found  between the V1.91 Hash Board & the earlier versions and will welcome any inputs as to the role that these play in enabling undervolting as a prelude to the next stage of the project.

Watch this space...  

Rich

[VirosaGITS]

So let's take stock on where we are on the project? Underclocking and Undervolting has proved possible on two S5 Miners with V1.91 Hash boards. There are differences in performance between the 4 boards tested which needs to be understood at some point.

Improvement in J/GH is possible at all Frequencies when you have the ability to adjust the voltage. This includes the Standard Frequency of 350Mhz where the voltage can be reduced from 12V to 11.1V reducing the Watts at the Wall from 568W to 510W and improving the efficiency from 0.492 J/GH to 0.442J/GH

Best efficiency with reliable starting and operation was at 150MHz,  9.4V supply (Core Voltage 0.63V) consuming 166W at the Wall giving 495GH with an efficiency of 0.335J/GH.

Best efficiency seen by winding down the voltage after hashing had started, but not sustainable was at 100MHz, 8.4V supply (Core Voltage 0.56V) consuming 97W at the wall giving 330GH with an efficiency of 0.294J/GH.

I feel that I have not yet found the best solution to an efficient, cost effective variable voltage PSU. Best that I have is 2 x 5V Meanwell PSU's in series. These give the full adjustment range needed for undervolting, but are a little low on efficiency at 82% and also a little low on power as they can only be used up to 300Mhz.

The DPS 600 Server PSU allowed adjustment of the output voltage from 10.6V to 12.25V but was disappointing, as when the voltage was reduced so did the power available.

I have also tested the DC-DC Buck Converter that Phil identified, on the output of a 12V Server PSU. The efficiency was extremely good at 96% however they were only good for 10 Amps as after that the load regulation was poor. However the voltage adjustment range was good and they are an option for lower power on a single S5 Hash board. (More detail available if people are interested?

So what next? I think the fun is now over and I need to knuckle down and work out why the V1.91 can be undervolted and the earlier versions can not? Once that is understood I want to explore if it's possible to improve on the results I have and if there is a way of enabling the earlier versions of the Hash board to be undervolted.

When I have worked up the detail I will describe the differences that I have found  between the V1.91 Hash Board & the earlier versions and will welcome any inputs as to the role that these play in enabling undervolting as a prelude to the next stage of the project.

Watch this space...  

Rich

Hmm indeed, personally i would like to find high quality PSU with easy variable volt control. I would sure love to see EVGA G2+(or such quality) with a led display and a turn knob that let you downvolt. If not then a higher powered PSU that runs 10-11.9v since those are the range i am interested in.

I would love to see how high you can overclock the unit while keeping the stock 590w~ consumption. And then find the most efficient speed that consume 450w~ per S5.
Personally i would want to run 3 S5 per 15a 120v breaker and this project can make it happen, tho obtaining specifically 1.91v's may prove challenging.

Meanwhile i hope you get to understand the 1.91v better, maybe it could lead to better volt control, then again maybe not, circuitry is above my paygrade.

[RichBC]

I would love to see how high you can overclock the unit while keeping the stock 590w~ consumption. And then find the most efficient speed that consume 450w~ per S5.

My guess for 590W would be to get close to 400MHz with 11.5V giving 1320GH? At 450W the closest measurement I have is at 300MHz with 10.6V Supply (Core Voltage 0.741V) 400W at the wall giving a Hash of 990GH and 0.404J/GH. I could not push it any higher as I felt I was at the limit of the Meanwell 5V PSU/s, but I am sure that 1TH would be possible with 450W. 

Rich

[VirosaGITS]

I would love to see how high you can overclock the unit while keeping the stock 590w~ consumption. And then find the most efficient speed that consume 450w~ per S5.

My guess for 590W would be to get close to 400MHz with 11.5V giving 1320GH? At 450W the closest measurement I have is at 300MHz with 10.6V Supply (Core Voltage 0.741V) 400W at the wall giving a Hash of 990GH and 0.404J/GH. I could not push it any higher as I felt I was at the limit of the Meanwell 5V PSU/s, but I am sure that 1TH would be possible with 450W. 

Rich

At 400Mhz i have hit diminishing returns. Oddly enough the HWE% does not raise by much, only by <0.001%, however i only get a 5gh increase up from 393.75mhz which is 34GH/s up from 387.5hz.

I tried pushing it even further up. At 425hz i had 1.355Th/s for 692Watts and at 437.5 the hashrate dropped to 1.2TH/s, consumption 714watt and HWE% 0.3%. So i am not sure what is causing the hashrate drop. But it is not proprotionate to the HWE%.

Since then i found a trick to keep both board at about the same temperature, i just use a piece of paper to direct air from from a box fan vertically through the "fins" in front.

I may try again during winter with low temps. But i don't think that is the issue since 60c does not seem that higher to me and the temp was not any higher than the few previous clocks.

I wonder if in your research on the 1.91v you will be able to understand why the S5 behave this way at higher clocks?

[RichBC]

I have not documented them very well but have seen examples of when the Hash Rate has not met expectations despite the HW error rate being relatively low. These have however usually been when I am quite undervolted relative to the voltage needed for a "normal" error rate. I usually aim for 0.002% or lower. However I have done very little testing at the higher frequencies as that has not been my main objective.

Hopefully the more work I do the better my overall understanding and that additional useful info and ideas will be found 

Bad news for me, with my 15c electricity cost, is that even with my most ambitious undervolt results they will only see me through making money until the Halving. This assumes that the 4% increase in difficulty continues. After that my only hope is that the S7 has become cheaper or that some emerge onto the 2nd hand market.

Rich

[RichBC]

VirosaGITS thanks for you continued interest I wonder however if anyone else has any interest in my ramblings now that the S7 is released?

Anyway to continue   I am still working on the assumption that S5 Miners prior to the V1.19 Hash Board cannot be undervolted? This is in no way confirmed but is backed up by several members posting over the Months since release that they have tried and failed. I am assuming that their tests predated V1.91, here are a couple of examples :-

My results are 0.44w/GH par 11.1v
Under 11v. Miner starts but dies after a few minutes.
At 10v and lower, it doesn't start hashing., but you have access to the web interface.
Stock efficiency with the same PSU is 0.54w/GH

The test unit also seems to cut out at 9.45V @ 200MHz even if I start at a higher voltage, and 9.75V @ 250MHz.

If anyone else has had any luck starting these at 10V or under I'd be interested in hearing it.

The V1.91 Hash board has two significant changes and I am assuming that one or both of them are responsible for enabling hashing at the lower Core voltage in the BM1384 spec sheet? I will describe these changes in my next post but it is interesting first to speculate what happened at Bitmain?

When first released the publicity included this statement.

When better power efficiency is needed in the future due to higher network difficulties, you may want to buy some special PSUs 9V DC with more than 10A output, which will allow you to have a 0.2J/GH mining efficiency, but at lower hashing speed.

Clearly they expected the unit to operate with a 9V supply, which equates to a Core Voltage of 0.6V. The 0.2J/GH was however slightly ambitious relative to the chip spec which is 0.249J/GH.  

This statement disappeared at some point with as far as I know no explanation as to why? Simple best guess would be that they found that with the release version the Miner did not hash at much below 11V. You could also speculate that someone in Marketing got a bit carried away with some engineering data and then it dawned on them that anything that prolongs the life of a miner is not good for future sales?

The more interesting point to me is what motivated them quite late on in the S5 life to make some changes that enable the unit to be undervolted, particularly as they chose to make no mention of the value of the change?

My theory is that Bitmain likes to experiment with ideas in a shipping product that will then be used in a new product. Examples of this are the S3++ which trialled String Supply later then seen in the S5 and the changes to the S5 board layout that then enabled board side mini heatsinks to be fitted, then seen in the S5+.

I will explain the 2 changes in my next post and then speculate further.  

Rich

[VirosaGITS]

VirosaGITS thanks for you continued interest I wonder however if anyone else has any interest in my ramblings now that the S7 is released?

Anyway to continue  I am still working on the assumption that S5 Miners prior to the V1.19 Hash Board cannot be undervolted? This is in no way confirmed but is backed up by several members posting over the Months since release that they have tried and failed. I am assuming that their tests predated V1.91, here are a couple of examples :-

My results are 0.44w/GH par 11.1v
Under 11v. Miner starts but dies after a few minutes.
At 10v and lower, it doesn't start hashing., but you have access to the web interface.
Stock efficiency with the same PSU is 0.54w/GH

The test unit also seems to cut out at 9.45V @ 200MHz even if I start at a higher voltage, and 9.75V @ 250MHz.

If anyone else has had any luck starting these at 10V or under I'd be interested in hearing it.

The V1.91 Hash board has two significant changes and I am assuming that one or both of them are responsible for enabling hashing at the lower Core voltage in the BM1384 spec sheet? I will describe these changes in my next post but it is interesting first to speculate what happened at Bitmain?

When first released the publicity included this statement.

When better power efficiency is needed in the future due to higher network difficulties, you may want to buy some special PSUs 9V DC with more than 10A output, which will allow you to have a 0.2J/GH mining efficiency, but at lower hashing speed.

Clearly they expected the unit to operate with a 9V supply, which equates to a Core Voltage of 0.6V. The 0.2J/GH was however slightly ambitious relative to the chip spec which is 0.249J/GH. 

This statement disappeared at some point with as far as I know no explanation as to why? Simple best guess would be that they found that with the release version the Miner did not hash at much below 11V. You could also speculate that someone in Marketing got a bit carried away with some engineering data and then it dawned on them that anything that prolongs the life of a miner is not good for future sales?

The more interesting point to me is what motivated them quite late on in the S5 life to make some changes that enable the unit to be undervolted, particularly as they chose to make no mention of the value of the change?

My theory is that Bitmain likes to experiment with ideas in a shipping product that will then be used in a new product. Examples of this are the S3++ which trialled String Supply later then seen in the S5 and the changes to the S5 board layout that then enabled board side mini heatsinks to be fitted, then seen in the S5+.

I will explain the changes in my next post and then speculate further. 

Rich

Cheers, no problem, its a good thing to to put interest in and just as people are interested in undervolting S3's at the moment, S5 are coming up next.

I'm not particularly in a rush to underclock the S5 because it is still an efficient unit and i think i pay like 1/4 of what you pay for your electricity iirc you said you pay 0.15? But it still has a parallel effect to what i wish to do.

Of course the best would be to underclock it without using specials PSU but beside finding a bulk step down that has a high efficiency and current capacity, it ain,t happening. Maybe Jabberwock or Sidehack could make some PCI-E step down, i'm not sure if adjustable volt control could be a thing, but it might resolve our issue completely as for obtaining the required hardware.

For the S5 having low voltage support removed, i think Bitmain just cheaped out and rushed out production and sold the S5 for cheap to kill the competition. That would be my market movement speculation.

[RichBC]

Cheers, no problem, its a good thing to to put interest in and just as people are interested in undervolting S3's at the moment, S5 are coming up next.

Three more 4% difficulty changes will see my most undervolted S3 with 0.64V & 125MHz giving 0.49J/GH at break even so I will be putting them on ebay as soon as I get round to it.  


So moving to the changes to the V1.91 hash boards. A bit complex so needs a careful read.  

1) The PCB was changed at V1.9 to make provision for multiple Xtal Oscillators, these were then fitted at the V1.91 rev. In all the earlier boards there was a single Xtal on the bottom chip in the chain  and then the clock signal was then chained through the 30 chips.

V1.91 has 4 Xtals, the original one on the bottom chip in the chain and then an additional Xtal after every 4 chip pairs.

As this is a cost addition I am sure that Bitmain must have had a good reason to fit the additional Xtals. Why this might improve operation when the miner is undervolted I do not know? However it is a contender for the improved performance.


2) Second change is a strange one. In the original design each chip pair in the chain had an LDO Voltage Regulator that supplies the PLL and IO circuitry.

The LDO was fed from the Core voltage of the chips 3 steps up the chain. This means with a 12V supply that the voltage into the LDO is 3 x 0.8V = 2.4V. Output voltage from the LDO is 1.8V for the IO & Analog PLL with then a potential divider to give 0.9V for the digital PLL.

The LDO for the last 3 chips in the chain are supplied from a small 14V Buck Converter, this voltage, when the supply is at 12V, equates to 2V  into the LDO for the last chips, 2.8V for one back & 3.6V for two back.

When the Miner is undervolted the input voltage to the LDO will reduce and with a supply of 9V the voltage into the LDO is 3 x 0.6V = 1.8V. This could be part of our problem when undervolting as the dropout voltage of the LDO is 0.3V. So this would mean that any voltage less than 2.1V would be at risk of being insufficient for the circuitry? 2.1V would be reached when the supply is reduced from 12V to 10.5V.


The change in V1.91 is to do away with the LDO's. The supply voltage for the analog  PLL and IO circuitry is now taken directly from Core Voltage of the stage above. The same potential divider is retained to feed the digital PLL The last stage in the chain retains an LDO driven from the 14V Buck converter.

So now looking at some numbers associated with the new means of supplying the PLL & IO circuitry is where things get confusing in relation to their value as an undervolt enabler.

So at 12V the voltage will be Core Voltage x2 = 1.6V. This is already very close to the data sheet minimum of 1.62V but almost certainly ok. However when we are undervolted to  9V Core Voltage x2 is only 1.2V which is well below the data sheet minimum.

It is just possible that as the core voltage is reduced the chip "likes" a matched reduction in the supply voltage for the IO & PLL circuitry, or it could just be a Bitmain cost reduction?

So at this stage all I can say is that this is a significant change with a significant effect on voltages when the miner is undervolted, so again may be the key to why undervolting is possible?

What would be good to know is what have Bitmain done on these two design points in the S5+ & S7 ? Comments welcomed on any of the above?

Rich

[VirosaGITS]

I can't really comment on the technical stuff.

Conceptually i'd ask, is it possible to change some regulator, resistors and shit to raise the volt of the thingabob that need a certain minimum while undervolting the overall board?

For the S3, check how much you'd get from Ebay and see what it'd cost to mail it to Canada (Probably prohibitive) but if the end price is comparable to stuff here i could be interested. If not i'm sure someone in the US would be.

[RichBC]

I suspect that shipping to Canada would be prohibitively expensive but will look into it. Fortunately there still seem to be plenty of people around who have not done the maths on what is and will be profitable.  

Once I can get to the bottom of why the V1.91 board can be undervolted then there may be a way of making changes to the earlier boards. However at the moment I am just exploring the differences in the hope of finding out the reasons for the changes and to further my understanding of how the board has been designed and the key parameters that enable operation at different Frequencies and Voltages.

Rich

[RichBC]

I tested one of the 15A DC-DC Buck converters, identified by Phil,  that are widely available on the net & ebay from $8.



Don't you just love the Insurance Tube.  

Here is the test setup using an Electronic Load.



Here the results with 12V Input 10A Output at some typical Undervolt levels.

Code:

Volts In	12	12	12
Current In	9.4	8.6	7.8
Watts In	112.8	103.2	93.6
			
Volts Out	11	10	9
Current Out	10	10	10
Watts Out	110	100	90
			
Efficiency	98%	97%	96%

As you can see efficiency is very good with just a slight drop as you reduce the output voltage. My only criticism, relative to the listed spec, is that Load Regulation at much over 10 Amps rapidly degrades.

Code:

Amps	Volts
0	10.16
1	10.14
2	10.12
3	10.11
4	10.1
5	10.08
6	10.07
7	10.05
8	10.03
9	10.02
10	10
11	9.98
12	9.94
13	9.71
14	9.36
15	8.6

You will need one for each Hash Board in an S5 and the only real problem is that even then they do not have enough power to support frequencies above 225MHz.

In conclusion. Good price, good efficiency, ok for lower frequencies needing 10V supply or less. Above 11 Amps the load regulation is poor but if you are careful this "feature" can be used to provide a higher voltage for start up than running allowing you to undervolt further.  

Rich

[RichBC]

As I said the the software is not at all my thing, but I assume there is a file that you can SSH into and edit on the controller board that will have the frequency divider & delay values? Same as there is for the S3?

Rich

Yes. I use WinSCP myself, pretty nice as you can edit everything from your windows computer more normally. Anyhow the json config file is /config/cgminer.conf. Open it and you will see the normal JSON style config.

Relevant line is;
"bitmain-freq" : ""

Example of proper value "bitmain-freq" : "3:393.75:1f06".

Am away from home but took a controller board with me and have just got round to SSH into it. Have found the line you refer to above and as far as i can see the first character "3" is the delay with "3" being around 10mS, this number increases as you drop the frequency and is 10 when you get to 100MHz. Second number "393.75" is the frequency however I think it might just be the text that appears in cgminer? The third Hex number "1f06" is to set up the divider and corresponds to the data in the BM1384 data sheet.

So I think this is where the current selected Frequency and associated setup information for Delay and Divider ratio are stored? So yes editing them will probably allow a different frequency to be set.

However there must also be a file somewhere which has the complete table of frequencies and with access to that an additional Frequency, Delay and Divider Ratio could be inserted which would then allow selection from within cgminer?

Question is does anyone know where this table is? 

Rich

[VirosaGITS]

As I said the the software is not at all my thing, but I assume there is a file that you can SSH into and edit on the controller board that will have the frequency divider & delay values? Same as there is for the S3?

Rich

Yes. I use WinSCP myself, pretty nice as you can edit everything from your windows computer more normally. Anyhow the json config file is /config/cgminer.conf. Open it and you will see the normal JSON style config.

Relevant line is;
"bitmain-freq" : ""

Example of proper value "bitmain-freq" : "3:393.75:1f06".

Am away from home but took a controller board with me and have just got round to SSH into it. Have found the line you refer to above and as far as i can see the first character "3" is the delay with "3" being around 10mS, this number increases as you drop the frequency and is 10 when you get to 100MHz. Second number "393.75" is the frequency however I think it might just be the text that appears in cgminer? The third Hex number "1f06" is to set up the divider and corresponds to the data in the BM1384 data sheet.

So I think this is where the current selected Frequency and associated setup information for Delay and Divider ratio are stored? So yes editing them will probably allow a different frequency to be set.

However there must also be a file somewhere which has the complete table of frequencies and with access to that an additional Frequency, Delay and Divider Ratio could be inserted which would then allow selection from within cgminer?

Question is does anyone know where this table is? 

Rich

I think not, you would have to use the table we already have and do try to manually figure out which delay to set with the hex.  It is indeed possible that the middle value is literally "just for show".

Since the cgminer use the typical json.config format, there's no need to have a list of code line that you ssh vi edit-freq etc and just #'out the unused line.

I understand what you mean there should be some dictionary for the firmware to automatically change it but i never found it and it might be compiled in the image in boot at start, as such you could try to open image such as cd/mnt/mm1 initramfs.bin.SD for the S4, but for the S5 seem even more elusive. I don't know where it is.

Perhaps someone could elucidate the mystery or give a hint but checking file by file is tedious and i did not manage to find anything relevant. =/

Also for the step down, is it possible to put 2 in parallel or something to allow something like 200w~ per blade? May sound expensive but meh, i'd like something around 11v and i guess if they were adjustable as well it would be a plus, but i don't know how step downs/buck converters work.

[flikflak]

Here are my results with the buck converters:

@175 MHz 10.06 Volts and 580 Gh/s ~215 Watts @ Wall -> 0.371 J/Ghs

So thats 27% less energy.

Despite I've had no HW errors @175MHz there were some @150MHz (100 per hour and less than 0.004%)








So thanks for the ideas to RichBC and all the other great guys.

[RichBC]

I think not, you would have to use the table we already have and do try to manually figure out which delay to set with the hex.  It is indeed possible that the middle value is literally "just for show".

Since the cgminer use the typical json.config format, there's no need to have a list of code line that you ssh vi edit-freq etc and just #'out the unused line.

I understand what you mean there should be some dictionary for the firmware to automatically change it but i never found it and it might be compiled in the image in boot at start, as such you could try to open image such as cd/mnt/mm1 initramfs.bin.SD for the S4, but for the S5 seem even more elusive. I don't know where it is.

Perhaps someone could elucidate the mystery or give a hint but checking file by file is tedious and i did not manage to find anything relevant. =/

Also for the step down, is it possible to put 2 in parallel or something to allow something like 200w~ per blade? May sound expensive but meh, i'd like something around 11v and i guess if they were adjustable as well it would be a plus, but i don't know how step downs/buck converters work.

I tried editing the line cgminer.conf and if you make any change even a single character then the frequency shown in the drop down menu in Miner Configuration is blank. I only have the control board with me so cannot see if the Miner hashes. Have had a bit of a search for a file with all the values in but no luck so suspect you are right that it is compiled.

Not sure about parallel connecting the converters would be concerned if they would power share?

flikflak nice results, which Buck converters are you using? Have you got a V1.91 hash Board?

Rich