HashFast BabyJet users thread

[perezoso]

Also, we did have some early BabyJets escape the factory with the fans installed the wrong way.  The top radiator fans should be blowing out (up) and the other chassis fans should be blowing in.  You should be able to reverse them with a phillips screwdriver if you feel comfortable.  This will help cooling.

-Phil  

Err... confirming:

The two fans on top of radiator:  BLOWING OUT
The two fans on the front:  BLOWING IN

easy enough...

The single fan on the back:  BLOWING IN (?)

There seems to be contradictory advice on this last one...

[HF-Engineer]

Don't worry about temperature.  While the silicon has a wide variation, in general they seem to be work best at between 70-85C.  My experience has been that below 70C there are more errors, so you'll usually see more until the system warms up.  Be sure that if you can set your difficulty, you set it for at least 256.  That's the ideal setting for a single board.  Dual boards can go to 512, and Sierras should be at about 1024.  On some pools, such as Eligius, every few minutes or so the pool looks at your hashrate and automatically bumps you to the optimal.

We are working tracking down the cause of some of these issues.  We are dealing with 4 different areas:

1. The Raspberry Pi USB subsystem (both hardware and driver code).
2. The cgminer software
3. The system control firmware (living in the on-board Microcontroller)
4. The Silicon itself.

We've identified some issues in all of these areas.  For instance, the CRC errors seem to be an anomaly in the RPI's USB stack.  Of course it's normal for silicon of this high-performance design to exhibit some errors, but it should be such a small percentage that it isn't statistically relevant.  However, it looks like all 4 parts are conspiring together to compound the problem.  We have identified some issues with the on-board firmware, so we will be pushing an update soon.  I'll let you know when it goes out.  If you are mining with a system other than our RPI distribution, then there will be some delays as we still need to finalize and test the update mechanism.  Until then you can always get a RPI and load our image (see a few posts above) and let it run long enough to update.  The update mechanism checks for updates every hour, and within 5 minutes of boot up, so you shouldn't need much longer than that to get an update once it's made live.

If this new version of cgminer (3.11.0hf1) seems worse, let me know.  I've found it more reliable here in the lab, but I'd love to hear your experience.  Like I said, for now, disregard it's reported hashrate and use what your pool is reporting.  And yes, it's normal to see around 5% variation throughout a day.  We should have new version to push soon, and if that doesn't improve things, I'll be happy to roll you back to the old one.  All of these versions contain experimental code to try to reduce these problems, that's what the "HF" designation means.

Right now, my main goal is to ensure you keep hashing away.  So anything we can put into place, even if it's only a temporary band-aid, we won't hesitate to do it.  We don't want you to lose any revenue while we get to the bottom of this!

I've been an Engineer for a long time, and I've never worked on a project this difficult, so we appreciate you patience.  There aren't too many chips out there the size of your fingernail that suck down over 100 amps a pop, and then we crammed 4 of them together!

BTW: We just hired 2 more people so we can improve support.  Hopefully you will soon see how awesome it's getting around here!

-Phil

 i

[joshv06]

When do we get out Upgrades and MPP?

[Melbustus]

Don't worry about temperature.  While the silicon has a wide variation, in general they seem to be work best at between 70-85C.  My experience has been that below 70C there are more errors, so you'll usually see more until the system warms up.  Be sure that if you can set your difficulty, you set it for at least 256.  That's the ideal setting for a single board.  Dual boards can go to 512, and Sierras should be at about 1024.  On some pools, such as Eligius, every few minutes or so the pool looks at your hashrate and automatically bumps you to the optimal.

We are working tracking down the cause of some of these issues.  We are dealing with 4 different areas:

1. The Raspberry Pi USB subsystem (both hardware and driver code).
2. The cgminer software
3. The system control firmware (living in the on-board Microcontroller)
4. The Silicon itself.

We've identified some issues in all of these areas.  For instance, the CRC errors seem to be an anomaly in the RPI's USB stack.  Of course it's normal for silicon of this high-performance design to exhibit some errors, but it should be such a small percentage that it isn't statistically relevant.  However, it looks like all 4 parts are conspiring together to compound the problem.  We have identified some issues with the on-board firmware, so we will be pushing an update soon.  I'll let you know when it goes out.  If you are mining with a system other than our RPI distribution, then there will be some delays as we still need to finalize and test the update mechanism.  Until then you can always get a RPI and load our image (see a few posts above) and let it run long enough to update.  The update mechanism checks for updates every hour, and within 5 minutes of boot up, so you shouldn't need much longer than that to get an update once it's made live.

If this new version of cgminer (3.11.0hf1) seems worse, let me know.  I've found it more reliable here in the lab, but I'd love to hear your experience.  Like I said, for now, disregard it's reported hashrate and use what your pool is reporting.  And yes, it's normal to see around 5% variation throughout a day.  We should have new version to push soon, and if that doesn't improve things, I'll be happy to roll you back to the old one.  All of these versions contain experimental code to try to reduce these problems, that's what the "HF" designation means.

Right now, my main goal is to ensure you keep hashing away.  So anything we can put into place, even if it's only a temporary band-aid, we won't hesitate to do it.  We don't want you to lose any revenue while we get to the bottom of this!

I've been an Engineer for a long time, and I've never worked on a project this difficult, so we appreciate you patience.  There aren't too many chips out there the size of your fingernail that suck down over 100 amps a pop, and then we crammed 4 of them together!

BTW: We just hired 2 more people so we can improve support.  Hopefully you will soon see how awesome it's getting around here!

-Phil

 i

Thanks for the details!

The version I'm running right now (Minepeon: 0.2.4.3hf4, cgminer3.11.0hf1) is definitely not working as well as whatever my machine was running about 17hrs ago (presumably when the last update went out). Yesterday I was getting a very consistent >400GH/s reported by Eligius, now my 12hr number is 391GH/s. And as noted in my earlier post, I hadn't seen a single "Bad CRC" error yesterday, and I see a couple every few times I look at the monitor today.

So at least in my case, cgminer3.11.0hf1 seems to be a regression (assuming it's cgminer that was updated). I'm curious to see if the upcoming v3.12 improves things for me. If not, I'd be interested to revert to whatever my machine was running yesterday.

Thanks again.

[jspielberg]

For what it is worth, I still see the CRC errors on my ubuntu miner controller running cgminer 3.12.0.  The hashrate also seems slightly better running off the PC than the raspberry pi.   The image on my pi doesn't allow for ssh access, so, I prefer using the PC.

I also see the hash modules disconnect and then reconnect as a new device occasionally.  It happens fast enough that it doesn't seem to be affecting hashrate at eligius.

With the raspberry pi, the CRC errors are much more frequent.

ps.  Phil, it is great to have a helpful HF representative on the forum.  Definitely a step in the right direction.  An official comment about MPP from HF would be great to hear as well :^)

[dbbit]

When do we get out Upgrades and MPP?

Off topic. See first post of this thread.

[xunknownx]

hey phil, few questions

i'm not really running into any issues but i would like to check my temperature.  how do i do that?
also how do i update? both minepeon and firmware. i dont see any update function in minepeon.  running stock minepeon 0.2.4.3 on the raspberry pi that came with the babyjet and i updated to cgminer3.10.0

[dbbit]

i'm not really running into any issues but i would like to check my temperature.  how do i do that?

See post #31 on this thread on how to ssh to the BabyJet and view the cgminer output, which includes the temperature.

[dentro]

Hi Phil - thanks for jumping into this thread.

In my case, the update a little over 14hrs ago (judging by my system's uptime) seems to have made my stability considerably worse than when I took it out of the box two days ago. For the first day, all cores ran between 69C and 72C, the monitor read a steady 424GH/s, and Eligius reported around 412-420GH/s.

After this update, now I'm getting:
1) Periodic "Bad CRC....discarding packet" errors. I didn't notice a single one of these before the update; now I get them periodically.
2) Temps are considerably higher: 74C-77C
3) Hashrate is lower: 775GH/s reported on the monitor, so I assume that's 2x, meaning: 387GH/s. Eligius reports 374GH/s.

So....any suggestions? What version of CGMiner did my unit ship with (or update to on first boot on Tuesday)? How do I revert?

Note that I haven't touched my unit since I first turned it on, nor has anything changed regarding ambient temp in the room, etc.

there are some posts about that behaviour earlier in this thread. The hashrate drop has something to do with the new cgminer version...

[HF-Engineer]

HF-Engineer - can you tell me  how are the dies are positioned on the chip? is core 0 top left? Trying to figure out which one is running hot.

Call me Phil.  Yes, one of the first patches to cgminer I made was to have it show all 4 temps/voltages so you can determine what's going on.  The standard release only shows one per board. (The max)

If you are looking at the board with the USB connector on the upper left (small black rectangular 5 pin), the dies are numbered clockwise starting with 0 (the first one) being the upper left.  So upper right is 1, lower RIGHT is 2, and lower left is 3.  It you chain more than one module, the top module (the master) which has the USB connection will have the first 4, and each slave (lower modules) will have +4.  My patch to cgminer will show all dies from a chain on one line.

Err... confirming:

The two fans on top of radiator:  BLOWING OUT
The two fans on the front:  BLOWING IN

easy enough...

The single fan on the back:  BLOWING IN (?)

There seems to be contradictory advice on this last one...

since hot air rises, we always want to be "going with the flow" rather than fighting it.  We also ideally want cool air for the board's power supply components, then all that air exhausts through the top.  So yes, ideally all chassis fans should blow cool air in.  The radiator fans then blow that air out the top.

Hi Phil - thanks for jumping into this thread.

In my case, the update a little over 14hrs ago (judging by my system's uptime) seems to have made my stability considerably worse than when I took it out of the box two days ago. For the first day, all cores ran between 69C and 72C, the monitor read a steady 424GH/s, and Eligius reported around 412-420GH/s.

After this update, now I'm getting:
1) Periodic "Bad CRC....discarding packet" errors. I didn't notice a single one of these before the update; now I get them periodically.
2) Temps are considerably higher: 74C-77C
3) Hashrate is lower: 775GH/s reported on the monitor, so I assume that's 2x, meaning: 387GH/s. Eligius reports 374GH/s.

So....any suggestions? What version of CGMiner did my unit ship with (or update to on first boot on Tuesday)? How do I revert?

Note that I haven't touched my unit since I first turned it on, nor has anything changed regarding ambient temp in the room, etc.

The CRC errors are expected on the RPI in general.  It really shouldn't be a high percent, and they always all seem to come in a big clump.  I think we might have moved some of the error logging from verbose only to standard, which may explain why you are seeing them now when you didn't before.  If your temps are higher when your ambient hasn't changed, that's probably a GOOD sign, as hotter chip means more hashing!  Your temps seem fine to me!  Stability should have improved, and by stability I mean that cgminer keeps on hashing regardless of errors, or at least it restarts in the event of a crash.

I have seen the RPI get more finicky after being up for more than a few days, so it's probably a good idea to reboot at least once or twice a week.  It's best to do this through the interface (main stats page has a reboot button).  Don't ever cycle power unless it hangs!

If you have some Linux experience you can manually add a cronjob to issue a reboot at a certain time.  I'll be sure to add an option in the web interface to do this as soon as I can.

-Phil

[kaerf]

When I try any clock speed above 600 I get:

Code:

 [2014-01-30 23:12:26] HFB 6 HFGetHeader usb read err:(-9) LIBUSB_ERROR_PIPE
 [2014-01-30 23:12:26] HFB 6: device disappeared, disabling                                                                     
 [2014-01-30 23:12:26] HFB 6 failure, disabling!                                                                                
 [2014-01-30 23:12:34] Hotplug: Hashfast added HFB 7                                                                            

Anyone seen this?

[ninjarobot]

When I try any clock speed above 600 I get:

Code:

 [2014-01-30 23:12:26] HFB 6 HFGetHeader usb read err:(-9) LIBUSB_ERROR_PIPE
 [2014-01-30 23:12:26] HFB 6: device disappeared, disabling                                                                     
 [2014-01-30 23:12:26] HFB 6 failure, disabling!                                                                                
 [2014-01-30 23:12:34] Hotplug: Hashfast added HFB 7                                                                            

Anyone seen this?

Yes, it is a known issue. I don't think it is related to overclocking though as I get those errors at stock speed.

Btw, my BJ hashrate chart on minepeon is starting to look pretty ridiculous now: https://i.imgur.com/0OM87Vl.png

But the system does appear to be stable and the mining pool reports the correct hashrate: 383.55 Gh/s

Temps have come down a lot too:

HFB 1: 76C/.79V 76C/.79V 70C/.79V 71C/.80V 708.2G/740.7Gh/s | A:918350 R:1500 HW:3045 WU:10347.7/m

[HF-Engineer]

hey phil, few questions

i'm not really running into any issues but i would like to check my temperature.  how do i do that?
also how do i update? both minepeon and firmware. i dont see any update function in minepeon.  running stock minepeon 0.2.4.3 on the raspberry pi that came with the babyjet and i updated to cgminer3.10.0

Some of the early systems were sent out with standard MinePeon 0.2.4.3 because our custom version wasn't ready yet, so many features, including automatic updates, are not in there.  In the version we are now providing, we have also included a "Monitor" tab so you can easily view the terminal screen without needing to SSH or mess with a Linux command line.  In addition, if you hook up a monitor or TV to the HDMI port on the RPI, you an watch it there.

Here's what the tab looks like:
http://setup.hashfast.com/files/hashfast-mp-mon-tab.png

Near the top there is a line that begins with "HFB0:".  On that line you'll find the temperatures (in Celsius) and voltages of each die in your system.   There is no need to bother with a command line.

Also, we didn't disable command line access, so you can still SSH just like the original MinePeon.

The version I'm running right now (Minepeon: 0.2.4.3hf4, cgminer3.11.0hf1) is definitely not working as well as whatever my machine was running about 17hrs ago (presumably when the last update went out). Yesterday I was getting a very consistent >400GH/s reported by Eligius, now my 12hr number is 391GH/s. And as noted in my earlier post, I hadn't seen a single "Bad CRC" error yesterday, and I see a couple every few times I look at the monitor today.

So at least in my case, cgminer3.11.0hf1 seems to be a regression (assuming it's cgminer that was updated). I'm curious to see if the upcoming v3.12 improves things for me. If not, I'd be interested to revert to whatever my machine was running yesterday.

Thanks again.

If the next release (3.12) doesn't improve things, I'll roll everyone back to the previous version. (3.9.0h2)  If you can't wait, you can download the previous one here: http://setup.hashfast.com/cgminer-3.9.0h2.tgz

When do we get out Upgrades and MPP?

Sorry, I'm just an Engineer.  I can't help you with anything related to orders, so don't bother asking me.  BTW, I'm doing this on my own, Supporting you guys is not in my job description, nor was I asked to by my bosses.

-Phil

[ninjarobot]

BTW, I'm doing this on my own, Supporting you guys is not in my job description, nor was I asked to by my bosses.

Phil, you are doing an awesome job. Everybody wants to get their miner running smoothly and understand what is going on. We know there are some issues to be worked out and that is totally ok. Hopefully, if we work together we get there sooner

[HF-Engineer]

When I try any clock speed above 600 I get:

Code:

 [2014-01-30 23:12:26] HFB 6 HFGetHeader usb read err:(-9) LIBUSB_ERROR_PIPE
 [2014-01-30 23:12:26] HFB 6: device disappeared, disabling                                                                     
 [2014-01-30 23:12:26] HFB 6 failure, disabling!                                                                                
 [2014-01-30 23:12:34] Hotplug: Hashfast added HFB 7                                                                            

Anyone seen this?

Yes, it is a known issue. I don't think it is related to overclocking though as I get those errors at stock speed.

Btw, my BJ hashrate chart on minepeon is starting to look pretty ridiculous now: https://i.imgur.com/0OM87Vl.png

But the system does appear to be stable and the mining pool reports the correct hashrate: 383.55 Gh/s

Temps have come down a lot too:

HFB 1: 76C/.79V 76C/.79V 70C/.79V 71C/.80V 708.2G/740.7Gh/s | A:918350 R:1500 HW:3045 WU:10347.7/m

We can't condone overclocking, and just by luck-of-the-draw, some dies work faster than others and some run hotter, etc.  What I can say is that we are working to improve the firmware which should result in much more stable performance, whether overclocked or not.  One of the things we did is when the dies get into an unknown state, there is a watchdog timer that resets the board after about 15 seconds.  This will cause the "Device disappeared, disabling" errors, and then it will pop right back up and cgminer thinks it's a new device.  Hopefully one of the changes soon to cgminer will track the devices by their serial numbers so that doesn't happen and you don't end up with a string of different devices over time.

If you do try overclocking, and I'm not saying you should, (it most definitely voids your warranty) a little birdie told me to tell you to explore many different speeds in small increments.  I've seen some boards not "like" a particular speed or range of speeds, and sometimes that means that a faster speed might even be more stable.  Even 5mhz can make a difference!

-Phil

[xunknownx]

hey phil, few questions

i'm not really running into any issues but i would like to check my temperature.  how do i do that?
also how do i update? both minepeon and firmware. i dont see any update function in minepeon.  running stock minepeon 0.2.4.3 on the raspberry pi that came with the babyjet and i updated to cgminer3.10.0

Some of the early systems were sent out with standard MinePeon 0.2.4.3 because our custom version wasn't ready yet, so many features, including automatic updates, are not in there.  In the version we are now providing, we have also included a "Monitor" tab so you can easily view the terminal screen without needing to SSH or mess with a Linux command line.  In addition, if you hook up a monitor or TV to the HDMI port on the RPI, you an watch it there.

Here's what the tab looks like:


Near the top there is a line that begins with "HFB0:".  On that line you'll find the temperatures (in Celsius) and voltages of each die in your system.   There is no need to bother with a command line.

Also, we didn't disable command line access, so you can still SSH just like the original MinePeon.

The version I'm running right now (Minepeon: 0.2.4.3hf4, cgminer3.11.0hf1) is definitely not working as well as whatever my machine was running about 17hrs ago (presumably when the last update went out). Yesterday I was getting a very consistent >400GH/s reported by Eligius, now my 12hr number is 391GH/s. And as noted in my earlier post, I hadn't seen a single "Bad CRC" error yesterday, and I see a couple every few times I look at the monitor today.

So at least in my case, cgminer3.11.0hf1 seems to be a regression (assuming it's cgminer that was updated). I'm curious to see if the upcoming v3.12 improves things for me. If not, I'd be interested to revert to whatever my machine was running yesterday.

Thanks again.

If the next release (3.12) doesn't improve things, I'll roll everyone back to the previous version. (3.9.0h2)  If you can't wait, you can download the previous one here: http://setup.hashfast.com/cgminer-3.9.0h2.tgz

When do we get out Upgrades and MPP?

Sorry, I'm just an Engineer.  I can't help you with anything related to orders, so don't bother asking me.  BTW, I'm doing this on my own, Supporting you guys is not in my job description, nor was I asked to by my bosses.

-Phil

yes, im the one that have the early version of minepeon.  how do i update it?

[jerseyjon]

Hi Phil

Really appreciate you being here on the forum.

Re the missing screw on the cooling - I have phoned, emailed etc and I have heard nothing.  As it is out of action until I resolve this issue, do you know how I can get a front and back screw for that 4th corner to get the cooling working?  I am uk based so if it is a standard part I can get locally that would help.  I looked at the Corsair cooling options but the design of the cooling head looks totally different size. 

Thanks
Jon

[perezoso]

We can't condone overclocking, and just by luck-of-the-draw, some dies work faster than others and some run hotter, etc.  What I can say is that we are working to improve the firmware which should result in much more stable performance, whether overclocked or not.  One of the things we did is when the dies get into an unknown state, there is a watchdog timer that resets the board after about 15 seconds.  This will cause the "Device disappeared, disabling" errors, and then it will pop right back up and cgminer thinks it's a new device.  Hopefully one of the changes soon to cgminer will track the devices by their serial numbers so that doesn't happen and you don't end up with a string of different devices over time.

I had cgminer 3.12 crash twice, once unfortunately overnight.  I happened to be looking at the screen when the other crash happened.  Cgminer popped up a message about imminent reboot and something about a watchdog, and errors were spewing across the screen faster than I could read them, but rather than the BJ restarting, cgminer crashed.  I'm suspecting a similar problem caused the other shutdown.

So I rolled back to cgminer 3.11, which is not prone to the same problem.  Unfortunately, 3.11 has the disadvantage - for me at least - of usually requiring the BJ to be physically hotplugged in order to get hashing each time the program starts up (e.g. after editing the config file), or each time there is a hiccup in my setup (e.g. wifi network goes down).

I don't use minepeon or the RPi.  This is a PC.

[defcon23]

hi , ive got an issue with my bj ... it's start to run , and the psu seems to stop after 30 sec...  , i pushe the little black button on the left of the board, the unit restart , and shutdown 30 second later...       any idea?    thanx

[SolarSilver]

hi , ive got an issue with my bj ... it's start to run , and the psu seems to stop after 30 sec...  , i pushe the little black button on the left of the board, the unit restart , and shutdown 30 second later...       any idea?    thanx

You mean it starts to run when you start up the cgminer process and then it shuts down after 30 seconds? What does the cgminer setting say? I have seen this when the freq is set too high