Okay so here's the latest in version hf8:

1. Fixed bug that was causing a reboot every hour.
2. Created new update mechanism that doesn't need to reboot RPI.
3. Restored cgminer 3.9.0h2 - This puts the hashrate calcs back and seems to be more reliable.
4. Added "Check Now" button to the update section on the settings page.  (This causes an immediate upgrade if one is available)
5. Added "boot_delay=1" to try to help some RPI's hanging at the "rainbow" when rebooting.  (Thanks rMuD!)

Everyone running the HashFast image will get this within the next hour provided your RPI's are online and Automatic Updates are enabled.  You'll be able to tell from the status page.  (Look for "0.2.4.3hf8" for MinePeon version)

Let me know if you have any problems or suggestions.

-Phil

Ok, thanks for the advice.  I added it to the next update (hf8) which is live now.

-Phil

Ok it sounds like the new 3.12 version isn't helping.   I'll put together another update back to 3.9.

Let me see if there is a way we can do this without needing to reboot.

-Phil

This isn't my area, but my understanding is the BJ upgrades are a second module that you chain to the first.  There is a second mounting location in the BJ for this new board and your BJ should have come with 2 extra power supply cables (look in the bottom of the case) that will plug into the new board.  We will provide the mounting hardware and the ribbon cable that chains the modules.  The web site indicates that is will come with a cooling system, so I assume it will be a second radiator that installs in the front of the case.

Sorry, but I don't know about the MPP at all.  I'm an Engineer, so my goal is to build the stuff and make it work as best as possible.  I can tell you everyone I know in the company I know is working their ass off.  Nights, Weekends, Holidays.

-Phil

Please follow the re-image instructions I've now posted several times.  Get a fresh start, it seems to fix a lot of issues people are having.  You shouldn't be rebooting.

I am running all my test systems off or RPI's.  Initially they were very unreliable, which is what got us on the path of customizing MinePeon.  Now it's pretty solid.  They have their niggles, and of course all things firmware and software are a work in progress, but they should have a 99.9% uptime now.

If you HAVE an extra PC, and can install Ubuntu on it (Windows is not reliable for mining), this is a bit more more stable, but there is a learning curve and you'll have to get cozy with a command line and shell scripts if you want a reliable system.

We hope to release a pre-configured image at some point similar to what we have on the RPI so it's plug-n-play.  Until then, unless you are comfortable with Linux, I recommend you stick with the RPI.

-Phil

It should not hang at the rainbow screen.  We now test every RPI that gets sent out, although there were some initially that didn't get tested.  If you restore the SD to the HashFast image, initially it's not set up for forced HDMI, but we update that in the first few updates.  Once you are at the current version, it should NOT hang on reboot, and HDMI should be on regardless of whether you started with a monitor connected.  If this isn't happening for you, I suggest you try burning to a new fresh SD card.  I've definitely seen the Pi hate on some SD cards!

Sorry for overwriting your configs, we are trying to make it easiest and most reliable for users that aren't very familiar with Linux, so it "just works" as well as possible.  It really sounds like somethings wrong on your system, as the current loads never hang.

-Phil
 

Here's a BJ that's been running great for weeks: 77C/.79V 75C/.79V 76C/.78V 79C/.78V

Here's a 2 board setup running great on my bench for stress test: 86C/.80V 87C/.80V 80C/.81V 80C/.80V 94C/.77V 93C/.77V 91C/.81V 89C/.77V

Unfortunately you can't buy the Rack DCLC model that we use retail.  The closest thing is the H100i, but it comes with extra stuff you don't really need.  My opinion is that this stuff is overpriced.  With all the ASIC companies out there, someone should start another cooler company!

Also, if you order the retail Corsair stuff, be sure to be very careful with the hoses.  The thick rubber ones they use like to pop off and leak green coolant all over everything.  (Don't ask me how I know!)

-Phil  

We are doing the best we can to improve the firmware.  One of the features we want to give you as soon as we can, is per-die speed control.  I can't promise it will give you more speed, but I'm hopeful it will help.

Engineering-wise, I can tell you with certainty that cooling it down more will NOT make it run better.  The only reason you'd want more cooling is if it's running too hot.  If it's going over 90C, then you can worry.  It will shutdown at 105C.

I've found almost without fail, the silicon likes to be warmer.  You'll see this by increased errors when you first bring it up and it's still "cold".  Once it gets around 70C it seems to get happy and the errors usually fall off.

-Phil
 

FYI, We are using the Arctic MX-2 paste.  Our cooler heads arrive with no paste, just bare copper, as we ordered them that way.  During development, I personally did a compare with a bunch of pastes, including the Noctua NT-H1 and the Xigmatek PTI-G4512.  We also experimented with special exotic indium foil.  I found that among the higher grade pastes, there was little difference.  There was a noticeable difference from the stuff they normally pre-apply on the Cool-IT heads, which is one reason why we didn't use that.

What does make a difference is the right pressure, so we modified the mounting to achieve a more even and consistent pressure.  It's important that the screws be tightened all the way down until the shoulders stop.  You can easily verify this by backing them off 1/4 turn and re-tightening them.  When it stops, don't force it.

The reason I am bringing this up, is that some people had screws work lose during shipping, so it's a good idea to check this if you are having uneven temps or overheating.  BE SURE TO TURN OFF THE PSU FIRST, but I recommend leaving it plugged into the wall so there is a ground path.  This will allow draining of any possible static charge that could damage your rig.  Do not attempt this unless you are comfortable working inside electronic equipment!  I also don't recommend you mess with it if it's working well.

DO NOT ever remove the cooler head, as you will need to re-apply paste first!

-Phil

 

Ok guys, I just pushed you Ckolivas' latest update; 3.12.0hf3 (with the RPI reliability patch).  Everyone should have it within the next hour.

We still have the strange hashrate reporting, but hopefully this will speed your rigs back up to were they were with 3.9.0, so let me know what you're seeing.

I also left 3.11 and put back 3.9 there too, and made a symlink so it's easy to switch versions back if you want to play with it.

No other changes.

-Phil

Hi NiteShdw,

Sounds like you have an old version.

I recommend you backup your MinePeon settings (bottom of settings page), then reflash a new image onto your SD card, let it boot, then after 3 minutes or so it should have all the updates and then you can go back to the settings page and restore the backup.

Here are the instructions:
1. If running MinePeon, first make a backup from the Settings tab (near the bottom).

2. Download this SD card image: Windows: http://setup.hashfast.com/hashfast-minepeon-1.0.zip  Linux: http://setup.hashfast.com/hashfast-minepeon-1.0.img.bz2

3. If running windows, we recommend the Win32DiskImager: http://sourceforge.net/projects/win32diskimager/, If running Linux, you can simply type:
(Where /dev/xxx is the device name of your SD card.  If unsure, type "dmesg" after inserting your card.)

4. Boot RPI with the new SD image.  Note that it may take up to 2 minutes to collect current updates from us before it will begin mining.  You can browse here to see when your system comes online and what it's IP address is: http://setup.hashfast.com/.  If for some reason it doesn't come up after a few minutes, power-cycle it.  (We've seen some brands of SD card cause it to hang on reboot)

5. Log into the web interface and restore your backed up MinePeon settings (if any).  Note that the RPI will generate a new wallet when you first boot from the new SD image, and will begin mining using that wallet.  It's important to switch your mining settings using the "Pools" tab so that your mining proceeds end up in the right wallet!

6. If you have any problems, please post them here and I'll do what I can to help.  I'll try to check in here at least once a day.

-Phil

We can't condone overclocking, and just by luck-of-the-draw, some dies work faster than others and some run hotter, etc.  What I can say is that we are working to improve the firmware which should result in much more stable performance, whether overclocked or not.  One of the things we did is when the dies get into an unknown state, there is a watchdog timer that resets the board after about 15 seconds.  This will cause the "Device disappeared, disabling" errors, and then it will pop right back up and cgminer thinks it's a new device.  Hopefully one of the changes soon to cgminer will track the devices by their serial numbers so that doesn't happen and you don't end up with a string of different devices over time.

If you do try overclocking, and I'm not saying you should, (it most definitely voids your warranty) a little birdie told me to tell you to explore many different speeds in small increments.  I've seen some boards not "like" a particular speed or range of speeds, and sometimes that means that a faster speed might even be more stable.  Even 5mhz can make a difference!

-Phil

Some of the early systems were sent out with standard MinePeon 0.2.4.3 because our custom version wasn't ready yet, so many features, including automatic updates, are not in there.  In the version we are now providing, we have also included a "Monitor" tab so you can easily view the terminal screen without needing to SSH or mess with a Linux command line.  In addition, if you hook up a monitor or TV to the HDMI port on the RPI, you an watch it there.

Here's what the tab looks like:


Near the top there is a line that begins with "HFB0:".  On that line you'll find the temperatures (in Celsius) and voltages of each die in your system.   There is no need to bother with a command line.

Also, we didn't disable command line access, so you can still SSH just like the original MinePeon.


If the next release (3.12) doesn't improve things, I'll roll everyone back to the previous version. (3.9.0h2)  If you can't wait, you can download the previous one here: http://setup.hashfast.com/cgminer-3.9.0h2.tgz

Sorry, I'm just an Engineer.  I can't help you with anything related to orders, so don't bother asking me.  BTW, I'm doing this on my own, Supporting you guys is not in my job description, nor was I asked to by my bosses.

-Phil

Call me Phil.  Yes, one of the first patches to cgminer I made was to have it show all 4 temps/voltages so you can determine what's going on.  The standard release only shows one per board. (The max)

If you are looking at the board with the USB connector on the upper left (small black rectangular 5 pin), the dies are numbered clockwise starting with 0 (the first one) being the upper left.  So upper right is 1, lower RIGHT is 2, and lower left is 3.  It you chain more than one module, the top module (the master) which has the USB connection will have the first 4, and each slave (lower modules) will have +4.  My patch to cgminer will show all dies from a chain on one line.

since hot air rises, we always want to be "going with the flow" rather than fighting it.  We also ideally want cool air for the board's power supply components, then all that air exhausts through the top.  So yes, ideally all chassis fans should blow cool air in.  The radiator fans then blow that air out the top.

The CRC errors are expected on the RPI in general.  It really shouldn't be a high percent, and they always all seem to come in a big clump.  I think we might have moved some of the error logging from verbose only to standard, which may explain why you are seeing them now when you didn't before.  If your temps are higher when your ambient hasn't changed, that's probably a GOOD sign, as hotter chip means more hashing!  Your temps seem fine to me!  Stability should have improved, and by stability I mean that cgminer keeps on hashing regardless of errors, or at least it restarts in the event of a crash.

I have seen the RPI get more finicky after being up for more than a few days, so it's probably a good idea to reboot at least once or twice a week.  It's best to do this through the interface (main stats page has a reboot button).  Don't ever cycle power unless it hangs!

If you have some Linux experience you can manually add a cronjob to issue a reboot at a certain time.  I'll be sure to add an option in the web interface to do this as soon as I can.

-Phil

Don't worry about temperature.  While the silicon has a wide variation, in general they seem to be work best at between 70-85C.  My experience has been that below 70C there are more errors, so you'll usually see more until the system warms up.  Be sure that if you can set your difficulty, you set it for at least 256.  That's the ideal setting for a single board.  Dual boards can go to 512, and Sierras should be at about 1024.  On some pools, such as Eligius, every few minutes or so the pool looks at your hashrate and automatically bumps you to the optimal.

We are working tracking down the cause of some of these issues.  We are dealing with 4 different areas:

1. The Raspberry Pi USB subsystem (both hardware and driver code).
2. The cgminer software
3. The system control firmware (living in the on-board Microcontroller)
4. The Silicon itself.

We've identified some issues in all of these areas.  For instance, the CRC errors seem to be an anomaly in the RPI's USB stack.  Of course it's normal for silicon of this high-performance design to exhibit some errors, but it should be such a small percentage that it isn't statistically relevant.  However, it looks like all 4 parts are conspiring together to compound the problem.  We have identified some issues with the on-board firmware, so we will be pushing an update soon.  I'll let you know when it goes out.  If you are mining with a system other than our RPI distribution, then there will be some delays as we still need to finalize and test the update mechanism.  Until then you can always get a RPI and load our image (see a few posts above) and let it run long enough to update.  The update mechanism checks for updates every hour, and within 5 minutes of boot up, so you shouldn't need much longer than that to get an update once it's made live.

If this new version of cgminer (3.11.0hf1) seems worse, let me know.  I've found it more reliable here in the lab, but I'd love to hear your experience.  Like I said, for now, disregard it's reported hashrate and use what your pool is reporting.  And yes, it's normal to see around 5% variation throughout a day.  We should have new version to push soon, and if that doesn't improve things, I'll be happy to roll you back to the old one.  All of these versions contain experimental code to try to reduce these problems, that's what the "HF" designation means.

Right now, my main goal is to ensure you keep hashing away.  So anything we can put into place, even if it's only a temporary band-aid, we won't hesitate to do it.  We don't want you to lose any revenue while we get to the bottom of this!

I've been an Engineer for a long time, and I've never worked on a project this difficult, so we appreciate you patience.  There aren't too many chips out there the size of your fingernail that suck down over 100 amps a pop, and then we crammed 4 of them together!

BTW: We just hired 2 more people so we can improve support.  Hopefully you will soon see how awesome it's getting around here!

-Phil

 i

Sorry, I have to combine my posts, the forum is restricting how much I can post.

ImI:  We will begin releasing utilities and updates soon.  Initially only for Linux-based systems though.  I'll post them as soon as they are ready!

ninjarobot: Thanks, let me know if you still have stability problems.

spectre2112: I'm not sure what aux connector you are referring to.  If you can attach a picture, that will help.  If it's the 5-pin white connector, that's the one that provides +5v standby power to the board, and also enables the board to turn on the PSU. (+12v)



If anyone else is having stability problems, it would help to have a detailed log of the issue.  You can create a temporary log by putting the following line in your "Extra cgminer prarmeters" which is located under the "Settings" tab in your MinePeon web interface:


Be sure to get the single quotes and spaces correct.  Incorrect data entered in this box can break your system!  Note that this will now be logging a large volume of text to your SD card, which is a bad idea, as it will rapidly wear out if it has to log all that data.  So be sure to restore the original settings (default is blank) after your logging session.

If your log is getting overwritten too much because the session keeps dying, simply add an additional ">" to the existing one, with no extra spaces.  This will cause it to append the logs instead of overwriting them.

Once you have seen the problem occur, you can access the log file at: http://ip.of.your.minepeon/log.txt

If you are not sure of your IP, you should be able to find it by visiting http://setup.hashfast.com/.

If you can then zip up the log file and upload it somewhere where you can share it, such as https://mega.co.nz/ , We will then be able to take a look and see what's going on.  This feedback is useful to us so we can improve things!


-Phil

Yes, if your cooler head is missing a screw, that's likely the problem that's causing the shutdown.  You'll see an entry in the cgminer log that shows "thermal overload".  The cooling head is designed to only work properly with about 60psi of even force across the 4 die substrate.  If it's unevenly loaded you will get thermal overload events.  Our firmware will detect the hot die and shut the system down to prevent damage.

Also, we did have some early BabyJets escape the factory with the fans installed the wrong way.  The top radiator fans should be blowing out (up) and the other chassis fans should be blowing in.  You should be able to reverse them with a phillips screwdriver if you feel comfortable.  This will help cooling.

-Phil 

Hi Guys,

My name is Phil, and I'm one of the engineers working crazy hard on getting your mining rigs to you and working great.

We've had to do a lot of work to get to this point (I've been averaging 4 hours a night sleep over the past few months), and we still are cranking as hard as we can, so please bear with us.

If you received a BabyJet and it's not working well, I advise you first to try our special MinePeon distribution on your Raspberry Pi.  It has a patched version of cgminer which helps stability, and also the distribution has a mechanism to enable firmware and software updates to be pushed out to you as we develop them.  Many of the first shipments didn't have this package, but you can download it.

Our #1 priority has been to get you your products, and now we are working hard on making sure they work well for you.  We are hiring people as fast as we can so that we can build a world-class support system for you.

Until then, I'll try and help as much as I can, time permitting.  The best use of my time is in the lab, as I'm sure you'll agree.

If you are running on the Raspberry Pi and your status screen doesn't show your MinePeon Version to be at least 0.2.4.3hf4, then you could benefit from an update.  It also includes cgminer version 3.11.0hf1, which should help solve some reliability issues until we (and Con Kolivas) can get a better version to you.  The known issues on this version are that reported hashrates are too high.  (I've seen it 2X)  Just use your pools stats, as they are going to be the most accurate.  Also, if there is a warm-boot on the RPI with your HashFast boards connected, something in the RPI's USB Hub/Controller gets left in a strange state and it sometimes takes a handful of tries before cgminer can successfully get connected.  We've set up the system to keep trying over and over again, and usually within a minute or two it "takes" and begins mining.  To avoid this, be sure to disconnect the USB before rebooting.  The goal in this special distribution was to ensure mining automatically runs and gets going again if there is a problem, that way you don't wake up to find it died right after you went to sleep.

We hope to get these issues  fixed soon!  Please let me know if you have any problems or possible ideas that may help, so we can try to incorporate them. 


To install the new version:
1. If running MinePeon, first make a backup from the Settings tab (near the bottom).

2. Download this SD card image: Windows: http://setup.hashfast.com/hashfast-minepeon-1.0.zip  Linux: http://setup.hashfast.com/hashfast-minepeon-1.0.img.bz2

3. If running windows, we recommend the Win32DiskImager: http://sourceforge.net/projects/win32diskimager/, If running Linux, you can simply type:
(Where /dev/xxx is the device name of your SD card.  If unsure, type "dmesg" after inserting your card.)

4. Boot RPI with the new SD image.  Note that it may take up to 2 minutes to collect current updates from us before it will begin mining.  You can browse here to see when your system comes online and what it's IP address is: http://setup.hashfast.com/.  If for some reason it doesn't come up after a few minutes, power-cycle it.  (We've seen some brands of SD card cause it to hang on reboot)

5. Log into the web interface and restore your backed up MinePeon settings (if any).  Note that the RPI will generate a new wallet when you first boot from the new SD image, and will begin mining using that wallet.  It's important to switch your mining settings using the "Pools" tab so that your mining proceeds end up in the right wallet!

6. If you have any problems, please post them here and I'll do what I can to help.  I'll try to check in here at least once a day.


If you don't want your RPI checking in for updates, you can disable this in the Settings tab, but I advise you to at least do it manually once in while.  We hope to have some new firmware and software soon!  We also advise you to set a password on MinePeon, especially if exposed to the outside internet.  (not behind a router/firewall)

I know the difficulty just went up and it stinks, but please bear with us.  All of us really are working our butts off to ensure we have the best mining products in the world!

-Phil