BAMT - Easy persistent USB key based linux for dedicated miners/mining farms

[lodcrappo]

Fix #21 - Offline interface + Tool updates + Bugfix

Lots of stuff in this one.  

First, fixes a small bug in the last update that causes error messages on LPs (does not effect mining).

All the BAMT tools are upgraded as well.  Some of the changes are things I had fixed/added long ago and never made it into the mainline. Other improvements are all new.  Lots of changes you probably won't notice but should help things work in more situations.  Most don't need you to change anything, but a couple things do need your attention to take advantage of them:

There are 2 new per GPU config options:

  # optional phoenix queuesize parameter: how many work units to keep queued at all times
  # queue_size: 2

  # optional phoenix avgsamples parameter: how many samples to use for hashrate average
  # avg_samples: 10

These correspond to the matching Phoenix parameters.  There is also a new optional item that can be specified on each pools file line to pass a backup pool URL to Phoenix.
This is the 4th paramter on an entry, after the primary, max shares, and pool timeout (max shares and pool timeout are optional but the place must be held).
So, to use a backup pool, make an entry like:

http://u:p@mypool.com,10,60,http://u:p@mybackuppool.com

or, if you don't want to set maxshares and pool timeout:

http://u:p@mypool.com,,,http://u:p@mybackuppool.com


You will find an example bamt.conf and pools file, both with all options described, in /opt/bamt/examples


Finally, I've added a new thing I call the BAMT Offline Config and Control Thing.  I'll try to explain it.

There is a new directory, "BAMT", that will show up on the USB key when it in inserted into a Windows or Mac PC (or another linux box too, but with a linux box you could just mount the real drive anyway..)  

Inside the BAMT directory are 3 more directories, their function described below.

CONFIG/

Files inside the CONFIG directory are synced with files in the /etc/bamt
directory inside the BAMT system.  Syncronization is done based on last write
timestamp, and is performed any time the /etc/init.d/mine script is executed.  
The newer version of the file will replace the older whether from this
directory or from /etc/bamt.  Please check that your system clocks are
reasonably accurate on all computers that touch either set of files, or
you may have unexpected results.

This allows you to access and modify the system's configuration from another
machine, or to create a ready to mine key with complete config before you ever
turn on the miner.  For people who are terrified of linux, you can now do all
your configuration with whatever OS you are comfortable in.


CONTROL/

Files inside the CONTROL directory are used to effect the operation of
the BAMT node.  The primary reason to use these settings is in the
recovery of a node that is locking up at boot or in debugging strange
issues.  In other words, if your rig is mining fine, nothing to see in
here.

Inside CONTROL you will find several flag files and a directory
called ACTIVE.  To activate a particular option, move the flag file into
the ACTIVE directory.  To disable an active option, remove it from the
ACTIVE directory (it is suggested you keep the flag files right outside
of the ACTIVE directory in the CONTROL directory).  If you delete or
lose the flag files, it is not a problem.  Simply create a file with
the name of the option you wish to use.  The contents are unimportant.
The names are not case sensitive.

Known options at the time this document was written:

noX             - Do not start X (Mining will not start without X)
noReXconf       - Do not regenerate /etc/X11/xorg.conf
noMine          - Do not start the mining processes (but do start X)
noOC            - Do not overclock any GPUs (but do start mining processes)
noOCGPUx        - Do not overclock GPU x
noGPUx          - Do not mine with GPU x
noStatus        - Do not gather status in offline output dir
noSync          - Do not sync offline and live configs


STATUS/

Contains various information that may be useful when troubleshooting problems.
Most of this is generated when the node boots up or when a serious error
is detected.  Information contained includes the output out various
diagnostics, log files, etc.

If your machine is locking up after boot, you might not be able to login and
look at what's going on.  The STATUS directory contains information that will
help you see what's happening.

OK... I think that's most of it.. enjoy.

[jamesg]

Looks like there is some extra output when running gpumon. Just wanted to let you know in case this was not intentional.

[lodcrappo]

Looks like there is some extra output when running gpumon. Just wanted to let you know in case this was not intentional.

if you mean the performance history that scrolls on the lower part of the screen, it is intentional.
to switch from overview to a specific GPU, press shift+# of gpu.

this allows you to easily see any changes in hashrate, temp, etc over several minutes.

to toggle between the new history view and the pool stats view, press h


or did you mean something else?

[jamesg]

Looks like there is some extra output when running gpumon. Just wanted to let you know in case this was not intentional.

if you mean the performance history that scrolls on the lower part of the screen, it is intentional.
to switch from overview to a specific GPU, press shift+# of gpu.

this allows you to easily see any changes in hashrate, temp, etc over several minutes.

to toggle between the new history view and the pool stats view, press h


or did you mean something else?

Nope. That's the one.

Thanks for starting back in on the project. I know I'll be donating!

[lodcrappo]

Fix #22 - mgpumon gets webified


This pretty much explains it...





The updated mgpumon takes an optional argument on the command line, this is a configuration file to use.

the fix will put an example configuration in /etc/bamt/mgpumon.conf that is setup to generate the web interface in the screenshot above at http://yourminer/mgpumon/

to start mgpumon with this config (as root):

mgpumon /etc/bamt/mgpumon.conf

you can adjust the value ranges that are considered problems and several other parameters in the config.  if you would like a mgpumon that generates the web interface and presents the console based interface at the same time, set both genhttp and screen to 1.  the other settings available should be pretty obvious.

note that mgpumon is not dependent on any other bamt files.  you can run it on anything that has perl, doesn't need to be a bamt node at all.  the config file will let you adjust all paths, etc, to work with whatever you have.

[abracadabra]

it just keeps getting better and better 

[deepDown]

Fix #22 - mgpumon gets webified


The updated mgpumon takes an optional argument on the command line, this is a configuration file to use.

the fix will put an example configuration in /etc/bamt/mgpumon.conf that is setup to generate the web interface in the screenshot above at http://yourminer/mgpumon/

to start mgpumon with this config (as root):

mgpumon /etc/bamt/mgpumon.conf

you can adjust the value ranges that are considered problems and several other parameters in the config.  if you would like a mgpumon that generates the web interface and presents the console based interface at the same time, set both genhttp and screen to 1.  the other settings available should be pretty obvious.

note that mgpumon is not dependent on any other bamt files.  you can run it on anything that has perl, doesn't need to be a bamt node at all.  the config file will let you adjust all paths, etc, to work with whatever you have.

What would be a way to put this web interface to a non-standard network port? I am guessing it is on standard HTTP port 80 right now?

[lodcrappo]

Fix #22 - mgpumon gets webified


The updated mgpumon takes an optional argument on the command line, this is a configuration file to use.

the fix will put an example configuration in /etc/bamt/mgpumon.conf that is setup to generate the web interface in the screenshot above at http://yourminer/mgpumon/

to start mgpumon with this config (as root):

mgpumon /etc/bamt/mgpumon.conf

you can adjust the value ranges that are considered problems and several other parameters in the config.  if you would like a mgpumon that generates the web interface and presents the console based interface at the same time, set both genhttp and screen to 1.  the other settings available should be pretty obvious.

note that mgpumon is not dependent on any other bamt files.  you can run it on anything that has perl, doesn't need to be a bamt node at all.  the config file will let you adjust all paths, etc, to work with whatever you have.

What would be a way to put this web interface to a non-standard network port? I am guessing it is on standard HTTP port 80 right now?

its bog standard apache.  you can configure it for any port you like.. standard apache config files in the usual places... if you're not familiar, google will show you how.

[lodcrappo]

Fix #23 - cgminer

yeah.. I said I wouldn't do it.  and I didn't think I would.   but..what the hell..  I did it.

this fix provides fairly complete support for using cgminer in place of phoenix, if that's your thing.

you must set a few things in the bamt.conf:

in settings:
 
  # start cgminer? set to 1 if you want bamt to manage the process
  cgminer: 1
 
  # options to cgminer (must have at least --api-listen for any bamt tools to work)
  # see cgminer documentation for options.  you may want to put cgminer config in
  # another file and simply put --config /that/file here.
  cgminer_opts: --api-listen -o some.pool:8332 -u user -p pass
 
  # tcp port for cgminer.. defaults to 4028
  # cgminer_port: 4028



and in each gpu that you want cgminer to manage (or really, that you don't want bamt to spawn phoenix on), you must add:

  cgminer: 1

as well. 

Do not screw up and let phoenix and cgminer use the same gpu at the same time.  You won't like the results.

That's about it.  All the standard BAMT stuff supports cgminer now.. gpumon, mgpumon, mining process control, etc.  You can switch back and forth by just setting a few 1 to 0 in bamt.conf and restarting mining.

This is a bit experimental as I have little knowledge of cgminer.  It seems a bit temperamental, locked up my machine once during testing (a machine that *never* locks up with phoenix).   So...  ymmv.   If you try this, report back with problems.

[bulanula]

WOW. Thank you for all the hard work !

Great thing adding cgminer !

[lodcrappo]

Fix #23 - cgminer

should work now, if you had trouble making cgminer start with 23, just run fixer and it'll give you 23b which fixes up a few more things.  if you hadn't already done 23, fixer will skip straight the the good one.  in any case, you'll end up with the right stuff.

[fitty]

Fix #20, containing a super charged version of Phoenix 1.7.4 is available now via the BAMT fixer.

Donation sent.

Donation sent too. =)

Has anyone ever tried to setup a lan boot server to host the images for each rig? The cheap ass flash drives I bought are dying left right and center. I'd also like to remotely reset the image for a rig if it was running into issues. I have a remote reboot setup already when it locks up, but if it's loading the same image that doesn't always fix the problem. I'd like to have one machine with a HD (maybe raid 1) to host the 10 rig images and have everything boot from LAN. I've found the fastest fix is just swap in a new flash drive and that fixes 99% of the problems I've run into. I'm not sure if it's the flash drive dying or something being corrupted, it's not worth figuring out since it takes 15 seconds to swap the flash drive. But if it's a remote farm that is a little more trouble. =)

Some rigs I switched to laptop HDs I had lying around and that works fine. I'm just wondering if anyone has gone a step further to help manage remote farms.  If no one else has done it does anyone know how hard this would be to setup? If it's not horribly difficult I'll start a bounty. =)

I have been looking at network boot.  it would not be too difficult.  the debian live tools can generate a netboot image from the same source that makes regular bamt. 

also, found some additional writes to disk that are eliminated in upcoming version, that will help keys live longer.

Awesome!

I've used BAMT on 8-15 rigs for what 8 months now? I bought the cheapest flash drives I found find and used any old ones I had lying around. Being used 24/7 for 6 months is probably way beyond what they were designed for. Regardless it's pretty minor. Thanks for the updates, donation coming!

[jamesg]

Setting up cgminer on all 18 rigs now!!!!!!!!!

Looks like lodcrappo earned the bounty I put up. Will be coming this weekend! 

[gnar1ta$]

Setting up cgminer on all 18 rigs now!!!!!!!!!

Looks like lodcrappo earned the bounty I put up. Will be coming this weekend! 

Did I pledge to that? Either way, I won't be able to test for over a month. Please post if cgminer is working correctly and I'll send some BTC.

[lodcrappo]

Setting up cgminer on all 18 rigs now!!!!!!!!!

Looks like lodcrappo earned the bounty I put up. Will be coming this weekend!  

well.. remember its experimental.. and maybe could be done better, so might change how we do it.  don't invest too much time is what i'm saying

i need feedback on general stability and just "does it work", but also on how it works.  right now, i integrated all the monitoring stuff like (m)gpumon, and I'm managing the cgminer process just like phoenix so you can attach via gpumon and all the start/stop controls work..
 but I left the configuration part completely up to you.   Regular bamt overclocking will still be done prior to starting cgminer, but after that its all cgminer's game, bamt leaves it alone and just monitors.  Pools and any other settings cgminer uses must be configured yourself.  You can save them all in a file and tell bamt to use that file as the --config paramter when starting cgminer, or for simple configs you can put all the arguments right into bamt.conf on the cgminer_opts: line.  

i guess the point is.. i *could* make bamt more intrusive, have it manhandle the entire config like it does with phoenix, even use the same pools file etc.... but that would be limiting the possible options in cgminer to basically what can be done in phoenix.  i am assuming thats not what you cgminer guys want   so... if this works.. great. or if you want something else, explain and i'll see what i can do.

(Edit) I'm just gunna edit this till I stop remembering other things..

A note that you can distribute the cgminer config files just like bamt.conf or pools files via the managed config command.  As long as they live in /etc/bamt, they can be synced and also will participate in the new offline control thing.  so if you did like

cgminer_opts: --config /etc/bamt/cgminer.conf

and then distribute cgminer.conf along with the others, you now have a centrally managed cgminer node.

[gnar1ta$]

So if I want cgminer to handle everything all I need is cgminer_opts: <...> in the .conf and cgminer: 1 on all the gpu's? No clocks, fans, pools, or kernels on each gpu?

[lodcrappo]

So if I want cgminer to handle everything all I need is cgminer_opts: <...> in the .conf and cgminer: 1 on all the gpu's? No clocks, fans, pools, or kernels on each gpu?

clocks and other overclocking are optional as always.  if specified, they will be set prior to starting cgminer, just like they are set prior to starting phoenix.  you can leave them out entirely if you prefer (always could).

pools and kernels are set in your cgminer_opts line, no need to set them again on each gpu.  if you want per gpu pool configuration, you'll need to configure it inside cgminer.  bamt does not interfere with or control cgminer's pool management.

you do also need a cgminer: 1 in the main settings section as well as in each gpu that cgminer will use.

[gnar1ta$]

pools and kernels are set in your cgminer_opts line, no need to set them again on each gpu.  if you want per gpu pool configuration, you'll need to configure it inside cgminer.  bamt does not interfere with or control cgminer's pool management.

cgminer doesn't allow for per gpu pool configuration, unless you run separate instances. You're only launching one right, otherwise you would need opts line at each gpu? I always deleted all the stuff I didn't use from the .conf, just made it easier.  Can't wait to get my new gear to test it on.

[lodcrappo]

pools and kernels are set in your cgminer_opts line, no need to set them again on each gpu.  if you want per gpu pool configuration, you'll need to configure it inside cgminer.  bamt does not interfere with or control cgminer's pool management.

cgminer doesn't allow for per gpu pool configuration, unless you run separate instances. You're only launching one right, otherwise you would need opts line at each gpu? I always deleted all the stuff I didn't use from the .conf, just made it easier.  Can't wait to get my new gear to test it on.

only launching one cgminer..  I assumed that since it mines with all the GPUs in one instance then it would be able to assign pools to the GPUs any way you wanted.  sorry i don't know much about cgminer.   would you rather have one per gpu, like phoenix?  makes little difference to me.

you can removr the  "cgminer:1" out of the settings: section entirely, but still put it in the gpu sections.  
this will tell bamt that you have gpus allocated to cgminer, but bamt will not start any cgminer or attempt to control it's process at all.  it will only pull stats from the gpus with cgminer: 1.   in this way you can manage the cgminer process(es) yourself and so run separate instances or experiment.

edit - maybe clearer way to say it:

cgminer: 1   in settings = bamt will start and manage a cgminer, using the cgminer_opts:.   0 or remove, bamt will not touch cgminer process

cgminer: 1   in gpu = bamt will try to pull stats from cgminer for that gpu (whether it started the cgminer or not).   0 or remove, bamt will try to pull stats from phoenix

[gnar1ta$]

Sounds good how it is. All my GPU's go to the same pool and login. Thanks for the work.