Smartcoin Linux mining administration. [MULTI-MACHINE SUPPORT NOW IN!]

[jondecker76]

Update r564e available.

- Just more underlying multi-machine stuff getting put in...

[jondecker76]

The internal method to determine CPU load  has changed since r500 to a more accurate method, which is why you are seeing a different CPU load.  You are running cgminer with 6 threads (one per CPU core) on a cpu device, which you can now see it maxes out all 6 cores

I thought it was showing load for gpu control only.
with my CPU mining it must be 100%

A load of 1.0 would be approximately 100% load on a single-core CPU.
A load of 2.0 would be approximately 100% load on each core of a dual core CPU...
A load of 6.0 would be approximately 100% load on each core of a 6-core CPU
etc...
etc...
Your report of 6.3 looks right to me, since you are running 6 CPU cores, and CG miner is using 6 threads

[plantucha]

The internal method to determine CPU load  has changed since r500 to a more accurate method, which is why you are seeing a different CPU load.  You are running cgminer with 6 threads (one per CPU core) on a cpu device, which you can now see it maxes out all 6 cores

I thought it was showing load for gpu control only.
with my CPU mining it must be 100%

A load of 1.0 would be approximately 100% load on a single-core CPU.
A load of 2.0 would be approximately 100% load on each core of a dual core CPU...
A load of 6.0 would be approximately 100% load on each core of a 6-core CPU
etc...
etc...
Your report of 6.3 looks right to me, since you are running 6 CPU cores, and CG miner is using 6 threads

i think CGminer load is not included at all

before I was using CPU mining and 1-3 was usual

And with catalyst 11.7 is load 13-15 => 1 core has double load  

[jondecker76]

Update r571e now available
- Many many many mutli-machine things taken care of.  The code that creates the new machines in the database, creates secure connection keys and propagates the keys to the remote machines is fully working and tested.

- Initial rough implementation of the auto-detection routines that will run on the remote machines to initially detect installed miners, gpus, etc.

[Alan Lupton]

Just to get things right: Do we need a smartcoin client on the remote machines, because it doesn't sound like we do?

[jondecker76]

Just to get things right: Do we need a smartcoin client on the remote machines, because it doesn't sound like we do?

No, smartcoin will run on one "master" machine. This keeps it simple as there is no need for any client-server architecture.  When you add a machine through the control screen (this isn't active yet, though I've test it), you give it some information on your remote machine (host address, port, username) and it will try to make a connection over SSH, and if successful a public/private RSA keypair is generated so that the remote commands can be run directly from the master machine. There will still be only 1 database.  As a bonus, the "miner" screen instance doesn't need smartcoin's monitoring functionallity to run, so if the master machine goes down for some reason, the other machines will still keep mining (though without advanced features such as failover, etc).
The architecture also allows for smartcoin to run on a computer that isn't even a miner at all (like a dedicated low-power netbook PC) and control all the miners remotely from there.

[Mistafreeze]

I'm holding off on switching the rest fo my rigs over to Linux until the multi-machine support is enabled. I use LinuxCoin with SmartCoin on 2 of them now, so I'm really, really anxious for the multi-machine support. Incredible work!

[jondecker76]

Just thought I would give a sneak-peak of a new feature that will be available once multi-machine support is in.
You will be able to define "Macros" (macro-profiles to be specific)
This will allow you to define a macro that basically says: (for example)
"Switch Machine 1 to profile 'A'"
"Switch Machine 2 to profile 'B'"
"Switch Machine 3 to profile 'A'"
"Leave machine 4 alone to continue to mining to its current profile"
"Switch Machine 5 to 'Failover'"
Switch Machine 6 to 'Automatic'"

Then by simply choosing one of the macros you defined, all machines will switch their profiles according to the rules you defined in your macro.  Of course, there will be some auto-created macros (Failover, Automatic and Manual Donate) which will switch all profiles on all machines to their Failover, Automatic or Manual Donate profile.

Just thought I'd post this information so you can start familiarizing yourselves with the concept.

[Fletch]

As there seems to be some confusion regarding the CPU load averages, I recommend that you change the text from "CPU Load" to "CPU load averages" and show all three averages instead of just the 1 minute number. That's what people are used to from running "w" and "top".

plantucha: If you're running CPU mining with CGminer, the load that puts on the CPU is included in the "CPU Load" that's displayed in smartcoin. It's the same number that's displayed when you run "top". It's not a percentage.

[jondecker76]

As there seems to be some confusion regarding the CPU load averages, I recommend that you change the text from "CPU Load" to "CPU load averages" and show all three averages instead of just the 1 minute number. That's what people are used to from running "w" and "top".

plantucha: If you're running CPU mining with CGminer, the load that puts on the CPU is included in the "CPU Load" that's displayed in smartcoin. It's the same number that's displayed when you run "top". It's not a percentage.

Yeah, good idea. I'll change the wording and display all 3 averages

[jondecker76]

Update r576e now available!
- General cleanup of the new AutoDetect routine
- 1, 5, and 15 minute load averages now displayed!
- Failover order can now exclude profiles.  Simply don't add the ones you don't want to the comma-separated-list.  The Set Failover Order menu option has been rewritten to clarify this as well.

[krzynek1]

Thank you for exclude option !

[jondecker76]

Update r590e now available!
- The ugly dependency of needing a phoenix path setting is now gone. (the setting is still in the Edit Settings menu for now, though it is now defunct).  When miners are launched, it is checked whether or not the miner is phoenix, and if so use the path already stored in the phoenix miner's database entry instead of from the settings table.

- Some lower-level changes were made that should eventually lead to being able to do miner instance reloads/killing/starting on the fly without killing and recreating the screen session.

- The failover routine was optimized a bit. It used to update the table every iteration. It now only updates the table if there was an actual  change.

- The status script and the monitoring functions have all now been made multi-machine aware.  Sadly, I haven't had a chance to test the parts that run the commands on remote machines but I'll be doing those tests soon. For now, I do know that it knows when a command should run on the local machine, and works as it should.



To give an idea of what still needs done for multi-machine support:
- I'm 90% finished with the new AutoDetect routine which will be run locally by the installer, and also remotely when a machine is added. Most of what I need to do now revolves around testing.

- The settings table needs a revamp so that it holds general settings, as well as settings specific to each machine (each machine could have a different AMD/ATI SDK location for example).  This will require a schema update that uses the breakpoint feature of the update system.  I may push out this shema update later today.  While I'm at it, I'll add an extra field to put better descriptions of the settings so when you edit them you will be given a more complete description of what the setting does.

- The new settings descriptions will need added to the routine that initially populates the database, as well as pushed out as an update patch to existing installs.

- The Configure Settings routine will need revamped to separate out general and specific machine settings, and use the new description feature.  This will actually be pretty simple.

- The Add Machine routine is finished and tested already. The part that adds the information permanently to the database is deactivated for now for my own testing purposes.  I just need to add a call to the new AutoDetection routine so that the miners/devices etc. add to the database automatically.

- I need to implement the Edit Machine and Delete Machine routines.  This isn't very hard, and I may even wait on these as they aren't all that important at the moment.

-Currently, the routine that launches commands on remote machines queries the database for the remote machine information each call.  I need to optimize this to load this information once at the beginning so that we don't pound the database for no good reason. (another easy task)

- Here is the biggest thing right now. I have to rewrite the routine that creates the screen session and actually launches the miner instances so that this happens on the appropriate machine (locally or remotely).  Quite actually, multi-machine would work right now if this were implemented, the AMD/ATI location was the same on all machines and I enabled the new features.

As you can see, we are getting very close. Please report any bugs that you may find with these latest releases, as this will ensure that I catch any problems in the new multi-machine code (which is fully active and being used for the last several updates). Technically, if it works for running commands locally, it should work just as well running commands remotely - so getting this testing out of the way now will help ensure that when multi-machine stuff is finished, it will be a smooth transition!

[jondecker76]

Update r613 now available!

- The settings table now goes through a verification process every boot. This process ensures that the correct settings are available in the database, and there are no duplicates.  This also ensures that it stays up to date with changes automatically(i can change the description information, or re-order the settings in an update)

- The settings now have an information field with more detailed instructions on what they do. This information is now displayed when you edit a setting.

- The entire settings system is now re-written and fully supports multiple machines.

- The control interface has also been rewritten.  You now "drill down" the settings more logically (Edit Settings->General Settings->SettingName in the case of general settings, OR EditSettings->MachineSettings->Machine#->SettingName in the case of machine settings)

- There is a partial update (brakpoint) in this update.  You will fist  update to r607. After a restart of smartcoin, Running the update again will take you from r607 to r613.  I had to do it this way to avoid some potential problems with the massive change of the settings system.


Please post any problems or suggestions.

[italeffect]

Just updated to r617e.

While accessing my miners from ssh...
Seeing a repeating message at the bottom of my screen sessions (screen #1) that says "aticonfig: This program must be run as root when no X server is active".

Tried running as:

smartcoin
screen -x smartcoin
DISPLAY=:0 smartcoin
DISPLAY=:0 screen -x smartcoin

Can't get it to go away. And of course the % load and temps do not show up with this error.

Thanks for any help.

[italeffect]

Sorry posted prematurely. I was able to fix it by fully killing smartcoin and then relaunching with DISPLAY=:0 smartcoin

[jondecker76]

italeffect:
thanks for the report.  I'm committing new code that may help

[Rob P.]

Update r613 now available!

Jon, assuming that's the "e" Experimental release?  I'm only seeing r496s in the stable branch.  True?
Or do I need to do a complete re-install to move into the r6xx stable branch?

[jondecker76]

Update r627e now available:
- More improvements to the way SQL queries are retried if they fail. It is even more robust now
- Many many many lower-level changes for multiple-machine support (99% complete now)
- Lots of work on new autodetect routines (99% complete also)
- Experimenting with a new way to set the DISPLAY=:0 before aticonfig commands are called. Please report if you have any problems.




FOR THOSE WISHING TO TEST MULTI_MACHINE SUPPORT:
You can now test the Adding and Deleting of machines (editing not supported yet)
There is a hidden control screen menu item #13. From there you can add/delete machines to test the new routines.  Note that you won't be able to actually control the machines yet, so I wouldn't bother adding profiles for the other machines yest (though you can for testing purposes).  This information will be useful to let me know that the remote information is being detected correctly, and that the low-level routines which communicate over ssh sockets work correctly.  Please report back your experiences if you do play around with it a little.

[jondecker76]

Update r613 now available!

Jon, assuming that's the "e" Experimental release?  I'm only seeing r496s in the stable branch.  True?
Or do I need to do a complete re-install to move into the r6xx stable branch?

Yes, 'e' stands for experimental, 's' stands for stable.  You can change the settings in Edit Settings from the control screen.
496 is the latest stable release - though pretty much everything new in the experimental branch has tested to be safe.  One thing you could do if you wanted, is:
1) Switch to the experimental branch from Edit Settings
2) Do an update
3) Switch back to stable from Edit settings

What this will do, is update you to the latest experimental revision, and then set you back to stable so that any new updates from then on will wait until the next stable update rolls out (essentially this means that you will be at r627s right now, and you will get no further experimental updates until I increment the stable release counter again.  This is an undocumented feature.