BAMT - Easy persistent USB key based linux for dedicated miners/mining farms

[jamesg]

Is there anyone who would be willing to look into what it would take to switch out the phoenix miner with cgminer in the bamt stack? Is this possible?

I am assuming the changes would need to include:

- Startup cgminer instead of phoenix at startup.
- gpumon could be replace with the cgminer interface or gpumon could read the output of cgminer.
- status reporting for mgpumon would need to read the statuses from cgminer.
- anything else i am unaware of.

I would be willing to pay a fixed price (BTC or $$$) to see this completed and working with the existing bamt toolset. It would also be nice if this could be a patch that can be run via /opt/bamt/fixer

[1714023777]

1714023777

[1714023777]

1714023777

[1714023777]

1714023777

[DeathAndTaxes]

IIRC the largest issue is that cgminer provides no mechanism to share data with other processes.  However one could potentially setup cgminer to write to log and they parse the log file.  Not sure if the log provides everything that BAMT currently provides.

[kirax]

Is there anyone who would be willing to look into what it would take to switch out the phoenix miner with cgminer in the bamt stack? Is this possible?

I am assuming the changes would need to include:

- Startup cgminer instead of phoenix at startup.
- gpumon could be replace with the cgminer interface or gpumon could read the output of cgminer.
- status reporting for mgpumon would need to read the statuses from cgminer.
- anything else i am unaware of.

I would be willing to pay a fixed price (BTC or $$$) to see this completed and working with the existing bamt toolset. It would also be nice if this could be a patch that can be run via /opt/bamt/fixer

Yeah, this was discussed earlier, cg miner doesn't really output a lot of the data needed: You would need to make changes to cgminer as well, I assume.

[lodcrappo]

IIRC the largest issue is that cgminer provides no mechanism to share data with other processes.  However one could potentially setup cgminer to write to log and they parse the log file.  Not sure if the log provides everything that BAMT currently provides.

Writing to a log constantly and parsing it will kill a USB key in no time, so that is not an option unfortunately.  Many requests have been made to the cgminer author for some more suitable mechanism, but as far as I know none of them have been answered.

[DeathAndTaxes]

IIRC the largest issue is that cgminer provides no mechanism to share data with other processes.  However one could potentially setup cgminer to write to log and they parse the log file.  Not sure if the log provides everything that BAMT currently provides.

Writing to a log constantly and parsing it will kill a USB key in no time, so that is not an option unfortunately.  Many requests have been made to the cgminer author for some more suitable mechanism, but as far as I know none of them have been answered.

It would be possible to use a ramdrive and simply write and read the log from ram.  Technically linux supports memory mapped files but since cgminer doesn't support that ramdrive is a good "hackaround" as it will be seen by cgminer as a normal drive.

[lodcrappo]

IIRC the largest issue is that cgminer provides no mechanism to share data with other processes.  However one could potentially setup cgminer to write to log and they parse the log file.  Not sure if the log provides everything that BAMT currently provides.

Writing to a log constantly and parsing it will kill a USB key in no time, so that is not an option unfortunately.  Many requests have been made to the cgminer author for some more suitable mechanism, but as far as I know none of them have been answered.

It would be possible to use a ramdrive and simply write and read the log from ram.  Technically linux supports memory mapped files but since cgminer doesn't support that ramdrive is a good "hackaround" as it will be seen by cgminer as a normal drive.

yeah that is a possibility.  i thought about doing that originally with phoenix but it felt like such a hack, I just put a socket in, and that turned out to be easy and work well. 

[DeathAndTaxes]

IIRC the largest issue is that cgminer provides no mechanism to share data with other processes.  However one could potentially setup cgminer to write to log and they parse the log file.  Not sure if the log provides everything that BAMT currently provides.

Writing to a log constantly and parsing it will kill a USB key in no time, so that is not an option unfortunately.  Many requests have been made to the cgminer author for some more suitable mechanism, but as far as I know none of them have been answered.

It would be possible to use a ramdrive and simply write and read the log from ram.  Technically linux supports memory mapped files but since cgminer doesn't support that ramdrive is a good "hackaround" as it will be seen by cgminer as a normal drive.

yeah that is a possibility.  i thought about doing that originally with phoenix but it felt like such a hack, I just put a socket in, and that turned out to be easy and work well. 

I agree ram drive is a 100% hack (no matter what it is used for).  I just thought of it because at work we had to get data out of a legacy app that we no longer had source code for.  It runs in a Windows XP virtual machine and write to a ramdrive so it can be quickly processed by a more modern system.  Sometimes hacks work.  Sockets or named pipes are far superior if you can easily modified the sender.  The issue with modifying cgminer is if changes aren't incorporated into mainclient you are stuck w/ patching and rebuilding it after every version and BAMT will be unable to function without the modifed version.  A ram drive while a hack provides seemless integration between the two projects.

[jamesg]

I agree ram drive is a 100% hack (no matter what it is used for).  I just thought of it because at work we had to get data out of a legacy app that we no longer had source code for.  It runs in a Windows XP virtual machine and write to a ramdrive so it can be quickly processed by a more modern system.  Sometimes hacks work.  Sockets or named pipes are far superior if you can easily modified the sender.  The issue with modifying cgminer is if changes aren't incorporated into mainclient you are stuck w/ patching and rebuilding it after every version and BAMT will be unable to function without the modifed version.  A ram drive while a hack provides seemless integration between the two projects.

So, to be clear, this can be done now to incorporate cgminer without modifications to cgminer?

[lodcrappo]

I agree ram drive is a 100% hack (no matter what it is used for).  I just thought of it because at work we had to get data out of a legacy app that we no longer had source code for.  It runs in a Windows XP virtual machine and write to a ramdrive so it can be quickly processed by a more modern system.  Sometimes hacks work.  Sockets or named pipes are far superior if you can easily modified the sender.  The issue with modifying cgminer is if changes aren't incorporated into mainclient you are stuck w/ patching and rebuilding it after every version and BAMT will be unable to function without the modifed version.  A ram drive while a hack provides seemless integration between the two projects.

So, to be clear, this can be done now to incorporate cgminer without modifications to cgminer?

it would allow a limited amount of functionality.  you would not be able to do centralized config management, but monitoring via (m)gpumon should be possible, assuming the logs provide the necessary information. 

[jamesg]

I agree ram drive is a 100% hack (no matter what it is used for).  I just thought of it because at work we had to get data out of a legacy app that we no longer had source code for.  It runs in a Windows XP virtual machine and write to a ramdrive so it can be quickly processed by a more modern system.  Sometimes hacks work.  Sockets or named pipes are far superior if you can easily modified the sender.  The issue with modifying cgminer is if changes aren't incorporated into mainclient you are stuck w/ patching and rebuilding it after every version and BAMT will be unable to function without the modifed version.  A ram drive while a hack provides seemless integration between the two projects.

So, to be clear, this can be done now to incorporate cgminer without modifications to cgminer?

it would allow a limited amount of functionality.  you would not be able to do centralized config management, but monitoring via (m)gpumon should be possible, assuming the logs provide the necessary information. 

Could we not start up cgminer using the bamt config? Could you have a wrapper for cgminer the way you wrap the phoenix process?

Sorry if these questions have obvious answers, I am not a rocket surgeon when it comes to linux.

[stryker]

is there an easy way to get bamt to use the whole memory stick/storage device?

Mine keeps running out of space 

[lodcrappo]

is there an easy way to get bamt to use the whole memory stick/storage device?

Mine keeps running out of space 

https://github.com/aaronwolfe/Big-A-Miner-Thing/wiki/Faq

[Transisto]

is there an easy way to get bamt to use the whole memory stick/storage device?

Mine keeps running out of space 

How can you run out of space, this thing is not to download torrents.

[jamesg]

Has some got the config sync working for the pools file from dropbox with wget to where it doesn't overwrite the file if it hasn't changed? If so, what's the args you are using?

[DeathAndTaxes]

I agree ram drive is a 100% hack (no matter what it is used for).  I just thought of it because at work we had to get data out of a legacy app that we no longer had source code for.  It runs in a Windows XP virtual machine and write to a ramdrive so it can be quickly processed by a more modern system.  Sometimes hacks work.  Sockets or named pipes are far superior if you can easily modified the sender.  The issue with modifying cgminer is if changes aren't incorporated into mainclient you are stuck w/ patching and rebuilding it after every version and BAMT will be unable to function without the modifed version.  A ram drive while a hack provides seemless integration between the two projects.

So, to be clear, this can be done now to incorporate cgminer without modifications to cgminer?

it would allow a limited amount of functionality.  you would not be able to do centralized config management, but monitoring via (m)gpumon should be possible, assuming the logs provide the necessary information. 

Could we not start up cgminer using the bamt config? Could you have a wrapper for cgminer the way you wrap the phoenix process?

Sorry if these questions have obvious answers, I am not a rocket surgeon when it comes to linux.

No we could.  Just be aware without RPC you won't have control over cgminer.  So it would be clunky.  You could start it up based on a config file, have it output a log and scan the log periodically to get stats,  if you need ot make a change have web front end make a change to config and then kill the cgminer process and restart it. 

It probably would work but likely would have glitches and gotchas.  I am thinking of forking the cgminer project to implement RPC calls so the miner could be controlled by outside process or even remotely (web front end, or 3rd party app).  That is the only way I see cgminer being viable.  Then you could just use RPC calls to change fans, clocks, pools, restart, get stats, etc.

[lodcrappo]

No we could.  Just be aware without RPC you won't have control over cgminer.  So it would be clunky.  You could start it up based on a config file, have it output a log and scan the log periodically to get stats,  if you need ot make a change have web front end make a change to config and then kill the cgminer process and restart it.  

It probably would work but likely would have glitches and gotchas.  I am thinking of forking the cgminer project to implement RPC calls so the miner could be controlled by outside process or even remotely (web front end, or 3rd party app).  That is the only way I see cgminer being viable.  Then you could just use RPC calls to change fans, clocks, pools, restart, get stats, etc.

This is exactly right, imho.  I came to the same conclusion and decided it wasn't worth the effort of adding rpc type interface to cgminer, since Phoenix worked fine for me.
Phoenix is so simple (a good thing, again imho) that hacking on a couple options and making a simple wrapper does the trick, but cgminer is a whole different beast.

[Transisto]

For those using bamt on 50+ gpu farm, One of the hurdle is using a quality router or using a proxy for distributing work. Because some pool require as much as 12 active connections per gpu, and most cheap router start to choke past 500 connections,  add to this a bittorrent client and web browsing and this figure can easily double.

Aaron had a Proxy in the making but did not release it.

I am using a well supported Hopping proxy and # of connections is ~80% less.

[jamesg]

For those using bamt on 50+ gpu farm, One of the hurdle is using a quality router or using a proxy for distributing work. Because some pool require as much as 12 active connections per gpu, and most cheap router start to choke past 500 connections,  add to this a bittorrent client and web browsing and this figure can easily double.

Aaron had a Proxy in the making but did not release it.

I am using a well supported Hopping proxy and # of connections is ~80% less.

What proxy is this? Also, do you just run the proxy on one of the rigs and point the others to it?

[DeathAndTaxes]

For those using bamt on 50+ gpu farm, One of the hurdle is using a quality router or using a proxy for distributing work. Because some pool require as much as 12 active connections per gpu, and most cheap router start to choke past 500 connections,  add to this a bittorrent client and web browsing and this figure can easily double.

Aaron had a Proxy in the making but did not release it.

I am using a well supported Hopping proxy and # of connections is ~80% less.

What proxy is this? Also, do you just run the proxy on one of the rigs and point the others to it?

Or any other machine.  Depending on your setup you could have a dedicated "proxy rig".

[lodcrappo]

For those using bamt on 50+ gpu farm, One of the hurdle is using a quality router or using a proxy for distributing work. Because some pool require as much as 12 active connections per gpu, and most cheap router start to choke past 500 connections,  add to this a bittorrent client and web browsing and this figure can easily double.

Aaron had a Proxy in the making but did not release it.

I am using a well supported Hopping proxy and # of connections is ~80% less.

What proxy is this? Also, do you just run the proxy on one of the rigs and point the others to it?

yes, the idea was that you run the proxy on one machine and the other nodes just pull work from it.   ideally the nodes would just find the proxy via broadcast and so could be zero configuration.

i used the proxy myself for about a month, but it has some issues that i never sorted out.  it is written in python and is very high performance, 100s of Ghash should be no problem for it.  if someone who knows python wants to take it over, i will give them what i have, but it is not suitable for public consumption at this time and i do not have the time to make it so.

you could also explore using one of the other proxies out there.. when i started the project all the proxies i could find pretty much sucked massively, but that may have changed by now.