Bitcoin Forum
December 09, 2016, 11:15:47 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Poll
Question: Would you use such a mining proxy?  (Voting closed: April 13, 2011, 10:26:05 PM)
Yes, it seems like a good idea. - 7 (63.6%)
Maybe. - 1 (9.1%)
No, I don't like the idea. - 3 (27.3%)
No, I use something similar already. - 0 (0%)
Total Voters: 11

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 [15] 16 17 »  All
  Print  
Author Topic: Flexible mining proxy  (Read 83966 times)
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
July 10, 2011, 09:03:49 AM
 #281

Do these comments mean this project doesn't work with Multiclone.us.to  ?  Because otherwise, this looks awesome.
Looking at the getwork response from Multiclone, I see no reason why it should not work with my proxy.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
1481325347
Hero Member
*
Offline Offline

Posts: 1481325347

View Profile Personal Message (Offline)

Ignore
1481325347
Reply with quote  #2

1481325347
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
nux
Newbie
*
Offline Offline

Activity: 24


View Profile
July 11, 2011, 04:36:01 PM
 #282

Anyone else having this issue?  I tried a fresh install of the proxy and still having the problem.

This only started after upgrading to  the latest poclbm.

Here's my output:

proxy.hostname.net:80 11/07/2011 11:31:01, Setting pool g11 @ proxy.hostname.net:80
                                                                                proxy.hostname.net:80 11/07/2011 11:31:02, Using new LP URL /index.php/1/aHR0cDovL3VzZWFzdC5idGNndWlsZC5jb206ODMzMi9MUA%3D%3D
proxy.hostname.net:80 11/07/2011 11:31:02, LP connected to proxy.hostname.net:80
proxy.hostname.net:80 229016 khash/snicating with bitcoin RPC 0 2
proxy.hostname.net:80 229331 khash/snicating with bitcoin RPC 0 2
proxy.hostname.net:80 11/07/2011 11:32:15, 02463d19, accepted 0 2
proxy.hostname.net:80 Problems communicating with bitcoin RPC 0 2
proxy.hostname.net:80 11/07/2011 11:32:34, c30efce9, accepted 0 2
proxy.hostname.net:80 228730 khash/snicating with bitcoin RPC 0 2
proxy.hostname.net:80 11/07/2011 11:33:56, 1888d04e, accepted 0 2
proxy.hostname.net:80 11/07/2011 11:34:42, 92512293, accepted 0 2
proxy.hostname.net:80 215716 khash/snicating with bitcoin RPC 0 2


I'm using the latest version of the proxy software, and the latest poclbm.  Since updating to the more verbose poclbm (week or two ago) I see this error all the time:

proxy.hostname.net:80 Problems communicating with bitcoin RPC 0 2

Then it recovers from 0mhash/s and looks like this

proxy.hostname.net:80 93870 khash/sunicating with bitcoin RPC 0 2

until it gets back up to the normal speed (which takes just a second or so).

I see this happen every few seconds give or take.  Has anyone else seen issues like this?  Since the last week or so I've had to stop using this awesome software as that error slows down my hashing.

Thanks

Edit: I'm running Apache on Debian.  Server is tuned to handle a lot of load, and is nowhere near capacity.  I'm not finding any issues in the error_log files and not sure how I can debug further at this point.
nux
Newbie
*
Offline Offline

Activity: 24


View Profile
July 11, 2011, 04:56:32 PM
 #283

After watching tshark, poclbm.py output, and tailing the access log for the proxy, I have determined that a low KeepAliveTimeout setting in the Apache config was the culprit.

The KeepAliveTimeout on my server was 3 seconds, and the ask rate was 5.  This caused the miner to try to use a KeepAlive session, only to find it timed out.  Then it had to establish a new connection.  This was giving the connection issues I was seeing.

Essentially what this means is the KeepAliveTimeout setting has to be higher than the ask rate.  I would suggest adding a few seconds on top to make sure there's a little wiggle room.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
July 11, 2011, 06:52:38 PM
 #284

After watching tshark, poclbm.py output, and tailing the access log for the proxy, I have determined that a low KeepAliveTimeout setting in the Apache config was the culprit.

The KeepAliveTimeout on my server was 3 seconds, and the ask rate was 5.  This caused the miner to try to use a KeepAlive session, only to find it timed out.  Then it had to establish a new connection.  This was giving the connection issues I was seeing.

Essentially what this means is the KeepAliveTimeout setting has to be higher than the ask rate.  I would suggest adding a few seconds on top to make sure there's a little wiggle room.
Thanks for the info!  I'll add this to the readme, and perhaps to the .htaccess file.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
nux
Newbie
*
Offline Offline

Activity: 24


View Profile
July 11, 2011, 07:36:49 PM
 #285

One feature I thought would be handy is something like a worker group.

Essentially a way to mass manage a handful of workers.  For example, create a pool, and update 9 workers at once to all use it with the same login, password, priority etc.

I currently just use SQL to do it manually.  Add a worker_id/pool_id combo manually, and then an UPDATE .. WHERE pool_id = to add a pool to every one of those workers.
NickW
Newbie
*
Offline Offline

Activity: 27


View Profile
July 12, 2011, 01:02:52 AM
 #286

I'm running this fine (so far) with nginx + php-fpm as the webserver.

One thing to note to get LP working is to set fastcgi_read_timeout in the nginx virtual host to something high (such as 3600) along with max_execution_time in the php.ini file.

You also need some fairly specific settings to get the php parameters such as path_info passed along correctly. This is what I have.

Code:
location ~ ^(.+\.php)(.*)$ {
  root    /web/root/here;

  fastcgi_index index.php;
  fastcgi_split_path_info ^(.+\.php)(.*)$;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  fastcgi_param PATH_INFO  $fastcgi_path_info;
  fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;
  fastcgi_read_timeout 3600;
  fastcgi_pass    127.0.0.1:9000;
  include fastcgi_params;

}

1DRCGzEkMhhzieEkDg3e8kUKt1mVM5uqNs
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
July 12, 2011, 12:22:41 PM
 #287

Everytime a block is solved i get this after long poll event:

Quote
[2011-07-12 22:22:02] JSON-RPC call failed: {
   "code": 0,
   "message": "No enabled pools responded to the work request."
}

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 22, 2011, 06:55:37 PM
 #288

I've been having some table locking issues with the curent database since the schema creates all of the tables as MyISAM. As a MySQL DBA per profession, I would recommend that anyone running more than a handful of miners to switch to InnoDB. You can run the following commands on the mysql command line to convert your tables.

Code:
alter table pool engine=innodb;
alter table settings engine=innodb;
alter table submitted_work engine=innodb;
alter table work_data engine=innodb;
alter table worker engine=innodb;
alter table worker_pool engine=innodb;

The schema file can be changed to create tables as InnoDB from the start by changing each line that says "ENGINE=MYISAM" to "ENGINE=INNODB".

<luke-jr> Catholics do not believe in freedom of religion.
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 23, 2011, 07:15:58 PM
 #289

I've created a health monitoring script and a new database table for the Proxy so that you can see your worker "clock speed, mem speed, fan speed, card temperature" from the Proxy Dashboard page.  I have the schema change (just an added table) and the health reporting script all finished. Just need to modify the dashboard to support a couple of new columns for that data.

I'll post the changes required when I'm done - hopefully people enjoy the improvements.

cdhowie - if you like my work with this can you include it in the github repo or should I branch the project? I really love the work you did with the proxy server, it's made my life a lot easier. Smiley

<luke-jr> Catholics do not believe in freedom of religion.
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 23, 2011, 07:17:22 PM
 #290

I'm running this fine (so far) with nginx + php-fpm as the webserver.

One thing to note to get LP working is to set fastcgi_read_timeout in the nginx virtual host to something high (such as 3600) along with max_execution_time in the php.ini file.

You also need some fairly specific settings to get the php parameters such as path_info passed along correctly. This is what I have.

Code:
location ~ ^(.+\.php)(.*)$ {
  root    /web/root/here;

  fastcgi_index index.php;
  fastcgi_split_path_info ^(.+\.php)(.*)$;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  fastcgi_param PATH_INFO  $fastcgi_path_info;
  fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;
  fastcgi_read_timeout 3600;
  fastcgi_pass    127.0.0.1:9000;
  include fastcgi_params;

}



Can you tell me why this stuff is required and why I would want to implement it over Apache (what I'm currently using to run the proxy app).

<luke-jr> Catholics do not believe in freedom of religion.
NickW
Newbie
*
Offline Offline

Activity: 27


View Profile
July 23, 2011, 11:10:22 PM
 #291

I'm running this fine (so far) with nginx + php-fpm as the webserver.

One thing to note to get LP working is to set fastcgi_read_timeout in the nginx virtual host to something high (such as 3600) along with max_execution_time in the php.ini file.

You also need some fairly specific settings to get the php parameters such as path_info passed along correctly. This is what I have.

Code:
location ~ ^(.+\.php)(.*)$ {
  root    /web/root/here;

  fastcgi_index index.php;
  fastcgi_split_path_info ^(.+\.php)(.*)$;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  fastcgi_param PATH_INFO  $fastcgi_path_info;
  fastcgi_param PATH_TRANSLATED $document_root$fastcgi_path_info;
  fastcgi_read_timeout 3600;
  fastcgi_pass    127.0.0.1:9000;
  include fastcgi_params;

}



Can you tell me why this stuff is required and why I would want to implement it over Apache (what I'm currently using to run the proxy app).

It's just the way Nginx works. It doesn't handle PHP directly like how Apache does, PHP is in an external process (fastcgi) instead and you have to tell Nginx how to pass the relevant parameters and forward the request to this fastcgi backend.

It's just an alternative web server and some people like myself prefer it over Apache. Primarily it is a high performance lightweight static file server and can easily handle 30000+ requests per second on static files, and 10000+ concurrent downloads, while using very little RAM (tens of megabytes) and works extremely well in these high load environments. It is not a fully featured beast like Apache, which is why you have to forward requests for things such as PHP to another backend process.

Why implement it over Apache? If it's just you using it then no reason at all. However I have set it up on one of my web servers which runs Nginx anyway. Also, it could be advantageous to run it if you have many (hundreds or even thousands) of miners connected due to the memory usage compared to Apache.

Check out some benchmarks for it Smiley.

1DRCGzEkMhhzieEkDg3e8kUKt1mVM5uqNs
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 24, 2011, 04:12:19 AM
 #292

I've created a health monitoring script and a new database table for the Proxy so that you can see your worker "clock speed, mem speed, fan speed, card temperature" from the Proxy Dashboard page.  I have the schema change (just an added table) and the health reporting script all finished. Just need to modify the dashboard to support a couple of new columns for that data.

I'll post the changes required when I'm done - hopefully people enjoy the improvements.

cdhowie - if you like my work with this can you include it in the github repo or should I branch the project? I really love the work you did with the proxy server, it's made my life a lot easier. Smiley


So I finished the health monitor script and integrated the status display with the main admin dashboard. Here's a screenshot showing the added columns: "Temp, Fan Speed, Clock(core) Speed, Mem Speed". Anyone interested in this being added to the github code? I have two tables on the database "worker_health_current" and "worker_health_archive" - the current table tracks the most recent entry for each GPU, and the archive table tracks all entries so that (when I code this later) I can display graphs for trending of GPU temperature over time.

Let me know what you all think.


<luke-jr> Catholics do not believe in freedom of religion.
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
July 25, 2011, 09:49:54 AM
 #293

Looks good, please fork and consider making it optional (via the conf file).


Is this being updated still?

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
exahash
Sr. Member
****
Offline Offline

Activity: 276



View Profile
July 25, 2011, 04:11:44 PM
 #294

So I finished the health monitor script and integrated the status display with the main admin dashboard. Here's a screenshot showing the added columns: "Temp, Fan Speed, Clock(core) Speed, Mem Speed". Anyone interested in this being added to the github code? I have two tables on the database "worker_health_current" and "worker_health_archive" - the current table tracks the most recent entry for each GPU, and the archive table tracks all entries so that (when I code this later) I can display graphs for trending of GPU temperature over time.

Let me know what you all think.

@Shotgun - I am VERY interested!  Please post a patch or a fork and let me know where to find it. 
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 25, 2011, 05:56:52 PM
 #295

So I finished the health monitor script and integrated the status display with the main admin dashboard. Here's a screenshot showing the added columns: "Temp, Fan Speed, Clock(core) Speed, Mem Speed". Anyone interested in this being added to the github code? I have two tables on the database "worker_health_current" and "worker_health_archive" - the current table tracks the most recent entry for each GPU, and the archive table tracks all entries so that (when I code this later) I can display graphs for trending of GPU temperature over time.

Let me know what you all think.

@Shotgun - I am VERY interested!  Please post a patch or a fork and let me know where to find it. 

Forked the project and added my files. The bin/worker_health.sh file has the instructions and pertinent info.
https://github.com/btc-shotgun/Bitcoin-mining-proxy

If you like it, feel free to send me some coins Wink
1BUV1p5Yr3xEtSGbixLSospmK6B8NCdqiW

<luke-jr> Catholics do not believe in freedom of religion.
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 26, 2011, 03:54:39 AM
 #296

I added support for graphing temperature. There's a link for each worker on the "temp" column which will show you the last 24 hours of temperatures for each card. You can get the changes via a git clone or update. Later on I'll add the ability to specify a date range for graphing as well as support for graphing the other health stats.

Here's an example. It uses DyGraphs.


<luke-jr> Catholics do not believe in freedom of religion.
dlasher
Sr. Member
****
Offline Offline

Activity: 468



View Profile WWW
July 26, 2011, 08:48:29 PM
 #297

I've created a health monitoring script and a new database table for the Proxy so that you can see your worker "clock speed, mem speed, fan speed, card temperature" from the Proxy Dashboard page.  I have the schema change (just an added table) and the health reporting script all finished. Just need to modify the dashboard to support a couple of new columns for that data.

I love what you're doing... that's awesome.. for my 2cents, I'd like to see the temps/load/etc separated from the "worker" name, since multiGPU boxes will generally run (1) worker per box... more than that gets unmanagable quickly.

Either that or you poll all the GPU's/CPU/etc in the box, and log the average..

dlasher
Sr. Member
****
Offline Offline

Activity: 468



View Profile WWW
July 26, 2011, 08:49:23 PM
 #298

I added support for graphing temperature. There's a link for each worker on the "temp" column which will show you the last 24 hours of temperatures for each card....

How tough would it be to add graphs for both "worker" and "aggregate" mining data, like Mhash, shares, rejects, etc?

shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 26, 2011, 10:57:40 PM
 #299

I added support for graphing temperature. There's a link for each worker on the "temp" column which will show you the last 24 hours of temperatures for each card....

How tough would it be to add graphs for both "worker" and "aggregate" mining data, like Mhash, shares, rejects, etc?



I'll have to look at the schema tables that store that data. Once I get a query to support gathering the data over time then pushing it to the graphing system is trivial. I'll update my findings soon.

<luke-jr> Catholics do not believe in freedom of religion.
shotgun
Member
**
Offline Offline

Activity: 98



View Profile
July 26, 2011, 11:25:45 PM
 #300


I love what you're doing... that's awesome.. for my 2cents, I'd like to see the temps/load/etc separated from the "worker" name, since multiGPU boxes will generally run (1) worker per box... more than that gets unmanagable quickly.

Either that or you poll all the GPU's/CPU/etc in the box, and log the average..

Glad you enjoy it Smiley

In regard to separating the workers.. well, that's not really possible with the current schema. Also I'm not sure it makes sense to store multiple GPU device temperature stats for one worker. It would require a 1:N schema which would require a completely different approach for the monitoring script, schema, and reporting aspects as well as graphing. You can get around this by making dummy workers in the proxy and having the health monitor script upload it's data for those dummy workers. So your single-worker-per-box can connect to the proxy as usual, but you'd have like "box-name-GPU0", "box-name-GPU1" etc and look at the graphs that way. But I can't store multiple GPU temperatures for a single worker.

As to administrative hassle of multiple workers per box, my boxes are all running 3-4 cards each with one instance of phoenix per GPU. Having an aggregate worker for all devices on a box makes it very difficult to see how an individual GPU is performing, and Phoenix can't use multiple devices per process instance. The only miner I know that will use multiple devices is Diablo, which runs about 5-10% slower on my GPUs than Phoenix.

Managing multiple devices per box is actually very simple, I have a script per instance that gets started via the LXDE autostart script:

Code:
> cat /etc/xdg/lxsession/LXDE/autostart
@lxpanel --profile LXDE
@pcmanfm --desktop --profile LXDE
@sleep 3
@/usr/bin/screen -dmS proxy0 /home/user/p0.sh
@/usr/bin/screen -dmS proxy1 /home/user/p1.sh
@/usr/bin/screen -dmS proxy2 /home/user/p2.sh
@/usr/bin/screen -dmS monitor /bin/ping 172.16.0.1
@lxterminal

Each of the /home/user/p<GPU_DEVICE_NUMBER>.sh script contains the following code to start up fans, core/mem speed, and connect to flexible proxy. If I reboot the box the miners get started automatically without any input from me. If my UPS dies the BIOS on each server is set to restore power to "LAST STATE", so they will turn on as soon as there is power available, then they boot LinuxCoin into init-level-5 and run the autostart script which in turn runs the miner via this code:

Code:
#!/bin/sh
#6870 OC Settings
CLOCK_CPU=950
CLOCK_MEM=1050
FANSPEED=100
WORKSIZE=128
AGGRESSION=11
DEVICE="0"
LOG="/home/user/log.$DEVICE.out"
URL="<flexible_proxy_ip_address>"
PORT="80"
USER="box#gpu#" #Worker naming schema: server_name<server-number>c<GPU#> like: ultra0gpu2 for "sun ultra40 workstation number zero, gpu device two"
PASS="password"

export DISPLAY=:0.$DEVICE
aticonfig --od-enable
echo "running: aticonfig --od-setclocks=$CLOCK_CPU,$CLOCK_MEM --adapter=$DEVICE"
aticonfig --od-setclocks=$CLOCK_CPU,$CLOCK_MEM --adapter=$DEVICE
aticonfig --odgt --adapter=$DEVICE
aticonfig --pplib-cmd "get fanspeed 0"
aticonfig --odgc --adapter=$DEVICE
aticonfig --pplib-cmd "set fanspeed 0 $FANSPEED"
aticonfig --pplib-cmd "get fanspeed 0"
echo "running: ./phoenix.py --url=http://${USER}:${PASS}@$URL:$PORT -k phatk VECTORS BFI_INT FASTLOOP=false WORKSIZE=$WORKSIZE AGGRESSION=$AGGRESSION DEVICE=$DEVICE"
python ./phoenix.py --logtotext=$LOG --url=http://${USER}:${PASS}@$URL:$PORT -k phatk VECTORS BFI_INT FASTLOOP=false WORKSIZE=$WORKSIZE AGGRESSION=$AGGRESSION DEVICE=$DEVICE


<luke-jr> Catholics do not believe in freedom of religion.
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 [15] 16 17 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!