p2pool -- How to monitor remotely from the console?

[a.miner]

P2Pool puts up a nice web page that you can view in a browser to monitor your pool performance.

However, my p2pool node are behind a firewall, and I only let ports 8333 and 9333 through that wall.  And 22, for SSH and SCP.

So, how can I monitor my p2pool node remotely under these conditions?

I would like to keep a log file on a remote machine, together with a real-time display, so if something goes wrong, I know it's time to go visit my hardware.

[1714491344]

1714491344

[1714491344]

1714491344

[1714491344]

1714491344

[squashpile]

SSH in and run screen is how I do it.
screen -d -m -S btcp2pool ~/p2pool/run_p2pool.py --give-author 0 blah 64646464615015484
screen -x btcp2pool
Hit control-A then D to exit screen and leave p2pool running. Type screen -r to see it again. If you hit control-c when in screen it will kill P2pool.

[norgan]

I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.

[a.miner]

SSH in and run screen is how I do it.
screen -d -m -S btcp2pool ~/p2pool/run_p2pool.py --give-author 0 blah 64646464615015484
screen -x btcp2pool
Hit control-A then D to exit screen and leave p2pool running. Type screen -r to see it again. If you hit control-c when in screen it will kill P2pool.

This sounds like a nice easy way to do things.  I think I will install screen and have a look at its man pages to figure out what the options you used are for.

[a.miner]

I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.

That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?

[a.miner]

Does anybody know where p2pool puts its log files and what it calls them?

[norgan]

Does anybody know where p2pool puts its log files and what it calls them?

you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log

[norgan]

I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.

That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?

haha yeah well it's a real mashup of the extended front end (https://github.com/hardcpp/P2PoolExtendedFrontEnd) and some of my own code. It's pretty straight forward for the layout and the javascript is all from the goblin stats project.

https://github.com/goblin/p2pool-stats/

[a.miner]

Does anybody know where p2pool puts its log files and what it calls them?

you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log

Aha.  Thanks.  Does it put the exact same stuff in the log file as it writes to the console?

[a.miner]

I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.

That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?

haha yeah well it's a real mashup of the extended front end (https://github.com/hardcpp/P2PoolExtendedFrontEnd) and some of my own code. It's pretty straight forward for the layout and the javascript is all from the goblin stats project.

https://github.com/goblin/p2pool-stats/

Thanks for the links.  I'll look into it.

[norgan]

Does anybody know where p2pool puts its log files and what it calls them?

you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log

Aha.  Thanks.  Does it put the exact same stuff in the log file as it writes to the console?

Yes it just dumps it's output.

[a.miner]

Yes it just dumps it's output.

Thanks again.

[a.miner]

There are a lot of interesting files in ~/p2pool/data/bitcoin with intriguing names like graph_db and log and stats and such like.

I could just grab these files with sftp and extract the information I want once I have them here.

Where can I find documentation for the file formats?

[norgan]

There are a lot of interesting files in ~/p2pool/data/bitcoin with intriguing names like graph_db and log and stats and such like.

I could just grab these files with sftp and extract the information I want once I have them here.

Where can I find documentation for the file formats?

that's its own databases for tracking shares etc after you restart a node. it'd be much easier to just pull the json files.

Code:

Web interface

Lots of data and useful tools are available at http://127.0.0.1:9332/something: 
 /static/ - Lots of information from shares to graphs to payouts. 
 /rate 
 /users 
 /fee 
 /current_payouts 
 /patron_sendmany - Gives sendmany outputs for fair donations to P2Pool 
 /global_stats 
 /local_stats 
 /peer_addresses 
 /payout_addr 
 /recent_blocks 
 /uptime 
 /web/log - Some different stats collected over the last day

[a.miner]

that's its own databases for tracking shares etc after you restart a node. it'd be much easier to just pull the json files.

Well, I looked at what those return, and I would still have to write code to parse out the contents of the JSON objects.

So I might as well parse out the data files directly.

Besides, the JSON interface doesn't seem to give me any way to plot a graph of local hash rate.  I assume the data for that is in the file graph_db.

The file format must be documented somewhere...

[norgan]

Code:

d3.json('http://pool.norgzpool.net.au:9332/global_stats', function(global_stats) {
						d3.select('#pool_rate').text(d3.format('.3s')(global_stats.pool_hash_rate) + 'H/s');
						d3.select('#pool_stale').text(d3.format('.2p')(global_stats.pool_stale_prop));
						d3.select('#difficulty').text(d3.format('.3r')(global_stats.min_difficulty));
						
						var time_to_block = local_stats.attempts_to_block/global_stats.pool_hash_rate;
						d3.select('#time_to_block').text(d3.format('.3r')(time_to_block/3600) + " hours");
						
						d3.select('#expected_payout_amount').text(d3.format('.3r')(local/global_stats.pool_hash_rate*local_stats.block_value*(1-local_stats.donation_proportion)));
										  
										  
						  /// ============================================
								var data = [
								['Local speed', local],['Local dead speed', local_dead], ['Global speed', global_stats.pool_hash_rate]
							  ];
								
                                                          if (plot2) {plot2.destroy();}
	                                                  $("#targetPlot2").remove();
	                                                  $("#SpeedChart").append("<div id='targetPlot2'></div>");
							  plot2 = jQuery.jqplot ('targetPlot2', [data],
								{ 
								  seriesDefaults: {
									// Make this a pie chart.
									renderer: jQuery.jqplot.PieRenderer, 
									rendererOptions: {
									  // Put data labels on the pie slices.
									  // By default, labels show the percentage of the slice.
									  showDataLabels: true
									}
								  }, 
								  legend: { show:true, location: 'e' }
								}
							  );
				  
					});
					
					/// ============================================
					var data = [
					['Good', local_stats.shares.total - (local_stats.shares.orphan + local_stats.shares.dead)],['Dead', local_stats.shares.dead], ['Orphaned', local_stats.shares.orphan]
				  ];
				  				
	                          if (plot1) {plot1.destroy();}
	                          $("#targetPlot1").remove();
                                  $("#ShareChart").append("<div id='targetPlot1'></div>");
	                          plot1 = jQuery.jqplot ('targetPlot1', [data],
					{ 
					  seriesDefaults: {
						// Make this a pie chart.
						renderer: jQuery.jqplot.PieRenderer, 
						rendererOptions: {
						  // Put data labels on the pie slices.
						  // By default, labels show the percentage of the slice.
						  showDataLabels: true
						}
					  }, 
					  legend: { show:true, location: 'e' }
					}
				  );
				  

				  
				  
				  
			    });
				/// pool speed grapg
				plot_later(d3.select("#main-local"), "H/s", "H", [
                    {"url": "http://pool.norgzpool.net.au:9332/web/graph_data/local_hash_rate/last_" + period, "color": "#0000FF", "label": "Total"},
                    {"url": "http://pool.norgzpool.net.au:9332/web/graph_data/local_dead_hash_rate/last_" + period, "color": "#FF0000", "label": "Dead"}
                ],1000,300);

		}

I think that's the bit that does the graph.