Bitcoin Forum

Bitcoin => Pools => Topic started by: a.miner on May 17, 2014, 07:21:27 PM



Title: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 17, 2014, 07:21:27 PM

P2Pool puts up a nice web page that you can view in a browser to monitor your pool performance.

However, my p2pool node are behind a firewall, and I only let ports 8333 and 9333 through that wall.  And 22, for SSH and SCP.

So, how can I monitor my p2pool node remotely under these conditions?

I would like to keep a log file on a remote machine, together with a real-time display, so if something goes wrong, I know it's time to go visit my hardware.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: squashpile on May 17, 2014, 07:52:21 PM
SSH in and run screen is how I do it.
screen -d -m -S btcp2pool ~/p2pool/run_p2pool.py --give-author 0 blah 64646464615015484
screen -x btcp2pool
Hit control-A then D to exit screen and leave p2pool running. Type screen -r to see it again. If you hit control-c when in screen it will kill P2pool.



Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 18, 2014, 04:22:46 AM
I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 19, 2014, 06:36:55 AM
SSH in and run screen is how I do it.
screen -d -m -S btcp2pool ~/p2pool/run_p2pool.py --give-author 0 blah 64646464615015484
screen -x btcp2pool
Hit control-A then D to exit screen and leave p2pool running. Type screen -r to see it again. If you hit control-c when in screen it will kill P2pool.
This sounds like a nice easy way to do things.  I think I will install screen and have a look at its man pages to figure out what the options you used are for.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 19, 2014, 06:48:32 AM
I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.
That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 19, 2014, 06:51:25 AM

Does anybody know where p2pool puts its log files and what it calls them?


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 19, 2014, 06:53:41 AM

Does anybody know where p2pool puts its log files and what it calls them?

you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 19, 2014, 06:55:48 AM
I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.
That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?

haha yeah well it's a real mashup of the extended front end (https://github.com/hardcpp/P2PoolExtendedFrontEnd) and some of my own code. It's pretty straight forward for the layout and the javascript is all from the goblin stats project.

https://github.com/goblin/p2pool-stats/


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 19, 2014, 06:36:01 PM
Does anybody know where p2pool puts its log files and what it calls them?
you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log
Aha.  Thanks.  Does it put the exact same stuff in the log file as it writes to the console?


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 19, 2014, 06:38:15 PM
I have my nodes dashboard hosted on a separate web server and can access stats via standard http port. The code locally as well as remote does a json call.
All you need to do is reference the pools URL instead of local json sources. There is a bit of into on this and I've based my front end on the alternate front end on the githib page.
Have a look at the link in my sig and see if that's what you are after.
Let me know if you need more help with it.
That looks really slick, norgan!

I had a look at the source code for your page... if there were any comments in it, I might be able to figure out what it does (my own code is at least half comments, otherwise I can't read it myself after a week or two).

Care to explain it to me?

haha yeah well it's a real mashup of the extended front end (https://github.com/hardcpp/P2PoolExtendedFrontEnd) and some of my own code. It's pretty straight forward for the layout and the javascript is all from the goblin stats project.

https://github.com/goblin/p2pool-stats/
Thanks for the links.  I'll look into it.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 19, 2014, 09:35:02 PM
Does anybody know where p2pool puts its log files and what it calls them?
you need to specify that in the start up string.

i.e. run_p2pool --logfile C:\p2pool-btc\logs\p2pool.log
Aha.  Thanks.  Does it put the exact same stuff in the log file as it writes to the console?

Yes it just dumps it's output.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 20, 2014, 10:30:54 PM
Yes it just dumps it's output.
Thanks again.


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 20, 2014, 10:35:37 PM
There are a lot of interesting files in ~/p2pool/data/bitcoin with intriguing names like graph_db and log and stats and such like.

I could just grab these files with sftp and extract the information I want once I have them here.

Where can I find documentation for the file formats?


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 20, 2014, 11:14:28 PM
There are a lot of interesting files in ~/p2pool/data/bitcoin with intriguing names like graph_db and log and stats and such like.

I could just grab these files with sftp and extract the information I want once I have them here.

Where can I find documentation for the file formats?


that's its own databases for tracking shares etc after you restart a node. it'd be much easier to just pull the json files.

Code:
Web interface

Lots of data and useful tools are available at http://127.0.0.1:9332/something:
 /static/ - Lots of information from shares to graphs to payouts.
 /rate
 /users
 /fee
 /current_payouts
 /patron_sendmany - Gives sendmany outputs for fair donations to P2Pool
 /global_stats
 /local_stats
 /peer_addresses
 /payout_addr
 /recent_blocks
 /uptime
 /web/log - Some different stats collected over the last day


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: a.miner on May 21, 2014, 12:12:02 AM
that's its own databases for tracking shares etc after you restart a node. it'd be much easier to just pull the json files.
Well, I looked at what those return, and I would still have to write code to parse out the contents of the JSON objects.

So I might as well parse out the data files directly.

Besides, the JSON interface doesn't seem to give me any way to plot a graph of local hash rate.  I assume the data for that is in the file graph_db.

The file format must be documented somewhere...


Title: Re: p2pool -- How to monitor remotely from the console?
Post by: norgan on May 21, 2014, 02:00:02 AM
Code:
d3.json('http://pool.norgzpool.net.au:9332/global_stats', function(global_stats) {
d3.select('#pool_rate').text(d3.format('.3s')(global_stats.pool_hash_rate) + 'H/s');
d3.select('#pool_stale').text(d3.format('.2p')(global_stats.pool_stale_prop));
d3.select('#difficulty').text(d3.format('.3r')(global_stats.min_difficulty));

var time_to_block = local_stats.attempts_to_block/global_stats.pool_hash_rate;
d3.select('#time_to_block').text(d3.format('.3r')(time_to_block/3600) + " hours");

d3.select('#expected_payout_amount').text(d3.format('.3r')(local/global_stats.pool_hash_rate*local_stats.block_value*(1-local_stats.donation_proportion)));
 
 
  /// ============================================
var data = [
['Local speed', local],['Local dead speed', local_dead], ['Global speed', global_stats.pool_hash_rate]
  ];

                                                          if (plot2) {plot2.destroy();}
                                                  $("#targetPlot2").remove();
                                                  $("#SpeedChart").append("<div id='targetPlot2'></div>");
  plot2 = jQuery.jqplot ('targetPlot2', [data],
{
  seriesDefaults: {
// Make this a pie chart.
renderer: jQuery.jqplot.PieRenderer,
rendererOptions: {
  // Put data labels on the pie slices.
  // By default, labels show the percentage of the slice.
  showDataLabels: true
}
  },
  legend: { show:true, location: 'e' }
}
  );
 
});

/// ============================================
var data = [
['Good', local_stats.shares.total - (local_stats.shares.orphan + local_stats.shares.dead)],['Dead', local_stats.shares.dead], ['Orphaned', local_stats.shares.orphan]
  ];
 
                          if (plot1) {plot1.destroy();}
                          $("#targetPlot1").remove();
                                  $("#ShareChart").append("<div id='targetPlot1'></div>");
                          plot1 = jQuery.jqplot ('targetPlot1', [data],
{
  seriesDefaults: {
// Make this a pie chart.
renderer: jQuery.jqplot.PieRenderer,
rendererOptions: {
  // Put data labels on the pie slices.
  // By default, labels show the percentage of the slice.
  showDataLabels: true
}
  },
  legend: { show:true, location: 'e' }
}
  );
 

 
 
 
    });
/// pool speed grapg
plot_later(d3.select("#main-local"), "H/s", "H", [
                    {"url": "http://pool.norgzpool.net.au:9332/web/graph_data/local_hash_rate/last_" + period, "color": "#0000FF", "label": "Total"},
                    {"url": "http://pool.norgzpool.net.au:9332/web/graph_data/local_dead_hash_rate/last_" + period, "color": "#FF0000", "label": "Dead"}
                ],1000,300);

}

I think that's the bit that does the graph.