Posted by Slovak on August 11, 2011 at 2:49pm
Is there a recommended tool/process/configuration to monitor traffic multiple sites running on nginx? I've looked at Munin's nginx plugins, but couldn't find anything that would fit the bill.
Example: Multiple sites running on a VPS with nginx. One of the sites is suddenly more popular and/or starts receiving higher traffic over time, perhaps necessitating separate VPS setup. Is there a way to parse logs or somehow monitor the traffic split amongst multiple sites?
Comments
Well
There's the stub status module that are the basis of the Munin plugins. I don't think that it's hard to script something around
stub-statusand have it make the deployment of a new instance if needed.php-fpm also has a status page, but I guess that's more useful for load balancing the FCGI backends.
There's also a third party HTTP acconting module. This thread should be of interest to you.
Note that I haven't tried the above module.
Well
There's the stub status module that is the basis of the Munin plugins. I don't think that it's hard to script something around
stub-statusand have it make the deployment of a new instance if needed.php-fpm also has a status page, but I guess that's more useful for load balancing the FCGI backends.
There's also a third party HTTP accounting module. This thread should be of interest to you.
Note that I haven't tried the above module.
I'd say most people use
I'd say most people use Munin. I use collectd, mostly chosen for its smaller memory footprint. collectd comes with an nginx plugin http://collectd.org/wiki/index.php/Plugin:nginx that stores and graphs number of requests, and also connections-reading/writing/waiting/active.
For php-fpm I use the collectd curl-json module http://collectd.org/wiki/index.php/Plugin:cURL-JSON. That stores and graphs the number of php-fpm children in use over time. This is my /etc/collectd/collectd.conf config for that:
<Plugin curl_json><URL "http://localhost:80/fpm-status?json">
Instance "php5-fpm"
<Key "active processes">
Type "gauge"
</Key>
</URL>
</Plugin>
That assumes an nginx server config like this:
# Listen on loopback so collectd can collect stats
server {
listen 127.0.0.1:80;
location /stats {
stub_status on;
access_log off;
}
location = /fpm-status {
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /fpm-status;
fastcgi_pass php;
access_log off;
}
}
and /etc/php5/fpm/pools/www.conf including this:
pm.status_path = /fpm-statusFor multiple sites, I assume you want each stub_status actually in the server config instead of one global one for the machine.
I don't have it set up, but I
I don't have it set up, but I assume multisite setup would be something like:
server {
listen 80;
server_name mysite.com;
location /stats {
stub_status on;
access_log off;
deny all;
allow 127.0.0.1;
}
...
}
and in collectd.conf something like:
<Plugin nginx>Instance "mysite.com"
URL "http://mysite.com/stats?auto"
</Plugin>
SCRIPT_FILENAME not needed
There's no need to define the
SCRIPT_FILENAMEif you havefastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_namein yourfastcgi_paramsfile. Here's my config for both status and ping.using the geo:
geo $dont_show_fpm_status {default 1;
127.0.0.1 0;
192.168.1.0/24 0;
}
And in your vhost config:
## The status page is at /fpm-status. Only local access is
## allowed. Non authorized access returns a 404 through the error_page
## directive.
location = /fpm-status {
if ($dont_show_fpm_status) {
return 404;
}
fastcgi_pass phpcgi;
}
## The ping page is at /ping and returns the string configured at the php-fpm level.
location = /ping {
if ($dont_show_fpm_status) {
return 404;
}
fastcgi_pass phpcgi;
}
More wood on the fire
Came across this tool that can get different machine logs (over SSH) into a central location.
So using this and a carefully chosen logging you can, even just using
wc -land a an elapsed time value get an idea of the traffic to the site. without making any Nginx stub_status calls.Hi Perusio,A free Amazon
Hi Perusio,
A free Amazon micro instance would be ideal to use as the central location, if only just to test this out. Cool!
While I'm here...I've only just started with using Drupal running on
NingxNginx (doh) and FastCGI with php-fpm. I want to say thanks for all the work and resources you've contributed towards making this an easier process, especially with the Github projects and this group. A big thank you to the other group organizers as well. I've seen projects maintained on Gitgub and Drupal by Omega8cc. Just yesterday I got smoothify's upload progress module running. A quick and big thank you to everyone for your support and contributions that make it easier for newbies like me.