Remote logging with nginx and php-fpm

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Torenware's picture

I'm looking once again into how best to monitor a large Drupal 7 app I've deployed on AWS, using nginx and PHP-FPM (5.3.21 currently). The app uses autoscaling, and the instances often don't live very long (a day is a long time). So I need some kind of logging solution that will get my logs off the instances and into some kind of centralized store.

What are people doing for this use case?

I've noticed that more than a year after I last looked into this, PHP-FPM appears to have not committed a patch to support

error_log = syslog

as is supported for apache / mod_php. The last entry in the PHP bug system suggests that as of Dec 2013, this was not in PHP 5.5, much less anything older.

Does anyone know of:

  • Usable patches to make PHP-FPM, well, behave.
  • Other kinds of approaches to this problem.

What is the simplest solution to this problem?

Thanks much,
Rob aka Torenware

Comments

I have a good solution to

Torenware's picture

I have a good solution to this issue, for the record.

There's a good patch available up on the PHP bug tracker (Bug ##66239 Fpm won't error_log to syslog, php works fine)

I'm using CentOS, and found it easiest to build a custom RPM. But once this is applied, PHP-FPM will talk to syslog.

Logstash?

Brian Altenhofel's picture

We don't bother patching the lack of syslog support in PHP-FPM; we just forward the logs to Logstash, and Logstash ships them to Elasticsearch to be indexed. We use Kibana to view them.

I wrote up a tutorial on getting started with that sort of set up quite a while back. It's slightly outdated now (lumberjack was renamed to logstash-forwarder, and both Logstash and Elasticsearch were moving targets at the time), but it could get you going in the right direction. I also presented on it at Drupalcon Portland. http://www.vmdoh.com/blog/centralizing-logs-lumberjack-logstash-and-elas...

Logstash can also ship log messages wherever you want. I even have some critical error messages going straight to PagerDuty.

Nginx

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: