Posted by superfedya on May 31, 2012 at 4:19pm
I need some help with PHP-FPM optimization for 4 core (8 with HT) server with 16GB.
Because some times I get "The connection was reset" with the heavy scripts (for example modules updates check).
My php-fpm.conf:
;;;;;;;;;;;;;;;;;;;;;
; FPM Configuration ;
;;;;;;;;;;;;;;;;;;;;;
; All relative paths in this configuration file are relative to PHP's install
; prefix (/usr). This prefix can be dynamicaly changed by using the
; '-p' argument from the command line.
; Include one or more files. If glob(3) exists, it is used to include a bunch of
; files from a glob(3) pattern. This directive can be used everywhere in the
; file.
; Relative path can also be used. They will be prefixed by:
; - the global prefix if it's been set (-p arguement)
; - /usr otherwise
;include=/etc/php-fpm.d/.conf
;;;;;;;;;;;;;;;;;;
; Global Options ;
;;;;;;;;;;;;;;;;;;
[global]
; Pid file
; Note: the default prefix is /var
; Default Value: none
pid = /var/run/php-fpm/php-fpm.pid
; Error log file
; Note: the default prefix is /var
; Default Value: log/php-fpm.log
error_log = /var/log/php-fpm/error.log
; Log level
; Possible Values: alert, error, warning, notice, debug
; Default Value: notice
log_level = debug
; If this number of child processes exit with SIGSEGV or SIGBUS within the time
; interval set by emergency_restart_interval then FPM will restart. A value
; of '0' means 'Off'.
; Default Value: 0
emergency_restart_threshold = 10
; Interval of time used by emergency_restart_interval to determine when
; a graceful restart will be initiated. This can be useful to work around
; accidental corruptions in an accelerator's shared memory.
; Available Units: s(econds), m(inutes), h(ours), or d(ays)
; Default Unit: seconds
; Default Value: 0
emergency_restart_interval = 1m
; Time limit for child processes to wait for a reaction on signals from master.
; Available units: s(econds), m(inutes), h(ours), or d(ays)
; Default Unit: seconds
; Default Value: 0
process_control_timeout = 5
; Send FPM to background. Set to 'no' to keep FPM in foreground for debugging.
; Default Value: yes
daemonize = yes
;;;;;;;;;;;;;;;;;;;;
; Pool Definitions ;
;;;;;;;;;;;;;;;;;;;;
; Multiple pools of child processes may be started with different listening
; ports and different management options. The name of the pool will be
; used in logs and stats. There is no limitation on the number of pools which
; FPM can handle. Your system will tell you anyway :)
; To configure the pools it is recommended to have one .conf file per
; pool in the following directory:
include=/etc/php-fpm.d/pool.d/.confI have 3 pools like this:
; Start a new pool named 'www'.
; the variable $pool can we used in any directive and will be replaced by the
; pool name ('www' here)
[www0]
; Per pool prefix
; It only applies on the following directives:
; - 'slowlog'
; - 'listen' (unixsocket)
; - 'chroot'
; - 'chdir'
; - 'php_values'
; - 'php_admin_values'
; When not set, the global prefix (or /usr) applies instead.
; Note: This directive can also be relative to the global prefix.
; Default Value: none
;prefix = /path/to/pools/$pool
; The address on which to accept FastCGI requests.
; Valid syntaxes are:
; 'ip.add.re.ss:port' - to listen on a TCP socket to a specific address on
; a specific port;
; 'port' - to listen on a TCP socket to all addresses on a
; specific port;
; '/path/to/unix/socket' - to listen on a unix socket.
; Note: This value is mandatory.
listen = /var/run/php-fpm.sock
; Set listen(2) backlog. A value of '-1' means unlimited.
; Default Value: 128 (-1 on FreeBSD and OpenBSD)
;listen.backlog = -1
; List of ipv4 addresses of FastCGI clients which are allowed to connect.
; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original
; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address
; must be separated by a comma. If this value is left blank, connections will be
; accepted from any ip address.
; Default Value: any
;listen.allowed_clients = 127.0.0.1
; Set permissions for unix socket, if one is used. In Linux, read/write
; permissions must be set in order to allow connections from a web server. Many
; BSD-derived systems allow connections regardless of permissions.
; Default Values: user and group are set as the running user
; mode is set to 0666
listen.owner = www-data
listen.group = www-data
listen.mode = 0600
; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user's group
; will be used.
user = www-data
group = www-data
; Choose how the process manager will control the number of child processes.
; Possible Values:
; static - a fixed number (pm.max_children) of child processes;
; dynamic - the number of child processes are set dynamically based on the
; following directives:
; pm.max_children - the maximum number of children that can
; be alive at the same time.
; pm.start_servers - the number of children created on startup.
; pm.min_spare_servers - the minimum number of children in 'idle'
; state (waiting to process). If the number
; of 'idle' processes is less than this
; number then some children will be created.
; pm.max_spare_servers - the maximum number of children in 'idle'
; state (waiting to process). If the number
; of 'idle' processes is greater than this
; number then some children will be killed.
; Note: This value is mandatory.
pm = dynamic
; The number of child processes to be created when pm is set to 'static' and the
; maximum number of child processes to be created when pm is set to 'dynamic'.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI.
; Note: Used when pm is set to either 'static' or 'dynamic'
; Note: This value is mandatory.
pm.max_children = 10
; The number of child processes created on startup.
; Note: Used only when pm is set to 'dynamic'
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 3
; The desired minimum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.min_spare_servers = 3
; The desired maximum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.max_spare_servers = 5
; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
pm.max_requests = 500
; The URI to view the FPM status page. If this value is not set, no URI will be
; recognized as a status page. By default, the status page shows the following
; information:
; accepted conn - the number of request accepted by the pool;
; pool - the name of the pool;
; process manager - static or dynamic;
; idle processes - the number of idle processes;
; active processes - the number of active processes;
; total processes - the number of idle + active processes.
; max children reached - number of times, the process limit has been reached,
; when pm tries to start more children (works only for
; pm 'dynamic')
; The values of 'idle processes', 'active processes' and 'total processes' are
; updated each second. The value of 'accepted conn' is updated in real time.
; Example output:
; accepted conn: 12073
; pool: www
; process manager: static
; idle processes: 35
; active processes: 65
; total processes: 100
; max children reached: 1
; By default the status page output is formatted as text/plain. Passing either
; 'html' or 'json' as a query string will return the corresponding output
; syntax. Example:
; http://www.foo.bar/status
; http://www.foo.bar/status?json
; http://www.foo.bar/status?html
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
pm.status_path = /fpm-status-zwei
; The ping URI to call the monitoring page of FPM. If this value is not set, no
; URI will be recognized as a ping page. This could be used to test from outside
; that FPM is alive and responding, or to
; - create a graph of FPM availability (rrd or such);
; - remove a server from a group if it is not responding (load balancing);
; - trigger alerts for the operating team (24/7).
; Note: The value must start with a leading slash (/). The value can be
; anything, but it may not be a good idea to use the .php extension or it
; may conflict with a real PHP file.
; Default Value: not set
ping.path = /ping-zwei
; This directive may be used to customize the response of a ping request. The
; response is formatted as text/plain with a 200 response code.
; Default Value: pong
;ping.response = pong
; The timeout for serving a single request after which the worker process will
; be killed. This option should be used when the 'max_execution_time' ini option
; does not stop script execution for some reason. A value of '0' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_terminate_timeout = 120s
; The timeout for serving a single request after which a PHP backtrace will be
; dumped to the 'slowlog' file. A value of '0s' means 'off'.
; Available units: s(econds)(default), m(inutes), h(ours), or d(ays)
; Default Value: 0
;request_slowlog_timeout = 5s
; The log file for slow requests
; Default Value: not set
; Note: slowlog is mandatory if request_slowlog_timeout is set
slowlog = /var/log/$pool.log.slow
; Set open file descriptor rlimit.
; Default Value: system defined value
rlimit_files = 4096
; Set max core size rlimit.
; Possible Values: 'unlimited' or an integer greater or equal to 0
; Default Value: system defined value
rlimit_core = 0
; Chroot to this directory at the start. This value must be defined as an
; absolute path. When this value is not set, chroot is not used.
; Note: you can prefix with '$prefix' to chroot to the pool prefix or one
; of its subdirectories. If the pool prefix is not set, the global prefix
; will be used instead.
; Note: chrooting is a great security feature and should be used whenever
; possible. However, all PHP paths will be relative to the chroot
; (error_log, sessions.save_path, ...).
; Default Value: not set
;chroot =
; Chdir to this directory at the start.
; Note: relative path can be used.
; Default Value: current directory or / when chroot
chdir = /var/www
; Redirect worker stdout and stderr into main error log. If not set, stdout and
; stderr will be redirected to /dev/null according to FastCGI specs.
; Note: on highloaded environement, this can cause some delay in the page
; process time (several ms).
; Default Value: no
catch_workers_output = yes
; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from
; the current environment.
; Default Value: clean env
;env[HOSTNAME] = $HOSTNAME
;env[PATH] = /usr/local/bin:/usr/bin:/bin
;env[TMP] = /tmp
;env[TMPDIR] = /tmp
;env[TEMP] = /tmp
; Additional php.ini defines, specific to this pool of workers. These settings
; overwrite the values previously defined in the php.ini. The directives are the
; same as the PHP SAPI:
; php_value/php_flag - you can set classic ini defines which can
; be overwritten from PHP call 'ini_set'.
; php_admin_value/php_admin_flag - these directives won't be overwritten by
; PHP call 'ini_set'
; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no.
; Defining 'extension' will load the corresponding shared extension from
; extension_dir. Defining 'disable_functions' or 'disable_classes' will not
; overwrite previously defined php.ini values, but will append the new value
; instead.
; Note: path INI options can be relative and will be expanded with the prefix
; (pool, global or /usr)
; Default Value: nothing is defined by default except the values in php.ini and
; specified at startup with the -d argument
;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com
;php_flag[display_errors] = off
;php_admin_value[error_log] = /var/log/fpm-php.www.log
;php_admin_flag[log_errors] = on
;php_admin_value[memory_limit] = 32MNginx related PHP config:
#-- mode: nginx; mode: flyspell-prog; ispell-local-dictionary: "american" --
### fastcgi configuration for serving private files.
## 1. Parameters.
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx/$nginx_version;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
## PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param REDIRECT_STATUS 200;
## 2. Nginx FCGI specific directives.
fastcgi_buffers 256 4k;
fastcgi_intercept_errors on;
## Allow 4 hrs - pass timeout responsibility to upstream.
fastcgi_read_timeout 14400;
fastcgi_index index.php;There is something that I can make better or add something?
Thanks
Comments
Here's an algorithm
Change from UNIX sockets to TCP.
Increase
max_requests.Increase the number of workers if needed.
Enable the
debuglog_level, stress the site and check the log.There's no turnkey solution, I'm afraid.
On a second approach
You should experiment with the timeouts also.
Thanks Done, no
Thanks
I raise the timeouts (of php-fpm workers) and it resolve my 502 problem. But I always have:
"The connection was reset, The connection to the server was reset while the page was loading." on the heavy scripts when page must be refreshed (like for Updated status page, the script check the updates and refresh the page after, for me it check the updates and show "The connection was reset" after, and I must refresh the page manuals).
ps. error like this can be related to fastcgi_buffers 256 4k?
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8388608 bytes)
No
the
fastcgi_buffersdirective is for tuning the buffering at the Nginx level. You have other problems besides buffer tuning.Increase the memory limit. Check how much it is, go to the reports section and check the PHP config as printed by
phpinfo()atadmin/reports/status/php.Fatal error: Allowed memory size of 134217728 bytes exhausted
The above is usually an issue with Suhosin (often part of php-fpm), the real enforcer is suhosin security patch applied to PHP5.
Set allowed memory size ( say 128M):
suhosin.memory_limit = 128
Simply, allowed memory for single PHP process is not enough. You need to increase it.
Highly likely that Suhosin extension is enabled on the server which prevent scripts from setting the memory limit beyond what it started with or as per the defined by suhosin.memory_limit value.
If Suhosin is enabled on the server, set the value of suhosin.memory_limit to 256MB and then increase the memory_limit value for the scripts.
--
Linux: Web Developer
Peter Bowey Computer Solutions
Australia: GMT+9:30
(¯`·..·[ Peter ]·..·´¯)
The connection was reset
The "connection was reset" issue is related to nginx line:
reset_timedout_connection on;
I comment it and all problems are gone.
I add this line to php.ini:
suhosin.memory_limit = 256M but no effect. (in phpinfo I see 256MB).
Thanks
You should read what that directive does
right here.
The problem is in the site. It takes a lot of time to reply. Do you have keepalive enabled for FCGI?
You're right. It seems to be
You're right. It seems to be a Dos Deflate script problem, this script ignore ignore.list and ban it's own server's IP when AdvAgg try to communicate...
Also I apply your grean Nginx config in the right way and now everything is much better.
Thanks
I'm getting a similiar issue
I'm getting a similiar issue to this which might in fact be the same. After adding a bunch of new roles and setting permissions I get an error going to the /admin/user/permissions page.
"Fatal error: Allowed memory size of 142606336 bytes exhausted (tried to allocate 4718653 bytes) in /data/all/002/pressflow-6.24.1-prod/includes/theme.inc on line 1429 array(4) { ["type"]=> int(1) ["message"]=> string(82) "Allowed memory size of 142606336 bytes exhausted (tried to allocate 4718653 bytes)" ["file"]=> string(54) "/data/all/002/pressflow-6.24.1-prod/includes/theme.inc" ["line"]=> int(1429) } n/a"
The set up is on BOA 2.0.3 so I'm hesitant making custom changes that will be overwritten on next BOA upgrade. I'm afraid this is beyond my skill level.
Is my issue the same/similar to this? If so, would you be so kind as to point me in the right direction (using laymans terms! ;))
Too much data
My guess is that you have too much data to be displayed.
For example, you have a view in a block and instead of showing just 4 nodes, the query is processing tens of thousands of nodes, and fetching the data from the database exhausts the buffers.
There could be other causes, such as too many modules enabled, but I would start with query result sets first.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.