httpload: testing tool for Nginx served sites

in advance of drupalcon London. I'm going to propose two BoFs on Nginx+Drupal. In the meantime, the examples on the manpage or on the README, can give you an idea of the range.

httpload -parallel 10 -seconds 10 test_urls.txt

16191 fetches, 10 max parallel, 696213 bytes, in 10.0079 seconds
43 mean bytes/connection
1617.83 fetches/sec, 69566.6 bytes/sec
msecs/connect: 0.171786 mean, 1.208 max, 0.032 min
msecs/first-response: 0.453629 mean, 1.836 max, 0.233 min
HTTP response codes:
code 200 -- 16191

This is requesting the in memory 1x1 GIF of the Empty GIF module.

Here's also a message on the "nginx ssl slow" thread of July on the Nginx english ML. Quoting.

Hello!

On Tue, Jul 12, 2011 at 01:58:38AM -0600, Mark Maunder wrote:

> Igor I did SSL benchmarks with 10 worker processes on a very fast
> multicore machine with multiple ssl_session_cache configs to try and
> disprove this post. My results were also slow:
>
> On a 4 core Xeon E5410 using:
>
> ab -c 50 -n 5000
>
> with 64 bit ubuntu 10.10 and kernel 2.6.35 I get:
>
> For a 43 byte transparent gif image on regular HTTP:
>
> Requests per second: 11703.19 [#/sec] (mean)
>
> Same file via HTTPS with various ssl_session_cache params set:
>
> ssl_session_cache shared:SSL:10m;
> Requests per second: 180.13 [#/sec] (mean)
>
> ssl_session_cache builtin:1000 shared:SSL:10m;
> Requests per second: 183.53 [#/sec] (mean)
>
> ssl_session_cache builtin:1000;
> Requests per second: 182.63 [#/sec] (mean)
>
> No ssl_session_cache:
> Requests per second: 184.67 [#/sec] (mean)
>
>
> I'm assuming the session cache has no effect since each 'ab' request
> is a new session. But I thought I'd try it anyway.

Yes, ab won't reuse sessions.

> 180 per second for a machine this fast compared to 11,703 per second
> on regular HTTP seems like a big difference. 'ab' was run on the
> local machine (it takes very little CPU) so there was zero network
> latency.

I've did some tests on 2 x X5355 (4 cores each, 8 cores total)
server, it should be comparable to yours E5410. I've used
empty_gif to test as well.

First of all, ab wasn't even able to saturate *regular* http while
eating 100% cpu (i.e. the whole cpu core, it just can't eat more
as it's single thread/single process). That is, it only shows
something about 13k r/s, while with 5 ab processes nginx is
actually able to handle 50k r/s over loopback.

So about "ab takes very little CPU": no it doesn't, it's awfully
CPU bound. If you see low numbers in top - make sure top shows
%CPU for a core, not for all cores in system, or you'll see small
number like 12.5% (100%/8) for a "whole core loaded, can't eat
more" case. Under linux it should be switchable by pressing 'I'
(Irix mode vs. Solaris mode).

> Let me know if there's anything I should try to speed it up.

The same as the above applies to https as well. While using 1024
bit RSA key and DHE-RSA-AES256-SHA cipher, with 8 ab processes
from another host I see 1800 r/s while system being 100% busy.

Other results include:

1024 bit key, DHE-RSA-AES256-SHA - 1800 r/s
2048 bit key, DHE-RSA-AES256-SHA - 1050 r/s
4096 bit key, DHE-RSA-AES256-SHA - 270 r/s

With ECDHE ciphers (and patch already mentioned in this thread):

1024 bit key, ECDHE-RSA-AES256-SHA - 2740 r/s
2048 bit key, ECDHE-RSA-AES256-SHA - 1340 r/s
4096 bit key, ECDHE-RSA-AES256-SHA - 285 r/s

This is with fairy trivial nginx config:

worker_processes 8;

error_log /path/to/error_log;

events {
worker_connections 10240;
}

http {
include mime.types;
default_type application/octet-stream;

access_log /path/to/access_log;

server {
listen 8443;
server_name localhost;

ssl on;
ssl_certificate cert.pem;
ssl_certificate_key cert.key;

location / {
empty_gif;
}
}
}

Obviously using other cipher suites will produce much different
results.

Just to compare, here are results from stunnel on the same
machine:

1024 bit key, DHE-RSA-AES256-SHA - 1990 r/s
2048 bit key, DHE-RSA-AES256-SHA - 1220 r/s
4096 bit key, DHE-RSA-AES256-SHA - 280 r/s

1024 bit key, ECDHE-RSA-AES256-SHA - 2285 r/s
2048 bit key, ECDHE-RSA-AES256-SHA - 1223 r/s
4096 bit key, ECDHE-RSA-AES256-SHA - 285 r/s

It looks a bit faster with DHE ciphers, and the reason is not
using SSL_OP_SINGLE_DH_USE option by default. Setting "options
SINGLE_DH_USE" in config results in the following DHE performance
of stunnel:

1024 bit key, DHE-RSA-AES256-SHA - 1480 r/s
2048 bit key, DHE-RSA-AES256-SHA - 953 r/s
4096 bit key, DHE-RSA-AES256-SHA - 260 r/s

Maxim Dounin

More to come. There's an accompanying script for generating 1kB/1MB sized files I'll look into it. I haven't tried ab since I've been apache clean for 18+ months. I don't have the apache2-utils package install. I get by with thttp-util for all my needs. But with siege I was unable to reach those figures.

PS: Slightly OT. Nginx 1.1.0 supports ECDH based ciphers.