Drupal Performance on IIS7 vs Apache

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Canadaka's picture

I've been doing a bunch of performance testing for one of my Drupal sites Trailforks.com and decided to compare IIS7 vs Apache 2.4. I host several Drupal websites on Windows Web Server 2008 with IIS7 and love it. I've read more and more articles that show IIS outperforms Apache and other web servers in many situations. But I wanted to make sure I wasn't loosing possible performance by hosting my site with IIS compared to Apache.

I created a local copy of my site and database on my desktop which is an Intel Core i7 920 OC@ 3.3Ghz with 6GB ram and an SSD. I installed the following software: IIS7, MySQL 5.5, PHP 5.3.13 & Apache 2.4. For IIS I also installed Microsofts URL Rewrite plugin. Besides that the IIS7 configuration is default. PHP settings are default besides upping the memory_limit to 256MB. For my first tests I also left Apache with its default config.

I used Microsoft's cool testing tool called WCAT. My tests were to see how many pages each server could load in a 20 second period, with a 5 second warmup. The following is my wcat .ubr settings

[Configuration]
WarmupTime 5s
NumClientMachines 1
MaxRecvBuffer 64K
CooldownTime 5s
ThinkTime 0s
NumClientThreads 80
Duration 20s
Comment Test Server

[Script]
SET KeepAlive = TRUE

NEW TRANSACTION
classId = 1
weight = 100
server = "localhost"
NEW REQUEST HTTP
URL = "http://trailforks.local/"

For the Drupal side, I tested my Trailforks site which contains over 100 modules & a custom theme running the newest release of Drupal 7. I created a blank page with a blank template using a custom module, so its the most basic drupal page. I disabled drupals cache and the Boost module cache. These are the initial results.

IIS - 192
Apache - 53

I was rather surprised with the results, IIS blew Apache out of the water. The next test was to enable boost and test serving basically just a simple html file.

IIS - 207,677
Apache - 84,436

Again IIS more than doubles Apache's numbers.

I also tried with IIS's built-in "Output Caching" and the result was prety close to Boost at 219,374. Now another advantage to IIS is that the default configuation of IIS7 and of PHP 5.3 when using the PHP Installer is pretty good, I wouldn't change much. Its been several years since I had used Apache, but I knew that the default Apache config is not great. So I did a bit of looking around and added a couple things to the httpd.conf along with disabling all but the minimum Apache modules needed to run Drupal.

KeepAlive On
MaxKeepAliveRequests 150
KeepAliveTimeout 5

ThreadsPerChild 250
MaxRequestsPerChild 0

This increased the Apache number only slightly from 53 to 62.
I also wanted to test each platforms most popular PHP Opcode cache extension, Wincache & APC. I installed the newest version for each and gave both lots of memory (same amount for both).

IIS - 683
Apache - 544

So once again IIS with Wincache beats out Apache with APC. Also nice to see that the opcode more than triples the performance!

Another quick test was to enable a "web garden" on the IIS application pool for the site. Changed the max worker process to 10. this increased the IIS number even further to 936.

Doing these tests has defiantly reaffirmed my choice to use Windows & IIS for hosting my sites. Not only does it outperform Apache in these specific tests but I find it MUCH easier to manage with the excellent UI's and tools. Ever since Microsoft moved the HTTP.sys into the Windows kernal and rewrote IIS from 6 to 7, its been a solid competitive option. With Windows you also don't have to worry about tweaking TCP like you have to on most Linus distros for optimal web hosting performance. Microsoft has also invested a lot into PHP & FastCGI on Windows over the past several years and PHP on Windows/IIS is just as good as LAMP now. One of my favourite things about IIS is "application pools" and how each site is isolated from everything else, so if it crashes, or is running slow, it doesn't effect the others. This also helps for security and application pools can be configured to recycle when it detects problems. Another huge bonus is most config changes you make with the IIS Admin GUI or by hand editing a web.config file take effect instantly! No restarting Apache every time you change a setting.

I know doing any kind of benchmarks is never perfect and I want to do more tests. I'm planning on running more tests with a higher concurrent to see if that changes the comparative results at all. Additionaly if anyone has any suggestions on how Apache might be better tweaked please let me know!

Has anyone else tried comparing benchmarks of Drupal 7 on IIS & Apache? If so what were your results?

Easy WCAT setup I created with simple .BAT files to run creating the server & client for testing: http://www.trevormay.ca/wp-content/uploads/2012/05/wcat_setup.zip

Comments

Great post!

alexpilotti's picture

Thanks for this great post!

The benchmarks I did so far were also leading to similar results. As you correctly pointed out, the IIS kernel level HTTP.sys is one of the key advantages of IIS over Apache in terms of performance.

There's IMHO an additional key benchmark test for Drupal on WIndows:

Drupal with vanilla PHP 5.3 vs Drupal with a custom PGO (Profile Guided Optimization) PHP 5.3 build.

Here's a webcast we recently did on FastCGI and PGO. The PGO part is towards the end.

I'd also consider load testing against a real user site navigation case (anonymous and not) beside the tests against the home page.

Finally, I have in my TODO list to do benchmarks for Drupal + MySQL vs Drupal + SQL Server

Alessandro Pilotti
MVP ASP.Net / IIS
Windows Azure Insider

Nice Comparison

mikeytown2's picture

The big question is how does it compare to a server running linux with nginx.

I was having the same

brianV's picture

I was having the same thoughts... or even IIS vs. nginx / php-fpm on cygwin.

Brian Vuyk
Senior Developer, PINGV Creative
bv@pingv.com | (315) 849-9733 | Skype: brianvuyk

this is true, but running

Canadaka's picture

this is true, but running benchmarks with the same hardware setup is not something I have the resources to do. I could install linux in a virtualbox, but that wouldn't be the same. I guess I could dual boot, which distro is the most popular for web server these days? CentOS?

I would wager that Windows + IIS will still beat out Apache on *nix. IIS is faster, Windows TCP & http.sys are faster. Although I think I heard that TCP is pre-tuned on CentOS.

This PHP PGO stuff looks interesting, defiantly going to try that out.

What were the HW specs then?

joshk's picture

running benchmarks with the same hardware setup is not something I have the resources to do.

So you didn't test these with equivalent hardware? Did you have APC enabled for both? If so, that reduces the value of the numbers significantly.

The real question here isn't "IIS vs Apache vs Nginx". The key question is the question is how do you get the best PHP runtime performance. If you want to serve static caches, you can do that with technologies meant for that purpose.

I know MS has put a fair bit of work into optimizing their PHP support, and you've been testing that by trying to load an un-cached page. But it does beg the question as to the specs and configuration of the two different systems.

;) If you read my first post

Canadaka's picture

;) If you read my first post I list the hardware setup and yes the IIS vs Apache tests above were all run on the same hardware, but under Windows.

If you read I also did tests with and without opcode cache, which is APC for Apache and Wincache for IIS. APC doesn't work with FastCGI. Wincache is the equivant php opcode cache developed by Microsoft, its a open source project.

Well PHP performance specific to Drupal is what I'm really wanting to test, but static performance is till important, especially if you use Boost on a drupal site. I was shocked at how much IIS beat Apache at serving these static files.

I may re-run all the tests all within VirtualBox's with the same setup, so then I can compare Debian or CentOS with apache & NGinx.

Sorry

joshk's picture

I didn't parse your opening paragraph very well.

This isn't terribly surprising. Nobody would recommend Apache under windows for a production setup that I know of.

I'm installing Debian in a

Canadaka's picture

I'm installing Debian in a https://www.virtualbox.org/ now, but I'm not that knowledgeable anymore about tuning linux for best web server performance. If someone wants to setup a Debian or CentOS install with tweaked Apache & PHP, then send me the virtual disk image (.vdi), I will use that for testing.

I'm going to setup 1 Virtualbox with Windows Web Server 2008 SP1 and one with Debian 6.0.5.

Well I've done some more

Canadaka's picture

Well I've done some more tests running Apache on linux in a Virtualbox. These results are the opposite from my opening post and I'm not sure why.

Could Apache really be that much faster on Linux than Windows?

I used https://www.virtualbox.org/ to create 3 virtual machines, all with the same settings and 2GB of RAM.

1 - Windows Web Server 2008
2 - CentOS 6.2
3 - Debian (pre-tuned web server http://vdi-server.com/)

1 & 2 have pretty much default setups of IIS/Apache & PHP 5.3. All 3 setups are basic PHP, no APC or memcache.

I decided to first test a simple PHP page on all\

<?php
phpinfo
();
phpinfo(INFO_MODULES);
?>

Using AB command "ab -n 1000 -c 1 http://localhost/test.php" Unlike my first post I'm comparing 'requests per second' because AB runs the test differently than WCAT.

Windows 2008 - 160
CentOS - 266
Debian - 1231

Apache on linux seems to be smoking IIS now... especially the pre-tuned CentOS. I also ran the same AB command from a VM against IIS and Apache on the host Windows 7.

Windows/Apache - 137
Windows/IIS - 235

So IIS still beats Apache hosted on Windows with a simple phpinfo test, but the results from Debian are shocking and rather confusing... makes me think something is wrong.

I did run some basic tests of the blank drupal7 site as well.

Windows 2008 - 13.5
Debian - 100

I setup the DEVEL module on these blank Drupal7 sites. On the Windows 2008 VM an authenticated user loads the home page in 68ms. In the the Debian VM an authenticated user loads the home page in 148ms. So I'm confused why DEVEL reports that the IIS/windows site is faster than the Apache one, but the AB benchmark says the opposite. In the browser to the human eye the IIS page loads noticeable faster.

An additional note, it would be interesting to also test the performance of Drupal on Windows Web Server 2008 running Server Core (no GUI, command line only). As both the CentOS and Debian VM's I used were command line only.

These guys ran some great benchmarks of IIS vs all others, and their results put IIS on top http://www.webperformance.com/load-testing/blog/2011/11/what-is-the-fast...

Did you run apache bench from

ben.bunk's picture

Did you run apache bench from the same machine for all of the tests? Say on the machine hosting the VM's and not from one of the VM's.

Also, what affect does up'ing the concurrency in apache bench do to your numbers?

I tried both, running the AB

Canadaka's picture

I tried both, running the AB tests on the VM being tested, or from another VM. It didn't really change the results at all.

removed invalid results

ahhh I think I answered my

Canadaka's picture

ahhh I think I answered my own question and it seems all my Apache results are bad and is the reason they are higher. Its not serving http 200 pages.

http://www.trevormay.ca/stuff/apache_ab_output.PNG

shows "non-2xx responses: 1000" which seems to mean that 100% of the requests were not properly served, maybe error 404 or 503? The same output for the AB test against IIS doesn't contain this line. Or am I reading this wrong?

Accessing the site i'm testing from any other machine on my network works, so i'm not sure why AB would not be working.

OK I knew something was up, I

Canadaka's picture

OK I knew something was up, I confirmed that the apache AB tests weren't actualy rendering any PHP but returning 404 pages. Something weird with the DNS, change the test to use the IP address and it works now.

So here are the REAL results. Disregard my last 3 posts.

[100 Concurrency] Drupal 7

Debian (VM) - 6.23
Windows 2008 (VM) - 13.17

[100 Concurrency] phpinfo()

Debian\Apache (VM) - 101
Windows 2008\IIS (VM) - 191


[1 Concurrency] Drupal 7

Debian\Apache (VM) - 6.13
Windows 2008\IIS (VM) - 13.12

[1 Concurrency] phpinfo()

Debian\Apache (VM) - 152
Windows 2008\IIS (VM) - 133

So this is more of what I expected, IIS clearly in the lead, even against the tweaked Debian setup.

I'm still not convinced

dazweeja's picture

I'm still not convinced you're comparing Apache on Linux to IIS on Windows. This is all on a Windows 2008 host right? Are you sure there are no differences in speed in VirtualBox between the same OS for host and guest and mismatched host/guests? I'd be surprised if there are no optimisations in VirtualBox for when the host and guest are the same/similar OSes.

I really think you'd have to run Linux from a separate partition on the same hard drive as Windows 2008 to draw any meaningful conclusions.

I've got to say I do appreciate all the effort you've gone to though!

I don't know about the

Canadaka's picture

I don't know about the technical workings of VirtualBox and what you say is possible I guess. But I'm fairly certain that the speed of Windows 2008 running in VM is the same as Linux running in VM. Both have the same virtualbox settings and resources. VirtualBox is also not natively a Windows app, its original for linux. So if your conserned about a host OS bias, one could be considered about a linux bias. But i don't think either exist.

It would take a lot more time to setup 2 new partitions on my desktop and install Win2008 and debian. But I just ordered a new Dell rackserver, when it arrives I can run some tests and just reformat it after.

Really looking forward to

obrienmd's picture

Really looking forward to your results! I'd love to see nginx and php-fpm vs. IIS on the same hardware.

Currently I'm experiencing a

jonh's picture

Currently I'm experiencing a performance issue with Drupal on IIS7, but it's not related to how many pages it can serve in a minute or anything like that. The issue is related to how quickly the pages load after they have not been accessed for a period of time, like 15 minutes. What can I do to speed that up? I've seen it take as long as 43 seconds.

AppDomain

alexpilotti's picture

Change your web site's AppDomain settings in order not to shutdown during idle periods.

Best,

Alessandro

Re: AppDomain

jonh's picture

Thanks. Are you refering to the setting in the answer here: http://stackoverflow.com/questions/838318/how-to-keep-asp-net-assemblies...

Thanks!
Jon

Idle Timeout

alexpilotti's picture

Hi Jon,

yes, you need to set the AppDomain's "Idle Timeout" property to 0. here's some documentation about how to set it.

Best,

Alessandro

Re: Idle Timeout

jonh's picture

Got it. Thank you very much! Another thing I noticed that was causing my issue was MS SQL 2008 has its databases set to auto close by default.