Concerned about performances / Benchmark on 3 different hosting.

Posted by jvieille on January 12, 2009 at 10:45pm

I explsed this thread in the main Drupal forum, but this mone might be more appropriate

I am very concerned by Drupal performance as it directly affects the productivity of people contributing to the content
(It is a Content Management System, isn't it?)

Beside "productivity", ie the time it takes to view, create or updtate content elements, reactivity is a key factor to keep contributor acting. A lazy system will discourge them to collaborate.

So I am mainly - only - interested in improving performance for logged in users. Drupal basic caching does not help.

This introduction is for explaining my furious search for decent performances, I mean getting a screen poping for normal contributions - < 1" - , and fast enough for more demanding administrative tasks.

Having started on a cheap shared hosting in US (I live in France), I was said that I should go for dedicated of VPS.

So I tried successively

RPS, dedicated-like server (Real Private server, sharing iSCSCI drives),
Fully dedicated server,
"high performance" shared hosting.
I currently run the same web site on 3 different environments:
(BH) "cheap" shared hosting : www.bluehost.com (US)
Real Private server with shared iSCSI drives : OVH RPS1 http://www.ovh.com/fr/particulier/produits/rps1.xml (FR, 200 Km) Intel Celeron 32/64 1,2GHz, RAM 512Ko (no numbers for this trial, quite poor performances because of the slow iSCSI drive)
(OV) Dedicated hoting: OVH 2XL http://www.kimsufi.com/ (FR, 200 Km) Intel E2180 64bits 2*2GHz, RAM 2GO
(SH) "High performance" shared hosting : www.simplehelix.com (US)

All are Xcache enabled

The numbers bellow are given by Yslow and Devel

The columns are the numbers for a given page:
(1) node/add/stoty
(2) admin/build/block/list
(3) plain text, no views.
(4) admin/build/modules/list

The rows are the numbers for the different environements as defined above
BH = Bluehost
OV = OVH
SH = SimpleHelix

Yslow with/ devel (Seconds)
____1____2___3____4
BH 03.0 03.5 02.0 10.8
OV 02.5 04.3 02.2 07.5
SH 02.4 03.5 01.8 06.8

Yslow with devel (Seconds)
___1____2___3____4
BH 07.0 04.0 04.1 111*
OV 05.0 04.2 03.7 054*
SH 04.7 05.2 02.9 042*
* Firefox breaks

Number of queries
___1___2___3___4
BH 365 304 283 5831
OV 609 560 522 6171
SH 524 528 510 5329

Total Queries ms
____1__2___3___4
BH 140 121 101 2533
OV 260 250 231 2463
SH 164 180 139 1923

Page ms
___1____2____3____4
BH 0900 1200 0700 11019
OV 1085 1175 0992 06163
SH 0735 0900 0424 04872

Slowest query ms
___1___2___3__4
BH 31.0 4.3 4.3 69
OV 02.7 2.6 2.7 49
SH 04.3 4.3 4.3 69

Fastest query ms
___1____2___3____4
BH 0,10 0,10 0,10 0,10
OV 0,16 0,13 0,15 0,16
SH 0,06 0,05 0,05 0,05

Pricing/month
BH 8$
OV 40$
SH 20$

Conclusions:
- whatever power is involved, Drupal is SLOOOOOOOOOOOOOOOW
- Dedicated hosting is cumbersome, for little, or no improvement

1) How is it possible that Drupal processes so many queries (> always more than 500, somethimes 10) - Is this normal?
2) Bluehost always reports less queries, sometimes 50% less. Where could this come from???
3) Are these numbers normal for Drupal, so I am simply exceptionnally demanding? (see introduction)
4) Should I try to increase the power, going to a 100$/month hosting for 43GHz /8Go to get acceptable performances?
5) I installed Cache Router, but never got it work, nor being able to to get any support. Could it improve things?
6) Any idea to improve this situation?

Thank you for sharing your opinion and giving any advice?

Comments

Hard to read

Posted by kbahey on January 13, 2009 at 4:47am

The data is very hard to read, but I will try.

In fact, I am presenting on Thursday on Troubleshooting a slow Drupal site: a case study!

Let us ignore the front end for the time being (Yslow) and focus on backend first.

A quick glance shows this:

Total Queries ms
____1__2___3___4
BH [140] 121 101 2533
OV [260] 250 231 2463
SH [164] 180 139 1923

Page ms
___1____2____3____4
BH [0900] 1200 0700 11019
OV [1085] 1175 0992 06163
SH [0735] 0900 0424 04872

If the queries only take 140 ms, but the overall page tales 900 ms, then you have a problem of code execution, not database.
When I see this, normally it is too many modules (100+), or no accelerator, or worse, both. The way I attack this is install APC. This can the time by half or even one quarter of the time! You say you have XCache enabled, which should be similar, but does it have enough memory allocated to cache all the modules you have enabled, or is it memory starved? What are the hits/misses ratio? I also copy the site on a physical server (no shared host, no VPS) and then measure things there on the bare metal and see if they are proportionally the same or significantly different.

Do you have some blocks that has a lot of code execution in it? Something like tagadelic? Try to disable some blocks and then see if the results are the same (at least node/add/story should be very fast.

Do you have a module that does fsockopen() or other operation over the network for every page load? Things like web 2.0 widgets (submit to digg, ...etc. that use the API). That can slow you down a lot.

Do you have the admin_menu module installed? Try to disable that as it has some sort that takes a lot of time in it.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Admin Menu - Physical server - XCache

Posted by jvieille on January 13, 2009 at 8:21am

1) Admin menu
This very useful add on is one of the culprit.
In the worst case (#4), the numbers change like that:

Yslow with devel ; 54 -> 30 seconds
Number of queries : 6171 -> 3504
Total queries time : 2463 -> 1113
Page time : 6163 -> 3585
Slowest query: 49->38,1ms
Fastest query: 0,16-> 0,06 ms

2) Number of modules
I have about 42, some are disabled

3) Physical sever
the second row prefixed "OV" is a physical server, as decribed. This is the one that gives the numbers above.

4)APC
I'll try it

5) XCache - This is what I get on the Xcache admin page

Caches:

-	Slots	Size	Avail	Compiling	Misses	Protected	Cached	GC
php#0	8.00 K	50.00 M	49.84 M	no	3	no	3
php#1	8.00 K	50.00 M	49.94 M	no	1	no	1
var#0	8.00 K	50.00 M	49.94 M	-	0	no	0	300
var#1	8.00 K	50.00 M	49.94 M	-	0	no	0	300

Free Blocks:

php#0 size offset	49.84 M 162.27 K

php#1 size offset	49.94 M 65.44 K

var#0 size offset	49.94 M 64.20 K

var#1 size offset	49.94 M 64.20 K

php Cached

Cache	entry	Size	SrcSize	Modify	device	inode
php#0 0	/home/controlc/www/xcache/common.php	27.86 K	1.83 K	1 Y	2050	9224222
php#0 1	/home/controlc/www/xcache/index.php	867.00 b	30.00 b	1 Y	2050	9224635
php#0 2	/home/controlc/www/xcache/xcache.php	69.34 K	4.65 K	1 Y	2050	9224638
php#1 3	/home/controlc/www/xcache/common-en.lang.php	1.23 K	50.00 b	1 Y	2050	9224219

Thank you for your vauable help!

physical server?

Posted by joshk on January 13, 2009 at 6:47pm

3) Physical sever
the second row prefixed "OV" is a physical server, as decribed. This is the one that gives the numbers above.

I can't read the page that you reference, but I strongly doubt you are actually getting a dedicated server as described for $40/mo. This may be mis-representation by the host, or maybe a mis-understanding of terms, but $40 a month is generally a price for a lower-end/oversubscribed VPS account.

http://www.chapterthree.com | http://www.outlandishjosh.com

https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com

Xcache: not showing active for Drupal / APC installed: the same

Posted by jvieille on January 13, 2009 at 2:44pm

The XCache information above seems to show nop activity at all regarding Drupal operations.

The newly installed APC seems to behave the same. Look at this:

General Cache Information

APC Version	3.0.19
PHP Version	5.2.5-pl1-gentoo
APC Host	www.controlchaingroup.fr
Server Software	Apache/2.0.59 (Unix) mod_ssl/2.0.59 OpenSSL/0.9.8g
Shared Memory	1 Segment(s) with 64.0 MBytes (mmap memory, pthread mutex locking)
Start Time	2009/01/13 15:38:54
Uptime	0 minutes
File Upload Support	1

File Cache Information

Cached Files	1 (435.4 KBytes)
Hits	1
Misses	1
Request Rate (hits, misses)	2.00 cache requests/second
Hit Rate	1.00 cache requests/second
Miss Rate	1.00 cache requests/second
Insert Rate	1.00 cache requests/second
Cache full count	0

User Cache Information

Cached Variables	0 ( 0.0 Bytes)
Hits	0
Misses	0
Request Rate (hits, misses)	0.00 cache requests/second
Hit Rate	0.00 cache requests/second
Miss Rate	0.00 cache requests/second
Insert Rate	0.00 cache requests/second
Cache full count	0

System Cache Entries

Script Filename	Hits	Size	Last accessed	Last modified	Created at	Deleted at
/home/controlc/www/apc.php	0	445828	2009/01/13 15:38:25	2009/01/13 14:18:30	2009/01/13 15:38:25

User Cache Entries

User Entry Label	Hits	Size	Last accessed	Last modified	Created at	Timeout	Deleted at
No data

What I am missing?

Thanks for your help

Have you made the

Posted by Jamie Holly on January 13, 2009 at 2:59pm

Have you made the appropriate changes to your settings.php file? Make sure you are including the new cache engine ($conf['cache_inc'] ). If your using cacherouter, make sure you have the cacherouter configuarion setup also ($conf['cacherouter'] ).

HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.

Some progress

Posted by kbahey on January 13, 2009 at 3:24pm

Well, 42 modules is not a lot. It is actually very reasonable. So it is not a case of over use of modules.

However, APC is doing nothing for Drupal. Are you configured with PHP as CGI? That can explain it. Please point us to a URL to see the output of the full phpinfo().

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Here it is

Posted by jvieille on January 13, 2009 at 3:38pm

http://www.controlchaingroup.fr/phpinfo.php
http://www.controlchaingroup.fr/xcache/
http://www.controlchaingroup.fr/apc.php

Eureka!

Posted by kbahey on January 13, 2009 at 3:44pm

Eureka!

phpinfo shows that you are running PHP as CGI. This means that you cannot use APC nor Xcache to accelerate it.

Switch to PHP as mod_php, and you will see an improvement with APC/Xcache.

Please post the info in a more readable format (a proper table, or surround it with code tag).

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

And don't enable them both

Posted by kbahey on January 13, 2009 at 3:45pm

And don't enable both APC and Xcache at the same time.

Use just one of them, measure, evaluate, then disable it and enable the other one.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Consider eAccelerator

Posted by jaydub on January 13, 2009 at 4:00pm

eAccelerator can be used with FastCGI. We've used it with no problems and performance for eAccelerator is typically benchmarked at the same speeds as APC and Xcache.

http://www.eaccelerator.net

Not really

Posted by kbahey on January 13, 2009 at 4:44pm

The issue is not that. He does not have FastCGI, just the plain old "fork-a-new-process-per-request" CGI, which no accelerator works with at all.

APC also works with FastCGI.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

How to switch from php-cgi to mod-php?

Posted by jvieille on January 13, 2009 at 4:05pm

Thanks a ot for your insight.

By the way, I am not really comfortable with al that PHP stuff.

Can you tell me how I can change that? Is it a simple setting somewhre, or does it needs to reinstall PHP, or install an additional program?
Any side effect?

Thanks

Depends

Posted by kbahey on January 13, 2009 at 4:42pm

This depends on the specifics of the Distro you are using and whether it is a self managed or managed setup.

If it is Debian or Ubuntu, you can try this guide: installing PHP and APC on GNU/linux Ubuntu Gutsy Gibbon 7.10 and Debian.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

What am I supposed to put in settings.php

Posted by jvieille on January 13, 2009 at 3:29pm

What I am supose to put in the settings.php file?
How to set ($conf['cache_inc'] ) ?

Cache Router really doesn't work for me.
I cannot get support either.

http://drupal.org/node/357837
http://drupal.org/node/358193
http://drupal.org/node/356555
http://drupal.org/node/353815

Different things

Posted by kbahey on January 13, 2009 at 3:42pm

I think intoxication was confusing APC/Xcache as an op-code cache vs. using them with cacherouter for data caching.

Since you are trying to optimize for logged in users, cacherouter does not apply.

Let us focus on getting APC caching Drupal scripts.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

In his first post he

Posted by Jamie Holly on January 13, 2009 at 3:53pm

In his first post he mentioned using XCache on all the benchmarks, but then posted a status report from XCache and it's showing that nothing is being cached. It sounds like there is some configuration (either module or server level) issue causing things not to be properly cached. That would also explain the high query count.

HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.

Not entirely true

Posted by slantview on January 14, 2009 at 9:28am

That is not entirely true that Cache Router doesn't apply. With Drupal 6 all the caches are in the DB, if your host is complaining about too many db queries, Cache Router with APC will definitely help prevent so many queries for cache* tables in the database and potentially speed up the site as well.

As noted many times, Cache Router is in development and only beta at this time. The most stable engine is memcache and behind that is APC. There will be a new release with a ton of bug fixes coming soon. (If you care to, you can download the CVS version of DRUPAL-6--1 and get a lot of the changes, but I make no promises to what condition CVS code is at during any point, I do check in broken stuff sometimes)

Slantview Media http://www.slantviewmedia.com/ | Blog http://www.slantview.com/

The error messages you are

Posted by Jamie Holly on January 13, 2009 at 3:47pm

The error messages you are posting there for DB engine are common, but cacherouter is still in it's beta phase, so errors do occur.

Also the dashboard you are referring to in those service requests is only in the dev branch, as stated on the project page under the todo list.

I have ran cacherouter successfully on a number of sites. One is using memcache, and three are using APC right now. For APC it's rather simple:

$conf['cache_inc'] = './sites/all/modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
  'default' => array(
    'engine' => 'apc',
    'server' => array(),
    'shared' => FALSE,
    'prefix' => '',
  ),
);

For page caching, you also need to enable the actual cacherouter module.

HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.

Please, please do not open

Posted by slantview on January 14, 2009 at 9:22am

jvieille:

Please, please do not open so many tickets. If you have a specific issue, open it, but read the previous open tickets first. Unfortunately this is open source software, we are not always here to serve you on your schedule.

As I wrote in the other ticket, don't use XCache. Not only is it broken in Cache Router, it's just broken in general. There have been 2 minor releases in 2 years, and frankly I have been less than impressed by it's performance.

I would highly suggest using APC. I am not intimately familiar with the configuration in non-typical setups like using FastCGI, etc. I have only run it with mod_php. If you have a question about additional setup you can post it, but you might not get the answer you are looking for.

If you don't feel that you are getting the support you need with the Drupal community, I would suggest that you either open a bounty or buy a paid subscription to a service like Acquia or hire a consultant. Performance tuning Drupal is a lot of work and depends on what you need to do with it, how much traffic you have, etc.

I can't imagine that you couldn't just ask for a recommendation in the forums for a Drupal friendly hosting company and just host with them. I built a site that is hosted with Dreamhost on cheap shared hosting and gets over 2000 unique visitors every day with no optimizations, etc on Drupal 5.

Slantview Media http://www.slantviewmedia.com/ | Blog http://www.slantview.com/

Thank you very much to

Posted by jvieille on January 13, 2009 at 4:55pm

Thank you very much to everybody
I will try to get mod-php and try all that

3 last questions

Posted by jvieille on January 13, 2009 at 9:08pm

Based on that discussion, can we say that:

1- there is no point running several optimizers at the same time. At some point I'll have to chose between eaccelerator, xcache, APC or whatever

2- My server runs php on CGI. Because of the particular "ready to use" distribution (Gentoo Release 2 by OVH) I chosed for speeding my setup, I cannot install something else nor upgrading without breaking the package. (I had much difficulties installing XCahce and APC... for nothing)

That means
* I cannot have fast-cgci: I suppose that Fast CGI would run my site faster even without any optimizer. But I cant even get that
* I cannot install mod_php (PHP as Apache extension?) which should be also faster than PHP on CGI, or even Fast CGI if I am right
* I cannot benefit of any optimizer with such configuration. My server runs in the worst conditions for Drupal

So I would have to reinstall my server with a more appropriate distribution...

3- What about PHP6?
If I can get it on my server, will it need these optimizers? Will Drupal still work?

Comments

Posted by kbahey on January 13, 2009 at 9:42pm

1- there is no point running several optimizers at the same time. At some point I'll have to chose between eaccelerator, xcache, APC or whatever

Yes.

As for which one is better, different people have different preferences, but APC is the one that is developed by the core PHP team and hence will be in sync with PHP releases.

2- My server runs php on CGI. Because of the particular "ready to use" distribution (Gentoo Release 2 by OVH) I chosed for speeding my setup, I cannot install something else nor upgrading without breaking the package. (I had much difficulties installing XCahce and APC... for nothing)

Do they have other distros? Ubuntu Server LTS 8.04.1 is very good, and you can install packages easily on it using aptitude.

That means

* I cannot have fast-cgci: I suppose that Fast CGI would run my site faster even without any optimizer. But I cant even get that

No.

FastCGI by itself will not be much faster than CGI. It is just more resource efficient in that it will not fork a new process for every page request. Instead, PHP will be running as a separate process and the request from the web server goes on a socket to that process.

The good news is that FastCGI allows you to use APC.

To keep things simple, start with mod_php5.

* I cannot install mod_php (PHP as Apache extension?) which should be also faster than PHP on CGI, or even Fast CGI if I am right

Yes, this is the fastest way for PHP to run: inside an Apache process.

* I cannot benefit of any optimizer with such configuration. My server runs in the worst conditions for Drupal

CGI does not allow accelerators. mod_php and FastCGI do allow it though.

So I would have to reinstall my server with a more appropriate distribution...

That is a good option. See my comment on Ubuntu above.

3- What about PHP6?
If I can get it on my server, will it need these optimizers? Will Drupal still work?

I would not recommend that. Not many people have tried Drupal in that way, let alone tested and debugged everything.

Stay with PHP 5.2.x for best results.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

You're

Posted by jvieille on January 14, 2009 at 7:13am

You're fantastic!
Thanks
Jean

Solved?

Posted by kbahey on January 14, 2009 at 3:40pm

So, were you able to solve it yet?

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Half

Posted by jvieille on January 14, 2009 at 9:06pm

A problem well exposed is half-solved...

Thanks
Jean

Concerned about performances / Benchmark on 3 different hosting.

So I tried successively

Comments

Hard to read

Admin Menu - Physical server - XCache

physical server?

Xcache: not showing active for Drupal / APC installed: the same

General Cache Information

File Cache Information

User Cache Information

System Cache Entries

User Cache Entries

Have you made the

Some progress

Here it is

Eureka!

And don't enable them both

Consider eAccelerator

Not really

How to switch from php-cgi to mod-php?

Depends

What am I supposed to put in settings.php

Different things

In his first post he

Not entirely true

The error messages you are

Please, please do not open

Thank you very much to

3 last questions

Comments

You're

Solved?

Half

High performance

Group organizers

New groups

Group notifications

Hot content this week