Posted by svenrissmann on December 12, 2008 at 12:08pm
Hi
I'm thinking about to port a site running on self build scripts to Drupal Version 6.
Here're some facts:
Registered Users: ~117.000
Averrage Online: ~6.500
Hits (monthly): ~2.200.000
Most used Modules:
- Forum (14.000+ threads)
- Private Messaging
- Image Gallery
- Buddylist
At the moment it all runs at a Clustered Hosting Plattform.
The Performance is very badly.
My goal is to write import scripts to Drupal and move to a Server like this:
Hardware:
- Intel Core2Duo 2x2,4Giga
- 2Gb of DDR Ram
- 1TB Mirrored SATA
- 2x1Gb Intel LAN
Software:
- Debian (in a very light version)
- Lighttpd
- PHP5 FastCGI
- MySQL (or Postgre?)
I've no experience with things like this!!!
Please try to tell me some helpful tips...
Kind regards
Uhu
Comments
Please provide description
Please provide description for "Clustered Hosting Plattform"
Few steps for optizatons:
how ofter users changes profiles settins?
top urls, top sql queries
calculate resouces needed for top queries and pages for top urls
database cluster
static pre-generated pages(hard cache)
memcache
apc, eAccelerator
Hi The
Hi
The Clustered-Hosting-Plattform is www.servage.net
I absolutly don't know how often the settings are changed per user.
The top queries I don't know, too. The top URLs can I figure out from the Statistics...
Do you realy mean I'll need a clustered Database?
Greetings
I research
I research Clustered-Hosting-Plattform(www.servage.net) - look like common shared hosting.
I said that database cluster is option which need to be reviewed. Database do two common operations read and write(simple look) - so if your server have 60% select queries and 40% and your server have not resources to handle more you need master/slave(write/read) cluster - this will allow you to optimize read and write operations separately.
devel.module - will be good point to start. Before do any optimizations it will be good for you to know what, when and why happened.
Registered User Load?
If you're saying that at any given point you have an average of 6,500 registered users online (at a time), you're very likely going to need more than a single server. At a minimum, you'll want to have a dedicated database server, with more power (particularly more RAM) than you're describing here.
If that's a per-day total, you can likely get away with one box if you have a solid caching strategy. Given the list of features you describe, I suggest Drupal 6 so you can take advantage of the block-level caching. I would also still say "get more memory" just to be on the safe side, and look into memcached to optimize all your cached page requests (as well as minimizing overhead from variable and menu cache requests for logged-in users).
Also, you should definitely use a PHP opcode compiler like APC or eAccelerator.
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Did you know that Drupal 5
Did you know that Drupal 5 faster than Drupal 6? - can you comment this?
Where do you get this idea?
What's your source?
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
eAccelerator - can not be
eAccelerator - can not be used with FastCGI
Maybe you're thinking of
Maybe you're thinking of cgi. eAccelerator works great with FastCGI and lighttpd. More stable for me than APC for some reason.
http://eaccelerator.net/wiki/InstallFromSource
yes, I thinking about cgi.
yes, I thinking about cgi. My mistake, sorry.
Hi Of course Servage is a
Hi
Of course Servage is a shared hosting Plattform!!!
But it offers me enough Disk-Space and mostly enough Traffic.
And as I've begun with ~20 Users the Performance was alright ;)
But now the performance is very bad!!!
In the evening the amount of users are online at the same time is something like ~4.800.
At the weekend the average user amount is up to 6.500 !!
Why do you mean Drupal5 is faster than 6 ???
regards
Uhu
There are hosters which
There are hosters which offer unlimited disk space and traffic (including me). But IO, CPU and maximum transfer speed limited in most cases.
Ah
If you read all of Kahled's commentary, you can see there are a lot of details to be considered.
First of which, Drupal 7 is still in unstable/pre-release state.
Second of which, Drupal 6 adds a great deal of support for more advanced caching solutions (which you want if you do high-volume anon pageviews)
Third of all, Drupal 6 adds huge boosts for bandwidth (compression, js aggregation), which decreases page sizes, which improves the user experience of speed over and above how many requests the server can chew through.
Finally, Drupal 6 also adds block-level caching, which speeds up performance for logged-in users significantly.
Druapl.org still uses drupal 5 because the project system is complex and un-ported, not for performance reasons.
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
compression can be enabled
compression can be enabled via .htaccess(mod_gzip - mod_deflate). Compression take CPU, so how will comment CPU vs bandwidth(as request count) loads? My opinion that client side cache (mod_expires - simple way) for static content decrease request count dramatically.
js aggregation - 8 lines in theme and I have this with Drupal 5
block-level caching - my themes + custom module for Drupal 5 actively use cache(memcache, APC), so it isn't problem for me.
"Drupal 6 adds a great deal of support for more advanced caching solutions" - please point me where I can read more about this.
If you're using memcache
You're patching drupal core. ;)
You're also not doing block caching of drupal blocks.
Basically, Kahled's benchmarks are for a very simple use-case which doesn't actually reflect most real-world situations. Your avg. high-performance site isn't running vanilla drupal core with 10 plain nodes on the frontpage.
For more information:
http://highervisibilitywebsites.com/drupal-6-benchmarking-and-block-cach...
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Drupal.org is on 5, but ...
Drupal.org is still using 5 not because of performance, but rather because of crucial components yet to be ported to 6.x, most importantly the project suite of modules that is the heart of the Drupal community.
We want to move drupal.org to 6.x as soon as possible, but this is dependent on the above.
As for 6.x being slower than 5.x, yes, on a raw level this is true. But there are several things in 6.x that mitigates that. For example, we have an integrated block cache, and we don't need to patch core for implementing advanced caching mechanisms such as memcache.
The bulk of slowness in real life is mostly not in core, but things like: too many contrib modules, poorly written contrib modules, large datasets, low hosting specs, ...etc.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Interesting...
This sounds interesting :) It's good that you wrote the home baked scripts as it makes things easy for pulling data from the source system.
Looking at your stats I notice your number of 'hits' looks low... I run a D5 system with only 1200-ish users of which 100 tops online at a time. With only 6500 posts at the mo (23K comments) we get almost 2m hits/month already! We use random image block + ad on each page but even without these think our hits would still be 1m+ easy.
The site I mention was (before a recent upgrade) running easily on an Athlon 64 3500+ CPU with 3GB RAM and had load average which hovered around 0.7 or so, so you'll be fine with a dual core 2.4Ghz. RAM is dirt cheap though, get at least 4Gb.
You'll probably want to use a custom theme to get the comments looking more 'forum' like.
I'm curious about why you wouldn't just stick with a LAMP stack, lighttp + Post looks to be a bit of an obscure choice. Not that there's anything wrong with obscure ;)
Hey all Thanks for
Hey all
Thanks for disputing!
First, if I believe in the posted bench it means to me that Drupal is getting more and more LAME!
I can´t believe that D5 does nearly 4x more requests in less time than D7 !?!?
And the irony of this is: "If I want to reach more 'Web 2.0' using D6-7 it goes on the back of the performance..."
About Lighty I just can say, I like the kind of minimalism he represents!
His conf-style is just so easy to use and understand.
And the performance he delivers...
http://immike.net/blog/2007/05/07/the-showdown-apache-vs-lighttpd/
http://csshyamsundar.wordpress.com/2006/10/29/apache-vs-lighttpd-a-compa...
http://www.howtoforge.com/benchmark-apache2-vs-lighttpd-images
Also, talking about eAccelerator shouldn't extrude XCache.
Of course I'll use custom theme and modules!
May be I'll take some time playing around with the Devel module in the evening...
Kind regards
Uhu
o, rly
Comments like that are a great way to become a pariah in the community. You will quickly be having conversations by yourself and not get any advice. Intelligent criticism is welcome, but this is...not intelligent.
Raw page serving performance for anonymous users is one of many aspects to total system performance. Other aspects like total page download time and performance for different users under different scenarios are equally important but more difficult to measure and therefore less likely to be compared between versions.
Drupal 6 has JavaScript aggregation that is not available in 5.x. It also has the ability to work properly behind a caching proxy server. And it has a block cache for authenticated users. These features are available as patches or modules for 5.x but are not as solid as what made it into 6.x.
Lighty is popular as a performance solution, but when it is benchmarked against Apache with MPM prefork and only the absolutely required Apache modules then Lighty and Apache perform similarly.
So, before going around saying something is "LAME" be sure that your own advice is well researched and not itself LAME.
One other comment since noone has mentioned it: make sure your custom theme takes advantage of CSS Sprites.
--
Growing Venture Solutions | Drupal Dashboard | Learn more about Drupal - buy a Drupal Book
knaddison blog | Morris Animal Foundation
not to mention patches going in
Since Khalid's benchmarks, registry queries are cached - this gave about a 6% improvement on the front page. And node_load_multiple() went in - which gives 10-30% improvement on the front page, taxonomy/term, rss feeds etc. for logged-in users. There's plenty of work to be done on Drupal 7's performance, but Khalid's benchmarks are literally that - a point from where progress can be tracked, not the final word by any means.
Hi It's ugly to say I'll
Hi
It's ugly to say I'll become a pariah!
I just said the truth...
But as I've now read here the posted bench didn't factor in the acceleration of chaching for logged in users.
And it would've been more indicated to explain this to me then posting so prig sentences!
Also saying Drupal.org is using D5 because of the performance is very fool!!
If you read the posts on Drupal.org constantly you would've figured out that the big problem porting to D6 is the Project-Module(s)!!!
Of course this was already said above.
By the way, disputing this way is highly destructive!!!
Kind regards
"Also saying Drupal.org is
"Also saying Drupal.org is using D5 because of the performance is very fool!!" - I was sure in this. But I open for discussion and facts sais that Drupal 6 have more optimizations for loged users(I verified this).
I use Drupal 5 as platform for all my projects, but I am PHP coder and patches for core isn't problem for me.