Hello,
Maybe next week I will setup a dedicated server.
Server hardware:
Dual core ATOM 330
2GB RAM
Swap: 2.2GB
OS: CentOS 5.3
Drupal:
-Use MySQL DB
-Use boost
(some of my website use statistic, some not, but all use boost)
-Lots of anonymous user, with high read & low write
For this serer, what I want is:
-highest request/sec
-lowest RAM & CPU usage
Below is what will I do, but because I dont have experience in nginx, so I hope there will be suggestion (Which one has duplicated function and can be ignored, or not enough something...)
-Any guide will be greatly appreciated
nginx:
-Use nginx cache
In PHP:
-Use PHP-FPM
(This is FastCGI Process Manager. But I still not clear about this because it was today I heard about this)
-Use eAccelerator
Something I have no idea:
-Use memcache/memcached?
(I dunno what memcache/memcached for? for MySQL or for PHP? or for both?..)
Thanks in advance
Comments
2GB RAM Swap: 2.2GB You
You don't need so much swap. While on a desktop, it is normal to swap from time to time, a web server (any server) should not swap at all (actually, it can, but just a little).
The only advantage in a lot of swap is that your server will not die if you misconfigure its RAM usage (but will become very slow). I am not sure this is an advantage actually.
Boost supports statistics collection, even when Drupal is not involved in the page creation.
Perfect scenario for enjoying boost's boost
<plug> try http://github.com/yhager/nginx_drupal for an easy start with nginx.</plug>
I am not sure this is needed.
There's a lot of buzz about php-fpm, but it is still in alpha mode (dubbed 'experimental'), and I am reluctant on using a patched PHP on production. I simply install PHP and run php-cgi.
My personal preference is APC, but I don't have a strong reasoning against eAccelerator.
memcache is a RAM based cache that Drupal uses to save the results of heavy computations or searches. By default this cache is in the database, but caching using memcache is faster.
Using memcache does not conform to your low RAM requirement - so I recommend to start without it, and move to memcache if you see performance issues that can be improved with a faster cache.
Thanks. that helps a lot The
Thanks. that helps a lot
The swap space is from suggestion in RHEL. There is many formula for swap space
-some suggest Swap = RAM *2 or *1 or *0.5
-some suggest Swap = RAM + 2GB
I dont know which one to follow, so I just choose amount that I think safe enough
That's nice and dandy, but
That's nice and dandy, but you need to understand what swap space is used for. It is used as an extension for RAM (when your RAM is out), and is 1000 times slower.
Guaranteed, when your server starts swapping, your users will wait. a lot.
The recommendation for RAM*2 is from the times we paid a lot to get 212 MB RAM (15 years or so) - some things have changed since then, but this recommendation is still copied over.
Hello Here is a list of
Hello
Here is a list of software installed
PostSQL 8.4
Memcache
Nginx
Varnish
Boost
Pressflow
It is especially very well configure the software for high performance
nginx: -Use nginx
Nginx does have built in caching and can function somewhat like Varnish as a reverse proxy, however no one on these boards uses that functionality, and I don't recommend it. Boost or Varnish are the preferred methods of static page caching.
Unlike Apache which can run the PHP language interpreter right inside the Apache process, nginx (and Apache-worker) requires that a PHP interpreter process be running independantly. nginx then passes the PHP requests to the PHP process using http and cgi and the PHP process passes the result back. The PHP project develops a "php-cgi" package to make this possible.
What php-cgi does not come with is a way to manage php-cgi processes and start them with various options like user and number of children. There's three commonly used methods to control php-cgi processes: run php-cgi directly, spawn-fcgi or php-fpm. Spawn-fcgi is a package in the (ubuntu/debian) repositories and adds a convenient init script for starting and stopping php-cgi processes and changing the runtime user. (in older versions of debian/ubuntu it's part of the lightttp pacakge) Php-fpm requires that you recompile php. The official php devs have recognized the need for this functionality and php-fpm has been officially moved to the php development VCS and will be integrated at some point, but it isn't today. I'm using spawn-fcgi myself.
-Use eAccelerator
PHP is an interpreted language, it is not pre-compiled into binary machine code before run time. Those .php scripts exist in plain text. That means that each time you run a PHP script it gets interpreted by the php interpreter before it can be run as machine code. Since the scripts rarely change, people use "code" or "opcode" caches which compile the PHP script into machine code and then cache it. Makes things much faster. The popular code caches are APC, eAccelerator and Xcache. You can use any, but only APC is in the repositories, so it's very easy to setup. The other two must be compiled each time you update your php version. You can google speed comparisons between the three, but they're all fairly close. Most people here use APC for simplicity, as do the Mercury/Pantheon images and so do I.
-Use memcache/memcached?
Another type of cache that Drupal uses is a "data" cache or "user" cache. The Drupal system holds certain temporary metadata to speed things up. This is data about menus, forms, search terms, etc. Drupal caches this metadata in tables called cache_form, cache_menu, etc. (and also entire pages if you're not using boost or varnish for that purpose). It's fine if this info gets wiped out because Drupal can just rebuild it from the actual data. By default, all these caches are stored as tables in your database. However, the cache tables get accessed several times every page (check out the query log using the devel module and you'll see many cache_set and cache_get and update_cache queries). Drupal 6 has a pluggable cache architecture and allows you to offload these "data" caches onto something other than (or in addition to) the database, and in some cases increase speed and scalability.
Memcached is a simple server for caching data and works well a Drupal data cache. You need to install and run the memcached server from the repositories and also download the memcache (or cacherouter) Drupal module. Then you put the appropriate lines in your settings.php to use the alternate caching mechanism for data caching. APC can also be used as a "data/user" cache in addition to using it as a "code" cache (which leads to some confusion). With APC you must use the cacherouter module.
Mercury/Pantheon (and I) use memcached for this purpose. I do so because of some weirdness with the APC/cacherouter setup which annoyed me and my impression that the memcache Drupal module is more actively maintained than the cacherouter module. (Even though I'd like the extra features of cacherouter.) Mercury/Pantheon probably does it because memcached can scale to multiple memcached servers, which APC cannot.
thanks,.. but is that a
thanks,.. but:
Is it a duplication when I use boost with varnish?
(Varnish stores data in virtual memory, while boost generate static files)
If Im going to use APC, what is advantage of using Cache Router instead of install APC through command?
(By the way, I not sure how Cache Router works... Cache Router will install APC itself.... or Cache Router will just manage the installed APC)
1. Is it a duplication when I
Yes, it's redundant. Both are accomplishing the same goal, i.e. caching full, static pages (generally). They just do it differently. I can't think of any reason to use both (though some edge case may exist) and it would add unnecessary configuration issues and complexity.
APC is a PHP feature that is not specific to Drupal. It can typically be installed through a distro repository (e.g. debian apt repository) or through PHP's own feature download system, i.e. PECL.
In order to tell Drupal to use APC or memcached as a data cache, you need to download a module that implements the alternate caching system instead of Drupal's default "cache in database" behavior. That module could be memcache module or cacherouter module (and possibly others).
Note that the alternate cache module is put into use by the line in your settings.php and not through the usual module activation. "Activating" the module through the UI only makes statistics available in the memcache module and does nothing as far as I know in the cacherouter module since the statistics in cacherouter have been disabled for a long time. It's still a good idea to activate the modules in the UI if only to get update notices when the module is updated.)
There's a difference in what
There's a difference in what Varnish and Boost do and how it affects performance. While Boost speeds up anonymously viewed pages by precompiling and storing them locally instead of going through the whole Drupal bootstrap, Varnish not only achieves the same goal but also caches requests of static ressources, such as images, CSS and JS files and keeps them away from the webserver. Varnish (or Squid) are in general faster serving their cached ressources than a webserver, which has to do some kind of bootstrapping itself for every incoming request to decide if, what and how a ressource may be served.
I guess by "webserver" you
I guess by "webserver" you mean "Apache". I think that with nginx the difference between boost & varnish should be minor, if at all distinguishable.
Are there any benchmarks on that?
I take the liberty of putting
I take the liberty of putting a schema, which seems to me ideal.
Eaccelerator is to recommend by report APC
Boost is to complicate with it to configurer, but has her effectiveness. There is also a Drupal module interesting " Parallel "
To earn some more performance, put discs SSD in hiding place and reading, and hard disks mechanical for writings.
Do you know distribution linux CAOS ? OS very high performance server !. http://www.caoslinux.org/
Sorry , my english is bad . The schema does not pass well
I'm incredibly curious as to
I'm incredibly curious as to what "berry of stocking" is supposed to be.
The Boise Drupal Guy!
reverse proxy php apache nginx
This may seem counterintuitive, but I have found success running nginx as the main webs server to server all static content and then reverse proxying all php pages to apache. Apache and php do very well together - you can also specify the number of apache processes that are spawned to keep memory in check. They also scale quite well. I have found that it also spares the anguish of configuring and/or worrying about php-fpm or php-cgi. And if I recall correctly, you don't have to futz around with clean urls.
Let me know if you'd like me to post a sample configuration file.
That sounds like basically
That sounds like basically the same way as varnish is setup. You're basically just replacing varnish with nginx.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
yes please
Yes. I would love to see you config file.
Do you use imagecache? How you server cached images by imagecache and then redirect to apache if can't find in imagecache directoy? I am confused because if I try to serve as images (.jpg) file then it won't redirect to apache-imagecache if it can't find and if I try to serve through php to take care of imagecache, then I am serving very little static images. Any thoughts?
ImageCache is easy
ImageCache is easy.
server {
listen 80;
server_name example.com;
root /var/www/example/html;
index index.php;
access_log /var/log/nginx/example.access.log;
location ~ .php$ {
proxy_pass http://127.0.0.1:8080;
break;
}
location ~/.ht {
deny all;
}
location / {
if (!-e $request_filename) {
proxy_pass http://127.0.0.1:8080;
break;
}
}
}
In this config, any request for an existing file (that's not .php or an Apache config file) gets served by nginx. If the file can't be found, it's passed on to Apache (running on port 8080) so Apache can do whatever it wants with that path - render an ImageCache thumbnail, display a Drupal page, 404, whatever.
As ImageCache renders static images into the filesystem, the first request for an ImageCache URL goes to Apache, and all subsequent ones go just to nginx for the now-static file.
I don't like passing 404s for
I don't like passing 404s for static files to php because then drupal creates a full page in response but the client never sees it.
Lets say you have a valid drupal page, but there's a stale proxy page so that one css file is missing and lets also say that there are 3 images on the page but their address is wrong or the underlying files were deleted. That means that loading that page will perform 4 additional drupal bootstraps and if you have search404 module installed it will perform 4 searches. And the client will never see any of that. Those pages will be sent in response to an img or link tag and won't render at all. (though you can see them in firebug) Wasted bandwidth and cycles.
I use a location block for css|js|jpg|png requests that sends a generic 404 not found reply. Most people have that block anyways to set a far forward expires for those types of files. Then you break out a second location block for /files/imagecache that serves files directly but redirects back to your php backend on 404s so the imagecache module will create non-existent image files.
Check out the sample config at:
http://github.com/omega8cc/nginx-for-drupal/blob/master/nginx.conf
One caveat though. You won't see Not Found messages in your drupal watchdog log. I use awstats anyways, so they appear there and then I know to fix the broken links.
Good points, thanks. :-)
Good points, thanks. :-)
Exclude apache?
I recently moved to a VPS server instead of dedicated server, I had a dedicated server with 2GB but it wasn't able to keep up and I think its because the hosting company put me on a very bad hardware. However, I moved to a VPS system with 2GB RAM; I have noticed one thing and that is that it seems that my memory is constantly being eaten up and it seems to be due to PHP, however, after diving into it a bit I'm wondering if its Apache+PHP that consumes my memory.
I have nginx handle everything on port 80 and allows Apache handle php but I'm now wondering if I should exclude Apache and let nginx do everything, does anyone know if there is a backside of that? I have boost configured on my several drupal sites and I'm thinking I should also install APC.
Does someone have experience doing it this way on a cPanel/vmh configuration, CentOS 5.4 with Apache not running, will cPanel still work via nginx?
Would appreciate if someone had suggestions on how to setup my mysql (my.cnf), nginx configuration for high traffic with low memory impact and can share.
/Chris
Regards,
Chris W
You first need to research
You first need to research what is consuming your memory. Do not automatically assume this is Apache.
On command line, use 'top' then hit shift-m to sort by memory consumption.
Apache is quite a bloat, but in my experience, not so much of a memory hog.
and I'm thinking I should
Having APC or eaccelerator or xcache installed as an opcode cache can make a big difference in response times. Definitely look into that.
Removing apache as a backend will remove much of CPanel functionality, unfortunately.
Is nginx configured to serve static files (css|js|png|gif|jpg)? Boost files?
Is your site crashing or experiencing long slowdowns or using large amounts of swap memory?
Re: and I'm thinking I should
Currently my nginx is configured to handle static files, not sure how to configure it to handle boost files as well but if you know let me know or I will google it.
My server has gone into restarting services due to all the memory being consumed so thats why I'm considering removing apache but as yhager said, I have to test to see if its apache consuming the memory or if its something else but when I do run top -d2 I see that PHP is consuming a fair amount along with mysql, I have started to tinker around with my.cnf but I need to find someone that is on a VPS and have figured out a good configuration for mysql at 2GB RAM and copy their cnf file.
I appreciate any help because adding memory is the next step but if I don't understand what is taking my memory now without returning it then adding memory could be a never ending story and since adding memory would come at a cost of almost an additional $48 a month for an additional 2 GB RAM. Its amazing how expensive RAM is when its hosted but if you buy it your self you can buy a TB for less than you spend a month at hosting.
Regards,
Chris W
Unless you're hosting a lot
Unless you're hosting a lot of stuff, you really shouldn't need 2GB of RAM in your VPS. Our company is doing just fine hosting a couple dozen sites on a 512MB VPS (average of maybe 5-10rps), without having to utilize swap much - so the thought of exhausting 2GB of memory on a web server strikes me as surprising unless you're doing some really intense stuff.
Now one thing that surprised me when one of our clients got a VPS account through a third-party hosting company was that it had a whole bunch of crap - Apache, mail and FTP daemons, etc - already installed and running. This was quite different from our VPS, where pretty much nothing except the operating system and the SSH daemon were installed and configured for us, and we had to install everything else on the server ourselves. I think you could definitely save a lot of RAM (as well as disk space and security headaches) by taking a bottom-up approach like that and only installing the things you really need. For example, if everyone using the server is capable of using rsync and/or SFTP, then there's no need to install an FTP daemon. Or opt to use a hosted email service instead of installing a mail daemon and then being responsible for mitigating spam to your users (or from, in the case of virus-pwnt Windows users). Bam - instant memory savings simply by not installing stuff you don't need.
And yeah… ditch Apache. And once D7 comes around, you can ditch MySQL too.
The Boise Drupal Guy!
Yeah. If he's got CPanel then
Yeah. If he's got CPanel then he probably does have mail and ftp. And possibly bind with WHM? Maybe a memory hog like spamassassin?
top should show you how much memory mysql is taking up and you may want to run the mysqltuner.pl and tuning-primer.sh scripts until you get a feel for how much memory you might use at peak connections and how much is assigned to various caches and buffers. It is unlikely that someone here will have a my.cnf optimized for a matching CPanel/nginx/apache 2GB VPS.
Omega8cc's nginx config on git at http://github.com/omega8cc/nginx-for-drupal has the necessary locations to serve Boost files directly from nginx. It'd have to be adjusted for apache.
Hosting a lot
We are hosting about 7-10 drupal sites; additionally, we are running our own dns, mail (spamd,exim) and ftp as well so I guess that is taking part of our memory. I'm not sure what my cost savings would be to have godaddy handle my dns and mail for my different domains and if they are good at it.
I will look into rsync sftp since I'm usually the only one logging into the system apart from some of my developers and they are savy so they can use that as well and perhaps cut out the ftp deamon.
I would love to be able to host my 7-10 drupal sites effectivly on my 2GB RAM and not add more RAM to it so thats why I'm here nagging you guys :)
I contemplating asking my VPS hosting company if I could have them spin up a Ubuntu image instead of CentOS without cPanel, cust my cost a bit but also avoid having all the extra cream on the top and instead use webmin; perahaps that will operate in a slimmer fashion and webmin I am sure would run on top of nginx.
I didn't know that D7 would eliminate MySQL, what is the replacement and will it compare to MySQL?
/Chris
Regards,
Chris W
Yeah, we aren't hosting DNS
Yeah, we aren't hosting DNS either. Most good domain name registrars, as well as Go Daddy (lol u see what i did thar?), will let you use their own DNS servers for no additional cost, and offer a web-based interface to set up records and such. We are also using Go Daddy for mail here at work; the service itself is fine, but like all other things Go Daddy, they really try to nickel-and-dime and upsell you for various things.
As for cPanel and webmin and such, you can definitely save some resources (as well as increase your geek-fu) by simply configuring things by hand. But I can understand how that wouldn't appeal to everyone.
Drupal 7 will make PostgreSQL and SQLite first-class database citizens alongside MySQL. If you use SQLite instead of the other two, you no longer have to run a database daemon, as SQLite stores its data in simple files which it accesses directly instead of using a client/server architecture. It also tends to be faster for read operations than the other two, though slower for write operations - but since most web sites do a lot more reading than writing, I think it will be an acceptable trade-off for most sites. So MySQL support won't be eliminated, but there won't be much incentive to use it anymore (though I'm sure most people still will out of force of habit).
The Boise Drupal Guy!
Yeah. Domain registrar for
Yeah. Domain registrar for DNS or I use DNS provided by my VPS company (linode). I also use Google Apps for Domains for mail which is free for under 50 email addresses and has top notch spam filter, web interface, pop, imap, etc. If you don't mind using big brother google, it's a pretty darn good system.
sshd can handle your sftp
sshd can handle your sftp needs without installing another daemon. For you as superuser it's all setup by default. For your devs you can either rely on file permissions or look into the chroot features of sshd. With a few lines in your ssh_config you can limit them to certain directories. (but it's not as easy to setup as a control panel interface.)
I am hosting 4-5 Drupal web
I am hosting 4-5 Drupal web sites (multisite) using nginx+php-cgi+apc, one wordpress site and one rails site. In addition this is a mail server (qmail) with spam filtering (spamassasin) and imap (dovecot).
All I have is 512 MB.
Not that I recommend this config on a daily basis, I just like to push things to the edge a bit :)
All I am saying is that exploiting 2GB for 7-10 sites is wrong - check your config. Check how many apache processes are running (MaxClients IIRC) and check how many open connection MySQL allows. Both apache and mysql take quite a bit of memory for each connection, so keeping this number low can have tremendous effects.
All I am saying is that
That depends entirely on the sites involved . A single well tunned site on well tunned hardware could require 64GB across 6 different boxes depending on the pattern of traffic and data involved.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
DNS and such
I will look into using go dadda (see, no point to them here, no way :) ) for my dns management and perhaps cut down my dns deamon and the fotprint of that. Not sure I'm willing to give into hosted mail server because it think it might be a equal trade of and I'm not sure go dadda would be my first choice for email but seeing how spamd is probably eating up a lot of my memory I could consider it.
Interesting stuff with D7, I wonder how sqlite would work for a site like e bey (messed them up as well ) since I would think it has both read and write quite frequently.
I'm going to check out Omega8cc's nginx script and see if I can optemize my configuration a bit more with boost and nginx.
I really appreciate all the great feedback from this Drupal (kudos to Drupal) community.
/Chris
Regards,
Chris W
I don't trust myself to run
I don't trust myself to run my email, which I consider more important than my web site. I don't want email to get rejected or lost because I got dugged or DDOSed or misconfigured some server script.
I can definitely understand
I can definitely understand this line of thought.
However, if your privacy is important to you, it is really hard to find someone to handle your mail. Handling mail in large volumes is quite difficult, so everybody turn to Google for this, which I refuse to do. If my VPS provider (linode) would offer mail service like they do for DNS, I'll use it without second thoughts.
On the other hand, configuring the mail system once and forgetting about it is not so difficult to do. Add some good iptables rules, and proper maintenance (which you should do anyway), and you'll be fine. Mail will not bounce even if your server is down for a couple of hours, and it will be delivered even if the server is down for a couple of days.
So yes, this is complex, but quite doable.
I'm currently experimenting
I'm currently experimenting with using Rackspace email hosting. At $2 per email box per month, minimum five boxes (so minimum $10 a month even if you don't use all five boxes), it's a bit expensive for personal use, but I've found their service to be nice and speedy and their webmail system competent, and both their webmail and IMAP servers support encryption. And they definitely don't index my email and show me ads based on it. Happy so far, but I'm not sure if I'll stick with them due to the cost. Might be a good choice for small biz, though.
The Boise Drupal Guy!
Long time ago someone
Long time ago someone recommended me http://www.rollernet.us.
I have not used their service cause currently I have everything set up the way I need, but maybe you should check them out. When my setup breaks (or just need an upgrade.. I am still with qmail, ugh...), I'll consider them.
I'll host your mail
:) I'll host your mail for $.50 per mail box, minimum 20 :)
I really like the option of having my own mail server and unlimited accounts and I can setup quota for everyone but myself :)
I am however, going to look into doing what yhager is doing with nginx+php-cgi+apc, I'm just not sure I understand the php-cgi part, could you elaborate on that yhager? If you can do 4-5 drupal sites with 512MB then I should be able to do my 7-10 on 2GB without an issue so I clearly have some work ahead of me.
Does anyone know if Ubuntu vs CentOS make a difference? It's all Linux based (I grew up doing slackware so I know my way around); I use to be Linux freek but my business side took over and I'm more focused on managing the business and sales now than being an admin but I'm going "garage style, boot strapping" on a project so I'm back to doing some admin as well.
Regards,
Chris W
confused again by the UI. My
confused again by the UI. My reply to you is here: http://groups.drupal.org/node/47790#comment-176178
The key here is to use a
The key here is to use a single Drupal install in a multisite setup. APC then bytecompiles the PHP files, and since you are using the same 'system.module' file for all of your sites, it is resident one time in APC cache. 30-60 MB should be enough for APC to give you fast response time.
The key
yhager, that is probably where I'm going wrong, I'm using individual drupal installs for each site instead of a multisite setup. I have never used the multisite setup and I'm a bit worried about seperating the data in the even of a crash but maybe thats because I don't understand or know how the multisite works.
However, I would be concerned that if something happened to your one install that all of your sub-multisites would be gone with it, am I wrong? I was just trying to click on the link you gave me but it took me right back here, I will review all of the latest postings to see where I might have missed your recommendations.
Regards,
Chris W
Yes, yes, yes, if you're
Yes, yes, yes, if you're running opcode optimizers, a mutli-site install is a must. But even if you're not, the convenience of doing so is excellent.
And you're right in that if you, say, update a module and it has a nasty bug, all sites (which use that module) will be affected. But that's why you make automatic site backups at least daily and also manually back things up before doing any major upgrades, right? Right? But really, we haven't had something like that happen too often - the convenience far outweighs the danger. There's a multisite group here on g.d.o which you can check out and post to if you have any questions.
The Boise Drupal Guy!
bad module?
Now if there is a bad module, only the sites that have enabled that module would be impacted by it becoming bad, is that right. The sites that doesn't have it but have access to it would not be impacted, would that be the right way to think about it?
Of course, I do automatic, daily backups like clockwork apart from the times when I don't (ok, I fezz up, I don't but I WILL)
It sounds like I need to regroup my whole server and just get a multisite configured instead because it would seem I would not consume the same amount of memory in that case (I think, it sounds like that).
This is a really good posting, I'm learning a boat load.
Regards,
Chris W
Now if there is a bad module,
Right, because sites for which the "bad" module is not enabled are not even going to load that module's code files.
I doubt that will completely get your memory usage back down to where it should be, but it certainly won't hurt.
The Boise Drupal Guy!
A lot of work?
My only concern is if it will take a lot of work and down time from my current "silo" approach to this approach but then again, if it makes things more stable then its worth it.
Regards,
Chris W
It's probably less work than
It's probably less work than you're thinking, actually. Peek through the Multisite group; there should already be a thread or two on converting multiple installs to a multi-site install. And if you still have questions, go ahead and start a thread of your own.
The Boise Drupal Guy!
Your probably right
Garrett, I'm sure your right and I'm going to get busy right way. Thank you for the suggestions and to everyone contributing to this forum-post, its an excellent learning arena.
I will let everyone know about my experience of going from silo to multi-site and the payoff which I'm expecting to address some of my issues and perhaps I can get closer to performance like yhager which would be awesome.
Regards,
Chris W
Multi-site how-to
I followed most of the steps outlined at http://justinhileman.info/articles/a-more-secure-drupal-multisite-install
I prefer to keep the drupal directory off the webroot and also the sites directory within the drupal structure symlinked. Makes for an easy minor version upgrade, with merely symlinking the whole directory instead of having to move files.
Oh, God, no, do not follow
Oh, God, no, do not follow those directions. Those are wrong, wrong, wrong… this person either had no idea what they were doing, or is doing it in an unnecessarily complicated and dumb way for some reason. In fact, any time you're working with Drupal and you feel the urge to symlink something, you are probably doing something wrong… like this.
Read the stuff in the comments of the default.settings.php file; there's a lot of good info there. And need I mention the Multisite g.d.o group again?
The Boise Drupal Guy!
I use a variant of Hileman's
I use a variant of Hileman's setup myself. It is an alternate setup, more complicated, and not necessary to use Drupal's multisite feature. It creates an extra layer of separation between sites, and has some security benefits, while still allowing the use of one drupal codebase.
Complicated, unnecessary, but not dumb.
Yes, I also was unable to
Yes, I also was unable to follow this line of thought. From skimming through the post, it seems like he managed to create a multi-single setup, using a single Drupal install, or something of some sort.
My own setup does include symlinking, but it is rather simple. I should write a detailed post about it (unless someone will tell me this is obvious and point me to an already written page).
Simplifying that a bit, I have, in directory /var/www:
drupal-6.16 <== This is clean Drupal coredrupal-6.16/sites -> /var/www/sites <== symlink
drupal -> drupal-6.16 <== symlink. Allows me to upgrade easily
sites <== the sites directory is here, contains an entry for each site
To upgrade to drupal-6.17, I extract that directory, wipe the original sites directory, replaces that to a link to my "sites", and when I am ready, I replace the main drupal->drupal-6.17 link.
There is even a rather easy method to upgrade each site separately, but I won't go into that here.
That makes sense. My personal
That makes sense.
My personal approach is a bit different. I take down the server daemon, then rename the web root directory (which just has Drupal in it) to something like
www-6.16. I'll make a database dump for backup purposes. Then I'll download the new Drupal release, rename it to the web root directory, then copy (not move) everything from the old web root directory to the new one. Then I'll use Drush to update all sites' databases. Finally, I bring the server daemon back online. Tada! Easy upgrade. Once I'm sure that the upgrade didn't foobar anything, I'll delete the older Drupal install directory.The Boise Drupal Guy!
That's more/less the same
That's more/less the same modifications I've made to the outlined process. Easy to upgrade, easy to keep everything separate.
sites <== the sites directory is here, contains an entry for each siteDo you point your web server to individual directories as the root for each domain? If so, do you have your index.php, cron.php, etc pointing back at the drupal install directory?
Do you point your web server
No. Every domain uses the Drupal directory as its root directory. Drupal itself figures out which site to serve by looking at which domain name is being requested. Again, check out the commented-out description of the process in settings.php.
The Boise Drupal Guy!
Detailed post...
I would be interested in reading your detailed post, let me know when its available.
Regards,
Chris W
It is important to note that
It is important to note that while it is a good idea to move from solo to multi on a VPS environment, this step by itself will not reduce the RAM usage on your server.
You have to research the source of RAM usage — I have not seen in this thread any indication to what it is, besides some guesses.
Memory usage
yhager; here is what my current memory top image looks like
top - 07:51:01 up 6 days, 1:23, 1 user, load average: 0.16, 0.12, 0.05
Tasks: 106 total, 1 running, 101 sleeping, 4 stopped, 0 zombie
Cpu(s): 7.3%us, 3.1%sy, 0.0%ni, 89.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2059492k total, 1611676k used, 447816k free, 361432k buffers
Swap: 2096440k total, 42012k used, 2054428k free, 804712k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15319 mysql 15 0 284m 130m 3312 S 0.0 6.5 84:11.87 mysqld
22294 root 15 0 104m 37m 2304 S 0.0 1.9 0:01.34 spamd
22296 root 16 0 105m 37m 1644 S 2.7 1.9 0:11.60 spamd
28669 root 18 0 49948 7628 1432 S 1.0 0.4 0:00.03 tailwatchd
21215 root 18 0 40456 6580 1956 S 0.0 0.3 0:00.09 leechprotect
29129 root 18 0 70136 4812 2892 S 0.0 0.2 0:00.64 httpd
3900 root 15 0 41760 4460 1156 S 0.0 0.2 5:00.15 tailwatchd
21224 nobody 16 0 70536 3920 1688 S 0.0 0.2 0:00.27 httpd
21223 nobody 15 0 70536 3916 1692 S 0.0 0.2 0:00.30 httpd
21225 nobody 15 0 70536 3916 1684 S 0.0 0.2 0:00.27 httpd
29515 root 15 0 86804 3824 492 S 0.0 0.2 1:27.42 cpsrvd-ssl
28533 root 15 0 94248 3364 2608 S 0.0 0.2 0:00.04 sshd
4142 named 24 0 161m 2244 848 S 0.0 0.1 9:43.62 named
21222 root 16 0 70136 2136 216 S 0.0 0.1 0:00.00 httpd
1660 dovecot 15 0 33996 1988 1504 S 0.0 0.1 0:01.91 pop3-login
1659 dovecot 15 0 33884 1888 1504 S 0.0 0.1 0:00.12 pop3-login
16933 root 18 0 32236 1784 560 S 0.0 0.1 3:59.13 queueprocd
1662 dovecot 15 0 33904 1724 1384 S 0.0 0.1 0:00.12 imap-login
1663 dovecot 15 0 33892 1724 1384 S 0.3 0.1 0:00.46 imap-login
26678 nobody 15 0 41896 1604 572 S 0.0 0.1 3:02.31 nginx
28552 root 15 0 66076 1560 1168 S 0.0 0.1 0:00.02 bash
15295 root 23 0 10796 1180 964 S 0.0 0.1 0:00.01 mysqld_safe
3136 mailnull 15 0 64336 1116 636 S 0.0 0.1 0:00.98 exim
28623 root 15 0 12740 1088 820 R 0.0 0.1 0:00.10 top
1632 root 15 0 14768 988 796 S 0.0 0.0 0:01.47 dovecot-auth
3347 haldaemo 15 0 31120 776 288 S 0.0 0.0 4:34.19 hald
2665 root 15 0 17536 752 592 S 0.0 0.0 1:43.09 vmware-guestd
3903 root 33 18 23504 584 264 S 0.0 0.0 0:00.43 cpanellogd
1598 root 15 0 6080 560 408 S 0.0 0.0 0:00.99 dovecot
3419 root 15 0 124m 464 304 S 0.0 0.0 13:55.32 automount
3065 root 7 -8 81816 448 176 S 0.0 0.0 5:55.30 audispd
Regards,
Chris W
Man, you are running a full
Man, you are running a full blown server there, with a full mail server, spam filter, DNS server, both apache and nginx and a lot of cpanel stuff going on there. Clean it up. You do not need complex setups, just something clean.
I am not sure what is this cpanel stuff good for, I never used it. If you require some server management software, seek something with a smaller footprint.
If I got it use it
I know, its as if I think things are going out of style and I have to use it now or lose it. :)
I have used the cPanel for managing the different accounts, dns, mail, etc just because my time has been limited to sitting and manually do all that stuff, cPanel gives me a quick in an out but comes at a price.
I am going to go through my server and see what things I can put elsewhere as the dns with go dadda but I doubt I will give up my mail server but maybe, not sure.
The apache is running because thats how I was reading to use it with nginx and cPanel needs it but I'm thinking of dropping cPanel and apache and instead just run nginx in the same fashion as you are. One thing I was struggling was the - sign that you had on your nginx+php-cgi+apc, what do you mean by php -cgi?
Regards,
Chris W
One thing I was struggling
It's not a minus, but a dash. nginx cannot run PHP scripts internally like Apache. nginx needs to talk to a PHP engine and send the inputs (script and parameters). The PHP engine is running the PHP code, and computes the result, which it sends back to nginx. This communication is going on internally, using a unix socket or a TCP socket.
The PHP engine is called "php-cgi" (this is the process name). This is quite a naked engine, it does not include spawning and killing processes. A few solutions exist around that, depending on your OS.
PHP-FPM is a PHP fork (which has been merged back to PHP 5.4) that aims to do process management efficiently and natively in PHP, and should perform even better than php-cgi.
PHP-FPM
So PHP-FPM should be the choice in order to not having to use apache for php processing then? I know now that php-cgi would also work but it seems that you are leaning towards php-fpm.
Are there any drupal specific benchmark on these do and what really works the best or its just up to your own taste and preference? (Google moment, I think).
Regards,
Chris W
I am actually leaning towards
I am actually leaning towards php-cgi.
PHP-FPM is a patch on PHP, and I prefer not to use patched software if I can. However, a lot of people praise PHP-FPM, so it must be quite good.
--y
PHP-FPM is getting pretty
PHP-FPM is getting pretty close to official.
In three weeks the Ubuntu 10.04 release will go live and will include PHP 5.3.2. The php.net experimental svn has php-fpm in it for 5.3.2 but it wasn't included in the official 5.3.2 release. It should be easy to add and I bet someone will have it in a PPA within days.
I've been making notes of contrib modules that need patches for PHP 5.3 over here: http://groups.drupal.org/node/56558
CPanel does provide a nice
CPanel does provide a nice skinnable web interface, tracks bandwidth, simplifies some complex things like mail aliases and forwards, integrates stuff like awstats, and has some useful scripts like emailing full site backups. It's also perfect for shared hosting sites where the server owner wants to restrict access to certain features or even disallow ssh.
Eventually you run up against its limitations (and annoyances) and you get your own VPS.
My own VPS
I have my own VPS so I'm using cPanel to handle my 7-10 different domains and sites just because I was to lazy to manage it by manual editing files left and right but it does come with a price and I think that price is partially RAM consumption.
However, I'm contemplating asking my current VPS hosting company to allow me a new Ubuntu image with no cpanel and I'll just get webmin installed and manage everything via that instead. The benefit is that I get to pay less and I will consume less memory and now if I want to add more memory I would end up paying my current price but at that point have 4GB RAM instead (nice trade off I think).
Regards,
Chris W
nginx compatible
So the multisite with nginx front-ending and apache backending is seamless? I'm planning on dropping apache eventually but initially that is a combination working just fine?
/Chris
Regards,
Chris W
yes, that works. I even saw
yes, that works. I even saw some benchmarks that there is not much gain from running PHP in the back of nginx than running a full apache there (but maybe PHP-FPM performs better here).
Multisite and boost
How does multisite work with boost since boost wants the cache directory to be in the root drupal directory?
Regards,
Chris W
boost creates a subdirectory
boost creates a subdirectory with the domain name.
multisite and svn
I was also wondering, is there any known issue with multisite and svn? I have all of my sites in SVN and I have configured it so that I can just run svn up when I have updated development and it updates production quit easy, does anyone have experience using this with multisite?
Regards,
Chris W
I don't think you should have
I don't think you should have any more issues than one normally has migrating to a production server, no.
The Boise Drupal Guy!
Nginx is a great solution,
Nginx is a great solution, quick guide on how to setup it with Drupal: http://drupalperformanceblog.com/nginx-and-drupal-configuration-high-per...