Doesn't look like I will make it to the Eastern Iowa meeting tonight after all... I'm having issues with my internet host that I need to deal with.
Yesterday I received an e-mail from my current host indicating that I've breached a "new" limit on my "unlimited" hosting plan.
Specifically, I have more than 1000 tables defined in my account. In fact, I have more than 2500 tables defined in my account, spread across 25 different databases, serving 22 different domains. The host has informed me that having more than 1000 tables in any single account degrades server performance. Really? They also claim that data in excess of 3 GB will degrade performance...my 2500+ tables still occupy less than only 0.7 GB of space.
I have 14 days in which to reduce the number of tables to 1000 or less or they will pull the plug on my account. This is simply not possible under the circumstances so I'm looking for another host.
I contacted one yesterday that looks promising. Anybody care to suggest a good host or have any Drupal experience with GreenGeeks or their load-balanced solution, SiteCloud.com. My circumstances and site characteristics didn't seem to bother them in the least.
Comments
A VPS is the way to go
Hi, I'd strongly suggest the host I use. It's a low-cost VPS service I've been using for about 4 months now and have had exceedingly good performance and uptime.
https://service.burst.net/aff.php?aff=1511 (this is my tell a friend link and gets me credit on my bill if you sign up) or just burst.net.
They have a 512MB Linux VPS for $5.99 /mo or $60 annually. a 1GB VPS is $9.99/mo or $99 annually. I created one with Ubuntu 10.04 LTS Server and it's performing superbly. I've not had a min of downtime and with default Ubuntu settings (40 max clients) I'm doing 2,000+ req /sec on a php "hello world" page.
If you're not used to running your own server, it's no sweat. Sign up for Ubuntu 10.04 LTS then SSH in and run sudo tasksel and install a LAMP setup. If you prefer a gui admin panel the free ISPConfig hosting panel is really great (I prefer version 2 rather than the newer v3). They have step by step instructions for installing it on Ubuntu Server on their website.
The only downside I've had with this host is that SSL performance is not stellar. I've gotten 11-12 req /sec doing SSL. I don't use SSL for much though so I'm fine with it.
--
Matthew Nuzum
newz2000 on freenode
Web Geeks For Hire: Hosting Provider
Web Geeks For Hire hosts many drupal sites already. We use DIY VPS hosting, similar to that mentioned above, but we have already done the work of configuring the environment to be suitable for drupal websites. If you want to contact me and give me an idea of your RAM, disk space, and bandwidth requirements, I can give you a quote.
Alternately, if you are into doing it yourself, I would recommend Slicehost (referral link: https://manage.slicehost.com/customers/new?referrer=3173bb04a3309c305937...) or Linode (referral link: http://www.linode.com/?r=89d9ff800a2d83c56ae5cba3b02020a5839f10c2). We have used slicehost for a while and they have pretty great support, although their pricing and hardware has started to fall a little bit behind. That caused us to look at Linode, which has newer hardware and more attractive pricing.
Good luck!
Sean Adams-Hiett
Web Geeks For Hire
www.webgeeksforhire.com
sean@webgeeksforhire.com
Thank You Gentlemen
Excellent feedback...and so quick! Thanks guys.
Sean,
So now maybe I will reinstate my sign-up for tonight's meeting. 8^) Sounds like I could really benefit from your experience (same goes for Matt, but the DrupalCorn meeting isn't tonight) so I will reconsider my attendance tonight if you think we might be able to find 15 minutes to discuss this before or after the meeting?
FYI... I just completed the setup and migration of http://www.s-tama.k12.ia.us from an old Ubuntu 8.04 server box to a new installation of Ubuntu 10.04 LTS server. I think I could be up for the challenge of handling a VPS.
Mark
Thanks for the discussion. I
Thanks for the discussion.
I am wondering if anyone could explain how to evaluate a vps. Everyone lists RAM and hard drive, but it seems the cpu is a factor not well disclosed.
Looking at burst.net, they have vps and premium vps.
Comparing VPS #4 with Premium VPS #2, it looks like VPS #4 is a better package, and it is $5 more. But only $5 more and more than double the guaranteed RAM, triple the hard drive, seems like a clear winner. Does this comment about the Premium VPS matter - LESS THAN 50% LOAD OF BUDGET VPS NODES
VPS PACKAGE #4
CPU: 2500MHZ GUARANTEED
Memory: 2.5GB GUARANTEED
Disk Space: 150GB (RAID BASED CONFIG)
Bandwidth: 2000GB/MONTH
FREE SETUP $29.95/MONTH
http://burst.net/linvps.shtml
PREMIUM VPS PACKAGE #2
CPU: 2500MHZ GUARANTEED
Memory: 1GB/2GB (GUARANTEED/BURST)
Disk Space: 50GB (RAID BASED CONFIG)
Bandwidth: 2000GB/MONTH (1GBPS PORT)
FREE SETUP $24.95/MONTH
http://burst.net/premiumvps.shtml
I recently switched to ServInt VPS and they have great tech support, but the package I am on is $89 per month and I would like to save money if I could. Also, the RAM seems fine but what I seem to run out of is CPU resources.
Any managed VPS recommendations would be appreciated. I tried Linode and could not figure it out, and their pricing appears close enough to ServInt to choose ServInt for the support.
http://www.servint.net/vps.php
There are big variances
There's a big difference between VPSs.
CPU is one. Virtualization technology is another.
If a site says that a resource is burstable (burst.net does this) then it means instead of true virtualization they're using a "container" style. That means they can oversell their resources. Therefore they say you get 1GB of RAM burstable to X or whatever. This style of hosting is the cheapest and performance can vary greatly. Stick with a vendor that you know from reputation or who you can get a trustworthy reference from.
Other virtualization technologies, such as that used by Linode, give you more consistent and predictable performance. You can be sure that you'll get exactly what is specified, there is no range. They can't oversell and therefore the cost is a bit higher. It's not necessarily better though.
Something else you may be paying for is management. This is usually more expensive than the hosting.
OK, so to get to your point, how much CPU do you need? I ran about 70 mostly static sites with a little bit of PHP and CGI on a 200MHz Cobalt RAQ with 256MB of RAM. I can run 120 req/sec constantly on a Django app on my burst.net 512MB VPS and server load stays below 1. I'll bet Drupal performance with xcache installed will be about similar to my Django app. Once your data set gets too big to hold in memory or the number of simultaneous users goes up performance will go drop. For example, a forum site I hosted that was very busy would melt a 512MB VPS.
Most of the websites people will run are not CPU intensive. They spend most of their time looking for data in the database and serving the data over the network. The CPU is largely idle because the processing is no big deal. In my experience running busy websites, the RAM and the disk performance are way more limiting. Each time a person attaches to Apache it uses RAM. Maybe as little as 2MB, maybe as much as 40MB. When the site has to read large data files off the disk often it causes delays serving the request to a user which causes more users to be connected and waiting for data which uses more RAM which causes less data to be cached which causes disk operations to go even slower, yadda yadda yadda.
--
Matthew Nuzum
newz2000 on freenode
Thanks. This is a snapshot
Thanks.
This is a snapshot from my server. It is with only myself on there and a few bots, and the RAM is still pretty high, at least in relation to 256 or 512MB.
Server load is low but when I do much at all it easily gets to 1.5 or 2, with 2 being the supposed limit.
Server load 0.05 (8 CPUs)
Memory Used 19.13% (601752 of 3145728)
Swap Used 0.00% (0 of 1)
I am new to Drupal and web site development and don't know if I am not configuring it correctly or if I just have a complicated site.
It's hard to tell from that
It's hard to tell from that how much ram is actually being used. The system may be using some of that as cache. It also may be that you have apache configured to use more RAM than it needs. For example, if your child processes use 40MB of RAM you can have max_children set to 10 and you'll use at most 400MB. If it takes 100ms to serve one request you could still have a theoretical 1,000 requests per second even with only 10 processes.
Finding the real amount of RAM can be notoriously tricky because the difference between virtual, resident, shared memory is quite obscure and the numbers can vary a lot.
The limit to the server load average is an interesting guide but shouldn't worry you too much. I've seen server's with a load average of over 100 that respond just fine. What that means is the number of processes waiting. If you have 30 people attached to your computer trying to download files over a slow connection it could be possible for your cpu and ram usage to be very low but your load average to be over 30.
Here's a performance trick I do on my own apps: I upload all of my static content to an S3 bucket and serve it from there, then I configure Apache as my app server and serve only dynamic content. If your site is www.something.com you can create a bucket named files.something.com and then point a DNS CNAME to apache's S3 service and then link to your files at http://files.something.com.
This gives you two benefits:
* Static content doesn't clog up your app server
* You split the files over two domains and therefore enable twice as many parallel downloads.
My S3 hosting bill for my wife's website ranges from $0.08 /mo to $0.24 /month. So ridiculously cheap that to me now it's a no-brainer.
--
Matthew Nuzum
newz2000 on freenode
Thanks for the comments
Thanks for the comments Matthew.
When you reference static content do you mean pdfs, jpgs, things like that? Because the cname idea seems logical to me, I had asked my shared host about it and they said it wouldn't work.
For my site, there is not much static content so caching does not improve it much. ServInt did install a common php cache program.
You have an interesting blog, I will try to learn more through that.
Here is a snapshot of server resources when flushing cache. That was my main problem with a shared host, it would regularly timeout. No longer times out but does take a good couple minutes to clear cache. Part of the annoyance may be simply I am developing the site and clearing cache frequently.
Server load 4.45 (8 CPUs)
Memory Used 37.26% (1172248 of 3145728)
Swap Used 0.00% (0 of 1)
Regarding static resources,
Regarding static resources, even your themes folder (CSS, Javascript and theme images) can benefit by moving them to a diff domain. However, I have a firm policy of not fixing things that aren't broken.
If your site is working fine and you like how things are going, I'd say keep on keeping on. If you have a particular issue that you want to fix, then some of the things we've discussed can be helpful.
The CNAME and S3 for static resources fixes two problems:
It allows more resources to be downloaded in parallel (many browsers limit the number of resources that can be downloaded at once from a single domain, using two domains effectively doubles the limit)
It allows you to optimize your server configuration for dynamic content. This means you may be able to get by with less server or increase your capacity without having to upgrade.
There are other issues and they have different solutions.
--
Matthew Nuzum
newz2000 on freenode
Thanks for all the comments
Thanks for all the comments everyone. Lots of ideas.
It's About RAM and Disk Performance... I Agree
I have to agree with newz2000 regarding RAM and disk performance as keys. In my experience with the server at http://www.s-tama.k12.ia.us I find that most visitors demand an average of about 40 MB per session. Yes, I have a fair (IMO) number of Drupal modules in play there but not overkill. The CPU load never seems to be an issue...RAM always seems to be the limiting factor.
I don't mean to hijack my own thread here but I have a related question...
One thing I struggle with on the s-tama site is my PHP session limit for memory. I think I'm currently set at 64 MB per session, and that seems more than adequate for my site visitors. However, I've noticed some admin tasks that come very close to tripping over that limit. I fear that adding another module, or even updating some, might push me over the limit in administration activities. I've noticed that there are a couple of php.ini files on the server...one for CLI and another for Apache, apparently.
I have to assume the Apache php.ini file settings control the resource limits for site visitors. What do the CLI (assuming that is command-line interface) parameters govern? More to the point, if I do my administration through Drush do the CLI parameters apply to my session rather than the Apache parameters? If that is the case then I'm certainly going to bump up the resource limits for CLI and maybe even ratchet them down to say 48 MB on the Apache side.
Great discussion folks. Thank you very much.
Mark
It's OK to set your memory
It's OK to set your memory limit higher than you need, it just means that Apache can use up to this amount of memory. I think Drupal recommends 96 MB.
Also, something I've done in the past to better utilize my memory is put a separate web server in front of Apache. In this case, with PHP embedded in Apache it's using a lot of RAM for every process. That makes it more of an application server than a traditional web server.
You can put Apache on a higher port number, for example 8080 and then put Nginx on the same server proxying to localhost:8080. You can then point the /files and other static file paths to the correct place on the server's filesystem so that it serves them directly. This does a few things to help.
An extreme example, say that a user requests a 2MB PDF file you've uploaded. If it takes them 30 seconds to download that file, if you're using Apache, that 40MB of RAM will be used up for 30 seconds delivering a PDF that doesn't need PHP at all. If your max_clients is set to 40 then that means you've decreased the number of requests by 1/40th for that time.
Ngninx will use far far less ram (probably under 2MB) to serve that PDF file. If you use proxying, Apache will hand it off to nginx instantly and let Nginx spin for 30 seconds delivering it freeing up the apache child process. If Nginx can deliver the file directly from the filesystem then it's even better.
You can also enable caching rules so that you can serve your site (or part of it) as if it were static content. I can tell you from experience this can be a life saver if you occasionally get bursts of traffic.
--
Matthew Nuzum
newz2000 on freenode
As to shared hosts and php
As to shared hosts and php limit, I was using Site Ground and they seem to have a higher php limit for shared accounts (over 64, 96 I think). I tried switching to BlueHost and had lots of problems related to php limit and switched back.
I also liked their customer support, it was all tickets no chat but they were quick and helpful.
https://www.siteground.com/