PHP memory consumption tips

Posted by raulc on February 18, 2009 at 1:29am

Hey everyone,

I imagine there should be a few of you out there having a similar issue like me. I am with a hosting provider that cannot increase the PHP memory limit over 32MG. At the moment I am facing two issues, I cannot access /admin/build/modules so I cannot enable the modules from there and if I try to just change the status from the database it seems like just by changing the status the module is not functioning right. I am also wanting to use image cache to have all the users' profile pictures to have the same dimensions but it is recommended to have 96MG of memory to use this module. My two questions are:

What approach should I take ti be able to enable modules in my live site so that I do not have to do ti on my development environment in the local host all the time and then having to transfer more stuff. IF I want to disable a few modules, which ones are the classic ones that would consume a lot of memory and that do not really add that much to my site?.
Would it eb risky to go live using imagecache module knowing that I have a memory limit of 32MG?. Since I am only wanting to use it for the profile photos, would there be any work around to resize and crop all pictures different than using imagecache and avoid having risks of memory that will lead to a white screen.

Any other tips to decrease memory usage would be welcome.

Thanks in advance

Comments

PHP Memory consumption tips

Posted by Abeaudrian on February 18, 2009 at 1:51am

Hi Raulc

I had this problem yesterday, and searched the forums at my hosting company's website.

What I discovered is a quick fix to that problem. what you need to do is to create a file named php.ini and put the following line in this file

memory_limit = 96M

Or simply substitute what ever megabyte value would work for you, I used 96M and that did the trick straight off for me, it may not need to be that high a value.

Place this file in the root directory of you file structure i.e., the public_html folder or (www) where your index.php file is stored.

Hope that helps.

Best Regards

Adrian Millington

Move to a bigger host

Posted by behindthepage on February 18, 2009 at 2:40am

Adrian is giving you good advice but it probably won't work on your server. On small shared servers they disable the ability to change the memory limit. Your best option is to move to a better plan. If it is for the Cancer Council I would move to a more expensive with more features shared hosting or a small VPS. It will save you headaches.

Did you receive my emails re the CC work?

Regards
Geoff Davies

Regards
Geoff
The user previously known as gpdinoz

Move on....

Posted by jdsaward on February 18, 2009 at 3:58am

I agree. Most hosting services I have used do allow re-configuration of the memory limit via the settings file or the htaccess file, as other folk have documented correctly here, but I have had the experience of the memory limit being 'fixed' in the host's configuration files and having no ability to over-ride.

There is much competition in hosting services, lot's of good deals around, and Geoff's advice is good. If your hosting services support cannot resolve this for you, move to a host that will deliver for you and your client.

Was doing social distancing before it was cool.

I've had the same thing. I

Posted by xstevex on February 18, 2009 at 3:01am

I've had the same thing.

I made the changes in the .htaccess file. Just find the code below and add the last line. Usually this works for shared hosting.

PHP 5, Apache 1 and 2.

php_value magic_quotes_gpc 0
php_value register_globals 0
php_value session.auto_start 0
php_value mbstring.http_input pass
php_value mbstring.http_output pass
php_value mbstring.encoding_translation 0
php_value memory_limit 96M

Will try php.ini

Posted by raulc on February 18, 2009 at 4:24am

Hello eveyone,

Thanks for the comments. I tried doing it on settings.php and .htacces files with no success and my hosting said they could not change the memory limit. This is not for our main website but for a smaller website of CC that will manage the online community. Thsi is also only an interim solution while we migrate all site into a dedicated server. Could I try php.ini or chances are that it will not be overriden?

Geoff: I did not get your emails, sorry. Feel free to write directly to raulc@nswcc.org.au

I will try the php.ini file to see if that has some effect.

Raul

Brilliant

Posted by raulc on February 18, 2009 at 4:30am

Reporting back. the inclusion of the php.ini file was able to override the php memory limit.

Thanks a lot

Raul

Email not getting through?

Posted by behindthepage on February 18, 2009 at 4:53am

Hi Raul,

I have sent two emails to your email address regarding helping with the CC site . One from my computer and one via your contact form and obviously neither got through. If you still need help please ring me on 0425 269 228 or 8211 9175.

Glad you got your problem sorted out with your webhost
Regards
Geoff

Regards
Geoff
The user previously known as gpdinoz

Re: Brilliant

Posted by Abeaudrian on February 18, 2009 at 5:12pm

You are welcome, glad I could help ;)

Best regards

Adrian

Thanks - the php.ini worked for me, too

Posted by beanworks on April 4, 2010 at 3:48pm

I noticed my hosting company also has to change the php memory, so I thought I'd try this first.
I created a basic txt file:

<?php
    memory_limit = 96M
?>

Saved it as php.ini and uploaded it to the root (public_html).

I don't know if it actually works, but least Drupal is happy. :-)

Need new host

Posted by shinz83 on November 2, 2010 at 1:21pm

Can anyone recommend a good Drupal host? Right now we are on "Blue H..." and they crash more than they should. I am also running into PHP memory consumption problems... which makes producing the new site just awful.

Please shoot me an e-mail at seth.hinz "at" michigandistrict.org

PHP memory consumption on my new VPS

Posted by chien_fu on January 20, 2011 at 3:34pm

I was using 1&1 for hosting my development site, but of course started running into the 32MB limit (this cannot be increased when using 1&1). So I recently set up a small VPS with 256MB of RAM.

I'm a novice at this, so I assumed that increasing from 32MB to 256MB would be a WORLD of difference and that I would be ALL set. This is however, not the case. It appears that just setting up Apache and having that and other supporting processes running consumes almost all of my 256... I can easily crash my sites just by refreshing two pages at once.

Nothing has changed about the drupal install or database or modules, it's just consuming ~230MB instead of ~35

Any hints on how to reduce this massive memory consumption?

PHP memory limit and VPS memory

Posted by mark matuschka on January 20, 2011 at 8:17pm

Are you getting confused between the PHP memory limit on shared hosting and the total memory required to run a virtual server?

PHP memory limit is the max memory a request can consume.

VPS memory runs the operating system, web server and database, let alone any extra memory required for PHP requests. 256 MB will barely run a vanilla LAMP VPS.

--
Mark Matuschka
Glo Digital

Ah hah...

Posted by chien_fu on January 21, 2011 at 3:50am

It seems a though I may not understand the difference between vps and php memory. I think I'm starting to understand and it all starts to add up. Sounds like I will need much more of a sever that I had at one point imagined. I don't plan to have millions of hits but I am aifor more than just a few per hour. It sounds like I wil probably need more like 1 gb or more oamoda When

Sorry for the typos. Swype won't let me erase...

So this all begs the question... Why would I want to use a vps and get effectively zero php memory when I could just use go daddy and at least get 64mb for 1/10 of the cost?

Memory vs. Bandwidth

Posted by InfinityMark on January 21, 2011 at 4:00am

There are others in my company who are more of an expert on this than I; however, I think you are confusing bandwidth with memory. The more modules you add to a Drupal site the more memory it takes to load that site - irrespective of the number of users who may be accessing your site at any given time. The memory requirement is how much RAM it takes to load your site ONE time - not for each user viewing your site. Once you get to the point that you are adding more than 25-30 modules you start to get beyond the capacity of a shared hosting solution - and it doesn't take long before you hit your head on virtual dedicated hosting solutions.

We spend about $220 a month for our dedicated servers with GoDaddy (each server) and about $750 per month with RackSpace - our GD servers are unsupported while our RackSpace servers are supported.

Not sure if that helps but...

Matt

Whilst it is true that every

Posted by ac on January 21, 2011 at 4:16am

Whilst it is true that most modules do increase memory usage for page loads, it is mostly insignificant (depending on the module of course). The bulk of memory usage by PHP processes is when enabling modules (especially if you enable a bunch at once). This is often experienced by people in shared/underpowered/poorly configured environments as a white screen when enabling a module. What this is is the PHP processes not having enough memory available to it (PHP memory limit). It can not finish executing the process as the system kills it when it tries to use more memory than it is allowed in php.ini. Whilst it is not a firm rule (i.e. a poorly written module may consume lots of memory on a page load and lead to a white screen) it is general enough to be applicable most of the time.

1) If you want to run your

Posted by ac on January 21, 2011 at 4:05am

1) If you want to run your own LAMP stack with anything more than a small amount of traffic you need a minimum of 512MB. This is still massively underpowered unless you know a bit about optimising Linux to consume less memory.
2) Optimising Linux to consume less memory is not for someone who is new to Linux.
3) As mentioned above the PHP memory limit has nothing to do with how much memory your server has available to it. PHP memory limit is how much of the available memory can be consumed by one PHP process. You shouldn't need the PHP memory limit to be any higher than 128MB and certainly for your situation 256MB is way too high.
4) If you really do want to use a VPS and have it available to the world I would suggest you spend some more time learning about Linux. Until then I would stick with a shared host that lets you increase the PHP memory limit and has more than 256MB of RAM available to it.

Cool

Posted by chien_fu on January 21, 2011 at 4:19am

I've been looking for a straight answer on this for quite a while now.thank you so much for clearing it up for me.i think I know what I have to do...
Thanks again!

You could also try a minimal

Posted by dazweeja on January 21, 2011 at 4:20am

You could also try a minimal lamp stack like Turnkey. It should have everything you need and is based on Ubuntu 8.04. They are tested with 256MB (but you will probably need more in a production environment):

http://www.turnkeylinux.org/lamp

Yes, you can get a full blown

Posted by DaveIsFluffy on January 20, 2011 at 4:01pm

Yes, you can get a full blown Drupal site up on 256 meg of RAM.
A few things I've picked up are:
You'll have to hack MySQL config to do it well though as it seems to assume we all run enterprise kit. Out of the box it will chug down about 40 meg of RAM before it actually does anything. Don't use innodb tables in a constrained environment without a good reason.
Some tasks are more hungry than others, so if you're generating A4 pdfs out of A3 jpegs on the fly you may need to cough up more money.
Have you configured swap space?

As always, please aware that this all works for my sites, but may not work for yours

How would you suggest hacking

Posted by chien_fu on January 20, 2011 at 4:28pm

How would you suggest hacking mysql to reduce memory, exactly?
I'm pretty novice at Linux currently and am working to get up to speed. Are there any tutorials or articles you can point me to that would help me understand how/why to change what/where about the mysql install to optimize it?

Not familiar with "swap space" so I'm going to take a stab and say that I have not configured any such thing.

I shouldn't be doing anything to swamp memory, like generating PDFs.. probably the heaviest task is uploading photos and creating thumbnails with imagecache.

Thanks!

What operating system is your

Posted by petercook on January 20, 2011 at 8:19pm

What operating system is your vps running and do you know is it 32 or 64 bit? I run a couple of low use drupal sites on a Ubuntu 64bit LAMP set up with 512MB without any problems.
Where are you getting your memory usage figures from? If I run >free at the command prompt I get the following:

> free
             total       used       free     shared    buffers     cached
Mem:        504352     480196      24156          0       6464     110824
-/+ buffers/cache:     362908     141444
Swap:      1048568      10708    1037860

It looks like I only have 24156 bytes free but as I understand it in linux processes stay in the ram even when they are done so they can be run again if needed without having to be loaded from the hard drive. They get removed when a new process needs the ram.

If you run >top you will get a dynamic table showing the processes running and their cpu and memory usage. You can have top running in a shell window while you load your webpages in a browser and see what processes are running and what resources they are using.

Hope this helps,
Peter Cook.

Here are a few pointers on

Posted by DaveIsFluffy on January 20, 2011 at 9:41pm

Here are a few pointers on optimising for constrained systems, which say pretty much the same sort of thing I would:
http://rimuhosting.com/howto/memory.jsp
http://paragasu.wordpress.com/2008/12/02/very-low-memory-mysql-5-mycnf-c...

Having said that, all the the optimizations in the world won't help you if you really need 256+MB RAM, I am just assuming you haven't built a massively heavyweight site and aren't planning on being hammered by tens of requests per second. Once you start worrying about more than a few requests per second you are going to have to start paying real money for your hosting.

The best way to learn about

Posted by Slovak on January 21, 2011 at 11:38pm

The best way to learn about running your own VPS is to invest in a 256MB one. OK, invest may be a big word for less than $100. It will pay back ten-fold!

Such a small VPS will force you only install what's absolutely needed, and then tweak those pieces to start with. No Webmin, no cPanel. Instead of Apache, run Nginx. Try, test, try. Then do it again. I've run a low-traffic multisite D6 install off a 256 to learn. Keep it around for testing and playing when you decide to upgrade.

Well Said

Posted by drupdrips on January 22, 2011 at 2:23am

very well said Slovak. Having a VPS will let you acquire many more skills and control over how your limited memory is used. But you also have to be willing to go down to command line level and do all the stuff.

If you are making big profits and have deep pockets, you can afford to be a bit wasteful or even your additional traffic may warrant the need for the higher end VPS/dedicated. But if you are trying to scale up from a very low end with as little of your own money, it is an art to learn how to start from AS LOW of a memory footprint as possible. nginx/lighttpd for webserver is definitely a good try and being aware of the # of thread/process you allow, their resident memory footprint as driven by the # of modules enabled and their max used point after running the site for a while including the admin functions. Using a php opcode cacher like xcache as well as file-based cacher like boost will help with performance on a low end (or high end) vps.

With a VPS and no cPanel or Managed hosting services, you have to be very diligent on administering your own security. There is a whole list to go down starting from iptables/firewall, hardening ssh and tcp/ip, fail2ban for avoiding brute-force logins, keeping an eye on open ports and close any non-needed ones - use non-standard port for ssh and also do not allow root logins, if you are running bind/DNS service - do not allow zone transfers also run bind from chroot/jail and stop recursion and .NS queries. Also look at file/folder permissions - dont run everything as root. If you are using mysql DB on same VPS which you would on a small budget / also better for performance, close the outside port and use bind-address 127.0.0.1. Speaking of mysql, if you use phpmyadmin for administering the DB, shutdown phpmyadmin after use and chmod 000 the phpmyadmin folder so no one can access it. There are a lot of script kiddies trying to hack into phpmyadmin. This is just some of the security aspects, there is quite a bit more, in each of the software layers, webserver, drupal also (for drupal try to see if you can avoid php execution in the files directory which you may not be able to avoid for modules like imagecache but there are other ways to avoid that , also i recommend the login_security module, and of course keep up with all the updates which is challenging in itself. Due to the changes/impact to a running environment you have to do a balancing act (these updates are a necessary evil based on your security needs and your extra effort of keeping a clean/secure env).

Make a choice; adjust; keep going.

Posted by jdsaward on January 22, 2011 at 2:46am

As an ex unix system administrator, I recognise the wisdom of drudrip's cautionary advice.

If one has the time, or money, or both, or let us say - 'the inclination', or 'interest' - to take all that security administration on, then for sure 'go for it'. It's all learningfull.

I for one now prefer all that O/S stuff to be taken care of for me, so that I can find enough time to concentrate on Drupal. However, I am very appreciative that at a certain time in my career I was dropped into the deep end of system admin and at least do understand and respect the creatures that lurk down there. ;-) Your listing in your 3rd par is great, drupdrip.

It's all a matter of where we wish to put our limited time. And what we currently wish to be challenged by. Cos in this industry whatever choice we make we will be challenged. That's a given.