Compression -- Drupal side or Apache side or both?

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Swampcritter's picture

Which would produce a better compression rate -- using mod_deflate on the Apache instance side for css/js files and/or using Gzip for Javascript/CSS on the Drupal side?

I am looking for overall site performance aspects as well (i.e. additional CPU time, memory allocation, etc).

Comments

The lower the better

kbahey's picture

Note that Drupal's "Optimize CSS" and "Optimize JS" options under admin/settings/performance does not gzip. So it is not the same as the compression in Apache.

If you are talking about custom code that is not in core, then I would imagine that, if configured the same, the end result would be the same compression wise. Both Apache and PHP have shared libraries that implement the compression. For example, on Ubuntu 8.04.2 LTS, both PHP and Apache use the same library : /usr/lib/libz.so.1.2.3.3

For Apache, mod_deflate can be configured in different ways, e.g. http://httpd.apache.org/docs/2.2/mod/mod_deflate.html (DeflateCompressionLevel).

"How much compression" should be weighed against "how costly it is to compress".

Pushing this to lower levels would be better at least from a resource point of view, meaning that Apache handling it is better than in PHP application code. If you have Apache configured as a threaded server, and PHP as FastCGI, then you tie less PHP processes by delegating this to Apache rather than doing it in Drupal.

So if you optimize css and js, you get the benefit of less connections per page load and users will notice a difference. In addition if you set mod_deflate's compression level to 9, you will get the same as putting it in PHP, but with less resources tied up for less time.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Right now I have Apache

Swampcritter's picture

Right now I have Apache doing the compression...

<IfModule mod_deflate.c>
   <FilesMatch ".(js|css)$">
     SetOutputFilter DEFLATE
   </FilesMatch>
</IfModule>

We thought about adding in the CSS_Gzip Project module as well (http://drupal.org/project/css_gzip), but didn't know if that would impact each other (i.e. double gzip compression).

I also found this article/thread (http://drupal.org/node/416808), but it gets a bit confusing on what should (or shouldn't) be used.

-- M

If you have Apache doing it ...

kbahey's picture

Unless you can measure and notice the added overhead of CPU utilization by Apache for compression, I would not think doing it on the fly vs. storing the gzip would matter much.

If you have Apache doing it, then don't bother doing it again at another level, PHP or otherwise.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

I am curious on your

ajayg's picture

I am curious on your thoughts comparing following senarios.

S1: Apache doing compression on the fly for each request of javascript and css

S2: Php (drupal contrib module perhaps) compressing and storing the gzip file once (or everytime it is updated). basically adding a compression step after css and javascript optimization. But not doing in apache.

Wouldn't S2 would be better (and save CPU cycles) since you are doing compression only once and storing rather than every request?

S2 wins

kbahey's picture

S2 wins in this case, since the file is compressed only once, and then stored for a long time, and served compressed as is.

But you have to create some rules in .htaccess to send the .gz as a css file.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Compress the whole page while you're at it

joshk's picture

As usual, I agree with Khalid: apache is the "best" place to do this. You can also have apache compress the html part of the page as well. I use this FilesMatch:

        <FilesMatch ".(js|css|html|)$">
          SetOutputFilter DEFLATE
        </FilesMatch>

And I turn off drupal's standard page compression. This gets the html body of the page compressed and saves PHP/Drupal the sweat of doing it.

http://www.chapterthree.com | http://www.outlandishjosh.com

Is there any reason NOT to do

acouch's picture

Is there any reason NOT to do this if you have your own server? Is this just an easy performance win for most sites if you have access to configure your vhosts?

You propably want to do

exlin's picture

You propably want to do both.

Drupal combines files into one big (takes unnecessary space out etc...) witch also helps with compability in some cases with Internet Explorer (can't remember if max css files were still something like 8 or 9 per site).

mod_deflate however adds true compression to file while for transfer. However, it is not so compatible with IE <= 6.

But if those conditions aren't stopping you, you propably want to do both.

No. Doing both is unecessary,

Fidelix's picture

No. Doing both is unecessary, and will only make your site slower.

Drupal's compression has nothing to do with css and JS aggregation.

Aggregation happens whenever you activate "Optimize CSS Files", as well with JS files. And Internet Explorer can take up to 32 css files.

mod_deflate is not incompatible with IE <= 6 because it automatically deactivates gzip for serving files to these browsers.
So yes, you always want to use Apache's GZIP instead of Drupal's.

No. Doing both is unecessary,

dalin's picture

No. Doing both is unecessary, and will only make your site slower.

@Fidelix any justification for this opinion? I disagree. A few things to note (as pointed out elsewhere on this thread):
- Drupal's compression setting is only for pages stored in the page cache (for anonymous users only). No compression is done for authenticated page views or static files.
- mod_deflate is smart enough not to try to re-compress something that is already encoded.
- compression takes time and will decrease the requests per second that your server can dish out (of course the benefit being that transferring those files to the client happens much more quickly).

Therefore it is most efficient to enable mod_deflate and compress within Drupal. Drupal will store compressed pages in the cache and subsequent views will not need to recompress them. Meanwhile mod_deflate will handle compression for everything else.

--


Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his

Wouldn't it be easier and

Alexander Langer's picture

Wouldn't it be easier and more efficient to use these two modules?
- http://drupal.org/project/javascript_aggregator
- http://drupal.org/project/css_gzip

They both store the aggregated and compressed file locally so apache only has to serve them as static files, reducing the overhead coming with mechanisms where apache has to compress an aggregated css or js file on each and every request.

Sure that will decrease the

dalin's picture

Sure that will decrease the work that mod_deflate needs to do. But you've still got authenticated users, AJAX requests, etc. I still use mod_deflate.

--


Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his

Alexander, Lighttpd caches

Garrett Albright's picture

Alexander, Lighttpd caches the compressed files it creates to disk. Then the next time it goes to serve them again, it will serve them directly from cache instead of recompressing them. I'm not sure how Apache's implementation works, but I'd be surprised if it wasn't caching too.

mod_deflate can slow down requests per second by 10x

mikeytown2's picture

http://drupal.org/node/101227#comment-3176382
Makes sense, compress it once and save vs stream compression.
Based off of the new rules in here I have changed the rewrite rules for boost so its 100% compatible with mod_deflate now; mod_deflate will only run if there is not a gz version of the request.

Tuning Apache mod_deflate

SqyD's picture

I would agree too Apache is the way to go. Cpu load increase is affordable in most use cases. If you're looking to improve compression levels of certain files or locations you can add some Apache directives, in your case something like:

<IfModule mod_deflate.c>
   <FilesMatch ".(js|css)$">
     SetOutputFilter DEFLATE
     DeflateCompressionLevel 9 
</IfModule>

Depending on resources (cpu, but mostly your time because these apache configs can get messy) you could consider enabling compression on some of the bigger generated (html,xml,rss,etc) files too. If you're having high loads compressing static files install a reverse proxy like varnish or squid in front of apache. (A good idea in any case). These don't compress themselves but can be configured to respect the Accept-Encoding header and store both uncompressed and compressed versions and serve out what the browser requests.

Aggregating and compacting your files (Drupals optimize css and js options) also has a big effect on your overall compression rates and total file sizes.

Update from all the feedback

Swampcritter's picture

Thanks all for the feedback. After finding out (via trial and error) what is and isn't allowed under Apache 2.2 and the mod_deflate module, this is what we can up with and it working like a charm for our environment:

<IfModule mod_deflate.c>
   AddOutputFilterByType DEFLATE text/plain
   AddOutputFilterByType DEFLATE text/html
   AddOutputFilterByType DEFLATE text/css
   AddOutputFilterByType DEFLATE application/xhtml+xml
   AddOutputFilterByType DEFLATE application/xml
   AddOutputFilterByType DEFLATE application/javascript
   AddOutputFilterByType DEFLATE application/x-javascript
   AddOutputFilterByType DEFLATE application/json
   DeflateCompressionLevel 9
   SetOutputFilter DEFLATE
   SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
   <FilesMatch ".(js|css|html|xml)$">
     AddOutputFilterByType DEFLATE text/plain
     AddOutputFilterByType DEFLATE text/html
     AddOutputFilterByType DEFLATE text/css
     AddOutputFilterByType DEFLATE application/xhtml+xml
     AddOutputFilterByType DEFLATE application/xml
     AddOutputFilterByType DEFLATE application/javascript
     AddOutputFilterByType DEFLATE application/x-javascript
     AddOutputFilterByType DEFLATE application/json
     SetOutputFilter DEFLATE
     SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
   </FilesMatch>
</IfModule>

Under CentOS/RHEL, we saved this as deflate.conf under /etc/httpd/conf.d thus don't have to have it directly embedded in the httpd.conf file (plus it allows us to tweak the code as needed).

-- M

FYI

rjbrown99's picture

The AddOutputFilterByType directive has been deprecated by the Apache team in favor of mod_filter, although it still works. Here's a reference doc:
http://httpd.apache.org/docs/2.2/mod/core.html#addoutputfilterbytype

Smashmagazine compression article

ObiWasabi's picture

Hi,

After finishing this article http://www.smashingmagazine.com/2009/07/19/how-to-automate-optimization-and-deployment-of-static-content/, I was wondering how at my newbie stage in DRUPAL development
to apply the techniques to a DRUPAL site? Any suggestions would appreciated

Tips

mikeytown2's picture
  1. Get even faster with subdomains

This is the only one that I don't see out there in the drupal community. The vast majority of these tips are built into drupal core, or there are modules/htaccess rules; minus the css sprites.

SmashMagazine Compression Tips

ObiWasabi's picture

I thought some of the article tips where included in the drupal core, as it is thoroughly planned. I'll research the modules/htaccess rules.

Many Thanks

Javascript loading block paralell loading...

redox's picture

I'm looking for the same solution... Javascript is loaded on the top of each page.

While that happens page loading is blocked to other things like loading images,flash and also could cause a gap while processing that code and restart loading the rest of the page...

Is there someone that know how to solve that?

thanks

God bless Drupal

calling $scripts before

PlayfulWolf's picture

calling $scripts before $closure will help, but you will need to check if all the modules work as supposed

p.s. getting "Your submission has triggered the spam filter and will not be accepted" - wtf is going on with groups?

drupal+me: jeweler portfolio

and what about Boost

Nigeria's picture

How does all this relate to Boost.

Boost has an option to turn on/off gzip compression depending on whether its already done in Apache.

Going by this thread it looks like it may be better to turn off Apache compression and let Boost do it and cache it at the same time.

Any thoughts or ideas on this?

If you let Boost compress

Alexander Langer@drupal.org's picture

If you let Boost compress its cache you'd save cpu cycles on cached pages. On the other hand you would send logged in users uncompressed pages.

So how you set up your server is up to your specific needs.

Alex

theatereleven's picture

I have been trying to figure out how to get this Apache compression to work. Some places say modify the .htaccess file while it seems like other places say something different.

I'm using Pantheon Project Mercury, and I looked in my .htaccess file and saw nothing similar to the below:

<FilesMatch ".(js|css|html|)$">
      SetOutputFilter DEFLATE
</FilesMatch>

How can I make my site compress files for faster load times for authenticated users? Any help greatly appreciated!

See Drupal Project Authcache

Peter Bowey's picture

See Drupal Project Authcache - > http://drupal.org/project/authcache
-or- wait until mikeytown2 adds 'auth users' to Boost?
-or- use Nginx with fastcgi cache + gzip and deal with user / admin cookies vs anonymous user

--
Linux: Web Developer
Peter Bowey Computer Solutions
Australia: GMT+9:30
(¯`·..·[ Peter ]·..·´¯)

Thanks! I can't use Nginx

theatereleven's picture

Thanks! I can't use Nginx because Pressflow (part of Pantheon) only uses MySQL.

Will the link you sent me be cool with the Pantheon installation? How is it better than Memcache and APC together?

Compress good, Cache better, CDN essential

Dave McCormick's picture

Hi

I agree that using mod_filter is an essential Apache config for improving your sites peformance, and I think that with performance in mind, there are a number of other performance improving steps (and modules) which you should also consider. Compressing content is good, but not sending content is even better! You need to make sure that you are setting good Cache-Control headers to allow users, proxies and content delivery networks to cache your content. I've written up a quick guide: http://www.practicalclouds.com/content/guide/accelerate-your-drupal-webs.... It also covers other techniques, such as automatically resizing pictures and non-blocking javascript loading.

Also, you really really should consider using a Content Delivery Network, so that your static data gets served from a location near to your users. This will also vastly improve your users experience of your website, if they are located a long way away from your webservers. I've written another simple guide to using Amazon CloudFront for this purpose here: http://www.practicalclouds.com/content/guide/cf-cloudfront

regards

Dave

http://www.practicalclouds.com
Host your Drupal website in EC2