Caching and 404s for aggregated js and css in D6

It's a recurring problem even without local caching, but especially with it. You go to update a module and then you run database updates. But part of that is clearing caches and now Drupal regenerates your aggregated js and css, which makes sense in case the updated module has changed its js or css files.

And then the problems occur. If you have a reverse proxy in front of your web server, it is still serving old cached pages that refer to old aggregated files. But those files no longer exist because Drupal has regenerated them and deleted the old ones.

Even if you don't have a caching proxy in front of your site you might still see 404s in your logs for your js and css files. There's plenty of discussion on it: http://www.google.com/search?q=site%3Adrupal.org+404+aggregated+js+css

Here's my current solution using nginx caching. Please comment to improve it or offer a better solution.

nginx provides native caching. It should be simple to just cache the js and css files and serve them from the cache for some very long period until they're no longer needed. They're small, so even if we save them for 6 months they won't take up much disk space, even though you might end up with several hundred of them as you update your modules.

But it's not so simple. The problem is that nginx caching only works for proxying to backends and not for static files. It's made to proxy a page from your php-fpm or apache server. If you want to cache a static file, you have basically 3 choices:

1. serve the static files from your backend like php5-fpm
2. use a module like slowfs cache
3. create a new server in nginx and have it proxy to itself

None are very elegant. The first is the simplest, and since you're only talking about a couple dozen requests, it's not very taxing on your backend, but it does feel like a waste of CPU cycles. The second is made to order since the module was designed for caching static files that might exist on another system mounted through NFS, but you have to recompile nginx with a 3rd party module which might or might not work with the next version of nginx. The third option requires no backend interaction and no third party modules, so I went with that.

First I created a new listening server on an arbitrary localhost port:

Since it's a localhost port it's not accessible from outside. It has two locations that serve only aggregated css and js files. It uses the $host variable so it'll be good for multiple domains. I don't use Drupal's multisite structure that includes sites/[domain.com]/files/css/js_ so my files are in /files/css/.

Then I just need the caching section in my regular server:

Because I'm using "priority" literal strings(^~) for efficiency, these sections will be matched before regex locations (~* or ~) so it doesn't matter where in the server section they occur. If you're using /sites/[domain.com]/files then you'll have to make them regex locations(~*) and place them prior to your regular css/js location. They pass the request to the local server and cache the file with a key that includes the domain, so we use one cache for all your domains. The cached file stays valid for 180 days and then purges.

Oh yeah, we need to define the cache in our http section like so:

Since we set the cache to be valid for 180 days, it never checks for the actual file after the first retrieval, so it doesn't matter if Drupal has deleted it. After 180 days it would check the actual location, but hopefully by 6 months all your caches are up to date.

So what do you guys think? Am I crazy to go through this trouble just for a few 404s? Or should you just purge every page from your reverse proxy cache every time you have to update a module?