I stumbled across this article which describes a useful technique for doing cache invalidation. This could work nicely and have many advantages over calculating the list of URLs to invalidate on the Drupal side, as the current combination of Varnish+Expire modules does. We could hook into entity CRUD functions and add headers to the response.
For example:
1. A user visits a node page. On that page, the node entity is loaded as well as the 5 comment entities. We send back the response headers as follows:
X-Invalidated-By: node-1,comment-1,comment-2,comment-3
2. We can cache this page indefinitely in Varnish.
3. When a content author updates node/1, the header returned from Drupal will be:
X-Invalidates: node-1
4. In default.vcl, we detect that the POST request has returned an X-Invalidates header, and ban any content that is invalidated by an update to that specific piece of content:
sub vcl_fetch {
if (beresp.status >= 200 && beresp.status < 400
&& (req.request == "PUT" || req.request == "POST" || req.request == "DELETE")) {
ban("obj.http.X-Invalidated-By ~ " + beresp.http.X-Invalidates);
}
}This is far more efficient than calculating expired URLs on the Drupal side, then making a request to Varnish to expire those URLs.
For multiple Varnish servers, the problem is sharing the ban list. One way to overcome that might be using a Varnish module that allows the servers in the cluster to communicate new bans to each other. For example, there is a Varnish cURL module that can do this.
Here is some basic code that adds the headers:
<?php
/**
* Code for the Varnish TCI module.
*/
/**
* Implements hook_entity_load().
*/
function varnish_tci_entity_load($entities, $type) {
foreach ($entities as $entity) {
list($id, $vid, $bundle) = entity_extract_ids($type, $entity);
drupal_add_http_header('X-Invalidated-By', "entity-$type-$id", TRUE);
}
}
/**
* Implements hook_entity_update().
*/
function varnish_tci_entity_update($entity, $type) {
list($id, $vid, $bundle) = entity_extract_ids($type, $entity);
drupal_add_http_header('X-Invalidates', "entity-$type-$id");
}
/**
* Implements hook_entity_insert().
*/
function varnish_tci_entity_insert($entity, $type) {
list($id, $vid, $bundle) = entity_extract_ids($type, $entity);
drupal_add_http_header('X-Invalidates', "entity-$type-$id");
}
?>
Comments
Multisites & Domain Access
The only issue I see with this idea is the NID on one host/multisite would be the same NID on another host; and to make the issue harder Domain Access means we have to tie mutiple hostnames and the NID together. Other than that this would be quite useful for things like views & panels.
Is this the varnish module you're talking about: https://github.com/varnish/libvmod-curl
Yes, that's the one. We could
Yes, that's the one. We could make a BAN http request using that module from within the VCL. I opened a ticket for supporting this method, but looking at the source of the vMod it would be quite trivial to do.
Drupal module for Varnish tag banning
We had the same issue. So we've created some simple Rules actions to integrate with the Purge module to add and purge based on tags. See Varnish Tag Invalidate for more details.