Posted by am4 on October 10, 2012 at 1:40pm
I've gotten Varnish working in front of Drupal 7. I've also got Pound in front of Varnish, and can successfully browse the site via HTTPS. However, whenever I try to log in over HTTPS, I get a 403 forbidden error, and I'm not entirely sure why. My guess is Varnish-related... but I'm not positive.
Anyone have any ideas/pointers as to what I'm missing? I'm glad to post Firebug output showing the POST to /user and the resulting 403 if it helps.
Relevant snippets from settings.php:
<?php
$conf['reverse_proxy'] = TRUE;
$conf['page_cache_invoke_hooks'] = false;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 300;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['https'] = TRUE;
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https') {
$_SERVER['HTTPS']='on';
}
?>Relevant snippets from default.vcl:
sub vcl_recv {
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# Do not allow outside access to cron.php or install.php
if (req.url ~ "^/(cron|install).php$" && client.ip !~ inside) {
# Have Varnish throw the error directly.
error 403;
}
# Do not allow outside access to /admin
if (req.url ~ "/user/?" && client.ip !~ inside) {
# Have Varnish throw the error directly.
error 403;
}
if (req.url ~ "/admin/?" || req.url ~ "/user/?") {
return (pass);
}
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove the "Drupal.toolbar.collapsed" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "Drupal.toolbar.collapsed=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^ *$") {
unset req.http.cookie;
}
# Cache static content unique to the theme (so no user uploaded images)
if (req.url ~ "^/themes/" && req.url ~ ".(css|js|png|gif|jp(e)?g)") {
unset req.http.cookie;
}
# Normalize Accept-Encoding header
if (req.http.Accept-Encoding) {
if (req.url ~ ".(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}
# Don't cache the install, update or cron files in Drupal
if (req.url ~ "install.php|update.php|cron.php") {
return (pass);
}
# Anything else left?
if (!req.http.cookie) {
unset req.http.cookie;
}
if (req.http.Authorization || req.http.Cookie) {
# Not cacheable by default
return (pass);
}
}
Comments
I would make sure that things
I would make sure that things are working well over HTTP before you throw HTTPS into the mix. It may just be that this snippet is blocking you:
# Do not allow outside access to /adminif (req.url ~ "/user/?" && client.ip !~ inside) {
# Have Varnish throw the error directly.
error 403;
}
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
My general rule when
My general rule when debugging something like this is to find the log entry. Either there is a log entry somewhere showing the 403 happening, or there's a log you haven't turned on that you can turn on, after which the 403 will generate a log entry.
Even if the log entry is otherwise unenlightening, it will tell you which layer of your stack is rejecting you.
Pound is terminating SSH, I presume?
Meanwhile, in the annals of premature guessing, my eye is drawn to these lines of your VCL:
Does passing through Pound mess with client.ip such that it no longer matches "inside"? Also, where is "inside" defined? Is that some VCL builtin that I just don't know about or is it defined elsewhere in the file?
@dalin - yup, HTTP logins
@dalin - yup, HTTP logins through Varnish work fine.
@mike - Correct, Pound is terminating SSL. The acl is defined at the beginning of the .vcl and is only used to restrict access to /user. It is as follows:
acl inside {"10.10.128.0"/17;
"127.0.0.1"/32;
}
Commenting out that block did nothing for HTTPS. 403 still thrown. (HTTPS logins directly through Apache work fine... but I want to be able to control clients using HTTPS via the use of Varnish.)
Here's the output from Firebug during an attempted login over HTTPS through Varnish/Pound:
POST user
Response Headers
Accept-Ranges bytesAge 0
Cache-Control no-cache, must-revalidate, post-check=0, pre-check=0
Connection keep-alive
Content-Length 0
Content-Type text/html; charset=UTF-8
Date Tue, 09 Oct 2012 17:49:08 GMT
Etag "1349804948"
Expires Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified Tue, 09 Oct 2012 17:49:08 +0000
Location https://drupal.domain.com/user/2
Server Apache
Set-Cookie SESSe15687525d17b8ec181665a71c88775c=EttXyyPvqESdU4RapW7xZkrRagGNHgRH5I9P6x0yRRE; expires=Thu, 01-Nov-2012 21:22:28 GMT; path=/; domain=.drupal.domain.com; httponly SSESSe15687525d17b8ec181665a71c88775c=mAdWa_a_OvcIIoWBuiVbLqFJzwyHiukfd_xBOVz_eaQ; expires=Thu, 01-Nov-2012 21:22:28 GMT; path=/; domain=.drupal.domain.com; secure; HttpOnly
Via 1.1 varnish
X-Drupal-Cache MISS
X-Varnish 2081364495
Request Headers
Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie has_js=1; __utma=194497400.1529654640.1349804906.1349804906.1349804906.1; __utmb=194497400.3.10.1349804906; __utmc=194497400; __utmz=194497400.1349804906.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
DNT 1
Host drupal.domain.com
Referer https://drupal.domain.com/user
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
GET 2
Response Headers
HTTP/1.1 403 ForbiddenServer: Apache
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Tue, 09 Oct 2012 17:49:08 +0000
Cache-Control: public, max-age=300
Etag: "1349804948-1"
Content-Language: en
X-Generator: Drupal 7 (http://drupal.org)
Set-Cookie: SSESSe15687525d17b8ec181665a71c88775c=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=.drupal.domain.com; secure; httponly
SESSe15687525d17b8ec181665a71c88775c=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=.drupal.domain.com; httponly
Vary: Cookie,Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Content-Length: 8916
Accept-Ranges: bytes
Date: Tue, 09 Oct 2012 17:49:09 GMT
X-Varnish: 2081364496
Age: 0
Via: 1.1 varnish
Connection: keep-alive
Request Headers
GET /user/2 HTTP/1.1Host: drupal.domain.com
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Referer: https://drupal.domain.com/user
Cookie: has_js=1; __utma=194497400.1529654640.1349804906.1349804906.1349804906.1; __utmb=194497400.3.10.1349804906; __utmc=194497400; __utmz=194497400.1349804906.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); SESSe15687525d17b8ec181665a71c88775c=EttXyyPvqESdU4RapW7xZkrRagGNHgRH5I9P6x0yRRE; SSESSe15687525d17b8ec181665a71c88775c=mAdWa_a_OvcIIoWBuiVbLqFJzwyHiukfd_xBOVz_eaQ
Further info - looks like
Further info - looks like those cookies are being deleted when trying to use Pound/Varnish. That doesn't happen when going to Apache directly for an HTTPS login.
VCL
Do you have vcl which removes cookies for certain state? You'll want to double check that.
Also, don't forget about varnishlog.
I've used NGINX in front of
I've used NGINX in front of varnish to handle https traffic before. Might be worth a look if pound is still giving you trouble. It took very little time to setup and has worked like a charm.
Google & https
Lots of folks are going to be looking at this due to Google's recent announcement about https & ranking http://googlewebmastercentral.blogspot.ca/2014/08/https-as-ranking-signa...
I'm still not sure how that gets around these concerns https://www.varnish-cache.org/docs/trunk/phk/ssl.html
Possibly using https://bensmann.no/seperate-varnish-caching-http-https/
--
OpenConcept | Twitter @mgifford | Drupal Security Guide
For Varnish + HTTPS sites
For Varnish + HTTPS sites that we've done in the past we have something in front of Varnish doing SSL termination (either a load balancer, or Nginx, or people used to use Squid for this too). Whatever does the termination should be passing some sort of HTTP header to indicate if the source was secure.
In this setup you don't get a secure connection between the terminator and the web server, but unless you're a bank you probably don't care.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Tried CloudFlare? Does both HTTP and HTTPS
Have you tried using CloudFlare which handles both HTTP and HTTPS in front of your Drupal site. They also recently announced SSL support on the free plans too. Think of it like a hosted NGINX service, with other benefits.
Agileware are an Australian Web Team specialising in Drupal, WordPress and CiviCRM.
Agileware provide development, support and hosting services.
https://agileware.com.au
Pound->Varnish->Drupal
I have Pound sitting in front of Varnish on my stack which allows all HTTPS pages to be cached. I've written about it over here and included full Drupal/Pound configurations as well as the relevant VCL configuration.
Pound terminates the SSL on port 443 and forwards to port 80 on Varnish. A header set in Pound tells Varnish that it was originally SSL traffic and Varnish is able to cache it separately to non-SSL. This is important if you're allowing mixed mode traffic as otherwise non-SSL resources can appear on SSL pages.
…and Varnish is able to cache
It's important for pure HTTPS too because you probably want to redirect HTTP to HTTPS. So to avoid infinite redirects Varnish needs to be able to treat them separately.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
I totally agree, terminating
I totally agree, terminating SSL before Varnish is key to a successful SSL deployment.
And with proper X-Forwarded-Proto headers, it is also simple to setup in Varnish / Apache using SetEnvIf.
Some ther SSL Varnish Links
I haven't done this yet, but looking into it again. Thinking about the EFF's efforts to encrypt the web. Anyways, here are some different links for consideration.
https://www.digitalocean.com/community/tutorials/how-to-configure-varnis...
http://blog.ajnicholls.com/varnish-apache-and-https/
http://edoceo.com/howto/nginx-varnish-ssl
http://mikkel.hoegh.org/2012/07/24/varnish-as-reverse-proxy-with-nginx-a...
I wish there were more details about Cloudflare's offerings:
https://www.cloudflare.com/plans
Has anyone used them?
--
OpenConcept | Twitter @mgifford | Drupal Security Guide
Cloudflare SSL
I've played with Cloudflare's new "universal SSL" a bit. It works well enough, but requires a modern browser that supports SNI. So if you have legacy users (like we do) who are running XP or IE6 or old Android versions (≤ICS) it will throw certificate errors.
Overall, Cloudflare is pretty nice. However they seem to be overly-sensitive to backend load, returning "the site can't be reached, giving you an old cached page" more often than needed.
I know this is an old thread,
I know this is an old thread, but it's the first that comes up when you search for Drupal SSL and Varnish, so thought I'd try to clarify some things.
The 403 you're seeing is generated by Drupal, Varnish doesn't touch it since you've authenticated yourself. All cookies use the source address ("client ip") in combination with some other things I can't recall. But since we're going through reverse proxies we can't reliably use the source address anymore. This is what X-Forwarded-For is used for. For every reverse proxy the request goes through it adds the source address to it, creating a list. The first, or left-most IP-address will be the original source address.
Drupal seems to be configured correctly, you're telling it that it's behind a reverse proxy and to look for the X-Forwarded-For header. When you're connecting over TLS the connection first goes to Pound (Or Nginx, HAProxy, whatever) which terminates the connection and forwards the requests downstream. It adds a X-Forwarded-For header, as it should, with the correct source address.
But, and this is a gotcha, you can never unset the X-Forwarded-For header later and then find the original source address. The default logic in varnish is to do just as Pound or any other reverse proxy, by adding the source address to the list. But in this case you're overriding the default logic and setting the X-Forwarded-For header to client.ip, which in this case then means the Pound-server. So when you sign in the cookie won't be generated correctly, and you will be thrown out as soon as you try to do anything. The first requests goes through as normal since you authenticate in the same step.
In Varnish you also have to hash things based on X-Forwarded-Proto, to make sure that you differentiate the caches for TLS and non-TLS. Otherwise you're going to run into some interesting issues. And in Drupal you can also reliably just nuke all cookies that are not SESS or SSESS, since they are always present when needed (forms, authenticated users).
--
Vegard