I have this weird issue and I'm not sure if it's me who's doing something wrong or I'm just missing something but maybe some of you can guide me a little bit here...
i setup a varnish server, within varnish itself i created 2 backends and using director i'm able to do round robin to my webheads. That parts seems to be working fine as I'm able to bring one of my webheads down and the other one can serve content to varnish and vice verse.
but here is a kicker... i noticed that sometimes page doesn't load properly so i start looking into why and I noticed some of the files are 304 or 404, so turns out somehow some of files in (files/css/.css or files/js/.js are missing), so I rsync them so now they all appears to have same content in those directories, but when i go to another page it seems like drupal generates more of those and since varnish goes to a round robin for any hit it serves some of page from one webhead and another part from another, so even though css,js file has been generated on first webhead it not generated on second and hence shows 304 or 404.
please advise
Comments
Your files directory needs to
Your files directory needs to be shared between your php servers. Lots of people use NFS. You could get fancy with glusterfs or use some form of nas but you should probably start with NFS.
Full Fat Things ( http://fullfatthings.com ), my Drupal consultancy that makes sites fast.
I am looking into NFS but I
I am looking into NFS but I want to know if there is a way for varnish itself to keep user on same server and not serve bits and peaces from all over...
alexus.org && alexus.biz
http://lmgtfy.com/?q=varnish+second+server+404
http://stackoverflow.com/questions/4327874/can-varnish-be-configured-to-...
of course you have to expand this logic a bit since request can hit serverA OR serverB - one in the example is set up only for serverA(404)->serverB
sub vcl_recv {if (req.restarts == 0) {
set req.backend = serverB;
} else {
set req.backend = serverA;
}
}
AdvAgg
Having a shared files directory is the correct way to do this. But core's handling of CSS/JS is broken in my opinion. The Advanced CSS/JS Aggregation module was built from the ground up with things like varnish in mind http://drupal.org/project/advagg. Say good bye to 404s on CSS files.
You could, but you probably don't want to
Technically your probably could do exactly what you say you want. Varnishs' VCL allows a lot of flexability. You could write some config to allways select for one specific backend for all /sites/default/files content and use round robin for all other dynamic content. Just having a shared filesystem with NFS or similar will save you a lot of time debugging wierd edge cases when processing logged in sessions, private filesystems and static files outside the /sites/default/files locations.
I just put in place NFS
I just put in place NFS server where files is being shared between servers, yet still seeing 304,404
[root@6prx01 ~]# varnishlog | grep Status
14 RxStatus b 200
12 TxStatus c 200
14 TxStatus c 304
12 TxStatus c 404
12 TxStatus c 304
14 TxStatus c 304
12 TxStatus c 304
16 TxStatus c 304
15 TxStatus c 304
18 TxStatus c 304
17 TxStatus c 304
17 TxStatus c 304
^C
[root@6prx01 ~]#
alexus.org && alexus.biz
304 is fine
304 Not Modified
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection
You been able to track down that lone 404?
well, after having nfs share
well... after having nfs share and sharing /files between 2 different VMs i'm back to where I started it...
here is 404 that i'm seein on server (sometimes i dont even see that)
172.16.0.151 - - [22/Dec/2011:13:16:50 -0500] "GET /files/css/css_0da6b9249592407aa55b9c93a380260f.css HTTP/1.1" 404 6259 "http://6dev.dev/where-we-stand" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7"
yet I see following:
-rw-rw-r-- 1 apache apache 233880 Dec 22 13:16 css/css_0da6b9249592407aa55b9c93a380260f.css
-rw-rw-r-- 1 apache apache 1034 Dec 22 12:20 css/css_3c590b89f670fe212c3602649f9cd757.css
-rw-rw-r-- 1 apache apache 233180 Dec 22 12:33 css/css_6283f0c2a73360b57cfdc0350bf22b3d.css
-rw-rw-r-- 1 apache apache 234003 Dec 22 12:24 css/css_6c4a45da3bf34be5ccbe3d328f1e85e0.css
-rw-rw-r-- 1 apache apache 233180 Dec 22 13:00 css/css_8ded81df262272be9a9e038490173c78.css
-rw-rw-r-- 1 apache apache 233181 Dec 22 12:39 css/css_cf69dd69376190c23253c098f1f0db48.css
-rw-rw-r-- 1 apache apache 233181 Dec 22 12:32 css/css_d8688f1071da51f459a7b4a187f88420.css
-rw-rw-r-- 1 apache apache 233180 Dec 22 12:20 css/css_da985bdd250e7b2da128190436e09abf.css
-rw-rw-r-- 1 apache apache 234005 Dec 22 12:34 css/css_ec759df853cdd6d278791424971adc71.css
so why is it saying 404 when file is there...
varnish request for this file
12 ReqStart c 10.52.208.221 52862 980385619
12 RxRequest c GET
12 RxURL c /files/css/css_0da6b9249592407aa55b9c93a380260f.css
12 RxProtocol c HTTP/1.1
12 RxHeader c Host: 6dev.dev
12 RxHeader c Connection: keep-alive
12 RxHeader c Cache-Control: max-age=0
12 RxHeader c User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
12 RxHeader c Accept: text/css,/;q=0.1
12 RxHeader c Referer: http://6dev.dev/where-we-stand
12 RxHeader c Accept-Encoding: gzip,deflate,sdch
12 RxHeader c Accept-Language: en-US,ru;q=0.8
12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
12 VCL_call c recv
12 VCL_acl c NO_MATCH unwanted
12 VCL_return c lookup
12 VCL_call c hash
12 Hash c /files/css/css_0da6b9249592407aa55b9c93a380260f.css
12 Hash c 6dev.dev
12 VCL_return c hash
12 Hit c 980385575
12 VCL_call c hit deliver
12 VCL_call c deliver deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 404
12 TxResponse c Not Found
12 TxHeader c Server: Apache/2.2.15 (Red Hat)
12 TxHeader c X-Powered-By: PHP/5.3.3
12 TxHeader c Cache-Control: public, max-age=1800
12 TxHeader c Last-Modified: Thu, 22 Dec 2011 18:16:50 +0000
12 TxHeader c Expires: Sun, 11 Mar 1984 12:00:00 GMT
12 TxHeader c Vary: Cookie,Accept-Encoding
12 TxHeader c Content-Encoding: gzip
12 TxHeader c Content-Type: text/html; charset=utf-8
12 TxHeader c Content-Length: 6259
12 TxHeader c Accept-Ranges: bytes
12 TxHeader c Date: Thu, 22 Dec 2011 18:20:21 GMT
12 TxHeader c X-Varnish: 980385619 980385575
12 TxHeader c Age: 214
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: keep-alive
12 TxHeader c X-Varnish-Cache: HIT
12 TxHeader c X-Cache-Hits: 5
12 Length c 6259
12 ReqEnd c 980385619 1324578021.208746672 1324578021.209323406 0.009578466 0.000101089 0.000475645
I think the problem is that Varnish already saw that as 404 (X-Varnish-Cache: HIT), so it wont even trying to hit the server to re-check it, yet file exists now and obviously didn't exist before (at time of first check that eventually lead to 404)
alexus.org && alexus.biz