We have a fairly normal multi-server setup: haproxy -> varnish -> drupal
We have drupal using a single memcache instance for all the caches except cache_form (we had 2 for redundancy, but since we're trying to debug this problem we're running it on a single instance).
All the urls that contain '/admin' are excluded from varnish (even though the cookie does that too, we just wanted to make sure this was not the problem).
So what happens is: a lot of the backend forms get cached.
We change some values on the form, click save, the page says success, we navigate somewhere else, come back (clicking, not with the browser arrows) and the old values show up.
The database shows the new values, so the change did get through.
cache_form is stored on database, but I can't find anything there (right now it's happening with 'admin/config/search/path/settings', when I change values for the 'Strings to Remove'.
It's not in varnish either.
My browser cache is also disabled with firebug.
I have no idea where to begin to fix this... any
Comments
The first step would be to
The first step would be to figure out where it's getting incorrectly cached. You can look at the HTTP headers to see if Varnish is serving you a cached page. If the problem is within Drupal it would help to know what the form is (node form? custom module that builds a form with FAPI?)
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Varnish reports MISS, so the
Varnish reports MISS, so the page is not being cached there. The module is pathauto, the 'Settings' page for pathauto, but I've had this problem with other settings pages on the backend...
Do you mean that forms cache
Do you mean that forms cache are stored in a memcached? Or that pages with forms are stored in a varnish cache? Or your forms cache is just doesn't stored anywhere?
Any url starting with
Any url starting with '/admin' is not cached by varnish. It shouldn't anyway, since to access admin pages you need to be logged in and the cookie bypasses varnish cache.
cache_form is stored in DB, as per setup instructions for the module memcache.
This is the relevant part of my settings.php
<?php
# APC ########################################################################
$conf['cache_backends'][] = 'sites/all/modules/apc/drupal_apc_cache.inc';
$conf['cache_class_cache'] = 'DrupalAPCCache';
$conf['cache_class_cache_bootstrap'] = 'DrupalAPCCache';
# MEMCACHE ###################################################################
$conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['memcache_servers'] = array(
'10.0.8.2:11211' => 'default'
);
$conf['memcache_bins'] = array(
'cache' => 'default'
);
?>
Config is correct. So the
Config is correct. So the problem is that you have too much cached forms in {cache_form} table? Am I right?
well, it doesn't seem like
well, it doesn't seem like it:
select count() from cache_form;+----------+
| count() |
+----------+
| 226 |
+----------+
1 row in set (0.00 sec)
The minimum cache lifetime at
The minimum cache lifetime at /admin/config/development/performance is known to cause issues with cache_form. Try setting it to zero to see if it helps. I thought that this bug was fixed, but perhaps you're using an older version of Drupal core.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
I believe I already have that
I believe I already have that set on the settings.php...
// Bypass Drupal bootstrap for anon users so Drupal sets max-age < 0.
$conf['page_cache_invoke_hooks'] = FALSE;
// Make sure that page cache is enabled.
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 600;
As for drupal, I'm running on 7.24
can you tell where the stale data is coming from?
If I understand correctly, the issue is:
1) you navigate to /admin/config/some-module
2) change value X to Y
3) submit form
4) navigate to /admin/config/some-module and value is still X
Does this happen w/ all admin forms? What about the node forms?
Does this happen on your dev and staging boxes or just production?
Does this happen every once and a while or every time?
I assume the values for some-module are stored permanently in the DB. Can you verify that the write succeeds in the DB and that X is changed to Y? Can you check the cache_get? (this might help running code interactively http://www.twinbit.it/en/blog/drupal-and-psysh-drupal-repl apparently phpsh is dead?)
Can you try clearing the layers of the cache independently to see where the hold up is? EG bounce apache, bounce memcache, bounce varnish. Does a page refresh clear the data? If another user logs in, do they see the stale data?
Have you tried visiting the site outside of Varnish and outside of the LB?
Both webheads have the same settings.php file? And you said there is only 1 memcache and 1 db? It doesn't seem like there is a chance of cache backends getting out of sync then or a mysql read delay.
As a last ditch effort, I would try DPMing the conf array in settings.php. I've seen it accidentally get overwritten; so if you spot checked it, it looked good but the output showed the missing info.
About 5 years ago, I had an insanity-inducing issue w/ APC stat=0 and svn. The deployments weren't changing the mtime inode, and I swear APC didn't clear unless we bounced the box. It took a while to figure out...I feel your pain.
Best of luck!
Loudon & Company Consulting
You aren't by chance using
You aren't by chance using strongarm or a similar module and forgot about it? That or have the variables been overridden in $conf somewhere? (might want to grep the code for pathauto_ignore_words and see)
The only other thing I could think of is something screwy in your browser with auto-fill. I have seen stuff like that before, usually from some addon.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.