Memcache module with Drupal 6.20 in a socket situation at WebFaction, and NGINX

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Miko.'s picture

From previous posting to this group the Special Olympics of Northern California and Nevada decided to try Webfaction for several reasons, but two of there were

a) that I was told it had memcache that we could use and also was
b) told that the nginx server would help with serving static files.

Well neither seems possible, unless someone can help. All help is appreciated by myself and the Special Olympics.

First, the memcache issue. I was instructed by WebFaction that we we share memcache with others of the same server, so IP access is a security risk to data stored in memcache. So they instructed me to create a SOCKET in the Drupal directory - memcache(d).sock. So I did that.

The problem is, when I installed the Drupal memcache module, it absolutely demands a port number, and doesn't accept port zero (0) or the non-passing of a port number. Apparently this may be my PECL version and inability to patch it without creating my own php stack and manging that?

Here's my server config line:

'memcache_servers' => array('/home/soncnv/memcached.sock' => 'default')

This is the code that the error refers to:

// Link all the servers to this cluster.
foreach ($memcache_servers as $s => $c) {
if ($c == $cluster) {
list($host, $port) = explode(':', $s);

      // This is a server that belongs to this cluster.
      if (!class_exists('Memcached') && !$init) {
        // If using PECL memcache extension, use ->connect for first server
        <b>if ($memcache->connect($host, $port)) {</b>
          $init = TRUE;
        }
      }
      else {
        if ($memcache->addServer($host, $port) && !$init) {
          $init = TRUE;
        }
      }
    }
  }

So the offended line is:

if ($memcache->connect($host, $port)) {

I tried hardwiring in port zero and removing the port argument here and in the recipient callede function code but it didn't help.

The error when I add a port zero (0):

Warning: Memcache::connect() expects parameter 2 to be long, string given in /home/soncnv/webapps/dru6/sites/all/modules/memcache/dmemcache.inc on line 309 Warning: Memcache::connect() expects parameter 2 to be long, string given in /home/soncnv/webapps/dru6/sites/all/modules/memcache/dmemcache.inc on line 309

Here's the error when I remove the port:

Warning: Memcache::connect() [memcache.connect]: Can't connect to /home/soncnv/memcached.sock:0, Failed to parse address "/home/soncnv/memcached.sock" (0) in/home/soncnv/webapps/dru6/sites/all/modules/memcache/dmemcache.inc on line 309 Warning: Memcache::connect() [memcache.connect]: Can't connect to /home/soncnv/memcached.sock:0, Failed to parse address "/home/soncnv/memcached.sock" (0) in/home/soncnv/webapps/dru6/sites/all/modules/memcache/dmemcache.inc on line 309

I saw online that Bob Vincent had solved this:

On 04/10/11 13:21, miko@mikosweb.com wrote:

pillarsdotnet,

Miko. (http://drupal.org/user/1153870) has sent you a message via your contact form (http://drupal.org/user/36148/contact) at drupal.org.

If you don't want to receive such e-mails, you can change your settings at http://drupal.org/user/36148.

Message:

Hi, I saw that you posted that you are connecting to memcache using memcached.sock with a PORT of "0". I am totally not able to connect this way! On my server, memcached.sock doesn't accept a port specification, I don't think. I will be in a state of shock if you're able to connect this wy! But maybe you can provide me some help somehow. Did you modify dmemcache.inc?

Bob's reply:
I modified both dmemcache.inc and the memcached library that the PHP extension is based on.

Links:

http://drupal.org/node/538426

http://drupal.org/files/issues/socket-connection-dmemcache.inc_.patch

http://pecl.php.net/package-changelog.php?package=memcached&release=2.0.0b1

https://bugs.launchpad.net/libmemcached/+bug/738208

-- Bob Vincent

The problem is that we're at the wrong memcache version I guess. Per WebFaction:

"Our core memcached installation is v1.2.6."

Our PHP memcache PECL extension is v2.2.4. That's http://pecl.php.net/package/memcache
and not http://pecl.php.net/package/memcached.

We've no plans to patch or upgrade any of those components at this time, so if
you need something that's provided by a newer version then you're back to
needing to build a private PHP stack (so that you can install the updated
PECL modules)."

Is anyone familiar with this issue? Is this going to be resolved if I build my own php stack? Or maybe there's a module that works at my level, working around the port issue? Has anyone gotten memcached to work with Drupal, via any module, using a SOCKET with no port, or with a port of ZERO?

If I have to build my own php stack, how do I install memcache into that?

NOW, REGARDING NGINX:

Well that was supposed to work also at Webfaction, but when I set up a symlink to a static-only app and add it as an "app" that functions to replaces "sites/default/files", we get recursion errors.

First of all, has anyone got NGINX to work with Drupal, especially at Webfaction? Or by creating your own symlinks or something? Or could i be that a module like "redirect" or "clean urls" is causing the recursion and we can configure it to avoid the issue? Or maybe even .htaccess with a REWRITE?

Meanwhile, the 20 year old Neerav at WebFaction, smart but oh so young and prone to error, said to me that we need to install something like the Drupal CDN module. The only problem is, even before thinking about how buggy THAT module is- the real problem is that it works with file extensions (e.g. .jpg) as the directive mechanism- but we have css and many things scattered, and really I just want to point to 5 or six FOLDERS and deliver EVERYTHING in them?

But le'ts say we use the CDN module to trick NGINX. It's not the preferred solution- but does it even work? I'm seeing a lot of issue reports tht I don'[t understand. Is there a better way?

Thanks, MIKO

Comments

Oh and which php pecl extension?

Miko.'s picture

If we go with a php stack of our own, if we MUST, which PHP memcache PECL extension is best for Drupal integration?

http://pecl.php.net/package/memcache
or
http://pecl.php.net/package/memcached

??

Thanks Again.

MIKO, Creative Technical Support and Developer
Miko's Support and Design Services, Seattle
http://mikosweb.com

Nginx Anecdote

As If's picture

For several years I managed development on a number of high-profile sites hosted at an expensive dedicated hosting company whose name I will not reveal here. Somewhere between 12 and 18 months ago, I began noticing that all these clients were now running on Nginx - apparently the hosting guys had decided it was the way to go for all Drupal sites and had moved all my clients onto Nginx servers. I don't think my clients even knew that this was done. I only discovered it myself when the sites began crashing and the Nginx error page came up.

The problems started almost immediately. Server load was terribly high, page load time was terribly slow, advanced caching solutions and complex code modifications had to be done to keep these sites from crashing all the time, and calls to technical support were a constant occurence. My clients began bleeding traffic. After months of this crap, they began migrating to new hosts. But before they could all get migrated, the host's network was cracked and wiped out. One client (who unfortunately had her site's DB and backups destroyed) was told that the attack came through a backdoor left by a disgruntled ex-staffer.

I don't know how many of our problems were directly related to Nginx, but the experience put a bad taste in my mouth that I may never get rid of. I don't ever want to touch Nginx again.

UPDATE:

Miko.'s picture

We went ahead and created our own php stack today, are installing APC and will install BOOST. We're hoping that the new phpstack will allow the new socket friendly version of memcached 2.0.0b1 patched by Bob Vincent. We are also hoping that the BOOST with APC will make NGINX unnecessary.

MIKO, Creative Technical Support and Developer
Miko's Support and Design Services, Seattle
http://mikosweb.com

NGINX Anecdote- reply

Miko.'s picture

Notably with NGINX NOT working, somehow CiviCRM import uploads are still being sent to usr/local/nginx temportay buffer area and causing the import to crash, so, Mr. As If, I'm NOT having ANY trouble believing you.

MIKO, Creative Technical Support and Developer
Miko's Support and Design Services, Seattle
http://mikosweb.com

Well

perusio's picture

if you don't know how to configure Nginx, then don't use it. It doesn't hold your hand like Apache does. You need to know how to configure it and understand its logic, very far from the mod_rewrite reverse logic.

Have you checked the said configurations that crashed the sites? If not then please abstain from general remarks regarding a particular piece of software. It just muddies the waters even further and increases the entropy unecessarily.

Thanks

CDN & Memcache

As If's picture

As for CDN & Memcache, I am using them both right now on a network of 5 Drupal sites running partially-shared databases, and it's working very well. Four public sites have their sites/default/files folders mounted to the sites/default/files folder of the 5th site, which is connected to the CDN. Sure, I needed to do some hacking (er, I mean customization), but that was expected. I have no major reservations against using these modules again.

Thank you, that's good to know.

Miko.'s picture

Thank you, that's good to know. I hope you'll let me ask a few questions in I have trouble with recursion or redirects after I try to set up CDN? But we'll only use CDN in we use NGINX and it's not sounding good.

Are you using memcached with a SOCKET (memcached.sock) in any of the instances?

MIKO, Creative Technical Support and Developer
Miko's Support and Design Services, Seattle
http://mikosweb.com

Nope (I don't mind), and nope

As If's picture

Nope (I don't mind), and nope (socket). The network I'm describing sits on 2 dedicated servers.

You do not want to run

vegardx's picture

You do not want to run memcached on an shared environment. You can do some security through obscurity, but if someone just guesses your bin-name they have access to more or less all of your data. Depending on what kind of site you're running, this may be a huge factor to take into account.

The difference between memcache and memcached is just that the latter one is a daemon that can be run stand alone on another server and accessed via network. It's a very effective way to do caching if you have multiple webservers, as they can use the same cache, thus resulting in less overhead in memory usage on all the webservers. But personally I'd wait with memcached until i had real scaling issues, at least slap a reverse caching proxy in front of it first, as that would offload more or less all your anonymous traffic.

I have a site running with nginx to serve static content, now I've moved on to Varnish instead, as I find Apache2 to be much better ducumented, and with varnish infront there is little or no hidden pitfalls, worst case scenario: you send all your traffic to your backend. Properly configured and you'll see a huge performance increase. You can also make all sorts of assumptions, like, if the user tries to access anything in /files it will unset any cookies and cache the content. You can easily pull sub-10k requests per second with varnish, with hardly any load at all, leaving more memory and cpu cycles to other non-cacheable content.

My personal suggestion would be to stick with the platform you feel comfortable with and solve any scaling issues on that end. Nginx will not solve all your problems, it's neglieble. You can use the CDN-module to offload Apache2 on static content, but quite frankly I think Varnish is a better and easier to configure properly choice.

When it comes to nginx security it's a no brainer. More or less all problems related to nginx is PEBCAK-related and has nothing to do with Nginx it self. It's a bloody fast webserver, secure and highly configurable. But that said, it's not that well documented compared to, say, apache2. But if you're familiar with some of the inner workings of things, it should be no problem setting it up and getting it to work.

--
Vegard

memcached security

Miko.'s picture

We are not going to even consider running memcached using a listening port, and a socket has no port- do you see the same bin access security issue with a socket? Webfaction says to use the socket because it's secure?

MIKO, Creative Technical Support and Developer
Miko's Support and Design Services, Seattle
http://mikosweb.com