Drush and Multisite

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Garrett Albright's picture

EDIT: The below applies to Drush version 2. Drush 3's site aliasing makes life a lot easier, as there's a built-in alias which means "all sites." So running a command across all sites in a multisite is as easy as:

drush @sites <Drush command>

You'll be interactively prompted on whether you really want to run the command on all your sites; to automatically skip this (for non-interactive cron run scripts, for example), add a `-y` flag to the command. For example, you can create a root crontab entry which looks like:

5 5,11,15,21 * * * </path/to/php> -f </path/to/drush.php> -- -r </path/to/Drupal/root> @sites core-cron -y

…to run cron four times a day. Much easier! (But note this bug.)

Now back to the original article.

I recently experimented with the Aegir project. It didn't seem to work very well for us, though to be fair it may be due to our nonstandard server setup (FreeBSD instead of Linux; Lighttpd instead of Apache) than any fault with Aegir.

Anyway, installing Aegir meant I had to install Drush. I've been meaning to try Drush one of these days, but it never came about. Of course, now that I have, I'm wishing I would have tried it a long time ago!

One of the most interesting features of Aegir to me was its ability to run cron and database updates across multiple sites, and even keep modules updated. What it's really doing, though, is acting as a wrapper to Drush, which itself is capable of doing this, but to only one site at a time. But it should be possible to do that with the magic of the Unix command line, right? After a bit of experimentation, I found the magic formula…

find <path/to/sites/dir> -depth 1 -type d -not -name all | sed -e 's/.*\///g' | xargs -I foo -t <path/to/drush.php> <Drush command> -r <path/to/root/drupal/dir> -l foo

EDIT: The above ought to work on BSD, but if you're using Linux, you'll probably have to replace -depth 1 with -mindepth 1 -maxdepth 1 as per naught101's comment below.

You should be able to run this without installing any special packages or anything on your system; if it can run Drupal, it can run this. But for God's sake, you're going to back up your site's code and databases before running this, right? Just because it worked for me doesn't mean it's not going to cause three million dollars' worth of fire damage at your datacenter when you run it.

Let's break it down so that you understand what it does. (You're smart and don't just copy-and-paste stuff from web sites into your terminal without understanding what it does… right? Right?) The find command finds files meeting certain criteria; in this case, we're finding the paths of each individual site directory. Its parameters/flags are odd because the file path comes before the flags, and the flags use -flag param format instead of -f param format or --flag="param" format (are there technical terms for those formats?)… Anyway, we want it to find things in our Drupal installation's sites directory, so we pass that as the first parameter. The -depth 1 part tells it not to search any deeper than the sites directory. The -type d part tells it to find directories; other sorts of files (including symlinks) will be excluded. The -not -name all part tells it to exclude the "all" directory, since technically there is no "all" site. (The "default" directory will be included, but that's a good thing, since there is a default site.) If you run just this command by itself, the output will look something like

/path/to/sites/dir/default
/path/to/sites/dir/example.com
/path/to/sites/dir/example.org

That's all well and good, but we only want to pass the last part of each line ("default", "example.com", "example.org") to Drush. So the next part uses the sed program with a regular expression to strip out any slashes, and any characters which are before a slash. sed is a bit of an odd program; it's like a non-interactive text editor which only works by taking text form standard input, running commands on it, and spitting it out to standard out. It's pretty complicated and can do some pretty arcane stuff if you can master it… but there's no shame in just wussing out like me and just using it to run a standard regular expression. Anyway, we now have

default
example.com
example.org

Next is xargs. xargs is another odd command which lets us use stuff from standard in as arguments for other commands. In this case, with the -I foo part, we're telling it to take one line from standard input and plug it in for "foo" where it appears in the command to come. The -t simply tells xargs to print the full command it's creating right before it's running, which aids in debugging and such - you may omit it if you wish. Finally, we specify the path to drush.php and give it a command. (When testing, I recommend using a relatively innocuous command like "status" - this won't change your site's database or files any when run. If you're new to Drush, run the drush.php script without any commands or flags and it'll print out a list of commands and some brief descriptions of what they do.) The -r <path/to/root/drupal/dir> tells Drush where your Drupal directory is, and the -l foo part tells Drush which site to run the command on - remember, "foo" will be replaced with the site. Sticking with our previous examples, Drush will be run three times - once with -l default, once with -l example.com, and finally once with -l example.org.

I've successfully used this to run cron tasks across all of our sites on our new server at once. I haven't tried it with module or core updates yet - mostly because none have appeared for our sites since I figured this out - but I imagine it'll work so long as we switch to root first or sneak in sudo somewhere in there.

Comments

"But for God's sake, you're

btopro's picture

"But for God's sake, you're going to back up your site's code and databases before running this, right?" -- lol, famous last words :)

Cool find, I've been meaning to play around with Drush as well and have just never gotten around to it. Need to try out aegir too as I've written some multi-site management code myself. Nice write up!

"Plaguing the world with Drupal; One Plone, Moodle, Wordpress, Joomla user at a time since 2005." ~ btopro

http://elearning.psu.edu/
http://elearning.psu.edu/projects/
http://elearning.psu.edu/drupalineducation/

Bits of drush came from Aegir

Boris Mann's picture

Aegir development folks ended up moving a bunch of their code into Drush - so they sort of evolved symbiotically. Some Drush code used to be in some of the bits of Aegir.

So, no worries, if Aegir doesn't work for you. In general -- if you are running lots of different sites off a similar code base of modules (e.g. install profile), then Aegir will be a good fit for you. The benefits probably kick in if you know you'll be running 10+ sites.

If you're running lots of custom sites all with different modules, then Aegir may not be the right fit.

Anyway, yes, Drush++ :P

grr.. linux

naught101-gdo's picture

Pretty funky. I was just beginning to look for ways to do this.

I'm on linux, and one problem I hit was that GNU find doesn't take arguments for -depth. I had to use -maxdepth 1 -mindepth 1. Seems a bit stupid really.

Might be worth writing up a script that takes the base dir, and the command you want, and runs this. I'll have a whack at it..

Thanks for that. Since it's

Garrett Albright's picture

Thanks for that. Since it's safe to assume more people are going to be trying to use this on Linux, I'll add a note about this to the article. (I devised this while working on FreeBSD.)

script

naught101-gdo's picture

This works, for any command, with drush_multi [-d base_directory] drush_command. It's a bit ratty - I'm no pro, but it works.

Edited, 2009-06-17 16:45pm UTC: this still needs some work, and commands won't work if they require user input unless you use -y or --yes. Feedback welcome.

#!/bin/bash
#Script for running drush on ALL sites in a multi-site directory. BEWARE.

if ! which drush > /dev/null ; then
  echo "Drush executable not found in your path."
  exit 1
fi

basedir=./

#see if option -d exists
while getopts "d:" o; do
case $o in 
  d) basedir=$OPTARG
esac
done

shift `echo $OPTIND-1 | bc`
commands=$@

cd $basedir

if [ $# -lt 1 ]; then
  echo Usage: drush_multi [-d base_directory] command
  echo eg: drush_multi modulestatus
  echo Run drush --help for more info.
  exit 2
fi

echo -n "Are you absolutely sure you want to run 'drush $@' on ALL your sites in `pwd`/sites/? (y/N): " 
read check
if [ $check != 'yes' -a $check != 'y' ]; then
  echo "Quitting."
  exit 3
fi

#Find all sites dirs. ignore "all", and "default", if it doesn't exist.
if [ -f $basedir/sites/default/setting.php ]; then
  find sites/ -maxdepth 1 -mindepth 1 -type d -o -type l |egrep -v '\ball\b' | \
    sed 's/.*\///g'|xargs -I site_dir -t drush -l site_dir $commands
else
  find sites/ -maxdepth 1 -mindepth 1 -type d -o -type l |egrep -v '\b(default|all)\b' | \
    sed 's/.*\///g'|xargs -I site_dir -t drush -l site_dir $commands
fi

stupid auto formatting

I've updated the article to

Garrett Albright's picture

I've updated the article to reflect improvements made in Drush 3 which make running commands on all sites in a multisite installation much, much easier.

Hmm. But what to do when

Sunshiney's picture

Hmm. But what to do when sites in sites dir are not given the domain name? We give local dev and hosted-elsewhere production site the same name, e.g. water. A D6 patch creates an array that enables the prod site to point to water.com and local to water.local No file changes in database are needed when file transfers occur. This patch will be in 7, I am told. Looking now for best method to run cron.php for all sites in a multi install.

I don't think I entirely

Garrett Albright's picture

I don't think I entirely understand what you're asking, but to answer your last sentence… well, see above.

Have you tried the @sites trick with Drush 3 and not had it work for you?

@Garrett -- My php knowledge

Sunshiney's picture

@Garrett -- My php knowledge is quite limited. However, my understanding of the script above is that it goes to the sites directory and "picks up" the name of the sites folder (e.g. clouds.com) in order to run http://clouds.com/cron.php

If my understanding is correct, then I'm concluding intuitively that it won't work for us and I'm still searching for a multisite solution.

The reason is that instead of:
/path/to/sites/dir/example.com
/path/to/sites/dir/example.org

We have:
/path/to/sites/dir/example
/path/to/sites/dir/clouds

That works with the patch I referred to in my OP, which is here: http://drupal.org/node/231298#comment-1420180 That patch is part of Drupal 7

If by "the script above" you

Garrett Albright's picture

If by "the script above" you mean Drush, then no, it doesn't go into the sites directory, though it does need to be able to find it. That's accomplished by either using a command-line parameter to point to it directly, or by cd-ing into your Drupal installation's directory before running Drush.

As for not working for you… I'm not sure, but it might. I don't think the directories in your "sites" directory need to be actual domain names in order for Drush to work (mainly because I don't think there's any smart way for Drush to intuit that, nor does it need to). Before writing it off, give it a try.