Multi-server configuration lessons learned

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Anonymous's picture

Thought I would share some useful info (before I ask for support help) on a few lessons I have learned for a Aegir multi-server configuration I have running at Linode (IMO the only Linux VPS for hardcore Drupal deployment). My ultimate goal is to be able deploy Drupal sites in a HA environment without having a complicated mess of more than 5 servers and running services like clustered MySQL.
The main problems I encountered in getting a multi-server configuration to the point where I could deploy a site and migrate it around to different nodes were;

  • When setting up node server in hosting web interface, SQL errors with host resolution (lesson 1)
  • When setting up node server in hosting web interface, SQL login error from host not allowed (lesson 2)
  • When setting up node server in hosting web interface, errors out due to ssh not working between servers (lesson 3)
  • When deploying a site to a node server (including Open Atrium) it would error out (lesson 4,5,6)
  • I have not quite figured out cluster servers yet (asking that question over in the support forum)

My setup:

  • Installed Aegir version 04-alpha12 on Ubuntu 10.04 64bit and have two other node servers built per the multi-server section in doc wiki.
  • Built server_master (hosting master) first with Linode Stackscript (https://www.linode.com/stackscripts/view/?StackScriptID=1027).
  • Built other two servers with LAMP.
    Server-1 Master (LAMP)
    Server-2 node member (LAMP)
    Server-3 node member (LAMP)
  • All servers have same aegir user and exact directory structure. Aegir home at /var/aegir/

  • Lesson 1: Add host names to /etc/hosts/ before installing LAMP and make sure the hostname of each server is in the hosts file

  • Lesson 2: Grant the correct permissions, on each node servers MySQL, for the MySQL user that will be used from Server-1 (user server-1@”%”)

  • Lesson 3: When setting-up ssh follow this simple process (main lesson here is to accept defaults when generating the key and to ssh with the aegir account into the node server, from Server-1, before adding it as a server in the hosting web interface).
    1- Switch to the aegir user (su aegir) from root login
    2- Run command ssh-keygen –t dsa on Server-1 (accept all defaults by hitting enter until done)
    3- Copy pub key to Server-2 and 3 (scp id_dsa.pub root@server-1:/tmp
    4- Login to Server-2 and 3 follow-steps 1 and 2 or just create a .ssh directory in aegir home
    5- Change to /tmp on Server-2 and 3 and run command “cat id_dsa.pub ~/.ssh/authorized_keys”
    6- It is not really necessary to copy the pub from Server-2 and 3 to Server-1, but will not hurt if you do so

  • Lesson 4: Increase memory for PHP and PHP-CLI to at least 96M (/etc/php/apache2/php.ini) and (/etc/php/cli/php.ini)

  • Lesson 5: make sure the directory structure, and platform, is same for each node server and Server-1 before creating a site that will live on a node server. What I am basically saying here is that even though you are using a node server as the platform (choosing Server-1 as database and node server as web) you still need the same platform on both servers.

  • Lesson 6: The only way I could get large install profiles like Open Atrium to install on a sever node was to:
    1- Build the site completely on Server-1
    2- Verify the site
    3- Migrate the site to the node server

If you have any useful info to share on how to build and run a multi-server configuration that allows one to have the same copy of a site running on multiple web servers and near-time database replication, please comment here in the clearest of layman terms you can.

Peace,
Michael Clendening
(the older I get…the more I just don’t know)

Comments

your use of acl

bwood's picture

100       if [ "$SETUP_ACL" = "Yes" ]
101     then
102            # Using ACL setup guide here: http://www.debianhelp.co.uk/acl.htm
103           apt-get install acl
104         mount -o remount,acl /dev/xvda
105          sed -i'-orig' 's/noatime,errors=remount-ro/acl,noatime,errors=remount-ro/' /etc/fstab
106   
107           # add Aegir group ACL permissions
108           setfacl -R -m g:aegir:rwx /var/aegir
109            setfacl -R -m d:g:aegir:rwx /var/aegir
110      else
111            echo "Skipping ACL setup because of user preference."
112         echo "setup_acl : $SETUP_ACL"
113     fi

I'm curious about why you're using setfacl here. (I didn't know about this utility and was just looking at the man page...) Does acl ensure that new files directories get create with some version of the permissions on the parent directory? If not, could you accomplish the same thing you are doing above with chown/chmod? Are you setting up users in the group aegir who need to create files under the aegir platform directories?

-b flag in Install.txt

timwood's picture

From the latest Install.txt:

Also note that those instructions setup a complete Aegir system. If you want to only setup a new server, it should be sufficient to install requirement (step 1) and configure them (step 2). You will just need the -b flag to avoid installing the frontend on the server.

The -b flag is mentioned, but I don't see what command this applies to. The install.sh script has no reference to a -b flag and appears to just blindly install the frontend on the second webserver at the very end of the script. Isn't there an automated way to install the necessary components onto a second webserver then this Wiki page: http://groups.drupal.org/node/21966?

Thanks,
-Tim

$@

bwood's picture

from install.sh.txt:

msg "Installing the frontend"
$DRUSH hostmaster-install --aegir_host=$AEGIR_HOST --aegir_db_user=$AEGIR_DB_USER --aegir_db_pass=$AEGIR_DB_PASS --version=$AEGIR_VERSION $@

so if you pass -b to install.sh.txt it will append -b to the hostmaster-install drush command.

If you look at .drush/provision/install.hostmaster.inc you can see the --backend-only option. I believe -b is the short version of that.

Thanks!

timwood's picture

Thanks bwood. I guess I should learn more about shell scripting.

So in your documentation here: http://groups.drupal.org/node/21966, I could just skip all the manual steps of installing Drush and Provision and just run the install script with the -b option?

Yes

bwood's picture

Yes that is supposed to work. I have have been running 'drush hostmaster-install --backend-only' (with the appropriate arguments on my test machines so I haven't tried this via install.sh.

(Just made a quick edit to the old page you referred to.)

there is no automatic way

adrian's picture

you dont install drush or provision on the second webserver any more.

adrian to the rescue!

timwood's picture

Hope you are enjoying your vacation! If you have a moment, would you point me to the proper instructions for a second host? I believe the install.txt is still only for single-server.

Thanks,
-Tim

See docs/INSTALL.txt, there

omega8cc's picture

See docs/INSTALL.txt, there is a "Remote web servers" basic how-to.

Link: http://git.aegirproject.org/?p=provision.git;a=blob;f=docs/INSTALL.txt;h...

version control?

John_Buehrer's picture

Asking ironically, is there a way to version-control this Install doc, in the sense that if I look at a printed copy (or PDF) I can tell its version number and publication date? Sure, the git stuff works for check-in and check-out, but I've seen already several URLs to different versions of "install.txt" around here and there's no way to tell what's most recent. Help! Can you organize putting a version number & timestamp into the content itself? Thanks.

Another goody would be clarification, what means "HINT_* files alongside this document". It's not obvious (to me, anyway) what the alongside-URLs would be. Thanks.

For now

timwood's picture

For now you should be able to tell the corresponding version of Install.txt by looking at the wget (download) line for Install.sh in section "3. Run the install script". If it refers to "hb=HEAD" then it's the HEAD version, if it has a version number it will look like "hb=provision-0.4-alpha12".

Good idea though. Should be noted at the top of the file or something.

-Tim

Huh?

John_Buehrer's picture

Let's say I start from the official docs page:

http://groups.drupal.org/aegir-hosting-system/documentation

Not finding a resemblance to installation info, I finally click the link "Visit Generic, the canonical INSTALL.txt instructions".

Now, how am I supposed to figure out whether this is heads or tails or yesterday's or last year's instructions? I'm trying to follow instructions here on the Drupal site, rather than from wget output, else there's a chicken-and-egg problem...

Apparently there are multiple URLs within Drupal.org & Google searches, pointing to different versions of this file. So a mechanism to keep this sorted out seems more like a requirement than just a good idea. How do the rest of you avoid confusion?

I agree with it being a

redpuma's picture

I agree with it being a requirement too.

adrian's picture

basically all the parts in install.txt up to BEFORE install.sh.txt gets involved. you do not download install.sh or run it on the external web server.

ie:
1) you install the packages
2) make symlink to /etc/apache2/conf.d
3) create aegir user
4) add aegir user to visudo for apache2ctl
5) add aegir user to the www-data group.

above that, you also change the aegir user shell using chsh so they can be logged into and you put the ssh public key from the hostmaster server onto the aegir user's .ssh/authorized_keys on the external web server.

you do not install drush, provision or any other parts of aegir onto the server.

I followed this section of Install.txt but...

timwood's picture

...it was missing instructions for setting up the remote databases user and permissions. I am familiar with how to do this, so I did it manually, but it seems like the steps should be documented in Install.txt or a separate document, or part of an install script for remote hosts, similar to the master install.sh.txt. It could prompt for whether you wanted the host to be a web or db or both and take appropriate actions.

Section "Database configuration" of Install.txt touches on this, but then refers to "The installer will prompt you for your MySQL root user password", which isn't the case for remote hosts since you typically won't be running the installer.

-Tim

One more thing

timwood's picture

One more thing to followup on. Could someone explain how the Master Aegir instance goes about running the cron process for all the remote sites? I've enabled the Cron Queue feature in my Aegir and my Master host has a crontab entry. Is that all that is needed?

Thanks,
-Tim

Getting stuck on symlink

3rdLOF's picture

Hello dear sages:

I got everything correct except this symlink part. I am truly confused by it.

Install.txt says:

"In Debian-based systems, you should symlink this file inside
/etc/apache2/conf.d

Doing ln -s /var/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf

throws off an error upon restarting Apache because....the files does not exist. (there is just /var/aegir)

Since we are not installing Aegir on the remote, then how does this file will exist? Am I missing something?

Don't restart apache, yet

timwood's picture

I believe the key is to not restart apache after making the symlink. Apache should be restarted later, by Aegir, once you've moved a platform/site to the remote server. At this point Aegir will have copied, using ssh, the necessary Apache configuration files to that path on the remote host and the symlink will now be valid.

The parts about adding the Aegir user, home directory in /var/aegir and ssh keys to the remote host enabled this to work eventhough the Aegir front-end or back-end is not actually installed on the remote host.

The following two excellent posts by eft might help visualize this:

http://groups.drupal.org/node/93849
http://groups.drupal.org/node/93854

THank you so very much. I had

3rdLOF's picture

THank you so very much. I had a feeling that was the case.

No password

that0n3guy's picture

Just to make sure, you have to leave the sshkey password blank, right? There isn't a way to do it with the password, correct?

Re: No password

skwashd's picture

You can run ssh-agent as the same user who is running drush and that will unlock the key for you. This is a more secure option, and you only need to type the passphrase once per session to unlock the key.

keychain and .ssh/authorized_keys

pearcec's picture

Running keychain would be a good method to having a passphrase on a key that gets unlocked one per reboot. Another protection would be to limit remote execution of commands to drush on the remote servers. Provided all the commands are running through drush on the remote server.

--
Christian

ssh-agent?

john.karahalis's picture

I'm still confused by this.

It seems that ssh-agent / ssh-add (or keychain) would work if you were logging into the shell of the aegir user directly, but how can you set this up so that the private key would be added in such a way that the Aegir website could use it?

Aegir hosting system

Group organizers

Group categories

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week