Hostmaster2

A guided tour through the hostmaster installation wizard

adrian — Fri, 27 Jun 2008 19:01:28 +0000

I just spent some time documenting the new installation wizard for hostmaster 2, and annotating it on this hostmaster wizard photoset on flickr.

Example screenshot :

Hostmaster2

New hostmaster feature : Queue Dispatcher

adrian — Fri, 27 Jun 2008 16:28:54 +0000

I've built a new API for use in hm2, which allows you to define queues that would like to get triggered. I've built the api so that defining 2 functions hook_hosting_queues and then the queue handler, gives you an impressive amount of free process accounting, and also configurability.

What the queue dispatcher does, is it keeps a list of queues, when last they were triggered (dispatched), how frequently they need
to dispatch, how long have they taken (are they hung?) and exactly what they are busy with. The primary example of a queue
is the task queue, which translates all changes on the front ends, down to the back ends.

Additionally, the dispatcher supports two types of queues, your typical serial queue (process x items every y seconds), but also
the far more complex batching type of queue. Batch queues are like the cron queue. You need to ensure that all your sites are
cronned every hour for instance, but it is unfeasible to try and cron all your sites every hour at a certain time.

What batching does, is it knows how many items it needs to process, and in what amount of time it needs to do so. It balances the amount
of items being processed in one instance, until the amount of items per instance become too great, then they add another instance.

As an example, if you have 100 sites that need to be cronned in an hour, it will cron 20 sites every 10 minutes, but once the number of sites per cron reaches 100, it will add another instance, until it reaches it's maximum amount of instances per interval. At 1000 sites,
it will have grown to do 100 sites every 5 minutes, keeping the load even the whole time.

This will allow you the flexibility of configuring how frequently sites are backed up, statistics gathered, system monitoring take place,
and the api handles configuration, error handling and all the rest.

It could even be possible (though most php internals people i've spoken to don't recommend it) to run the dispatcher as a daemon,
for times when you need quick turnover of tasks -> sites.

Hostmaster2

Hostmaster 2 0.1 alpha2 released.

adrian — Fri, 27 Jun 2008 16:19:44 +0000

I've just published the 0.1-alpha2 release of hm2, which features extensive changes from the first alpha release 3 weeks ago.
We are now nearly feature complete for the 0.1 release, but there's still a fair amount of loose ends to clean up.

Release announcements:

Hostmaster : http://drupal.org/node/275709
Provision : http://drupal.org/node/275710
Hosting : http://drupal.org/node/275708

Single Download:

For your convenience, I have packaged up the profile and all it's dependencies which you can download here : Hostmaster 0.1 alpha2

Extract this tarball into your profiles directory, and create a new site (not in the default directory) using the profile.
Remember, you need your own server and a unix based os to run hm2, and this is not production ready, so please don't
use it for anything critical yet.

This package includes the hosting and provision modules, as well as Drush 1.3, Cvs Deploy 1.1 and Views 1.6.

Hostmaster2

Managing permissions on a shared Drupal hosting

anarcat@drupal.org — Fri, 06 Jun 2008 19:12:47 +0000

I thought a bit about how to fix the problems regarding shared drupal hosting in the HM2 paradigm. I think I have a good solution and would like to have feedback on it before it gets implemented in full in HM2. But first things first...

The problem(s)

The way web hosting traditionally work is that all files are readable by the webserver and are thrown on the wire when the web client requests it. That's all fine and dandy when you're kernel.org and want your files out as much as possible, but when you start talking about Drupal, things get more complicated.

Read permissions

First, you don't want anyone to read your files. Not only the web client, but the other web applications shouldn't be able to read your (e.g.) settings.php file because that would mean they have read/write access to your database, which bypasses any access control you might have put on your nice Drupal website.

So basically, out of the box, LAMP hosting opens this can of worm

Write permissions

To support file uploads and other "fancy" things, the webserver needs to be able to write to some (and only some) parts of the filesystem. This is generally accomplished by allowing everyone (or just the webserver) write access to the files/ directory (wherever it is is mostly irrelevant here).

This creates an issue similar as the read permission issue: every other website can write to your files now.

Known solutions

Every shared hosting platform out there has their solution. I'll try to go around a few I know of.

open_basedir

open_basedir is a PHP configuration setting that restricts access to certain parts of the filesystem only. The "home directory" of the "user" is usually the only place where the scripts should have access to. All PHP function calls are then restricted to that directory.

The main issue with that approach is that it doesn't really work properly. Without safe_mode (see below), it's fairly trivial to workaround the basedir restrictions using system() calls, for example. Also, this approach assumes that all potential PHP extensions (curl, imap, etc...) respect this functionality, which is very often not the case.

This theorically resolves the read and write issues.

safe_mode

When safe_mode is on, PHP checks to see if the owner of the current script matches the owner of the file to be operated on by a file function or its directory. Some hosting providers set the gid or the uid of hosted files to keep seperate website to access each other's file. This can easily be worked around by using system calls, which is why safe_mode also comes with a safe_mode_exec_dir directive that restricts allowed executables to those in a directory.

Safe_mode has similar issues has the open_basedir directive in that it is wildly misimplemented through the PHP codebase and extensions. Relying on it for security is considered bad practice and the extension will be removed in PHP6.

This theorically resolves the read and write permission issues.

environment variables

In a controled environment like HM, it's possible to set environment variables from the Apache configuration that is read as root by apache and can therefore be hidden from the www-data user. Those variables are then read by a modified settings.php to setup the database connection.

That resolves the read issue, but not the write issue.

suexecphp or "real UNIX permissions"

This is, I hope, the "proper" solution to the problem. This implies using Apache's suexec support in conjunction with PHP's CGI mode (optionnally running with FastCGI for better performance) to make the webserver run as a "real user".

This is how I think it should be implemented:

each site has a "user" assigned to it (say "userA")
each hostmaster "client" is actually a "group" (say "groupA")
the PHP environment runs in a userA:groupA environment (as opposed to www-data:www-data)
the settings.php file is mode 600 (read/write only by the user: that takes care of the read problem)
uploaded files can be similarly protected from writes
the rest of the files in Drupal are owned by the hostmaster user and group, unless we want to give access to themes and modules to the "user"

I think this reliably resolves all of the issues documented here, at the cost of some issues:

performance cost: FastCGI is slower than mod_php, especially if it needs to have seperate users for seperate domains
additionnal complications in the Apache setup: a configuration file needs to be created for each virtual domain (which we already do anyways, but some hosting platforms don't need that)
a file manager gets hard to implement in an eventual control panel (that would run as the hostmaster user). The only solution I could find here was to use some web2FTP gateway: the file browser just becomes an FTP client. Note that in the context of the "general shared hosting solution", the files could be owned by a user that would be the same username as the group and would therefore be a different user than the apache user, enhancing security...

Note that this approach also has advantages from a generic shared hosting point of view. "users" have various privileges: FTP access, SSH access, Drupal site, email, all in a central database with ACLs to control which user has which accesses. "groups" become the "control panel user": the group itself is the login/password to the control panel and can create users, only within its own group of course.

Note finally that "suexec" is just one way to make sure that Apache runs as the proper user. There are other ways, namely mpm-itk which allows the use of mod_php...

It's a bit rough for now, but you probably get the idea. I'm interested in getting comments, feedbacks, flames, similar experiences, edit this page!

Comments? Questions?

Hostmaster2

Ubercart -- HostMaster2 integration notes

EclipseGc@drupal.org — Fri, 06 Jun 2008 17:38:58 +0000

This will be the page to describe potential ways of integrating ubercart for HostMaster2's billing/sales system. I'll update this page with my thoughts as I experiment.

Eclipse

Hostmaster2

HostMaster Easy One Page Install

Boris Mann — Fri, 06 Jun 2008 16:51:53 +0000

Please share your steps on installing and setting up HostMaster and related dependencies (Apache, MySQL, cron, etc.).

Dedicated Server

TODO

On your Mac Laptop

For download, feel free to substitute CVS checkout

Download Drupal 5 into your root web server docroot (e.g. /Library/WebServer/Documents)
Download the hostmaster install profile into the profiles folder, so that it is at /profiles/hostmaster
Download required modules: views, drush, ...
Create a new directory in the /sites folder with a name equivalent to your local machine -- this can be either 127.0.0.1, localhost, or your machine name (in my case, sleipnir.local) -- and copy over the settings.php file; NOTE: currently, the main hostmaster site can't be run out of the "default" folder
Navigate to the new site created in the last step by going to it in your local web browser -- e.g. http://sleipnir.local
You should now see the profile selection install screen -- select the Hostmaster option and install
Once installed, run through the wizard
The only snag you should hit is when it asks you what user/group to use for running the system; the default should show your currently logged in username (e.g. 'bmann'), and the help instructions explain how to use vigr to add that user to a group. This does not work on Mac OS X. This page explains dscl, which is what is used on OS X. On Leopard, you'll need to run the following command: sudo dscl . -append /Groups/_www GroupMembership bmann Alternately, you can also follow instructions on that page to ccreate a brand new 'hm' user and add that user to the '_www' group.

Additional notes

I ended up using our Bryght Basic Drupal 5 SVN checkout for my D5 base install -- since we backported watchdog you'll need to edit the hostmaster.profile to comment out watchdog as a required module

Hostmaster2

Hostmaster Roadmap

puregin — Fri, 06 Jun 2008 10:55:39 +0000

This is a brief outline of development plans for HM2 over the next few weeks. Planning at the weekly level will happen in the #hm2 room for irc.freenode.net, Fridays at 9:00AM PDT.

Now that the first release is out, we will be revving pretty rapidly to get the bug fixes in, and to make this more usable.

0.1 - Single platform

Solid release for Drupal 5 platform.
Will not be able to upgrade.
Production ready.

0.2 - Multiple platforms

Support for more than one Drupal release.
Initially just 5.x to 5.x upgrades, but after this release, we will be working on porting provision
to D6, which will allow major version upgrades. (as part of the 0.2 release).

0.3 - Multiple servers

Support ssh as communication channel between multiple servers.
Move sites between servers, and support for multiple db servers
and moving between them.

0.4 - User interface improvements

Clean up and make user interface friendlier.

0.5 - Drupal 6 release of front end. [tentative]

Port hosting to drupal 6, and implement complete views support.

As hosting is based on D5 atm, we have only implemented the
bare minimum of fields we need to create our default views.

Re-doing this for views2 would not be effective use of our time,
so we're waiting for a d6 port before fully supporting views.

1.0 - Frozen API.

A stable, published API for Hostmaster modules that will be frozen during the whole lifetime of the 1.0 branch

From there on...

Extension modules

There is also a host (pun intended) of provisioning tasks that it would be nice to see created:

LDAP (open LDAP)
Solr Search (Tomcat)
Jabber (ejabberd)
DNS (Bind -- Best done as hooks to bind Module?)
Pound (load balancer)
Email (Zimbra)
An RDF triple store (needs more research)
Asterisk (voicemail, proxy)

Mass site operation modules

Search (could be related to Solr)
Statistics (info on nodes/comments/users etc. across all / many sites)
Logging (centralized logging overview, might just be web interface for syslog)
Spam / Comment moderation (we had some discussion about writing a Mollom module that works across sites)

Feel free to ping us if you are interested in pitching in to work on one or more aspects of HM2.

Note that some actions were isolated in the last meeting.

Hostmaster2

Weekly hostmaster2 development meeting

adrian — Mon, 02 Jun 2008 19:48:56 +0000

We have a weekly development meeting on #hm2 on irc.freenode.net, to discuss development, and make our weekly release.

If you are interested in hostmaster 2, please join us.

http://www.timeanddate.com/worldclock/meetingdetails.html?year=2008&mont...

Hostmaster2

Hostmaster 2 now tracks packages installed on system

adrian — Wed, 26 Mar 2008 04:27:28 +0000

Just a quick note to let you guys know I implemented functionality whereby hostmaster finds all the modules / install profiles on any
platform it manages, and builds 'package' and 'package release' nodes out of them. These will be linked to sites through the package_instance junction table, so you will be able to see exactly what modules / themes are on the system, what version they are
and eventually whether they are enabled or not.

All this is exported to views, so you can build views that do things like show you all the sites with a specific version of a certain module installed (like when a security release becomes available , to see which sites are vulnerable).

The major thing wrong with it still, is i need to integrate cvs_deploy into the back end, as it has issues getting the package versions atm.

Also, I plan to integrate to drupal.org when http://drupal.org/node/157514 hits, I will be generating packages and releases directly from d.o, and the system will be able to much more easily notify you of wth is going on on your sites.

For security sake I will eventually keep track of checksums of all the files in projects to monitor when things have been changed.

I also added arguments for the embedded views, so they are working again now.

Hostmaster2

Hostmaster - Project Goals

adrian — Mon, 24 Mar 2008 16:31:20 +0000

Open

This project is developed on drupal.org, using the tools provided by the community. After many discussions with many different developers working in this space, it has become apparent to me that many people have developed their own custom set of utilities and scripts to manage many of the issues this project encompasses.

This project aims to be an open framework that will allow us to collaborate on a single set of community developed / maintained utilities, so that the quality and the capabilities will improve for all of us.

Easy

This functions on both a developer and a user level.

For the user, the intent is to make the system as simple to install and operate as possible, which means drastically lowering the requirements for installing and getting up and running with the system. To that end the system has an installation wizard that steps you through everything from registering your first account, to configuring and verifying your server, to importing any sites that may already exist.

The system will auto-detect as many of the settings it requires to run as is possible from within the confines of PHP, and will provide detailed and accurate inline documentation on the server configuration required, based on the configured settings.

This system aims to be self documenting, to aid in troubleshooting, in the event that an error occurs. This documentation is both translatable and extendable when new services are enabled.

For developers we aim to keep the implementation simple, clear and concise, and use as few as possible additional requirements outside of Drupal itself. We aim to provide extensive developer documentation, to lower the barrier for people wanting to integrate with the system.

The system will exist entirely in contrib, and will never require any additional core patches for the base functionality.

Basically, do everything possible to ensure that the user and developer experience is a positive one.

Useful

The system aims to help maintain many sites, not just when they are initially installed, but over the entire lifetime of the site.

It is designed to install new sites, provisioning all the required services (such as Apache, Mysql and DNS), with the flexibility to provision additional services (such as Mail, LDAP etc.) through optional contributed modules.

It provides the ability to revert back to previous versions of your sites, as well as re-deploy existing sites under different domain names (mysite.com copied to dev.mysite.com). These backups will be usable on any system with the provisioning back end installed (such as a developer's machine, or a staging server)

This aspect of the system will also be used to manage upgrades between multiple versions of Drupal, and will allow you to schedule upgrades. It will do it's utmost to ensure that your site has been successfully upgraded, and in future versions will feature dependency tracking to keep track of which sites are eligible to be upgraded.

It will provide a mechanism that ensures the cron process is run on all your sites within the specified time frame, and additionally will allow for scheduled backups of all your sites.

Through optional modules it will be able to extract usage statistics from your hosted sites and provide an overview of site usage through the administration interface, it will also be able to keep track of server health and load for each of the configured servers.

These are the stated feature goals, many of these features will not be available in the initial releases, as they are not considered a base requirement.

Secure

The system will make sure that it has the least amount of power available to it to accomplish it's task, and will ensure that all file permissions are set to the strictest possible levels they need to be, and that no sensitive information is web accessible.

None of the functionality or code required to accomplish it's tasks will ever be available in the provisioned sites, and the hosting front end itself will make extensive usage of Drupal's access control requirements to ensure that users will not have access to things they are not meant to see.

to be extended

Distributed

The system functions entirely on the command line, using unix pipes for inter-process communication.

As such, once ssh has been configured for the user the back end script is running as, it will be able to freely communicate with the external server through the only mechanism it knows how (the command line).

As an extension of the backup/restore functionality, you will be able to seamlessly move sites between servers, while providing a redirect to an always accessible site alias for users who have cached dns entries.

You will also be able to change database servers through simply modifying the site record on the user front end.

to be extended

Diagnostics

The system will provide accurate and complete logs of everything that has occurred on the system, as well as feature complete error reporting and notification of all issues that have occurred. It will first verify that it is able to complete the requested task, and then it will verify that the task has been completed successfully, before completing the task.

When an error occurs, it will rollback to it's last working state before exiting, so that failures will not negatively
affect sites.

All node types in the front end are revision controlled, to allow for complete history of all objects.

to be extended

Flexible

This system aims to be as flexible as Drupal itself is.

It will provide a complete and useful set of hooks to module developers, for both the front end and the back end, to accomplish whatever tasks they need.

The administration interface will provide complete views support for all data in the site, to allow for custom reporting on your network.

The core node types are all standard Drupal nodes, that can be extended through CCK by the implementor. This could be used for tracking additional client information or additional information on sites.

You will be able to integrate Drupal contributed modules such as Organic Groups or Ecommerce/Ubercart to build support or billing functionality directly into the hosting front end.

The system is completely white boxed, so that it will work with almost any Drupal theme, and as such can have your own corporate identity.

to be extended

Hostmaster2

Hostmaster - Design and Terminology

adrian — Sun, 23 Mar 2008 21:54:27 +0000

This page documents all the different terms used when referring to components of the Hostmaster system, and how the different entities relate to each other.

Front End

The user interface used to administrate your sites. The front end is provided by the Hostmaster install profile, and the Hosting contributed module. It defines a complete Drupal based data model for managing the various aspects of your installation.

Entities

These entities are used primarily in the front end to manage the configuration and data. Upon calling the back end, the system breaks
down the properties of these entities to be passed as options for the command line, so the back end only has a flat data structure to
work with. During this process, all relationships are automatically retrieved, making it a one step process for the developer.

Client: The person or group that runs the site.
This information is usually required for billing and access purposes, to assure
that only certain people are able to view the information for sites they run.
If you do not intend on having more than one client access the system,
you will not need to create any additional clients for your purposes.
Site: An instance of a hosted site.
It contains information relating to the site, most notably the domain name, database server
and platform it is being published on. A site may also have several aliases for additional
domains the site needs to be accessible on.
Task: The mechanism whereby Hostmaster keeps track of all changes that occurr to the system.
Each task acts as a command for the back end, and contains a full log of all changes that have occurred.
If an task should fail, the administrator will be notified with an explanation of exactly what went wrong,
and how to fix it.
Platform: The file system location on a specific web server on which to publish sites.
Multiple platforms can co-exist on the same web server, and need to do so for
upgrades to be managed, as this is accomplished by moving the site a platform
hosting an updated release.
Platforms are most commonly built for specific releases of Drupal.
Web server: The physical machine where files will be stored for publication.
Each web server hosts one or more platforms, which act as publishing points for the hosted sites.
If you are not intending to use Hostmaster in a distributed fashion, you will not need to create
additional web servers for your purposes..
Database server: The database server on which sites will host their date.
Most web servers and database servers are on the same machine, but for performance reasons
external database servers might be required. It is not uncommon for one database server
to be shared amongst all site instances.
If you are not intending to use an external database server, or multiple database servers, you
will not need to create any additional database servers for your purposes.
Package: An installable component for a site.
Hostmaster keeps tracks of all the modules, themes and install profiles installed across your entire network.
These packages most commonly link to their project on drupal.org, but might not in the case of custom modules.
Users are not able to create nodes of this type, as they are generated automatically during verify and sync tasks.
Package Release: The release of a specific package.
Each package on your network has at least one installed release, and each platform (including Drupal itself) may
have several releases installed across your network.
Package Instance: Where a package release has been installed.
A package release may be installed in multiple places on a platform, and sites may have access to multiple versions of packages.
This entity also tracks which modules are enabled, and what version of the schema (if any) they have installed.

Task types

These are "hosting commands" that the server can accomplish. These are mapped to back end commands by the task queue processor.

Verify platform: Verify that a platform is correctly installed. Automatically created when new platforms are added, to ensure they are capable of installing new sites on. Operates on: platforms
Import sites: Import existing sites on platform. Operates on: platforms
Install site: Install a new site. This is automatically generated when a site node is created. Operates on: sites
Sync site: Synchronize changes to the site record with the back end. Automatically generated on editing of the site node. Operates on: sites
Backup site: Generate a backup of an existing site. This backup contains everything needed to run the site. Operates on: sites
Restore backup: Restore a site to a previous backed up version. Operates on: sites
Disable site: Disable a running site. This is commonly used in the case of non-payment from clients. Operates on: sites
Enable site: Enable a disabled site. The opposite of Disable. Operates on: sites

This is an incomplete list, and will be updated as more tasks are added

Queues

The front end uses and maintains several queues, which are then processed through the drush.php hosting dispatch command.
More queues can be added by optional contributed modules, and are often used to provide regularly scheduled back end commands,
such as backing up sites.

Task Queue: The primary and most important queue. Whenever a change is made to the data set, the front end creates a "Task" node. These task relate to such things as installing new sites, regenerating the configuration files of sites, verifying that a platform has been correctly configured and importing existing sites on a newly added platform.
Cron Queue: This queue is used to manage the regular execution of the cron process required by most sites. It is configurable with a frequency you want all the sites to be cronned with in . This is commonly 1 hour, but might be higher or lower. higher frequency will cause higher system load. This command will iterate through all enabled sites, and batch the cron calls, based on how many sites are available, and how frequently it is running.
Backup Queue: This queue operates in the same way as the cron queue, in that it iterates through all installed sites, based on a configurable frequency. This executes the same functionality as requesting a site backup through the user interface. You are able to revert back
to a previous backup of your sites.
Statistics Queue: This queue operates on the same mechanism as the cron queue, and alows you to retrieve statistics of your installed sites. This will retrieve such metrics as number of registered and active users, and number of posts / comments, which will then be viewable on the front end.

Back End

The back end is a command line interface provided by the Provisioning Framework, of all the tasks it is capable of performing.

It is designed to operate separately from the front end, so that you can have multiple back ends on one or more servers.

The back end handles the server level configuration of the system, such as creating new databases and virtual host records and
restarting the apache process for new installations to take effect.

When the queues are processed by the front end, these are turned into calls to the back end, for whichever server they happen
to be on. The back end communicates to the front end through a very simple mechanism, whereby it returns a serialized string containing state information and log messages / error return codes.

Hostmaster2

Hostmaster - Overview

adrian — Sun, 23 Mar 2008 02:30:40 +0000

What is Hostmaster?

Hostmaster is a distributed instance management system, that allows authorized users to administer multiple Drupal sites across one or more servers, through a Drupal based administration front end. It is designed to help solve many of the common problems that administrators come across when providing mass hosting solutions, by providing the basis for turn-key deployment and management of Drupal sites and related services.

What do I need to run Hostmaster?

The primary requirement for being able to run the Hostmaster is having full access to the machine you intend to run Hostmaster on. Due to the types of access and configuration required, Hostmaster will not run in a shared hosting environment. A Unix environment is recommended, as many features might not available on a Windows based host.

Furthermore, because Hostmaster is Drupal based, it requires that you meet the requirements for running Drupal on your server.

Where can I get Hostmaster?

Hostmaster is developed and distributed on the Drupal contributions CVS repository.

There are three project pages for the three main components of the system.

Hostmaster project which contains the install profile for managing your sites. This is the primary project and all issues should be logged against this project, to simplify the issue queue.
Hosting project contains the support module used by the hostmaster project. This module defines all the necessary node types , relationships and functionality. Theoretically it can operate without the install profile, but this is unsupported at this point.
Provision project contains the back end provisioning framework. This project is designed to be useful separately from the front end.

At present there is no stable release of the project, and you will need to retrieve all source code from CVS. The front end additionally requires the views module, and both the back end and front end require the drush module.

Hostmaster project roadmap

We are planning for an initial release of the hostmaster code in late May 2008.
Following this, there will be weekly releases for a month or two until the code has stabilized. At that point, releases will be driven by critical issues or the addition of significant features.

History of the Hostmaster project

Hostmaster 2 is a completely new iteration of the original Hostmaster written by Adrian Rossouw for Bryght (now Raincity Studios). The original version has been running the hosted service provided at http://bryght.com for nearly 4 years, hosting literally thousands of sites, and has been implemented for several large clients.

Originally the system would only manage a single Drupal instance, on a single web server with a single database server, and as the requirements grew (concurrently running multiple versions of Drupal and then needing to manage several servers), it grew considerably in (unplanned) complexity.

There were also non-ideal implementation and licensing issues, such as the choice of Python and PostgreSQL for the backend and not developing the system as a pure GPL project, which severely limited the number of developers that were attracted to the project.

Because of the system requirements, it was also very difficult for new users to install, and had many unnecessary points of failure.

This new system has been designed with all these issues in mind, and the lessons learned have formed the basis of many of the goals of Hostmaster 2.

What are the goals of the Hostmaster project

The goals have been exhaustively documented on the Project Goals wiki entry

How is the system put together? What does it do?

This information is available on the Design and Terminology wiki entry

Hostmaster2

Nagios-type monitoring with hostmaster2 framework, GSOC

Ian Ward@drupal.org — Wed, 19 Mar 2008 13:55:01 +0000

UPDATE cross posting this into SOC 2008 for any feedback
First, thanks for the great BOF talk at DrupalCon, Adrian and Boris. I've got to install this and get it running, but stepping through some of the code I'm wondering whether this is a good framework to build out a monitoring system based in Drupal (something else on my mind)? Perhaps there are a few monitoring metrics that would not work well, but on the other hand, perhaps you can keep a very good eye on a lot of metrics about the servers themselves which have the hostmaster2 back end installed on them. Since I have not setup hostmaster2 myself yet, forgive me if I'm going down the wrong track here. So the idea is, since the hostmaster2 framework has the groundwork laid out to connect a frontend with multiple backends (multiple servers) and there's the hosting_stat hook which is invoked by the provisioning_stat module on each agent, could a suite of modules be developed to, at the general server level:

monitor disk usage
memory usage
apache benchmark each Drupal instance and aggregate stats
monitor disk i/o
raise flags for more case-specific cases like:
-- cron not successfully completed on a single Drupal site for over a day
-- # of new user accounts created in 24 hour period exceeds 100.
-- no new taxonomy terms created on the site in past 24 hours
(the idea w/ these is perhaps they're application-specific indicators of problems, and not necessarily a problem on every drupal site)

So, you could do some server-level monitoring, as well as what you've already started doing w/ the hosting_stat hook (application-level monitoring). Is the server level monitoring something others would find valuable? Are people using other tools for this, or do not have the need at all?

Next, I've never used Nagios personally, but I've heard it's a bear to setup, and for doing monitoring/alerts for simple things like excessive memory usage or disk space monitoring, it seems like it could potentially be faster to write simple scripts than get going on that application (though of course perhaps there are definite reasons to use it over something like this...)

Finally, w/ Google Summer of Code about to begin, is there anything in the hostmaster2 realm that would be great to get a student involved w/, like the BIND integration, or this server-level monitoring if it seems useful and practical to do w/ hostmaster2 framework?

thanks for any thoughts,
Ian

Hostmaster2

Existing Domains and Multiple Site Module Control

davidb — Tue, 19 Feb 2008 22:30:41 +0000

I'm excited to begin working with this module. It has been quite the task to enable a module or change a permissions setting across 40 sites, having to log into each one or via update.php.

So my question is, how would this integrate into a collection of 30 - 40 established sites in a multisite configuration?
Is this something that needs to be implemented from stage 1?

I read the description of what the HM2 does but am still a little unclear how it handles modules.
Can I enable/disable modules via HM2?
Can I modify access controls across all domains and/or each domain via HM2?

Thanks,
Dave

Hostmaster2

hostmaster and autopilot

sirkitree — Sat, 05 Jan 2008 21:53:37 +0000

So I was reading about a module today called autopilot (http://www.workhabit.org/change-management-and-drupal-autopilot-news) and was wondering if anyone here is involved or knows anything about it and what sort of differences/similarities there are between it and Hostmaster2.

I'm very interested in the topic of change management and and such things, but just getting my mind wrapped around these tools. I'm involved with some large Drupal deployments so tools like this are getting pretty high on my radar.

Hostmaster2

CivicSpaceOnDemand provisioning OpenAPIs

Amazon — Mon, 26 Nov 2007 21:10:54 +0000

Hello, as a step towards getting the support to open source
CivicSpaceOnDemand, we have further opened up the CSOD APIs. This
time we have exposed the provisioning system and everything that is
involved in setting up and configuring databases, mail servers, dns
servers, file systems, web server configurations, Civicrm files,
Drupal files, etc.

Please note that this system is entirely developed in PHP and MySQL and is compatible with Drupal coding styles. It has been in production for almost 15 months and deployed hundreds of sites.

Provisioning system:
http://customprofiles.civicspaceondemand.org/api/files/taskmanager

We had previously released the pre-configuration and update system
APIs here: http://customprofiles.civicspaceondemand.org/api/file

We hope this will encourage folks in the community who would like to
see a thousand Drupal ASPs flourish to better understand what's
involved and understand how they can help us to open source what we've
done.

If your organization would like to make CiviCRM or Drupal available
On Demand please contact me and I'll work with you to help make this
available.

Cheers,
Kieran

Hostmaster2

A mass hosting consortium?

nedjo — Sun, 18 Nov 2007 09:22:30 +0000

We've had various good efforts now at building mass provisioning solutions for Drupal. But too often we're building solutions in isolation, in competition, without enough resources to pull them off. We end up with duplication, half-finished products, privatized code. To put it mildly, it feels like we're not making the best use of the potential of open source.

I'm speaking from my own experience at CivicSpace, where I worked for a year and a half on enabling mass hosting. Through the process of wrestling with all the practical problems of producing a secure, high performance, flexible provisioning system, we got a tone of insight into what's needed and produced some creative and valuable solutions.

But they're not out there. They're not being used.

Looking e.g. at Hostmaster, it seems like a pretty similar story. A single company making a huge investment of talent and creativity into a codebase that ultimately may be a bigger task than they can support.

So I come out of this experience asking: how can we bring together the players here? How can we make the best use of what we've got in play--which is a lot, actually? Take just the pieces we produced at CivicSpace. Some of them aren't even open sourced as yet--specifically, the Taskmanager provisioning system and the Packages install profile system. But even those that are probably no one knows about. Like the Theme Factory project, http://drupal.org/project/themefactory. Here is an answer to a key mass hosting issue, how to provide powerful theme design tools without exposing the security issues of allowing clients PHP access. But who's there to even tell about it, let alone take on the (actually, I suspect, fairly minimal) work of shepherding it into something that can be broadly adopted?

And so I've been wondering, do we need something like a non-profit consortium? Call it "DropHost". A grouping that brings together the companies and other groups and leading individuals that are committed to solving the problems of mass hosting? Not to actually do the work of producing code. But to set the standards, to coordinate, to facilitate.

Does this make any sense?

Are there better ways to promote common solutions to the problems of mass provisioning, so we can confidently move forward? And maybe in the process actually get some broad value out of the significant pieces that have already been produced?

Hostmaster2

Mass hosting needs and existing solutions

nedjo — Sun, 18 Nov 2007 08:31:02 +0000

Mass hosting Drupal presents particular security, technical, network, integration, and other challenges that require specialized solutions.

There have been significant efforts by individual companies at providing provisioning systems, including Bryght’s Hostmaster platform and CivicSpace’s Civicspace On Demand platform.

This wiki page is an attempt to pull together a listing of the particular problems of mass hosting and what some existing pieces are.

Provisioning system

Details

Solutions needed to manage the creation, configuration, management, backup, and maintenance of client sites (files, databases, mail services, etc.)

Existing solutions

CivicSpace Taskmanager system.
Bryght’s Hostmaster and Hostmaster 2 projects.
May First / People Link (we've worked in their system, don't know it all yet ben-agaric)
Related
Sympal, http://drupal.org/project/sympal_scripts
Autopilot, http://drupal.org/project/autopilot
Drupal Automated Staging Toolkit, http://drupal.org/project/dast

Install and update modular custom profiles

Details

Ideally, install systems will be built modularly in such a way that a given install profile can be composed of several pieces that can be combined in various ways, e.g., an ‘administration’ package that handles . A key part of this functionality is ensuring that new objects (nodes, users, etc.) can be created

Existing solutions

CivicSpace’s “packages” system implements these features.
Install Profile API, http://drupal.org/project/install_profile_api

Prevent PHP access

Details

While Drupal typically includes PHP access for admin users, in a hosted environment providing all clients with PHP access presents security issues.

Question

Rolling a Drupal site without the possibility of PHP snippets in Views arguments, Block visbility, CCK computed fields, etc, reduces the scope of what one can accomplish in Drupal -- does the mass hosting you envision simply exclude that functionality, or look to replicate it somehow? ~billfitzgerald

Existing solutions

Paranoia module, http://drupal.org/project/paranoia
D6 introduced the PHP module, partially addressing the presence of PHP, but the process is incomplete as block configuration is still in PHP.
Patch to finish the isolation of PHP was not completed for D6, http://drupal.org/node/124158.

Theme design without access to PHP

Details

Custom Drupal theme design typically relies on (usually PHPTemplate) PHP files. Allowing clients to upload PHP files would be an unacceptable security risk.

Existing solutions

The Theme Factory module, http://drupal.org/project/themefactory, developed for CivicSpace On Demand, provides secure and fairly full featured theme design through Smarty.

Hide and require certain modules

Details

Often a hosted solution will need to require certain modules or hide them from client use (so that they cannot be turned off).

Existing solutions

System Mask module developed by CivicSpace, http://drupal.org/project/systemmask

Manage and track client registrations

Bryght has a module for this?
I think MayFirst's is built on Basebuilder (FOSS)

UI for site creation

Existing solutions

CivicSpace AJAX site creation wizard.

Hostmaster2

BIND module for managing DNS zones

Boris Mann — Fri, 12 Oct 2007 17:39:15 +0000

I just found the BIND module. Definitely something to look at running along with HM2.

Hostmaster2

Hostmaster2 status and drush

Frando — Thu, 27 Sep 2007 12:32:08 +0000

Hey everyone.

What's the current of Hostmaster2? Is there still someone working on it?

I won't be able to do much work on drush in the upcoming half year as I'll be a lot less active in this time. I'll still read this group and the drush issue queue from time to time, but won't be able to actually commit patches. Nevertheless, I still think that having Hostmaster2 based on drush is a great thing. So I'd suggest that whoever is working on Hostmaster2 just sends a short message to Arto asking him for CVS access. Moshe also did a few commits on drush recently, so he might be someone to talk to about drush related things, too.

Hostmaster2

Hostmaster Panel

discursives@drupal.org — Fri, 17 Aug 2007 18:21:02 +0000

I've had a few conversations about my own wishlist for a management panel for VPS containing multiple multi-site instances.

My use cases are pretty simple:

Use Case 1: Upgrade 1 (one) important module on one multi-site codebase (like CCK)
Use Case 2: Mass migration of multi-site 'site files' to new code base

Both of these require the same basic things, the main difference being extracting one site out of a multi-site instance versus a complete migration of all sites in a multi site instance (instance = codebase).

The core activities here are:

mysqldump
mysqlimport
editing of httpd.conf (through an interface preferrably)

Everything else is really just command line stuff in general. I'm not really needing this to talk to CVS or some other go-between for version checking of modules.

I'd like to continue to increase the value I provide to an SVN distro like Bryght public distro and, over time, see mroe modules integrated into that for versioning purposes.

Hostmaster2

OpenSRS module

Amazon — Wed, 18 Jul 2007 03:36:11 +0000

We have a OpenSRS module for allowing you to purchase domain names from OpenSRS.

Is anyone interested enough in providing domain name registration during site creation to port it to Drupal 5?

Hostmaster2

Initial HM2 code in Bryght hostmaster SVN repository

puregin — Fri, 29 Jun 2007 16:43:12 +0000

Just to let everyone know, Adrian's initial work on hostmaster2 is checked into the Bryght SVN hostmaster repo for your viewing and hacking pleasure.

Hostmaster2

Hostmaster 2 requirements on Drush

puregin — Fri, 29 Jun 2007 16:40:13 +0000

These are notes posted by Adrian, following some initial code development based in part on Drush.

The requirements I had for Drush were

1) required parameter handling. In the form of
./drush.php provision $urlhere

This is already implemented it turns out, so that when you define a command
called 'provision', any extra parameters are automatically passed to the callback.

2) multiple Drupal systems through a central drush.php

This also already exists, via a -path argument. More on this later.

3) Force loading of modules.

I decided this would be best done by implementing a .drushrc config file, which will
allow you to set any of the config settings. Most specifically, the include path, in the
standard form of 'path:/path:/path2' , and some code to manipulate module_list.

This will allow us to store the drush / provision code outside of the drupal tree entirely,
and stop us from needing to have a hide_modules module.

4) install mode bootstrap.

Since I use a copied chunk of code from install.php, I think I might be able to modify it to
install sites by switching into another database using db_set_active. This would simplify
the code we need to write in Drush, but it's also significantly different from what we have in
hostmaster at the moment.

This will require more research.

Hostmaster2