Aegir 0.4 : Roadmap for the next few alpha releases.

So other than some time I spent on a cleaner user interface for tasks last night, I also sat down with mig5 and anarcat to figure out what our next few weeks of goals are going to be.

Now that we have moved towards using drush make, we can finally start using some external modules that provide useful functionality to the hosting front end, obviously with the goal still to keep them to a minimum, but if they are reliable we should be able to use them.

I'm busy focusing on refactoring the UI somewhat (any suggestions welcome), and i've decided to go with http://drupal.org/project/modalframe , which is the basis of the overlay module in D7 so I will have an upgrade path.

We have a patch in the queue (http://drupal.org/node/588728) to add views 2 support for all our packages too, so we'll hopefully get that up and running.

Roadmap

0.4 Alpha 3

We are going to be spending this release (alpha3) focusing on bug fixes and getting the UI changes we would like to see implemented in. Keep in mind these are mostly cosmetic and not a major refactoring. I want to focus on improving some of the usability problems i see, to make the user experience more seamless.

Specifically I want to give the user more feedback about what is going on with their site, and be able to display messages from the backend on the front end , such as once your site has been installed to give the user a login link they can follow directly without having to open the log page and scroll down to the log message, copy, open new tab and paste and hit enter. Either that or wait for the email to be magically sent out.

This will also be useful because we want to create a 'reset password' task for sites.

Move to git

After alpha 3 is released, we are going to migrate to git proper. I think that for our purposes we don't actually want to keep the cvs backlink going, because the next phase of development especially is going to involve renaming a lot of files, and it will be a nightmare to do this correctly in cvs.

0.4 Alpha 4

we will be re-doing the way server nodes work, which means a major amount of table changes, db modifications and node type changes. Essentially we are going to be doing away with separate node types for database server and web server (and dns server, but more on that later).

In it's place will be a single server node type, which acts as a container for services. each of these services is of a specific type (http, db) and each service can have multiple engines (apache, nginx or mysql, postgresql). When creating a site you will no longer be relating to the db server node, web server node or even the server node itself, you will instead be creating a relationship to an instance of a service provided by a specific server.

This in it's place also has the ability to do some interesting mapping of networks. You may have multiple services that don't actually publish themselves to sites, but only to other servers. So you could have a server that acts as the entry point to an entire cluster of web servers for instance. When you choose to publish to that service you will be publishing on all the web front ends.

If at some point you want to add a web front end, you simple create the new server with it's associated service and associate it to the cluster instance. Things won't be getting that complex for this alpha release, but it gives us a way forward on some of these problems.

This also means we will be refactoring some of the backend in the near future to more cleanly resemble the design of the front end. So instead of provision apache, we will have provision_http , which will have extensions of it's own for the various type of web servers it support.

Now one of the major stumbling blocks w.r.t. migrating between servers is how to get the backups across. Essentially what is our problem is a spoke model versus a mesh model. Do you have a central backup repository, that all the servers need access to, or do you have a mesh model where all the servers can connect to each other.

One of the things we are going to be implementing in this alpha (although it might get pushed to the next alpha if it's taking too long) is a new service type , namely that of a 'file service'. In the simplest install, the file service will be associated with the master server, which would mean that all the sites would need to be able to communicate to the file service.

This service would also have various engines, ie: scp, sftp , subversion, git etc. so there's a lot more directions we can move in after this that make for some really interesting configurations, some of which sidestep the key propogation issues we would run into with scp entirely.

We will also be implementing new tasks that work on the server level, instead of just on platforms, which will greatly simplify a lot of the stuff we are working on (ie: db details changed?, no reason to reverify all the platforms). The first of these tasks will be a 'verify' task which will do diagnostics and detect which services are available already.

0.4 alpha 5

our next big problem and requirements for site migration between servers is going to be the fact that for site migrations to take effect the dns needs to change, which means that we will need dns server support for all this to fly. This is the main focus for this alpha, which also has some really interesting problems associated with it, such as the fact that migrating from one server to another will take a fair amount of time for the dns to refresh, which means we might need to stage server migrations so they lower the ttl a day or so before, and then do the move, and then once it's successful set the ttl higher again. or we could just live with a permanently lowered TTL.

ongoing

we'll be reviewing patches and working on drush and probably drush make to get it up to our requirements. views support will happen, and might involve some refactoring on our side to take advantage of things.

time estimate

Time estimate wise, i figure alpha 3 will take us another week or week and a half, depending on how many good interface suggestions we get and how hard they are to implement. I have my own personal hitlist, but there might be others.

The baseline alpha 4 changes (without the file server stuff) is probably a week worth of breaking things and then another week and a half of fixing all the issues we've unearthed. It's basically the largest bit of refactoring we have planned for this release, and the rest of the release cycle will be about taking advantage of the new functionality to finally map out the things we need to get done.

The alpha5 dns code is already in progress, but it is not nearly ready yet, and the alpha 4 changes are a big part of that.
there's a large amount of stuff that needs to be resolved for this.

This is a very loose estimate for now, but we'll know more once we start delving into the changes more deeply.