Hi everyone.
I'd like to propose some features for Aegir, which, to be honest, I'm not sure to be missing or if it was I who wasn't able to figure out how to use them. Forgive me if some or even all of these are already possible.
Here goes.
1 - It would be very useful to be able to import existing sites into Aegir simply by pointing to their current instalation directory and perhaps indicating which platform to use (importing a Drupal 6 site into a Drupal 5 platform wouldn't work very well). I do realize this brings up several issues, such as what to do about the modules that are installed, but that can perhaps be solved by running update.php and working in conjunction with my next two points.
2 - Upgrade modules for each or all sites at once would also save a lot of time and this was the feature I expected to see in Aegir from the first day I read about it. This also brings up some issues, such as what happens if a site is hacked in order to work correctly with a certain bug of certain version of a certain module? The simple answer would be to try it out and if things break apart, Aegir's migration and/or the backup and restore systems would be there to help. For the simpler cases, such as when the admin knows that all his sites are similar and that upgrading all the modules at once won't break them, being able to upgrade all the modules at once would be a huge time saver.
3 - Having a module manager that installs and manages all the necessary versions of the modules. This is just a wild idea that may not even make sense but perhaps there could be a directory where the real modules were stored, each version with it's own directory, and each site could link to the wanted version of each module. Managed modules could be added or removed by the admin (at the request of a client, for example) and Drupal would take care of downloading the updates into the correct directory. This would allow the upgrade of all the modules for one site but leaving all the other sites untouched, if necessary. Working together with the package system of Aegir, this could allow super-fast provisioning of modules for sites, as each added module could be added as a package to any site.
4 - Same as above but for themes.
5 - Same basic idea as above but for Drupal releases.
6 - Being able to specify the database name for each Drupal site.
So basically, apart from #6 above, what I'm proposing are some features that would further automate some tedious and repetitive tasks, common to many Drupal instalations, and provide a GUI for them.
Any thoughts?
Comments
1- is already possible: you
1- is already possible: you can import existing sites as a "platform" and migrate the sites to existing platforms if they are compatible.
2- is also possible with the migrate mechanism, not sure what you feel is missing there.
3- i would love some work progressing on this front. it's very sensitive however: we should only allow new modules to be installed and module to be removed only if not used by any site (same for themes). one feature I would like to see (and I think i already filed a feature request for) is the "platform cloning" mechanism (take this drupal 6.12 module list and clone it into a new drupal 6.13 platform), right now this needs to be performed "manually".
4- the above applies to themes too
Thanks for the reply,
Thanks for the reply, theanarcat.
I'll go through the points one by one, as you did.
1 - I think that creating a new platform is not the simplest way to go. It's not intuitive. I did not even think of it until I read your comment (thanks for the info, by the way). I have 11 Drupal sites on a server, so for each one of them, I'd have to create a new platform, migrate the site to a previously existing platform, and then delete the "imported" platform. Not intuitive or friendly at all. What I meant was something like an "Import site" feature, which would ask for a filesystem path where the Drupal site to be imported could be found, a target platform to migrate it to, and Aegir would take care of creating a new site using the already existing database (or perhaps copying it to a new one would be a better choice, allowing the user to chose the database name - see point #6) and importing the files into Aegir's directory.
2 - Again (if I understood it correctly), this requires creating a new platform with the desired modules and then migrating the sites to that platform. Not very intuitive or friendly. I'm talking about a button that says "Upgrade" and Aegir would take care of the rest. I know there are a lot of details involved and a lot of things to care for, but I believe it can be done.
3 & 4 - Yes, platform cloning would be great. And yes, the modules/themes should only be removed if not in use by any site.
5 - I think I didn't explain myself well on this one. I was talking about clicking a button and bam, having an upgraded Drupal. Let's forget about the Aegir concepts for a few moments - a site admin shouldn't have to care if it's a platform, a site or a peanut; the site admin has a bunch of sites hosted on Drupal 6.12 and suddenly there's Drupal 6.13, an important security release and obviously, the admin wants to upgrade all his sites. Instead of having to download Drupal, put it somewhere, create the platform by hand and specifying where he downloaded the new Drupal release to, then migrate each site, one by one, to the new platform... why not be able to select the platform he wants to upgrade and simply click a button? That's all the admin wants to do. I know that some other admins may want to do other things and those are not useless use cases, but I think Aegir is forgetting about this simple use case.
Perhaps part of this can be thought of as an automatic Platform creator, one that downloads the selected Drupal release and create a platform based on it.
As for the upgrade, I picture it happening as something like this: Aegir detects that there's a new Drupal version and marks the Platforms that use the previous version as upgradeable. The admin selects which platforms he wants to upgrade and Aegir downloads the new Drupal version, creates the new Platform based on it, and automatically migrates all the (or the selected ones) sites to the new platform.
Perhaps selecting several sites for migration is already possible (I haven't tried it yet) but not the automatic Platform creation, I think. Combining the two would wield an semi-automated upgrade process that would save a lot of time on simple cases where the admin knows for sure that his sites can be upgraded without any problems (or perhaps he's just irresponsible, but that's his problem :) ).
6 - I just added this because I forgot it when I first posted - being able to specify the database name for the sites would also be pretty nice, because names like "site_64" are not really explicit about which site that database belongs to. I know these names are supposed to be managed by Aegir but what if I need to do something on the database myself, and for some reason don't have access to Aegir? How would I know which database belongs to which site?
I hope I made some of my points clearer. :)
let's go point by point
let's go point by point again.
1) you're right, it's not intuitive and could be improved upon, patches welcome. :) note that there's already a feature request + some patches to allow aegir to import tarballs as sites... http://drupal.org/node/322788
2) Yes. I'm just describing the way it works right now. You could add a bunch of sugar on top of that, for sure.
3 / 4) skipped. see http://drupal.org/node/448692
5) you want http://drupal.org/node/373062 and batch tasks (could not find a feature request, maybe you could open it? :)
6) you know because it's the settings.php. :P but more seriously, when used standalone, the provision module chooses the database name based on the site name, and it works pretty well, but it can get quite messy too. i like better the numbered approach, for mass hosting. and anyways, drush can give you that SQL shell you're looking for...
1 - I knew I had seen
1 - I knew I had seen something like that some time ago! :) But I'd still like to see my use case covered (of course :P), which is to import already existing sites from the same server, meaning the admin would only need to point Aegir to the site directory and Aegir would do the rest. I'll try to look into Aegir's code and see what I can do.
2 - I love sugar! :P Seriously, I'll try to see what I can do about this one too.
3 / 4 - Looks like the patch you pointed to is for the Drupal 5 version of Aegir. Is this where patches should go first, and then they get rolled onto 6.x?
5 - I think I've read that post when adrian wrote it but didn't remember it. Makes things sound a lot more complicated than I pictured... :| briwood@drupal.org seems to have done some work in the direction I mentioned.
6 - Yes, the numbered approach is absolutely necessary, don't get me wrong. I'd just like to have a little optional field that can even be hidden by default, like the "advanced" options for the database configuration when installing Drupal, but available in case the user wants to enter a different name. Of course this also requires some checking so it doesn't allow names that already exist. This actually seems to be the easiest of the features I mentioned.
So, where should I start? Would someone be willing give me some guidance on this?
1 & 2 will be modules that
1 & 2 will be modules that wrap up existing functionalities. You really need to start with platform management for any of this to make sense, and this is already a big task to tackle, and will get you much more familiar with the code, so start there.
3 & 4: patches should be sent against head, so those patches will need to be ported, and D6 is the current development platform, d5 is deprecated.
5: i remember we have a few issues about refactoring the task display you may want to look at before digging into batch task mode:
http://drupal.org/node/454452
http://drupal.org/node/454400
http://drupal.org/node/422962
http://drupal.org/node/422970
6: options are always cool, and this could "just work". You need to know how the frontend passes options to the backend and add a flag that specifies the database name... Usually, the name-generation code kicks in when no site_id is provided, but that's not an option for you (otherwise the backend will loose track of the node id associated with the site, not good). So you may need a separate option. You will want to hack the hosting_site module (preferably with hooks) and the provision/platform scripts.
All work should happen on HEAD and you should try to use as much hooks as possible to create modules, not patches, to keep crazy stuff optional (or experimental).
Thanks! Right now I need to
Thanks!
Right now I need to go to sleep but I'll get to work on this tomorrow morning.
I also have another suggestion, which had already occurred to me and came to my mind once again when I saw the discussion about SSL support (more specifically, when you mentioned that 0.4 will also take care of DNS):
7 - Not everyone will want to have Aegir touching their vhosts or DNS configuration files. For the vhosts, I can imagine a situation where someone would want to use a web server other than Apache. For the DNS, some people may not run their own DNS servers (I don't) and thus have no access to the configuration files.
So my suggestion is to allow the user to chose wether he/she wants Aegir to take care of the configuration for the web or dns servers. A couple of checkboxes more.
Besides, this also starts to raise another point (which is also valid for the SSL discussion): what servers will Aegir support? For the web server, it requires Apache. What DNS server will it require? Why those and not others?
Perhaps it would be best if Aegir would have pluggable server support. Instead of hardcoding the Apache, DNS and SSL stuff into it, Aegir could have an API that could be implemented by modules that provide server configuration services. This would allow Aegir to support Apache, nginx, lighttpd, bind, djbdns, powerdns...
So DNS was removed from
So DNS was removed from 0.2/0.3 for exactly that reason: it wasn't possible to disable it properly in the backend. So we just teared it out of the code base, because it was breaking everything. DNS will be an optional feature, but it's a required feature. :) Also, the original DNS code was designed with multiple DNS server engines in the backend, with only BIND implemented. As I mentionned in the SSL/Pound thread, the objective in the long term is to be implementation agnostic, but we start with one implementation and have that work properly first.
Also, out of the box, Aegir doesn't touch your apache configuration file. You still have to, manually, during the install process, tell Apache to use Aegir's configuration files. If you don't, those vhost files will simply be ignored and apache will not be touched (aegir will try to restart apache with that sudo command, but you could put /bin/true in there and it would "just work").
It will be the same (and better) with Bind: there will be a directory with a bunch of bind config and zone files in it, and as long as you will not tell bind to use those files, Aegir will not affect bind. It will be better because you will be able to disable DNS support altogether, and the DNS code had multi-engine support (originally, not sure how it will come back from the dead though).
Some questions...
Hi,
1.) could someone give a short description of how to import a site as a new platform? When a site is on a different server is it possible to use
$ drush snyc
$ drush dump
$ drush load
Or can i use provision import?
Does aegir knows that a database exists?
2.) I'm quit new to drupal.... What is the proper place for modules when using aegir? /sites/mydomain.com/modules/ ? Or sites/all ?
3.) Is there a good "howto" about creating installation profiles? Than creating platforms all the time should be less work...
regards Volkan
Hi Volkan. 1 - I believe
Hi Volkan.
1 - I believe that importing a site would consist of specifying the site's installation directory when creating the new platform. I have no idea about the database, though - good question.
2 - I'd say that you should put them in /sites/mydomain.com/modules, unless you're certain that all your sites need the modules, and that when you upgrade them, you won't disrupt any of the sites.
3 - Check http://drupal.org/node/67921 and http://drupal.org/project/installation+profiles
If you have any more questions, I'd suggest you open new topics specifically for them, perhaps even support requests on the hosting or provision modules, since you'll have better chances of people seeing them.