Rewrite Drupal Update Process

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
johnmurray's picture

Overview:

I would like to rewrite the current Drupal update process to be more efficient in terms of time and bandwidth.

Description:

As of now, an update is performed by downloading a CVS snapshot and unpacking the files on your server or uploading the files to your server. This effectively overwrites your current Drupal Installation. Then an update file (update.php) is execute update the database to reflect all of the new changes. The main problem that I see here is the wasted bandwidth. When you download an entire snapshot, you don't necessarily need to update all of the files in your current Drupal installation. No doubt, there are some entire (core) themes that may not need to be changed or core files that are not even touched.

My proposal is to re-write the current Drupal update process to be more efficient. This could be implemented by maintaining a revision history (for each file) in the database. This table would tell the current version for the file being used. The updater could be called in the Cron to check for updates via a web service (specifically for this implementation) that would return a list of files and their versions. These version numbers will be the version numbers that "should" be in use. Once an update has been found (once your Drupal installation knows that there are newer versions of files available) the admin will be notified and the update process can begin. The update process would include connecting to and downloading the PHP files from the CVS repository, replacing the file(s), downloading the update script for the database, and executing the DB update script.

Using this method, bandwidth would be saved by not downloading unnecessary files. For example, even if a (core) theme were to be updated, it is unlikely that many image files (usually the larger files) would change and need to be downloaded. This method would also allow for automatic updates or manual updates. A major advantage to this method is that the user does not have to upload and/or unpack files on/to their server. I also believe that this method would make updates easier for beginner users that may be more timid when it comes to applying updates.

Mentors:

  • No one as of now, but open to anyone who might be interested!! :-)

Contact Details:

  • Email: murrayj5 at nku dot edu (best-method-of-contact!)
  • Phone: +1 502-442-6682

Difficulty:

Medium to Hard
How does your proposal meet the stated goals of the Knight Drupal Initiative program?: 

This project helps further the Knight Drupal Initiative by making the learning curve to Drupal maintenance lower. The current method of updating Drupal can be a bit intimidating to the non-technical Drupal users (who most likely set up their Drupal installation via a web hosting package). By eliminating the current method (or providing a good alternative) and giving the user a simple interface, the user can feel more empowered and in control of their site by being able to manage their updates.
How long will your project take to complete?: 

Entire Summer (Google Summer of Code)
Any additional work or maintenance I would like to maintain after the summer and become an active developer! :-)
How will you implement and distribute your project?: 

I would most likely implement this as a module for now. However, if it is accepted by the Drupal developer community, I would like to integrate this as a change to the core Drupal installation. It would be distributed through Google Code (via the conditions of GSoC) but it would also be available on the Drupal.org website (like I said earlier, most likely as a module).

Also, some beginner users might feel like an update would take a lot of time (and thus avoid it), but, with a simple interface, they might feel as if the process would be fast (or at least faster). However, that is just a opinion.

What is your total budget estimate and how much funding are you requesting: 

Funding will come from GSoC!

Comments

Are you reimplementing cvs then?

chx's picture

What you describe is practically cvs up. There are http://phpclasses.cfappsinc.ca/browse/package/4285.html http://code.google.com/p/phpnetcvs/ PHP CVS client classes already. Given these, what is going to take three months here...? Of course, one might say that integrating these poses the usual security weakness of Drupal writing itself.

But then what are we saving here? A previous GSoC got us Plugin Manager and it's now in core and that's good. It downloads Drupal tarballs to the server and then uploads back to the server over FTP or SSH in a more secure fashion than Apache writing PHP files. It's not like anyone needs to upload the ful ltarball from her home computer to the server over a slow uplink. I do not readily see why would we want to spend a GSoC slot on saving some miniscule amount of server time. Or are you trying to save OSU OSL bandwidth? That's a noble goal but I do not think Drupal tarballs consume any significant amount of ftp.osuosl.org's bandwidth given their small sizes compared to things like the Linux kernel, whole distributions, Firefox etc. I believe that directing traffic towards cvs.drupal.org instead of the monster FTP site actually would harm OSU OSL / Drupal than help anything.

The major goal of this is not

johnmurray's picture

The major goal of this is not only bandwidth, but also ease of use (the major focus being on ease of use). If bandwidth from the Drupal sites is not a problem, then that still leaves the issue of usability.

The best scenario is a, mostly, non-technical admin who has created a site for his local community/committee/band/etc. If they are using Drupal, then it is highly probable that they used an automated setup that came included in their hosting plan. An example would be a popular shared hosting company GoDaddy. They have a service called Hosting Connection (http://hostingconnection.godaddy.com/AllApplications.aspx?prog_id=GoDaddy), and as you can see, Drupal is one of the listed applications. These sort of included packages are a (sometimes) one-click install (or something close) that requires very little technical knowledge. It can not be said that all of these "simple setup" solutions use the most recent software or stay on top of vital updates (as they should). There is just no way in which this can be a guarantee. By creating a module withing Drupal that can update itself, this gives the users a way in which they can update the site through a simple (non-technical) interface. This avoids trusting the hosting companies to stay on top of their updates and patches. Also, beyond the user, if the module were to be integrated into the cron tasks, then the update process could be automated and no intervention required by the user. This means that hosting companies could configure the installation to update itself by default. The more-technical users could modify/disable this if they so desired once it was set up.

As far as other features go, the user could also have an option to roll back to a previous version of Drupal if there were some compatibility issues with newer versions and modules that they may have in use. The update module could also include module version checking to make sure that the version of Drupal that it is upgrading is compatible with the currently installed/active modules. If there are any conflicts, then an option could be shown to upgrade or to leave at current version.

I'm sure there is more that could be done with this module, and hopefully I can get some constructive feedback from the community to help evolve this idea.

--
Thanks!
John Murray

Modules updating by

Dave Reid's picture

Modules updating by themselves is not a very good idea and could easily bring a site down (see latest Drupal core release fails on PHP 4, modules that have had less than stellar upgrade paths like nodewords). Have you had a chance to evaluate the Drupal 7 update manager module? It seems like most of what you're proposing has already been accomplished (updating in browser and not having to touch the server).

Senior Drupal Developer for Lullabot | www.davereid.net | @davereid

update list for non-activated modules?

ClearXS's picture

I'm more interested in getting a list of my non-activated modules, that I can update, with the same info as for my activated modules.

I maintain a lot of non-activated modules (also for others to download as a package), but can't keep track of updating them...