Install/Update Modules/Themes within Drupal Interface

You might want to look into Wim Leer's CDN module for moving files around. It has a built in file transfer mechanism with pluggable interfaces for whatever you need (although currently only ftp is built.) I had been wanting to send him a patch which abstracts the file transfer stuff into its own distinct module for easier usage in situations exactly like this one.

http://drupal.org/project/cdn

A few things that I'd like to see...

• Ideally you'd refactor the code from CDN and/or media_mover so that the (s)ftp code is a separate module and can be re-used elsewhere.
• When the files get downloaded from d.o the md5 of the tar file should be checked prior to installation and checked against the values listed on d.o. I think those values are available via the xml that gets used by udpate(_status) but I'm not sure. You could also ask the user to paste the md5 into the form - that way the module is somewhat protected against DNS poisoning.

This seems like a great proposal to me and those two items are really "nice to have".

--
_{Open Prediction Markets | Drupal Dashboard}

I too find this to be highly intriguing. I would very much like to do work with this.

I would recommend taking a look at the Serendipity Project's SPARTACUS which uses XML+version control to do exactly this for s9y.

I was closely involved in the project when SPARTACUS was developed, and would be more than glad to answer questions.

For those who came in late ;-), Serendipity is an open-source blogging engine written in PHP.

Cheers,
-K

There is already xml data available for project releases. It is used by the update status module. I do not see that this would be particularly overtaxing on the d.o. server, as it is simply replacing manual downloads with automated ones.

.. why not simply use HTTP to transfer the files? The update status module already checks for updated versions of the modules you've already have installed and retrieves the direct link to the downloadable tarball for the module in question. So why not use something similar to retrieve the different URLs and then use file() and gzopen() or something similar to retrieve this module directly? The only challenge I see is how you overcome the fact that you probably haven't got write access to the file system, so some additional rights has to be set to be able to accomplish the actual unpacking of the tarball.

But as I said, I may be missing a part of the picture here :)

This should be automatic but not too automatic in that the modules should download but not be overwritten without the direct interaction of the admin. I like the idea of having to match MD5 codes - but only if the module provides a link to the d.o pages with the MD5 codes, I don't want to make it an all day process to find the codes.

Because we know that this will make Drupal a more open environment for less technical users we also need to address the backup question in this post as well. Can we backup the tables specifically altered by the specific update that is being installed and can we back up the folder for the specific module as well with the ability to revert to the old version of the module.

So the work flow looks like this in my mind.

1. Install update-helper module (name TBD)
2. Enable or disable automatic downloads via s(ftp) or ssh to a downloads area directory or disable to be able to fetch from an interface
3. Once files are downloaded module installed copies tables to be effected and the current module folder to archive
4. New module is copied over old module and update.php is ran

I am not a programmer and I could be oversimplifying some of this but I think that this would be a big help to make Drupal more availible to more users. I am lucky enough to have SSH from my host but not everyone has the luxury. This is a great SOC project.

I want to see if I understand the idea here. For example, where "drupalbox" is the server on which drupal is run:

1. I tell the magic Drupal Module/Theme installer (DMTI) I want to install package X.
2. I input my (non-root/apache) *nix login username/password for drupalbox.
3. DMTI downloads package into my homedir on drupalbox.
4. Using SFTP, it then does a local transfer of the package from my homedir to the drupal installation's "modules" or "themes" dir.
5. Somewhere along the way, it does "tar xzf" to unpack the module.

Is this about right? (Or would/could it be more along the lines of running a shell script as the non-root, non-apache user whose credentials were given?)

Many shared hosting use fcgi-suexec and suphp.
So module can check or ask about uploading to local filesystem.

I think it is very good idea because:
1 user can be in fcgi based system
2 user can be in suphp based system
3 user can be in mod_phpX based system then we upload to local (s)ftp
4 user can be in mod_phpX based system and have scp proto

Huge huge huge +1! I've been begging whoever would listen for a module like this for a long time... I think mostly through annoying, irrelevant comments on update_status issues. I'd love to code it myself, but I don't know if I have the time to do it justice. When I switched from Windows to Ubuntu a few years ago, the ease of installing software through Add/Remove Applications and Synaptic blew me away. It would be awesome if a Drupal package manager could have the same impact.

The implementer should consider "best practice" in package management (which is basically what this appears to be). Many Linux distributions have their own package managers: apt, rpm, and portage are mature examples. The PEAR library also includes a package manager. I realize that these are complex and multi-layered programs, but there are lots of lessons to be learned for the simpler Drupal context. For example:

• Most package managers can fetch a list of all available packages from a repository (here, drupal.org). Does the XML-RPC protocol used by update_status support this? Maybe it should.
• Packages can also be installed from the local disk. E.g. in Drupal, a user could upload a .tar.gz through the web, and the package manager would take care of installing it.
• Apt uses MD5 sums, as suggested by many here. These are provided in the aforementioned package lists. There are also encryption keys to ensure the package manager is talking to the actual repository (not a man-in-the-middle).
• Multiple repositories can be used (maybe not urgent, because modules are mostly centralized on drupal.org).
• @denverdataman: Options for pinning (not doing any upgrades for specified packages), automatic upgrades (at some interval, via cron), and policies (automatically update the package list only OR automatically install security fixes only OR automatically install all updates). Note that the "automatic package list update" is essentially what the update_status module does; but it only fetches a partial list.
• @tjholowaychuk: The package manager is a package itself in most distributions, and can therefore update itself.
• Automated update times are slightly randomized to avoid hammering the repository on, say, Friday at midnight GMT.
• Dependency resolution, usually with prompts to the user to install packages required by the one they asked for. This information could be extracted from module .info files and provided with a package list.

...what I'm trying to say is that other people have already wrestled with many of the ideas brought up here. Drupal doesn't have to — and shouldn't — copy their solutions, but it's certainly wise to do some research.

Like houndbee, I'd be glad to talk further with anyone interested in implementing this.

I use drush. I'd be happy to walk clients through a nice UI wrapper around drush.

I like debian's apt model: The server (ftp.drupal.org?) looks into all the modules and builds the dependency tree. This is then compressed and made available for download.

Drupal clients poll this file, download it if outdated, and uses it deciding what to install for project X.

So, +1 for best practices. :)