Module and Theme Browser in Drupal Core

Greetings,

My name is Leighton Whiting and I worked with Bojhan Somers and Ryan Szrama in GSOC 2010 last year to build the Bulk Product Creation module. It was a blast and I learned a lot. I am still in school and would like to do another project to help Drupal get some new usability improvements.

This proposal is to port Drupal's Plugin Manager module to Drupal 7, as well as add some new features and add repository API support, and other usability improvements that could be included in Drupal 8 Core.

Introduction

One of the greatest strength of Drupal lies in its great collection of Contributed Modules. Anyone who has worked with Drupal knows that it is very hard to create a site using only Drupal Core. This is true of any great CMS. So it is common practice for each and every user of Drupal to install a number of modules for each Drupal site they build.

One of the things that I really feel is missing in Drupal 7 is the ability to search for contributed modules and themes and install them from within Drupal's admin interface. Such a feature allows site developers and admins alike to easily and quickly build and improve their sites. Drupal 6's Plugin Manager module (http://drupal.org/project/plugin_manager) provided this very useful feature, and I found myself installing this module first each and every time I started a new Drupal site.

Thankfully, most of the Plugin Manager's functionality has been ported into Drupal 7 Core, so we can easily install and update modules from the admin pages. One noticeably missing feature though is the ability to search for modules. This is something that I (and everyone I've asked) really love about WordPress. For quite some time, it has had an immensely useful Search feature to allow admins to install modules from their admin pages.

I did a bunch of searching into the issue, to see if there is a satisfactory module already existing which could be used as a starting point/springboard for ideas. I found that Plugin Manager is defunct now and the Drupal 7 version doesn't seem to be working (I installed it but couldn't get anything to happen). I found a nice discussion and mockup by Bojhan Somers at his blog here: http://www.bojhan.nl/module-theme-browser-interface . This was an excellent discussion and exactly what I was looking for. I am very glad that others out there are thinking about and interested in this great usability improvement.

Bojhan provided a link to a proof of concept module created by Stijn Vanden Brande. I eagerly installed the code from GitHub and it looks like a great starting point for this kind of functionality. The Module Browser module by Stijn has some key functionality that can be reused, and the XMLRPC module is right along the lines of what I was thinking.

Overview

I would like to port Plugin Manager to Drupal 7, as well as add some key new features and provide an extensible API to allow for external repositories. Hopefully this module would allow the Module Browser functionality to be in Drupal 8 Core eventually. Having it available in the Drupal 8 Core modules (perhaps as a disabled by default Optional module) would be the easiest way to get people on their feet and running. Such a solution could integrate into the 'Install Module' page (which currently only shows the upload file or copy-paste download link) to show a search box to let users search for a modules that way from the Install page.

I would like to emulate as much as possible the WordPress workflow and presentation for this functionality. The idea is that users enter one or more keywords and are presented with a list of modules which match them, sorted by relevancy, with some basic information, possibly ratings (from drupalmodules.com) and an install link (or add to install queue). Clicking on a module could open up a modal which shows more in depth information about the project.

Drupal 7 Features

Some of the possible features that I would like to include are:

• Integrate the Search functionality into the admin/modules/install page (perhaps using Drupal 7's page_alter hooks)
• Provide API/hooks for checking remote repositories for modules and themes (opening up the possibility to search modules from other places - not just Drupal.org)
• Provide a Module/Theme browser which will allow users to browse the contribs by category and other filters
• Integrate with or require the Backup/Migrate module (http://drupal.org/project/backup_migrate) to automatically backup the site before installing new modules
• Provision for installing a list of modules at once
• The Search results should have useful, concise information and a More link/button that would either expand or show a modal which has the project information in detail

Future Features for Drupal 8 eventually (beyond the scope of this project)

• Expose new XMLRPC call from Drupal.org to be used for fetching project information, searching modules, etc
• Show any reviews (perhaps taken from http://www.drupalmodules.com as well) for the modules on the Search page in the more details modal
• Automatic detection of module dependencies and provision for installing all missing dependencies when a module is to be installed (would be possible with the XMLRPC method detailed above)
• Provide some kind of rating system to let users know which modules are well-known and used. Possibly this could be scrubbed from http://www.drupalmodules.com, but it would be nice to have this feature available on Drupal.org, and expose it with an XMLRPC call as well

Issues

There are many things to take into consideration for a project like this. One of the major ones would be how to get the search results for the modules. Currently the only reliable way to do so and be up to date is to download the large projects file here: http://updates.drupal.org/release-history/project-list/all and parse it.

This solution works and will work going forward, but it would be nice to have drupal.org expose some XMLRPC methods so that the release information and other things can be fetched when needed. Stijn has the xmlrpc module in his code that illustrates how this could work, which follows my line of thinking exactly. His module has a link to his hosted repository list hard coded in, but we could hopefully get something like this going on Drupal.org in the future and use that. Exposing such a call could be a part of this project as well.

An issue with this approach is the potential server resources that would be needed for numerous Drupal sites searching for the modules. Personally, I don't really see this as a big issue, since installing new modules isn't something that every site will do every day. Also, the Update module does a similar thing which runs much more often than the this module would, and it doesn't seem to be much of an issue.

Timeline

• May 23 - 31: Coding menu and pages, getting the skeleton worked out
• June 1 - 7: Adding new Functionality to the Project module on d.o to expose the necessary data
• June 8 - 17: Creating a condensed version of the Backup/Migrate module to backup the database before installing new modules
• June 18 - 30: Coding the batch install-backup-enable process
• July 1 - July 12: Redesigning the Search Page (including js, css, and php)
• July 11: Submit Midterm
• July 16 - August 6: Coding the Module and Theme browser and filters
• August 7 - 14: Providing an API for other module repositories to integrate with
• August 15: Begin polishing up the module, fix bugs, write documentation
• August 22: Submit Final Report to Google

Conclusion

I believe that this functionality is a very important user experience improvement that will benefit every user of Drupal, and should be pursued and implemented. Not only will this make the Drupal experience that much of a joy, it will also open up a new portion of the CMS market and expand the userbase.

Related Links

Discussion - Bojhan Somers
http://www.bojhan.nl/module-theme-browser-interface

Module Browser on GitHub - Stijn Vanden Brande
https://github.com/Krimson/Module-Browser

Plugin Manager - Not working on D7 (see http://drupal.org/node/1001976)
http://drupal.org/project/plugin_manager

Backup/Migrate Module
http://drupal.org/project/backup_migrate