New Release System

Future work

dww — Sat, 11 Nov 2006 07:40:44 +0000

Here's a list of all the issue's I've already submitted for future work -- things about the current system I'd like to improve, and issues other people have suggested. None of these were critical enough to delay the initial deployment of the system, but they should all be fixed eventually. I'm keeping this list updated (and roughly in priority order, as i see it) so there's a central place to keep track of future work.

If you want to support any of these efforts financially, please donate to the cause. There's a PayPal button in the announcement about the release system going live.

Issues that will hopefully be addressed by Google Summer of Code proposal on translation tools:

Resolved/fixed items:

#97173 (bug): Download link targets other project's tarball Fixed on 2006-11-12
#87298: Add version info to modules page (5.x modules only) Fixed on 2006-11-21
#100256 (bug): xcvs-taginfo doesn't properly associate core tags with project 3060 Fixed on 2006-11-29
#98277: add project identification to .info files in packaging script Fixed on 2006-12-01
#89538: need branch-aware notion of "default" or "latest" releases Fixed on 2006-12-06
#89539: table of downloads on the project node, not just a "default release" link Fixed on 2006-12-06
#97780 (bug): "Last updated:" does not get updated Fixed on 2006-12-07
#101887: add setting to hide dev snapshot table on project nodes Fixed on 2006-12-09
#101601: table of downloads when filtering by [all] versions in project browsing pages Fixed on 2006-12-17
#97095 (bug): Issue tracker depends on project_release.module Fixed on 2006-12-25
#108493 (bug): file_date updates from package-release-nodes.php not always saved to the DB correctly Fixed on 2007-01-19
#97337 (bug): Module branch is not available as release target Fixed on 2007-01-21
#111323: allow maintainers to see unpublished release nodes Fixed on 2007-01-29
#94154: module page could say if new releases are available Fixed on 2007-07-11
#105224: download table is broken if a release has no file attached Fixed on 2007-07-14
#109181: provide feedback to owners when there are errors packaging releases Fixed on 2007-10-06

New Release System

cvs contrib procedure

aufumy@drupal.org — Tue, 16 Sep 2008 18:53:25 +0000

When I gained cvs access, I read the drupal book pages on cvs and also merlinofchaos blog, and polled irc drupal-support.

What I came up with was that there are 2 ways of maintaining releases for contributed modules.

1) Always have the latest code in head, and only release a branch when the new version of drupal comes out. For example when working with module and it is for drupal5 the code is found under HEAD, when the module is upgraded for drupal6, create a branch DRUPAL-5 for the drupal5 module.

2) Don't worry about having HEAD updated, and only work in the branches for the code specific to that drupal version.

I came to prefer option #2. This issue came back to my forefront after talking with mroswell.

Any thoughts, opinions?

Drupalchix

Commit restrictions in distributed version control

jpetso@drupal.org — Tue, 29 Jan 2008 16:47:17 +0000

I just spent some time in #git to further investigate how our CVS access control scripts would translate to distributed version control systems, in order to help determine the right direction for our new GHOP-powered Git and Mercurial backends (currently being worked on, more on that to come later). Short answer: keep out of that altogether - it's not DVCS style to restrict project maintainers like that. Read on for a more detailed analysis.

For those who are not totally familiar with the CVS access control scripts, they're currently doing two things basically:
1. They only allow the commit to happen if the commit location is part of those projects where the user is registered as project maintainer. In addition, sandbox commits are allowed to every CVS account owner.
2. They forbid non-standard tag and branch names.
3. Not implemented currently, but would also be possible: checks on the code itself, say, for making sure that the code adheres to coding standards or satisfies all unit tests.

Is it necessary?

Now, repository layouts work very differently in distributed version control: instead of having a huge repository where everybody commits to, there are usually lots of small repositories for each project. That kinda makes 1. obsolete: users don't have commit access to other projects anyways because those are contained in wholly different repositories. Also, users would probably have their own sandbox as separate repository, or even more probable, they'd have a separate repository for each of their sandbox projects.

As for 2., controlling branch and tag names needs to be done in CVS because branches and tags exist repository wide, so granting users more freedom on that would cause chaos all over the whole repository. However, in distributed version control, branches and tags are always local to the respective project and can easily be deleted too (which means that the chaos won't appear without active maliciousness of the maintainer), so this main reason for restricting branch and tag names does not apply here.

Of course, branch and tag names are also used on drupal.org for determining how release tarballs should be named, and restricting names means that one can rely on the fact that each branch or tag can be transformed into a release name (and subsequently be released). The folks on #git didn't consider that alone a good reason to disallow branches and tags that are differently named, and indeed the lack of super strict name control could enable more possibilities for developers, the most prominent one being feature branches that can be developed in the open before being merged back into the master or stable branch. This wouldn't hurt other projects or developers - the only precondition is that the release scripts don't try to package everything but only consider tags and branches with "release compatible" names.

Which leaves us with 3. as the only remaining "valid" justification for access control, and doing it for just this purpose been countered by the people on #git with a couple of points:

Stuff like style checks can be done locally in the commit hook instead of server-side on receiving the commits. The latter would be a problem anyways as there are many potential commits in a "push" operation, and if, say, the first one is "invalid" then all the rest of the locally committed changes has to be reverted and redone. Not good.
If people intentionally remove the commit hooks from their local repository copies, they don't deserve project maintainership and their commit access should be revoked anyways.
Running unit or compilation tests on each commit is likely very hard on the server, and would be better suited to run as cronjob (together with a "dashboard" to report errors or other issues).

Conclusion

So, overall, the design and workflow of distributed VCS makes a good point against pretty much all of the arguments why access control scripts would be needed for those repositories. drupal.org is already one of the most restricted repositories that I know of, and if we could get by without these measures then anybody can. (Also mind that admins and organizations that disagree here could still do all of these access control checks, of course - it just wouldn't be integrated with Drupal.)

Revision Control Systems

A complete solution for task/project/issue/case/ticket management with Drupal

mikey_p — Wed, 26 Dec 2007 23:07:10 +0000

I've been evaluating solutions for Project management (for the duration of this post, that includes what i describe as project, issue, ticketing, case tracking, and pretty much anything that falls in that category) solutions with Drupal, over a year actually. I keep being enticed by the features of each individual solution, and new promises that are announced for each module(s) and trying them out and coming to the same conclusion with each of them. And yes, all of them seem to have the same problems that I'm hitting repeatedly.

They are designed with too specific a project system in mind, and are not ultimately extensible.

Project module and it's family are designed mainly to run drupal.org, and it does an amazing job at it. However project attributes are not easily modified or extended. I know that dww has taken on what is no doubtly the most crucial element to the entire Drupal project[1] with this module, and he has made amazing progress with it. However, one constraint to this awesome capability is that new and advanced features...no that's not right, dww and crew and getting tons of new features and such into project, the issue I'm noticing, is that features that wouldn't be used on d.o, are less likely to be committed. This is no fault against the project maintainers as they are already plenty busy, keeping up with the necessary features for d.o. Don't get me wrong here, Project* is amazing, and I'm grateful to the maintainers, however its very requirements but some limitations on it. (of course I know that I am welcome to submit patches to project*, but I fear that would just place added burden on the maintainers to review and commit them)
Enter Casetracker. It's newer and seems shiny and featured packed at first glance, but once again it ultimately fails in terms of extensibility and features. As in most cases of Drupal modules, it begins with a fairly specific task in mind, becomes more configurable, but ultimately doesn't give the user all the options they desire.
Taslklist and Tasklist advanced are another option, designed for more personal level. There's some good stuff in these, but they just aren't designed for the same type of collaborative uses, and extensibility that casetracker and project* are. And they suffer the same problems that project* and casetracker do as well.

My solution to this problem as I keep trying to write new glue-modules to tie functionality between all these, is that I need to just stop and rewrite the whole module from scratch to do what I need. This will solve all of these problems once and for all. At least as far as I'm concerned anyway. But surely, I've realized, someone must have had the same conclusions I did? And their module to solve all these problems would be different from mine wouldn't it? What do we do in Drupal when we start having several projects covering similar and duplicate functionality?

An API you say?

Enter the idea of what I have termed the TaskAPI (could just as easily be Project API, but that seemed to close to existing project*). Why not write a module that handles helper modules to assign project attributes to a node, and their display and then a module to display all that content different ways? Finally, extensible project/task management!

For the uninformed, I've just summed up CCK/Views rather well. They already do this. There is already an issue to convert Project* module to CCK and it's being written to use Views as we speak. So is that it? Can we already to everything we need with other Drupal modules? Pretty much. I'd even say for probably 80% of sites out there (maybe more) CCK/Views can do almost all of their custom content needs. So is that the end of the TaskAPI? I don't think so.

Actually I think it is still needed now, more than ever, as the glue between all these modules, and to provide the ability to hook, non-data, non-display task behavior to tasks through various means. I can't explain that very well at the moment, but I already have several use cases, that would not work well, with just CCK/Views. Also, some new field types/formatters may be needed, maybe not. I'm not even sure of all the functionality that the TaskAPI would provide, other than to be widely extensible.

So now that the idea is out there, I'd like thoughts and ideas on it. Specifically, what behavior would you add to CCK/Views to have your dream task manager? Is there interest in this type of project? Would anyone like to contribute? Finally, if this model proves to be successful, what would have to be done, in order to make it a central API to extending Drupal? Would project* or casetracker ever require the Task API?

Please note, that while currently no code exists in the repository, or my sandbox on this, that doesn't mean that it hasn't ever been attempted. This article is the summary of 3 attempts to provide the basic functionality described. Once, I tried hacking casetracker and organic groups to get the functionality I required, another time I built a site specific module to combine functionality which didn't really work. The last attempt, was a generic Casetracker/Organic Groups integration module, which I intended to release, but the features just weren't workable, without hacking Casetracker or Organic Groups. This time around, I'm trying to get input, and advice from the community, and I am trying to assess which current codebases could be workable to modify as well.

So let me know what you think!

[1] Without Project module and it's family of helpers, drupal.org wouldn't have an issue queue, and all development on Drupal itself would grind to a sudden halt.

Project Management

Project* roadmap for D6

dww — Tue, 18 Sep 2007 08:38:58 +0000

Now that Drupal 6 is out, getting the project* family of modules ported to D6 is going to become an urgent task. We want to ensure that by the time the final release candidates are out, we're ready to upgrade drupal.org to D6 (which must happen before the official 6.0 release is possible). That's going to take a lot of effort, and hunmonk and I have some very specific plans for how it should all happen. This page will be the place to keep track of what has to get done, in what order, so that anyone who wants to help knows where to put their efforts. It's obviously a work-in-progress, so feel free to help keep it updated, add issue links where appropriate, cross off things that are completed, or add other steps that need to happen.

I'm cross-posting this to a lot of groups since: a) converting project* to use views is a big part of this effort (so we have significantly less code to support in project* and port to newer versions of core), b) since we need lots of volunteers/help, and c) in case anyone is able to help sponsor some of this work to ensure that everything is completed well before the core maintainers would like to ship 6.0. Thanks!

~~#103790, #103791, #103795 and #177271: Make project* E_ALL compliant (since we have to do it eventually, anyway, and it'll be easier to do this before we make more branches, etc).~~
~~finish up existing project module bugs here: http://tinyurl.com/ynq2n6 and also here: http://groups.drupal.org/node/7396~~ Finished on 2008-01-06
~~Project: cut a 5.x-1.2 release of everything~~ (finished on 2008-01-06), and finally create the DRUPAL-5 branch.
~~Project issue tracker: Fix issues at http://groups.drupal.org/node/7396, cut a 5.x-1.2 release of everything,~~ ~~and finally create the DRUPAL-5 branch.~~ Finished on 2008-01-06
~~#18920: Convert issue followups to comments (mostly done, just needs more help with the upgrade path, testing, and more reviews).~~
~~Project subscriptions or other aspects may benefit from this proposed enhancement to Views: http://drupal.org/node/103171.~~ Note: there is now a module called Views Bulk Operations that makes this patch unnecessary.
Test the Version Control API, CVS backend and Project integration version control modules. The release node integration needs to be completed (probably in versioncontrol_project). [This can all happen in parallel with most of the rest of this list, unless we decide it'll be significantly easier to do the release node FAPI integration via D6 FAPI]. -- postponed until after d.o is running 6.x, see http://groups.drupal.org/node/8102

Work to do for the actual 6.x ports

#157694: Port project to 6.x
#157693: Port project_issue to 6.x
#76725: Convert the issue queues into views (project_issue_views.inc) -- I started this for my day job -- however, the goal would be to actually rip out all the existing code and make project_issue require views, which will take a little more work (but remove *tons* of code we'll no longer have to maintain).
#76726: Convert the project browsing pages into views (project_views.inc)
~~#209408: Port CVS module to 6.x~~
#225059: test CVS 6.x project components
Deploy everything on s.d.o

Work to do after the initial 6.x port

Port versioncontrol* to 6.x. Probably sounds like a lot, but it's not much more code than the existing CVS integration module, and it's a) much better code and b) much more flexible for the future, and c) potentially useful to other sites. [Note: this can happen in parallel with project_issue porting]. -- postponed until after d.o is running 6.x, see http://groups.drupal.org/node/8102

New Release System

Garrett Dimon discusses his issue tracking design

alpritt — Tue, 21 Aug 2007 13:58:47 +0000

Garrett Dimon is writing a series of posts on the design of his issue tracking system. The design if for smaller projects than Drupal, but it may provide some inspiration.

So far:

http://garrettdimon.com/archives/2007/8/14/bug_issue_tracking/
http://garrettdimon.com/archives/2007/8/20/the_tracker_status_bar/
http://garrettdimon.com/archives/2007/8/20/tracker_status_amp_comments/

New Release System

Project* TODO list

dww — Tue, 07 Aug 2007 19:55:00 +0000

Instead of keeping potentially stale copies of a TODO list in the README.txt files in CVS, I'm moving these list to a wiki page here so there's just a single canonical copy. This is general stuff about the modules. Other to-do lists of interest include:

Project module

Improve themeability of project listings.
Integrate image module galleries for screenshots.

Project issue tracking module

~~Using the existing Drupal "comment" system for issue replies instead of the customized issue "follow-ups" currently in place. http://drupal.org/node/18920~~
Better mailhandler integration for creating and replying to issues via email.
Add method for project owner to assign issues to others.
Add a setting where you can select what node type should be the "project" that project_issues are associated with.
Improve themeability of listings and issues.
Allow projects to specify their own categories (bug report, feature request, etc).
Tool for administrators to be able to update the status of several issues at once (mass closing).
Make components sub-projects with their own defaults.

New Release System

Issues to resolve in project* before a 5.x-1.0 release

dww — Sun, 05 Aug 2007 00:00:12 +0000

At http://drupal.org/node/150278 we've started serious work on an official 5.x-1.0 release of project* (project, project_issue, and cvslog).

Here's my current list of issues I'd like to see us resolve for the 5.x-1.0 releases, if possible. Mostly, these are either a) critical bugs, or b) patches I've already written for relatively easy-to-solve problems. Feel free to add comments or update this list. And, of course, please work on any of the issues below -- reviewing, testing, rolling patches, etc. Thanks! -Derek

Bugs we should definitely fix

~~http://drupal.org/node/119860~~
"Release download returns 'Page Not Found'" (for sites with private file downloads -- patch needs work, but not much).
Fixed on 2007-08-05
~~http://drupal.org/node/133052~~
"remove dependency on taxonomy" (project_release will depend on it for 5.x-1.0, but project + project_issue shouldn't need to).
Fixed on 2007-08-09
~~http://drupal.org/node/91195~~
"cvs_get_version_from_tag() assumed to exist, but lives in cvs_local.inc" (easy fix, patch already done)
Fixed on 2007-08-16
~~http://drupal.org/node/163586~~
"Project browsing shouldn't exclude projects without a file" (easy fix, patch already done).
Fixed on 2007-08-06
~~http://drupal.org/node/114281~~
"'Access own project issues' access privilege not working as expected" (too many project* menu paths don't honor it, so you end up needing the full 'access project issues' permission to use the issue queues, etc, anyway).
~~http://drupal.org/node/106654~~
"add index for sid" (has a patch, needs review).
~~http://drupal.org/node/165975~~
"fatal error when adding new project issue status in postgres"
Fixed on 2007-08-09
~~http://drupal.org/node/166496~~
~~http://drupal.org/node/167105~~
"Missing hook_uninstall()"
Fixed on 2007-08-15
~~http://drupal.org/node/159419~~
Add pgsql support to cvs.module
Fixed on 2007-08-15
~~http://drupal.org/node/164958~~
Add missing pgsql indexes to project_issue
Fixed on 2007-08-15
~~http://drupal.org/node/159286~~
"Extensions validity incorrectly tested for attaching files to issues"
Fixed on 2007-08-17
~~http://drupal.org/node/167838~~
"Should only add CCK fields to followup form, not issue form."
Fixed on 2007-08-16

Bugs/tasks we should fix if we can, but aren't blockers

~~http://drupal.org/node/159892~~
"Rip out menu callback for serving release .xml files"
Fixed on 2007-08-07
~~http://drupal.org/node/164846~~
"Rip out XML-RPC server code for update_status 5.x-1.* clients"
Fixed on 2007-08-07
~~http://drupal.org/node/142122~~
"make sure only users with valid mail addresses can subscribe"
Fixed on 2007-08-15
~~http://drupal.org/node/104190~~
"Submit issues cannot find projects" (problem when project_issue is installed *after* project nodes are created).
Fixed on 2007-08-15
~~http://drupal.org/node/103806~~
"pager count query not correct on CVS browsing pages"
Fixed on 2007-08-18
~~http://drupal.org/node/107706~~
"make query use index"
Fixed on 2007-08-16
~~http://drupal.org/node/59039~~
"attachements missing after pressing preview" (for issues)
Fixed on 2007-08-18
~~http://drupal.org/node/155697~~
"Attached file not added" (for release nodes -- probably duplicate with (G) due to re-using similar code)

New Release System

Still new?

moshe weitzman — Thu, 19 Jul 2007 21:30:17 +0000

how long is a release system allowed to describe itself as new?

New Release System

How to help with the project* modules

dww — Mon, 16 Apr 2007 09:45:37 +0000

The project* collection of modules (Project, Project issue tracking, and CVS integration) is the largest set of code running on drupal.org besides Drupal core. They are the key tools that power all Drupal development, including the Drupal issue queue and Drupal release system. Because of the huge user base, high visibility, complex requirements and feature requests, and size and scope of the existing code base (and issue queues) there is a ton of work to be done. This wiki page is how I (dww, the primary maintainer of project*) will try to communicate the best ways for other people to get involved in helping. One of the ways to help is to improve this list, so please add your own ideas here.

This list is roughly in order of what kind of skills you'd need, but not exactly. It's not in priority order, since all of it would be helpful, and it's hard to prioritize given there might be many people with different skills or interests. So, just read the whole list and decide what parts of it you'd like to help with. In terms of the issue queues, you can obviously prioritize things marked as critical.

Join the New Release System group.
Read the handbook pages about the project* modules on drupal.org. Figure out how to install your own local test site running 5.x Drupal core and the latest versions of these modules. Improve and/or extend the handbook pages as you go.
Read this post about the drupal.org testing installation profile. See if you can help make that profile more complete, if you can get it to work, if you can contribute documentation on how to use it, etc.
Read these 2 issues about testing on http://scratch.drupal.org :
- 5.0 release candidate now running on scratch.drupal.org (http://drupal.org/node/105004)
- Testing the 5.0 port of the release system on scratch.drupal.org (http://drupal.org/node/105016)
They're slightly out-dated, but the basic idea is still valid: see if you can find any new bugs, and if so, report them. Both of those pages include detailed instructions on where to report any bugs or problems you find. Please don't submit duplicates, see below. ;)
Read through the Project* issue queues:
Familiarize yourself with what's there now (this is a fairly huge task on its own). Then, here are the ways you can help clean-up the queues to make them more manageable for other users and developers:
- Find duplicate issues and mark the newer issues duplicate, including a link to the oldest issue in the queue about the same thing.
- Older issues that are no longer relevant should be marked "Fixed" (e.g. things that are now working because of the new release system, etc).
- If an issue's title isn't clear, try to figure out what the issue is about and give it a more descriptive and acurate title.
- See if you can reproduce any of the bug reports on a local test site. Start digging into the code and see if you can figure out anything about what's going wrong. No matter what, report your findings in a follow-up in the issue so other people can see your results. Just knowing if you can (and how to) reproduce a bug on a local test site is a huge step towards fixing the problem.
- ... (feel free to add other suggestions here)
Keep an eye on the webmasters and infrastructure issue queues for bug reports and feature requests that really belong in one of the project* issue queues:
- Drupal.org webmasters
- Drupal.org server administrators
Since project* is so central to the functionality on drupal.org, people often request features or report bugs to these issue queues, not necessarily realizing that there are separate queues for the project* modules themselves, and these bugs or features almost always exist or would work on other sites running the project* modules. If you find issues in either of these queues that is general to Project* code, not specific to how it is setup on drupal.org, move the issue into the appropriate project* queue.
Keep an eye on the drupal.org forums for questions about project* and see if you can answer them, point the person to the appropriate issue if it's a known bug or feature request, etc.
Read the CVS handbook. See if you can find ways to make any of the pages more clear. If you've got documentation powers on drupal.org, try to fold comments into the body of the page and delete them. If you're unsure, just ask me (see below).
Read the Maintaining a project on drupal.org handbook. See if it makes sense. See how much of it should be folded into the general project* documentation, or if there's a better way to organize this information.
If you haven't already, read the patch handbook. Then, take a look at the "patch queue" for these modules (all issues marked as "Patch (code needs review)"):
See if the patch still applies to the version it's reported against. If not, set the issue to "Patch (code needs work)". If you can, resolve the conflicts and re-roll the patch for that branch, post your new patch to the issue, and set the status back to "code needs review". If it's a patch for a feature request, see if it applies against HEAD, or the DRUPAL-4-7--2 branch, and set the version of the issue accordingly. No matter what, if the patch applies, test it. Again the patch handbook has great instructions on how to apply, review and test patches, so read that if you're unsure what to do.
Test all of these modules using PostgreSQL. There are some known issues in the queue, so review, apply, and test those patches first. If you find any new problems, submit detailed bug reports about them.
Dig into the code and see if you can understand any of it. ;) If you figure out something that seemed cryptic at first, write some helpful comments in the code and provide a patch for it. If you find a function without doxygen comments that explains what the function does, what kind of input it expects, and what output it produces, write the comment and post a patch (feel free to include more than one function's comments in the same issue/patch).
Check out the list of Future work for the new release system, and see if any of it interests you enough that you'd like to start working on a patch for something. If so, follow-up to the issue, assign it to yourself, and start sketching out your thoughts on what needs to happen. Feel free to just ask questions in the issue about implementation details, etc.
Add to this list.

I'm often available on IRC in #drupal (my nick is also "dww"), and I try to lurk in #drupal-dojo whenever I get the chance. Feel free to add a comment below if you want to get my attention about something. If you have a question about an issue, just followup to the issue itself, since other people might be able to answer it.

That should be more than enough to get anyone started, no matter what skills you have to offer. ;) Please dive right in and help in whatever ways you can. Drupal.org and project* need you!

Thanks,
-Derek

Projects Needing Volunteer Developers

Help Me to Help You

alpritt — Fri, 13 Apr 2007 20:44:13 +0000

Hi,

I'm thinking this may be a good place to start seriously contributing to Drupal. Basically I'm looking for a project that needs help (!) and that will help me to get a strong grasp of how Drupal works. I know Project* fits the first criteria, but I'm a little unsure how well it will help me grok Drupal as a whole.

I'm particularly interested in User Interface, but I'll work happily on anything under the hood since that is what I need to learn. My plan is to do Drupal development professionally once I have enough experience, so what I work on now really needs to help me learn with that in mind.

I'm a beginner going on intermediate so I'm not sure how helpful I'll be at first. But I'm currently finding about 35 hours a week to learn Drupal so hopefully I'll improve fast. I figure if I give a portion of that learning time over to working on a specific project I'll learn faster.

Does it make sense for me to just dive straight into this project, or should I have mastered anything in particular first? The thought of working on this is a little scary since I really don't know what is going on, but that is why I think it might be a good idea -- a chance to push myself.

Any thoughts/directions/help will be much appreciated.

Thank you,

Alan

New Release System

Improve module categorization

Frando — Sun, 01 Apr 2007 13:34:26 +0000

It is a known fact that the drupal modules page (http://drupal.org/project/Modules) is not perfect. There's a lot of discussion going on about how to gather module quality metrics, whether to enable a voting module or not, etc.
It's quite true that it's currently rather difficult to find what you're looking for. I think that by now I've got quite a good overview about the "hot" modules, but still for me, I often find the modules by using the advanced search or by checking cvs commit track page of the user of which I know he created the module.
If you're trying to look for new stuff, it's getting even harder. Most categories contain more than 50 modules, some even up to over a hundred, and many are attached to several of the just 26 module categories.

I think the discussion about sorting modules by their "quality" is a very worthy one, but most solutions involve coding and therefore will need quite some time to get deployed.

However, there's quite a simple thing we can do right now without any coding (at least I think so) that would increase the usability of the module page enormously: Improve the categorization. Drupal is famous for its taxonomy system, so why shouldn't we use it in a more sophisticated way?

For example, there are at least 10 modules out there that provide more usable menu blocks, usually by using DHTML (examples are nice_menus, dhtml_menu, lucid_menu, simplemenu, ..), yet they're scattered all over the different categories.

So, what I propose is to refine the module categorization. For example by using at least two levels of hierarchy.

This is a wiki page, so anyone is welcome to update and improve this working draft for a better categorization of drupal modules. Whenever you edit this page, please add a comment that quickly describes your changes!

Working draft for a better categorization of drupal modules

(very early working draft)

Usability enhancements
- Menus
- Javascript-based usability enhancements
- General usability enhancements
Administration tools
- General administration usability
- Taxonomy management
- Content management
- Comment management
- Import / Expor tools
- Backup tools
- Statistics
Caching and Performance
Content
- Content rating systems
- Event managment
- Content listing
- Paging
- Input filters
- Editors
- Image modules
- Media integration
- Project management
- Wiki related
- Forum related
Content access
- Taxonomy-based
- Forum-based
- Menu based
- other
Internationalization
Communication
- Mailing
- Newsletters
- Messaging
- Chat
E-Commerce
Aggregation & Syndication
- OpenSearch
- Content aggregation
- Feed creation
File managment
User management
- Authorization
- other
Theme related
3rd party integration
- Web services
- Authorization
- Bridges
Module specific
- Views
- Organic groups
- VotingAPI
- CCK fields
- other CCK related modules
Developer modules
API-only modules

New Release System

Proposal: drupal.org testsite install profile

dww — Sun, 25 Mar 2007 11:06:51 +0000

this is an idea i've been thinking about. posting it here so other folks can add to it, provide feedback, etc.

Summary

we could really use an install profile that setup a test site similarly to how drupal.org is configured (especially the project* modules and CVS integration). obviously, it wouldn't have the bluebeach theme, but otherwise, it would look fairly similar to d.o in terms of structure, content, permissions, etc.

this would greatly help people who wanted to reproduce bugs and test patches for the project* family of modules, which is something we desperately need more of. it also might be a popular profile as a starting point for people looking to setup software-collaboration sites (though ideally we'd have a separate "software development site" profile eventually).

here's what i envision (but please edit this to expand or modify it):

enables and configures all the project* modules and the CVS repo stuff
sets up the d.o project taxonomy stuff (project types, module categories, etc)
enables and configures the devel.module
configures the d.o roles and permissions
generates a bunch of random users
generates a few targetted users for each of the main roles on d.o, with documented usernames and initial passwords
generates a skeleton set of project nodes (core, some of the common contribs, cross-section of different project types (modules, themes, translations, etc))
generates a bunch of random releases
generates a bunch of random issues
(optionally) generates some random forums and comments
...

so, you download all the right versions of things, install this profile, and you're ready to start testing patches for the d.o infrastructure.

in his "State of Drupal" talk, Dries was making a plea for more people to help with making our home on d.o more like a palace. this should significantly facilitate getting more "builders" involved in the construction. ;)

Details

specific modules

see http://drupal.org/node/27367 to start with. ;) otherwise, contribs i'm specifically interested in, are:
* cvslog
* project (including project_release)
* project_issue
* code filter

taxonomy

The project taxonomy stuff is a little weird. you get the basic idea: there's a "project" vocabulary, which is hierarchical. top level terms are "Module", "Theme", "Installation profile", etc, etc. At this point, only Module has sub-terms, which are the module categories (see http://drupal.org/project/Modules for a list of them). Once http://drupal.org/node/105986 is done, and contrib translations are in their own projects, we'll have sub-terms for each language under the "Translations" term, but that's getting ahead of myself. ;)

There's also the project release vocabulary -- "Drupal Core compatibility" (for the "5.x", "4.7.x", etc terms on release nodes, which is also used in a bunch of the download pages, etc).

I see 2 possible solutions for this:

you just manually re-create a subset of this structure (not every single module category, etc)
we actually export the relevent records from the {term_*} tables directly (and regularly) from d.o. not sure this idea will fly, but perhaps we could have a cron job that just does a selective mysqldump and sticks the .mysql file somewhere folks can just download. ;) i'm *only* talking about the structure of the project (and project release) vocabularies, nothing more.

cvs repository

ideally, the install profile would include a tarball of a pre-configured cvs repository that had the right basic directory structure, and some files for the project nodes that would get created with the profile. all the contributions/modules/cvslog/xcvs* stuff would be configured in the CVSROOT of this repo, and would be customized during install to point to the right drupal database. there's a pretty thorough README.txt in there now. we'd probably want to create 2 repos, in fact -- the contrib and core ones.
then, of course, the drupal site would be configured to know about each repo (via [site]/admin/project/cvs-repositories).

specific nodes

i don't think it's feasable or desireable to actually replicate all of the project, issue, and follow-up content from d.o. i was thinking we'djust creating a small handful (~10) different projects that are representative of the structure on d.o. these would be the same small handful of directories in the stub CVS repository. they'd have some specific CVS tags and branches, too, and the corresponding release nodes. once that's in place, i'd say we just randomly generate a bunch of issues and comments automatically.

roles and permissions

i'm not sure how much of this i can share, nor how much of it is really necessary for the needs of this profile. basically, we should just configure the anonymous and authenticated users with the appropriate perms, and then make an uber-site-admin role with all perms enabled. the whole "document maintainer" vs. "site maintainer" vs. "site admin" stuff probably isn't essential -- though it might be nice to tackle in a future version once the other parts are working smoothly.

other settings

i'm not sure what's the best way to deal with this: if i should (can?) just provide a dump of the relevent entries in the {variables} table, like i'm proposing for the {term_*} stuff. i'll try to get some of the more senior powers that be to comment on this part, too.

2 important Project module and drupal.org improving SoC ideas

dww — Sat, 17 Mar 2007 09:11:06 +0000

Myself, Webchick and Killes have come up with two proposals for this year's SoC which involve the drupal.org infrastructure and the project module. Both of which could solve some of the worst drupal.org usability problems, and yet are interesting, flexible projects where someone with a lot of creativity and talent could have a huge, highly visible (and valuable) impact on the entire Drupal community.

I'm hoping to generate interest (so that the right applicatants take on the projects) and to solicit feedback from other SoC mentors or interested parties in the scope and direction of each proposal:

http://drupal.org/node/128546
Create project quality metrics system

http://drupal.org/node/128616
Better translation support for Drupal contributions

Please see each proposal for more details, and feel free to just comment on those directly instead of this post.
Thanks!

Issue tracking and software releases

Help test 5.x core (and project*) on scratch.drupal.org -- become a project ninja!

dww — Thu, 28 Dec 2006 08:44:38 +0000

the Drupal Dojo sounds great, and i'm happy to join the efforts (as both student and teacher, it's a whole spectrum, not a bit). anyone who wants to dive into Drupal development right now can help by mercilessly beating on http://scratch.drupal.org to test out the latest 5.x release candidate code, and submitting issues (with patches, if possible!) about any problems you find.

i'm particularly interested in people testing out the project* set of modules (the biggest set of code running on the [s].d.o infrastructure that isn't Drupal core itself). this code is essential for all Drupal development, but is a huge, and (unfortunately) fairly complex chunk of code, which is mostly maintained entirely by one person: me. since it's not core, the pool of people testing it (aside from live use on drupal.org) and contributing code and documentation for it is tiny. if the Dojo wants to really do some good in the Drupal world (and it seems like it certainly will!), one of its big goals should be expanding the pool of people who understand, test, contribute patches to, and eventually help commit code and fully maintain the project suite of modules.

here are some of the many ways you can help:

read these 2 issues about testing on http://scratch.drupal.org
- 5.0 release candidate now running on scratch.drupal.org (http://drupal.org/node/105004)
- Testing the 5.0 port of the release system on scratch.drupal.org (http://drupal.org/node/105016)
both of those pages include detailed instructions on where to report any bugs or problems you find.
join the New Release System group.
figure out how to install your own local test site running 5.x Drupal core and the latest versions of the
Project, Project issue tracker, CVS management/tracker modules. document as you go and make some handbook pages about them on drupal.org... ;) see http://drupal.org/node/87175 for an issue about doing this in the project issue queue.
check out the issue queues for the "HEAD" or "5.x-0.x-dev" versions of the following modules:
see if you can reproduce any of the bugs on a local test site. start digging into the code and see if you can figure out anything about what's going wrong, or how you might implement a new feature.
check out the list of Future work (http://groups.drupal.org/node/1830) for the new release system, and see if any of it interests you to start working on a patch for something. if so, followup to the issue, assign it to yourself, and start sketching out your thoughts on what needs to happen. feel free to just ask questions in the issue queue about implementation details

i'll try to start hanging out in #drupal-dojo whenever i'm in #drupal. i might sometimes come across a little scary, but i'm *always* thrilled for anyone who wants to help with project*, cvs.module, etc.... i can be very nice and helpful when i want to be. ;)

looking forward to a productive collaboration... huge thanks to josh for getting this group started!!!

-derek (dww)

Drupal Dojo

Additional resources about the release system

dww — Wed, 27 Dec 2006 07:09:21 +0000

I've noticed a couple of other helpful resources on there in The Wild(tm) relating to my efforts on the new release system. I figured it'd be nice to list them all here, so other folks can a) benefit from easy access to the additional info and b) add to the list as they see fit. Perhaps some of this can/should be linked to from the CVS handbook pages about the release system on d.o? Anyway, here's the list...

Lullabot Podcast #28: Derek Wright and the New Release System
My informal take on using the new release system (post by Earl Miles a.k.a. merlinofchaos).
Testing the new Drupal release system (post by Kjartan Mannes, original author of the project.module).
Project of the Month: Drupal Project.module

That's probably not exhaustive, so if anyone knows of other sources, please comment here and i'll incorporate them into the post.

Thanks!
-Derek

Issue tracking and software releases

Distributions: The prerequisites on drupal.org

Frando — Sat, 23 Dec 2006 12:32:49 +0000

Cross-posting an email I sent to infrastructure list a few weeks ago, together with Kjartan's response.
It would be great if drupal.org supported distributions by the time Drupal 5 will be released.

The two mails (by Kjartan and me) outline a way to support distributions in an easy way (both for developers and end users). So what are your opinions on this?

frando said:

1. For me, naming seems to be still unclear and ambigous. We should
clarify this (profile/installation profile/distribution).

2. Now on my idea what we could do on d.o., I'll call it 'distribution'
for now. However, as I stated above, I'm not yet clear about the naming.

a) Create a new top-level term and cvs folder "distributions"
b) A particular distribution shall then only consists out of the following:
- a distname.install installation profile
- README, UPGRADE and so on
- a new distname.info. In this info shall be a list of all contrib
modules (each with a specific version - once again kudos to derek) and
themes and version of core used in the distribution. E.g.

[mydist.info]
core=drupal-5.0
modules=cck-5.0-1.1,views-5.0-1.0,project-5.0-2.1
themes=mytheme-5.0-1.0

- the package script (the only part of project that would actually need
improvements for this proposal) would then grab all the files together
(via cvs export):

* specified version of drupal core
* all the modules and themes specified in the distname.info (only
specific versions (tags) should be allowed - no branch/development
snapshots/releases, so that the distribution maintainer retains full
control over which exact code will be in the final tarball)
* everything that's in contributions/distributions/distname under the
appropriate release tag (e.g. additional docs, a logo, a
distribution-specific theme, ...)

and create a tarball out of it. All contrib that's includet via the
distname.info gets placed in profiles/distname/modules|themes. The final
tarball is then available for download via a normal project release node.

If any distribution needs more than this, drupal.org should not host it
directly.

However, I think that a distribution can achieve a lot with this system,
and the whole managment is still on d.o. (release system, issue tracker,
cvs access). It does ease pains for distribution maintainers, as they
can easily move issues to the appropriate modules and create releases
via the project.module. We avoid duplication of code and potential forks
by not duplicating the already existing contrib code in the
contributions/distributions/* dirs. And we ease the whole thing for
normal users as they can download ready-packed tarballs directly from d.o.

regards,
frando

Kjartan Mannes said:

This is identical to the system I was thinking of suggesting. It would
give "distributions" the power to customize Drupal, while making sure
that it does not needlessly duplicate the existing code in any way. It
also encourages "distribution" maintainers to work with the module
authors instead of seeing duplicated maintenance processes

Just like the contrib module/themes we should have some required files,
which should probably be more strongly enforced. I'd also like to see
them somehow add a field on the admin/logs/status detailing its a
distribution, with a link to a more detailed description of which
modules/themes are used and where they are from.

I'm pretty sure we can do this in two or three phases:

Phase 0: Determine the proper name to use. Distributions feels wrong to
me. Install profiles is too limited, and while discussing them people
tend to just leave out the "install" bit and then they risk being
confused with the profile module. I doubt we can get away from Install
profiles though, seeing the directory where they are stored is called
profiles/.

Phase 1: Add a "distributions" folder to the CVS repos so people can
start working on install profiles. Users who want to use these profiles
will have to download the necessary modules/themes from contrib
themselves. This gives "distribution" maintainers a chance to start
working and exploring the possibilities without waiting for Derek or
someone else to code the necessary backend.

Phase 2: Support distribution.info and launch the packaging script.
Should make it easier for the everyone and hopefully everyone will be happy.

Phase 3: Take a look at how everything is being used and see if there is
anything else that can be done to make it all better.

New Release System

Dedicated project space for install profiles and distributions?

gusaus — Sat, 16 Dec 2006 02:52:18 +0000

Are there any existing guidelines or best practices when it comes to developing specific install profiles and distributions? As far as I can tell, I don't see info on this type of contribution (http://drupal.org/node/22286) or a means to set up a project similar to how you'd do it for a module or theme (http://drupal.org/node/7765).

I think this question is relevant to the context of this group. Any guidance would be much appreciated.

New Release System

New Release System

Future work

cvs contrib procedure

Commit restrictions in distributed version control

A complete solution for task/project/issue/case/ticket management with Drupal

Project* roadmap for D6

Work to do for the actual 6.x ports

Work to do after the initial 6.x port

Garrett Dimon discusses his issue tracking design

Project* TODO list

Project module

Project issue tracking module

Issues to resolve in project* before a 5.x-1.0 release

Bugs we should definitely fix

Bugs/tasks we should fix if we can, but aren't blockers

Still new?

How to help with the project* modules

Help Me to Help You

Improve module categorization

Working draft for a better categorization of drupal modules

Proposal: drupal.org testsite install profile

Summary

Details

specific modules

taxonomy

cvs repository

specific nodes

roles and permissions

other settings

Related Links

2 important Project module and drupal.org improving SoC ideas

Help test 5.x core (and project*) on scratch.drupal.org -- become a project ninja!

Additional resources about the release system

Distributions: The prerequisites on drupal.org

Dedicated project space for install profiles and distributions?