Evaluation discussion for how to move Drupal.org off of CVS

Was just fleshing this list out but my efforts have now 'been flagged as spam and will not be accepted'. So I'm giving up, which is a pity, because I had a lot more to add. No wonder so little has been added to this.

Can't we get away from the Not-invented-here syndrome and use tools that actually work? Like say http://etherpad.com/
Even google wave is better than this.

Immensely frustrating.

/me grins

webchick dear, some of that verbiage looks suspiciously like something I sent you in an email last fall...or is that just my imagination?

EDIT: oh wait. i think credit for including those goes to marvil07.

At chx's urging, I'm posting my opinion that whatever we do has to still facilitate a patch workflow for those who use other VCSs. I'm probably stating the obvious -- I'm pretty sure everyone agrees that patches should still be accepted. :) But I do think it's important. Whatever VCS we choose, somebody's going to hate it, and they'll probably start a mirror in their VCS of choice. That has to be okay.

Here's a proposal we kicked around in IRC for how this might work. It is probably totally wrong, but will create a starting point for discussion, anyway.

Regardless of how we set this up though, the main thing we absolutely do not want to lose is the collaborative problem-solving via the centralized issue queue workflow that we have now. I would argue that it is the very cornerstone of our awesome community. It provides a single place to look for any and all decisions made on any line of code in all of core/contrib, one set of tools for contributors to learn, metric tonnes of mentorship both direct and indirect from the discussions, etc.

So here's an idea of how a hybrid might work. Please ignore my legacy terminology below, which I've put in em tags, so we can later replace it with whatever bzr/git/whatever calls them. For now, I'm sticking with CVS terminology so we can all stay on the same page.

1. Bug reporter: Someone finds a bug. They go to the issue queue and create a new issue, say, issue #987654. Based on the issue metadata they select (project, version, etc.), this will automatically create a branch of that project called '987654' in our repository.
2. Patch author: Someone who wants to work on that issue will create a clone branch of that 987654 in their own drupal.org-hosted repository, at xxx.drupalcode.org/users/[username]/projects/[projectname]/987654, and start making lots of commits. When they hit a good stopping point, they'll post back to the issue and say "Hey, check this out! Here's a summary of what I changed..." and then fill in the commit ID (or URL or whatever) to their latest changeset in the reply form.
3. Patch reviewer: They go surfing through the issue queue and come across something interesting that's marked "needs review." They pop it open and scroll down to the latest patch issue reply. The reply includes a link to the full commit logs to this point, a link to the diff of changes, and (if I can have unicorns AND kittens), a tarball of the parent project + diff together so that folks like the UX team can easily test them. They apply the patch using $vcs, or they download a tarball and test it and report back.
4. Patch reviser: Someone else who wants to work on the same issue can either branch off of the commit indicated in the patch reply, and begin committing incremental improvements to their own xxx.drupalcode.org/users/[username]/projects/[projectname]/987654 directory, which the original author can choose to incorporate or not, or can say "No, you're totally wrong" and "branch" off of the main 987654 branch instead, to start all over again.

One concern I have is that as far as I can tell, in order to accomplish this, every single registered user on Drupal.org would get their own /username/ repository hosted on our central server, in order to satisfy two critically desirable situations that we currently have:

1. Absolutely anyone can make a patch to any project in all of Drupal using the same set of tools.
2. The entire back-story of all development decisions in every project in all of Drupal is here on drupal.org. (a couple of outliers notwithstanding)

So I assume we'd need some pretty fancy safeguards in place (extension checking, filesize checking, etc.) to ensure that this does not turn into a gigantic warez repository. :P

One of the reasons that most people right now are using Github is as more or less a "sandbox" area to shove code into. If Drupal.org's set-up could better facilitate this, I think that would help stem some of the bleeding in terms of people going elsewhere.

I also haven't quite figured out how we're going to handle granting commit access to the contributions repository, where our users go to get all of their modules and themes, and so therefore necessarily needs to be more tightly locked down. But whatever VCS we choose needs to support this kind of user-based role granting system like we currently have with CVS (the concept of maintainers and co-maintainers, etc.)

bzr is also python is seems (and launchpad is built on zope) so not sure why goggle would prefer hg to bzr if language was the sole issue.

Aside from its potentially better merging algorithms for renames, I think i'd favor bzr for drupal.org since it supports a checkout mode where it operates just like svn (update and commit talk directly to the remote branch) - also it can integrate with svn or git or hg remotes, comes with a GUI, etc.

Our action list has gotten a lot less action-y. :) So I got it back to concrete to-dos, along with ____ to represent blanks that need filling in. Please go nuts filling in said blanks, signing up for $vcs brigade, and investigating into drupal.org integration issues and posting back with your results.

I find it extremely interesting that despite the concerns about git being overly complex (which I have no doubt are true), the Git documentation resources listed so far are actually really, really great. And of the people involved in this thread so far who have voiced a preference (which presumably means they use these systems and would be able to help others who were having trouble), about 10 of them support Git, 2 of them support Bzr, and 1 supports Hg. This is a huge indicator to me, as someone whose primary concern is keeping the community intact and couldn't care less about what we choose as the technology, that Git is worth some serious consideration, and may indeed be the easiest path forward for our particular community.

Is this actually reflective of our community's preferences and general comfort level, or have the bzr/hg people just not heard of this initiative yet? I've Twittered about it a couple of times, but I'll also try and write up a Drupal Planet post early this week to get the word out better.

See also this Stack Overflow thread for a lot of good info.

Mollom won't let me add this but here's a CVS to HG link: http://mercurial.selenic.com/wiki/CvsConcepts

At this stage I think it is safe to rule out Mercurial.

I also added a summary of the existing contenders. I don't really think we can make a bad choice here. My "gut" gives Git a slight edge due to the enthusiasm and general community momentum behind it, but having people actually lined up to do the work on Bzr, along with the learning curve advantages place them about even-keeled, at least from where I sit.

I added web admins as "Users" in the use case section. Actually, I see no reason why drush shouldn't ship with the capability of cloning project repos instead of downloading tarballs. You can even re-apply existing private patches (or preliminary cherry-picked ones from upstream) by rebasing, instead of fuzzing with the regular patch tool over and over again. Pretty cool stuff can happen if you keep your site under version control!

Launchpad

• Based on Zope, Python, and PostgreSQL
• Tightly integrated with Bazaar but can import/mirror most other systems
• Focus on full development and release workflow, including blueprints, sprint planning, bug tracking (including cross-project and external)
• Questions and answers system
• Support for development/release series, in the same way we currently have them for 5.x, 6.x, and 7.x module versions
• No wiki system
• Tools for creating releases, managing tarballs, and linking bugs/blueprints/etc. to each release
• Merge proposal tools that clearly interleave comments, reviews, and updates to the code. A net diff is visible from the merge proposal; viewing detailed commit and branch content requires hopping to Loggerhead (linked from the branch).
• API that supports external testing tools
• OAuth for external app integration
• Supports translations, including committing the changes from the web-based work back to the Bazaar branches
• Integrates with Mailman to provides lists for projects that desire one
• Announcement tools with RSS/Atom/something feeds
• Branches link to Loggerhead (a tool not fully Launchpad-integrated) for branch and commit browsing
• Rich user profiles: https://launchpad.net/~jonobacon

Gitorious

• Based on Rails
• Tightly integrated with git, and imports are handled outside of Gitorious
• Mostly focused on merge workflow
• No bug tracking
• Wiki system
• Allows downloads of branch content, but not release-style tarballs
• Merge proposal tools that allow comments and multiple changes to the code, but without a clear relationship between comments and code changes other than both being timestamped. Each commit's diff is individually browsable directly from the merge proposal.
• Native branch and commit browsing
• Basic user profiles: http://gitorious.org/~mvo

What I'd like to do, for a while, is to pull changes directly from git and bzr and use them concurrently for a while. During this initial phase, we should continue to share patches in the issues too, but it gives people an opportunity to compare both. Would that be helpful?

I added the following to pro-Mercurial points for shifting Drupal.org but it's worth to add it here too. Remember that the choice of DVCS affects not only Drupal core but themes and modules too. The latter are where most contribution goes and Drupal has always been proud of its community and modules eco-system.

• Mercurial has excellent Windows support and this was the major reason git was not Mozilla's choice for their DVCS.
• Mercurial is just cool and can do the same.

Drupal (module and theme) developers are mostly not hard-core developers and the joy factor is very important. Mercurial delivers that. Git is created for Linux and developing for kernel automatically makes you hard-core developer that uses Linux. But web developers are very different and good Windows/Mac support is a must.

I'd have to agree with launchpad being a bit ugly and confusing - so I think there is a tradeoff is between maintaining (and coding) our own project* monster with bzr in place of cvs (and the workflows we are already familiar with) versus installing, configuring, and explaining something like launchpad.

I spent some time trying to find clear docs on setting up a multi-project, multi-user bzr server, and I found the bzr docs themselves totally lacking. One bit is that bzr is that branches can be available over http as well as bzr+ssh, so at even people behind a firewall can always do a checkout.

"More developers are already familiar with Git" is a very poor reason to choose Git in Drupal's case. If we want the developer base to grow, we should choose the system which is the easiest for newbies to use - people who are new to using DVCSes, who are new developers or maybe aren't developers at all (themers or translators). It's going to be a lot harder for these people to learn their first DVCS than it will be for an experienced DVCS user to learn how to use their second or third. Given this, a should choose the system with a proven record of ease of use and "it just works"-ness, and that would not be Git (shall I "go there" on the Windows issue?).

Can somebody please give us semi-newbs a little explanation as to what is meant when you talk about using Gitorious, Launchpad, etc., instead of project*? Would we be still be hosting the repo at d.o via a launchpad/gitorious install or would the repo just move to launchpad.com, etc.? Would we lose some integration with the rest of d.o? What would something like launchpad handle (e.g., will we move documentation and issues there as well) and what would be left for the Drupal part of d.o to handle?

Or are these all questions that we're still trying to decide upon?

Sorry, I'm just trying to keep up.

Though I am a long time lurker, I'm still a relatively newbie themer. Half of what you are discussing here goes over my head.
My development machines are ubuntu or windows based. I do my theming on windows.
Like many programmers prefer to diff from the shell, and many osx themers do everything in textmate, i like to work from my ide (Eclipse/Aptana).
These are links to the Eclipse plugin repositories & their documentation.

For Git:
Name - Egit
Update site - http://download.eclipse.org/egit/updates/
Documentation - http://wiki.eclipse.org/EGit/User_Guide

For Bazaar:
Name - bzr-eclipse
Update site - http://verterok.com.ar/bzr-eclipse/update-site/ (check http://wiki.bazaar.canonical.com/BzrEclipse/Installation for current link and additional requirements)
Documentation - http://wiki.bazaar.canonical.com/BzrEclipse

My personal experience with git has been interesting. I was originally rebutted by git's intricate commandline interface (about a year or two ago), laughing at stuff like "fsck" or "gc" being a possibility in a VCS. Since then, git has greatly matured, especially on the documentation side. There are still some oddities with the interface, as git is strongly model-driven (compare drupal nodes with git blobs), but those are smoothing out over time. To a newcomer, git is very useable, and Koumbit is switching to git (from svn and mercurial) everywhere. There's a windows client with a TortoiseGit interface. The recent clients have very good inline help, so the documentation side of git is fixed, in my opinion. And the fsck/gc commands are internal commands that are used to fix catastrophes or optimize the repo (can your VCS do that? ;).

I believe the project_issue part is also almost fixed. A lot of work has been done to make version_control_api work with git, and the only manifestation of that critical piece of infrastructure for bzr is this project, a "placeholder" without any time frame for development, and nobody here, as far as I could read, committed on developping that.

I remember a very important post in the handbook that webchick brought up outlining very basic requirements for leaving CVS: windows/mac support (both fulfill that requirement), Project module migration (only git fits the bill), documentation (both seem to have adequate documentation), release scripts have to be re-written (that needs to be done for both, but I already wrote a hook that creates tarballs automatically out of git tags).

Basically, I feel that there are key points lacking for bzr, while git has everything we need.

So this is perhaps an obvious statement to make...but: I am in favour of Bazaar. I first needed to use Bazaar for the Ubuntu Desktop Course project (i.e. documentation) in 2007ish. I currently use Git for StatusNet
documentation.

I found the very basic documentation for git to be lacking (i.e. how do I clone a project to start working on it) within the tool itself. I asked git users about the system the response was generally that it was "trivial" and I should find it easy to use. And did you know how powerful Git is? As a result Git made me angry inside. It made me feel stupid that I couldn't understand the very basics of how the system worked and as a result it made me not want to contribute because it was too embarrassing to not understand.

The culture of Bazaar users is quite different. The internal help system has been tested by new contributors to Ubuntu. It gives immediate hints on how to do the basic steps that most "casual" contributors will need. Yes, I have been working with Bazaar for longer. I have never felt bad about asking for Bazaar help because of the way the system has been designed. It encourages people to ask more questions.

Compare:
git --help <-- listed alphabetically, assuming you know what you need
bzr --help <-- listed by step-by-step task usage and grouped according to functionality

I've always received very patient help from Bazaar users and core developers. I find the channel to be very accessible. The first time I watched a bzr presentation it was by MySQL community manager Lenz Grimmer at DrupalCon, Szeged. He was enthusiastic and really liked showing off how bzr had made it much easier for their team to collaborate on the development of the database. It was as much about the "code" as it was the community of people. I like that. Having contributed some documentation to Bazaar as well as several (unpaid) conference presentations I then had the pleasure of working with the Bazaar team on the redesign of the front page of their Web site. I found the contributors to be passionate and interested in making the system accessible to all sorts of new people. Not just developers and coders.

For our purposes Git and Bazaar are probably about equal from a functionality point of view. They will both have some things they handle more naturally. The workflow of either system can be adapted to be centralized. I can't comment on how easy it is to script either system because I don't automate my tasks. My guess is that most casual contributors won't either. I also can't comment on how easy it is to work on more than two or three versions of a project or rebasing because I've found my contributions to be very task-oriented. I have no problems working with teams using Bazaar. I have never found it to be lacking in some functionality which prevents me from working and contributing on a project.

The version control system of choice will not change how I participate in Drupal; however, I feel that the Bazaar culture will be welcoming to casual and non-traditional contributors within our Drupal community such as documenters and designers.

I can't honestly remove the "no extra plugins required" statement from the wiki page because it's technically true. But, honestly, it's deceptive. Bazaar enables and ships for Mac and Windows with all the major plugins (including rebase), and on Linux systems they're available via packages. It's like criticizing Drupal for having menu navigation as "only a module" -- in both cases, the chunk of functionality is both shipped and enabled for almost all users.

Provides the benefits of a distributed version control system, while also supporting a traditional centralized workflow, which will make transition easier for people ...

From what I read about Bazaar, Git allows for a centralized workflow just as well as Bazaar does. In fact I use Git at work in a centralized manner. I'd remove that point as a pro for Bazaar.

I'd recommend using Git because it suits perfectly for the workflow we already have with projects at Drupal.org. Each project should have it's own Git repository. Each project can be given very fine grained access rights using Gitotis or any other standard way of controlling access to a Linux account over ssh or HTTP/S. With gitosis you can give read-only or write access to groups or individuals. It works with ssh using public keys, which is easy and secure. Have a look at the Gitosis example conf. And a very nice detail about Gitosis is that itself uses a Git repository for storing the configuration ... eat your own dogfood. :-)

As for Git guis I'd recommend GitX for Mac users, or Gitk on Linux (and Windows?).

From what I hear Git can do everything Bazaar can ... and more. As far as the documentation is concerned, I find it very good, but I understand that the initial learning curve might be perceived as steep for a newbie coming from Subversion. But there I must say: if the available documentation is not god enough, why not make it better or write our own custom tailored documentation that describes the common use cases found when working with Drupal?

For me as someone who doesn't know Bazaar, it seems like Bazaar has better GUI support on Windows. This is clearly a big pro for Bazaar, but don't be fooled by the current state of matters. Git is moving very very fast and has proven to take criticism serious, take a look a this list ofGit flaws I found on a Bazaar documentation. From this site:

Update: Since this page was created Git has come a long way. All of the issues listed have been addressed and this page should probably be retired.

Personally I expect (good) Git GUI to show up very soon ... I don't know if it's true, but I get the impression that Git is more widely use, and from what webchick has written it's especially true for Drupal.

I work on OS X (and Linux (Ubuntu)) I use either the shell or GitX which is a very handy and easy to use GUI, I really recommend it.

A friend just told me that there is a TortoiseGit which seems to be a very handy tool for those working on Windows.

A major TODO imo is to find out the future of Git for Windows.

October 22 was the last preview release / beta / very stable beta of the native Git implementation msysgit. (I don't consider Git on Cygwin a solution).

Will it get a release anytime soon? Will it chase the main git releases closely? Do the git maintainers take use on Windows into account when adding certain features?

On IRC there was a rumor that msysgit was lobbying to be merged back into the main project. If that happens, that's a boon for git, if not, will msysgit linger?

As to TortoiseBzr / TortoiseHg, contrary to TortoiseGit, these use (AFAIK) python as an Explorer shell extension. Is that true? I'm not really comfortable with that.

This topic has really grown quite large - I haven't had time to read through all arguments and I'm afraid I wont have time until it's too late.

Anyway - it seems like one difference between Git and Bazaar hasn't been mentioned - and that is how the integrity of the repositories are guaranteed. I like Git's approach of cryptographic authentication of history - by knowing the sha-1 hash of the latest commit in a branch you can be sure that it and all previous commits hasn't been changed by errors or intruders.

I don't know if Bazaar has something similar - they may have some kind of PGP-signing - but it seems like there's no automatic, simple, bullet proof way of knowing if the history of one repo really is exactly the same as another in Bazaar.

I would prefer Git and have added my name to the list. (I would also love to see a forkable future of Drupal.org that takes heavy inspiration from eg. GitHub. - but that would be a future feature request)

This is idea has the potential to keep everyone happy:

Why choose between git and bzr?
With git each project would need it's own repo and bzr can work the same way (we don't need to have all projects in one big repo). If we stick with project.module for issue tracking and release management we could support both git and bzr. We could actually let project owners choose at the time of project creation.

Yes it creates extra work in initial development, migration and ongoing maintenance. But we have plenty of people on both sides that seem willing to make their preferred VCS happen, this would neatly put us into a 'put up or shut up' situation. If either side fail to come through then one becomes the de-facto choice. Everyone gets to work on integrating their favourite VCS without fear that their work will be cast aside.

As for documentation, well we already describe an array of VCS workflows in the handbook, it'd be business as usual.
Git and Bzr are similar enough that I think any one individual will be able to adapt easily enough when getting involved with individual projects, especially with the help of decent documentation and a cheet sheet.

It would also mean the proposed code sprint at Drupalcon could begin to focus on implementation right away instead of wasting time in heated debate.

It would however leave us with a slightly contentious choice of which VCS to use for Drupal core...

So is this idea completely crazy? Or might it actually just work?

We recently crossed the 250-reply threshold mark on this monster, so time for another summary. :)

• One thing that is universally agreed is that CVS sucks, and drupal.org's usage of it is leading directly to community and resource fragmentation, and moving code off-site that never gets contributed back. A group of concerned Drupal netizens want this to stop, and we're trying to hash through what exactly it would take to get modernize d.o's infrastructure to stop the hemorrhaging, and who can come on board to help.

• We've ruled out moving to a better centralized version control system, such as Subversion, because though it is unquestionably more popular and makes more sense than CVS, it still shares the same limitations as CVS in terms of workflow, and the feature trade-off simply isn't worth the pain it will take to migrate.

• In terms of distributed version control systems, the big players are Git, Bazaar, and Mercurial. While Git and Bazaar users have turned out in droves to sign up for the tough grunt work to help with the migration, as well as handling the size-able documentation and training components (some of which has already started), we simply haven't seen that from the Mercurial crowd. Therefore, though it is undoubtedly a robust VCS with great features and a smooth learning curve, we've ruled it out as a contender. The only way it goes back is if a throng of Drupal community Hg fans pull off an organized effort on-par with what the Git and Bzr folks are doing, which feels doubtful at this point. Therefore, from a practical standpoint, we are basically looking at Git vs. Bazaar.

• A lot of discussion has gone on as to which of these two systems is best, and lots of resources have been gathered and linked to form the wiki, and pro/contra points posted to the discussion. For the most part, though, for the purposes of our community it really seems like they're both functionally equivalent. So we're currently in the process of working out exactly what the workflow would be like in both systems for each of our identified use cases: "major" and "minor" patch authors, module/theme developers, patch reviewers, and those deploying their Drupal sites from $vcs. This will allow us to compare apples to apples, and help evaluate the learning curve question, which is currently the biggest stated differences in the two systems.

• A wonderful idea was raised by David Strauss to have "real life" usability testing on each VCS at Drupalcon SF on one of the code sprint days, consisting of a group of guinea pigs who have no experience in either, and are primarily used to CVS/Subversion (or don't know what a version control system is at all). We're currently looking for volunteers to wrangle this "Bzr vs. Git Smackdown", including preparing documentation and exercises for victims to go through, determining the data to look for to judge the learning curve aspects and how to gather that data, etc. Please jump in and volunteer/help organize this!!

• On the infrastructure side, "phase 1" will basically consist of the simplest thing that could possibly work -- switching CVS for $vcs, and making whatever adjustments need to be made to our existing tools to get them showing $vcs commit logs instead of CVS commit logs, and packaging up $vcs branches/tags instead of CVS branches/tags, etc. Changes will still be handled with patches, and our existing workflow will change minimally. Itemizing exactly what this work is going to entail is an ongoing process that we need help with from folks who have knowledge of how that's all set up.

• We also need help determining what exactly phase 2 and phase 3.5 ;) might look like. Analyzing other providers like Launchpad and GitHub and envisioning a way to combine that with our community's tools would be a good initiative for some folks to take on.

• Speaking of those, we've also explored the idea of replacing Project* modules (which currently power our issue tracker and download system) with something like Launchpad, thus off-setting a major chunk of our infrastructure to folks who specialize in building these tools. This discussion is necessary, since a decision here would lock us in to picking one VCS vs the other. Overall though, this idea has been met with lukewarm reception at best, since the migration work would be quite significant, and it would add even more training requirement not only to our developers but to our entire community. There is also intense disagreement from prominent members of the drupal.org infrastructure team. Therefore, I also think we can safely rule out consideration of other project management systems from factoring into this decision, which brings us squarely back to Git vs. Bazaar on their own merits.

• Timing-wise, this whole shebang will not be happening before Drupal 7 is out the door, because there are too many potential wildcards. So if you want to help scratch this itch, but don't have the chops/time to actually help the infrastructure team pull it off, a great place to dive in is the Drupal 7 critical issue queue.