Move Git repositories to Github

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Know something about Github or Drupal.org? Please help flesh out the Drupal.org vs. Github feature list!

August 27: Here's an attempt at summarizing the high-points of discussion/research so far.

What's your idea?

Github has superior features such as Pull requests, forking projects, inline code editing, inline code commenting etc. that are far easier to understand for new contributors and more efficient for developers working with open source code daily. We should use drupal.org for things that it is good at: project releases, centralised repository of projects, the issue queue etc.

We should not host our own Git repositories and rather integrate our issue queue with Github pull requests and their API.

There is a similar proposal: Implement Pull requests on Drupal.org

What are the benefits?

Less time spent on building/maintaining our own Git infrastructure. Less code to upgrade during major version upgrades of Drupal.org. Lower barrier for people to contribute, higher visibility of Drupal code in the Github social ecosystem, more efficient contribution workflow.

Built-in features:

  • better git viewer
  • integrated some dreditor features (comment on line level)
  • inline editing of files
  • linking issues
  • pull requests
  • private repos possible (we only use this for bluecheese)
  • extensive rest api (how would we actually use this?)

What are the risks?

  • Relying on a commercial entity that can change whatever they want without our consent. However, Github has earned the trust of many open source projects already. In the worst case scenario we have to move back our Github repositories back to drupal.org or choose another provider (Gitorious, Bitbucket, ...)
  • Registering an account on *.drupal.org makes sense when you are a member of the Drupal community. Registering on Github and give away all your data (account data, activity, etc.) to an entity you do not particularly identify with might not.
  • The Github Pull request workflow may not work so well for us because multiple people working on one pull request have to be granted access to the forked repository. Another solution is to send a PR to a fork.
  • It is very easy to fork and it's questionable whether people would pull request back or just use their fork. But, people often have "forks" of Drupal modules locally currently. Would forks on Github change this one way or the other (i.e. are pull requests easier than patching?). May be already happening silently in d.o, see comment: https://groups.drupal.org/node/313068#comment-953538 and also: https://groups.drupal.org/node/313068#comment-953898
  • With the forking model it's very confusing to see which one is the 'real deal' and which is a fork. Solution: Keep d.o centralized project listing, as stated in the proposal. Publish in d.o project's page the link to the real repository. It is the model already used today. See comments: https://groups.drupal.org/node/313068#comment-954488 https://groups.drupal.org/node/313068#comment-953468 Counter: Solution does not address repository ownership transfer (via forking), and updating actual user sites to the new fork.
  • For core development, already on drupal.org we have sandboxes and finding sandboxes is a huge issue. The issue queue - patch based development model makes everyone's work visible but when people fork, it's just not on the radar, you have no idea what's going on.
  • Impact on long-term project involvement (Historically, we have seen less longevity and participation in any 'off-drupal.org' initiatives than those on drupal.org). It's unclear if Github would follow that same trend.
  • Impact on code quality/moderation/spam control in the repositories (loss of 'git vetted user' permission check)
  • Concept that 'all forks are created equal' will cause pain for new users and make it harder to manage projects on drupal.org. (i.e. transfer of repository ownership via a new fork, and challenge in updating drupal.org references/pointers; not to mention pointers for existing user sites which are not redirected).
  • Adds complexity to packaging, distributions, whitelisted libraries,
    download/usage tracking, automated testing, update manager, d.o project browser, project management, security advisories, etc.
  • Significant development efforts required to break drupal.org repository ties and refactor to leverage github APIs, and potential burnout as this would be the same resources who have been involved in the D7 drupal.org migration.
  • Github is not as accessible as drupal.org and we can't do anything to fix it
  • Additional risks related to moving issue queues to github (which some have argued for in the comments -- a summary is in this comment: https://groups.drupal.org/node/313068#comment-954428):
    • Impact on ability to host private security issues (security.drupal.org uses the project issue module now - we would need a similar system or to reinvent or flow for managing those issues)
    • Inability to differentiate between bug/support/feature request issues (we can use tags, but that feels like a hack and might not be done consistently between queues)
    • Inability to easily move issues between queues
    • Impact on ability to moderate issues (e.g. blocking a troll account)
    • Inability to upload files other than images to issues (don't make me query for how often this happens...) would result in hosting elsewhere and then linkrot when elsewhere goes offline.
    • Forcing users who just want to report and get updates on issues onto a separate, unfamiliar site (where they will presumably have to log-in again).

How can we measure the impact of this idea? (metrics)

By the average time it takes to write a patch and get it commited. By the number of people contributing to Drupal core and Drupal projects. By the amount of time that would be saved maintaining our own Git infrastructure. By the amount of time that would be saved during major version upgrades of Drupal.org.

Who directly benefits from / will use this improvement? (target audiences)

Developers.

Are additional resources available for discovery/implementation? (volunteer effort, financial backing, etc.)

In Gaelan's experience, GH has great support. Gaelan is trying to get them to weight in on this discussion.

Related Wiki Pages

Comments

Of course I can be wrong, but

urwen's picture

Of course I can be wrong, but personally I think that the efforts of Drupal should focus on drupal and Dev itself (project management, development, ...). Whatever you do not in this scope out is slowing the evolution as a tool/framework.

as i'm in favor of this move,

dasjo's picture

as i'm in favor of this move, i have compiled a list of articles that discuss moving our git repositories from d.o to github.

Drush - Migration From Drupal.org (to Github)
https://github.com/drush-ops/drush/wiki/Migration-From-Drupal.org

Github for social coding" by kim.pepper with reponses from webchick, dries, ...
https://groups.drupal.org/node/48818#comment-127138

Drupal sans Github: Missed Opportunity
http://www.echoditto.com/blog/drupal-sans-github-missed-opportunity

Several reasons why I prefer Github over Drupal for Hosting my Drupal Projects.
http://berk.es/2012/03/23/several-reasons-why-i-prefer-github-over-drupa...

agreed

attiks's picture

I see benefits in both options, but since github as tons of extras for developers, I think we - as a community - should really consider moving the code to github, or something similar. The other option is that we speed up the development for the D7 migration, but I think to be able to do this we probably will need (a lot of) resources/funding.

For the issues we might consider using rest api to integrate into d.o. and I think meta issues should stay on d.o.

AFAIK the integration with the packaging and the testbots should be pretty straight forward, for testbot we might considers using Travis.

PS an alternative might be https://enterprise.github.com or maybe http://blog.gitlab.org/gitlab-6-dot-0-released/ they have comparable features, but a different pricing :-)

I did a quick test using

attiks's picture

I did a quick test using gitlab and I must say I'm impressed, I tested it using http://bitnami.com/launch/gitlab (free for one hour)

Edit: I installed gitlab on my dev machine and did some more testing

I created a project for drupal core and for the picture module, and it took me only a couple of minutes to set up.

It looks very similar to github and will probably do a lot what the community wants:
- hosted on our own servers
- better git viewer
- integrated dreditor (comment on line level)
- inline editing of all files
- linking issues
- pull requests
- protected branches
- integrated CI server
- use something else for issue management
- private repos possible
- extensive rest api

  • ...

Issues in d.o?

Mac_Weber's picture

I'm really up for hosting repositories on GH or any other provider that offers similar quality of services.

Though, I'm not sold d.o is good for issue discussion. Discussion viewing code as in GH is a great feature.

Agreed on the issues

jhodgdon's picture

The issue tracker on github is really really terrible and we'd be losing pretty much all of our useful issue tracker features if we went with github.

If we don't want to keep issue tracking ourselves, we need to adopt a real issue tracking system like Bugzilla. If we adopt Github's issue tracker, I think it will become nearly impossible for me to do what I do now using the Drupal issue tracking system.

Builder of Drupal sites and modules, and Drupal tutor - http://poplarware.com
Drupal Core maintainer for documentation

not sold for issues on GH either

Mac_Weber's picture

To be honest I have mixed feelings about issues on github. I see advantages and also many drawbacks both in d.o and github issue queues.

Discussion with code is a great feature, but maybe it could be implemented it in d.o using their API?

I agree. Drupal.org wins

dkinzer's picture

I agree. Drupal.org wins hands down. But this does not preclude a move to Core; it just means that some work needs to get done first to integrate GitHub with the Drupal.org issue tracker.

I think GitHub is a great

Gaelan's picture

I think GitHub is a great service. GitHub also offers an API, so we can still create d.o project pages, based on the data in GitHub. There is already a Drupal mirror on GitHub (github.com/drupal/drupal), so 1) you can see what the code browser is like and 2) we might consider accepting GitHub PRs for a while, just to see what happens.

Pricing!

Gaelan's picture

Github: Free, unless we want private repos (Bluecheese?)
GHE: Not possible, per https://enterprise.github.com/faq#faq-7. Also very expensive. ($5000/yr for 20 accounts)
GitLab: Free for "CE", paid for "EE". Open source.

See plugins.query.com for an

mongolito404's picture

See http://plugins.jquery.com/ for an example of community using GitHub to host contributions, while still providing a central plugins registry.

My +1 to moving source code to GitHub

joelwallis's picture

Use the Issue Queue of Drupal.org to handle issues, and code with lots of patches attached in comments is just primitive. GitHub have much more productive tools to handle cummunity contributed softwares stuffs than what d.o has today, and we believe it should be used by the Drupal community.

Contributions authorships are also other thing that sucks in d.o, since authored commits are made by tagging the user with an @username tag. I know there are ways to made the commit with authorship information, but hey, there are GitHub which already have it implemented and lot of graphs of comminity contributions. Why not use it?

Drupal community must focus in the Drupal platform development itself, and leveraging resources like GitHub to streamline the contributions to the Drupal core. If we can save time and money by avoiding to build a big source code hosting solution, so I really believe we SHOULD do it.

It really would be great!

danylevskyi's picture

It really would be great!

I think moving to github is

igor.ro's picture

I think moving to github is mistake.

The first big difference between opensource projects on github and drupal.org is count.
On github we have PHP, Symfony, Hyphop - this is big single code base projects.
Symfony have extentions but there are about 10 of them (prof link: http://symfony.com/components)

Drupal.org have >1000 modules. So it would be more then 1000 small projects on github.

Now let's check github pull requests. PHP project on github have 980 forks and 56 pull requests, Symfony - 2500 forks / 131 pull request.
Even Rails have 6400 forks / 374 pull requests.

That means that "easy make fork" motive developers to make forks instead of pull requests. Current drupal.org workflow makes publishing patch easily then create own copy of module.

So > 1000 modules and such fork/pull request relation will slow community.

Let's think how we can reach the same goals with other tools.

For example.
Lower barrier for people to contribute and more efficient contribution workflow.
Community can provide developer infrastructure that makes development process easy and motivate developer to contribute.
To reach that goal we use does not store contribute code in project git. We store only drush make file. So contribute code have to be downloaded. But if developer find out a bug and need to fix it, he have to create patch, contribute and add to drush make file.

This is very efficient.

Other my suggestion is to remake drupal.org issue tracking. Currently patch is just attach to comment. But realy patch should be special entity, that have relation to issue, module and module version. It would allow us to make automated port patch to next version of module. Because there is a lot of small fixes on drupal.org that are ignored by maintainers.

We started to think about that for internal cases and made public project on github
https://github.com/Drupal-Omsk/patch_management_system
but it is in very early stage.

Drupal has, indeed, thousands

mongolito404's picture

Drupal has, indeed, thousands of projects (mostly themes and modules, but not only). A move to from Drupal.org to GitHub for code hosting does not means thousands of projects owned by a single organisation. Neither does it mean the end of project pages on Drupal.org with project overview, releases, etc. But rather thousands of projects owned by thousands of organisations and users. All listed and searchable on Drupal.org as they already are.

Yes easy forking on GitHub encourage developpers to create and then forget a lot of unmaintened, outdated and dead forks. But from my little experience, Drupal projects are forked by developpers who don't want to bother learning how to contribute patches all the time. But since they lack the tool to do it quickly on Drupal.org or anywhere else, they fork on their own internal VCS (or shared file server). So, unmaintened, outdated and dead forks exist in the Drupal, we just don't see them. Switching to GitHub will neither create this problem nor solve it. But at least, it provides an easy to use tools for those who want to publicaly fork, and then maintain, a Drupal project. By providing easy to use tools, it also encourage developper needing to fork a project to do it publicaly.

As for the eficiency of submitting a patch to Drupal.org, back in the days there was talk that CVS was efficient enough, that the migration to git was unecessary from a developper engagment perspective. I would love to see the number of contributions, contributors and new projects before and after the migration to git.

Make sure whatever concerns

webchick's picture

Make sure whatever concerns you have about this proposal are reflected in the "What are the risks?" section of the wiki page. We want people to be able to come into these discussions and glean the pros/cons without reading all X comments.

Those metrics are misleading

chx's picture

You forgot to count the closed PRs. Rails in reality had over 12 000 pull requests and the real number of forks is unknown because deleting a repo is not hard and so the number of deleted forks is unknown.

jp.stacey's picture

As far as I can tell, drush make (and drush dl) rely almost entirely on the metadata feeds at e.g.

http://updates.drupal.org/release-history/gmap/7.x

We've got a heavily drush-make-oriented workflow, and it would be a considerable loss of convenience to have to move from "projects[views] = 2.x" or whatever to "project[views][download] = git\nproject[views][url] = ..." etc.

For us, then, the essential requirement is for "drush to still work" with the shorthands of simple "drush dl"; the ideal way of fixing that would be not to put a kludge in drush, but to change the way d.o produces those release-history metadata fields persist, so they reference e.g. github package URLs.

As long as those metadata feeds remain, I'll follow the community wherever the download_link XML fields link off to. I have lots of additional opinions either way, but as I'm really happy that d.o continues to at least contemplate and discuss alternative technologies like github, I wouldn't want to contribute to any bikeshedding!

for some reason Berkshelf

cyberswat's picture

for some reason Berkshelf immediately jumped to mind when you mentioned this. The packaging system is adaptable enough to pull from multiple sources, not just a single feed. I've seen those sources be things like the chef community pages for their version of "modules" as well as github projects. Maybe something like that could be considered if a change were to happen. It almost seems like that would be necessary if travis-ci starts becoming a regular topic.

Side effects

mongolito404's picture

Note that a move to GitHub would also include features suggested in other proposals at no cost:
- Pull Request
- Back-link commit messages on issues (if issue are also moved to GitHub)
- Improved notifications system

Native Dreditor as well.

Gaelan's picture

Native Dreditor as well.

gitlab

attiks's picture

I created a new wiki to discuss to benefits of using gitlab at https://groups.drupal.org/node/313158

Why not issues as well?

webchick's picture

Would you be able to provide more justification for why you think we should move only the code repos to Github and not also the issue tracker?

Since one of the stated benefits of this proposal is "Lower barrier for people to contribute" (which I agree with), it's odd to me that we would then choose a non-standard collaboration method for those repos, and thus largely negate that benefit.

In other words, people who use Github use the whole enchilada: the code repos, the issue tracker, the wiki pages, etc. They're used to collaborating on other projects on Github that way. If they had to do something different for Drupal, and go get a separate login in some other site and fill out forms they've never used before, etc... don't we lose a lot of the advantages of doing this in the first place?

who's you

attiks's picture

webchick, who are you addressing your comment to?

In case it is me, I think moving issue queue as well is a good idea, and if we go the gitlab route, we might even easily migrate everything as well. Gitlab provides the option to use an external issue queue, but it doesn't mean we have to use it.

One problem we probably face when moving is that we might loose the link between the commit message and the nid of the issue.

Klausi, I think...

webchick's picture

Or whomever wrote:

We should use drupal.org for things that it is good at: project releases, centralised repository of projects, the issue queue etc.

We should not host our own Git repositories and rather integrate our issue queue with Github pull requests and their API.

...in the wiki summary.

I'm not familiar enough with

klausi's picture

I'm not familiar enough with the issue queue on Github, but I think you cannot have the full range of states like "needs review", "needs work" etc. as we have on drupal.org. I need those states to efficiently work with the issue queue, that's why I proposed integrating our powerful issue queue on drupal.org with the Github API to push/pull whatever information we need from pull requests for example.

I might be wrong though, not sure what can be done on Github and what not.

If we need to do the work to

jthorson's picture

If we need to do the work to integrate with Github, is there a reason we wouldn't simply perform that same integration work with our existing repositories? Having been in the guts of project_issue, PIFT, versioncontrol_project, and versioncontrol_git ... it seems to me that breaking and reimplementing the existing repository ties would introduce a significant amount of disruption and instability into the entire ecosystem; whereas (once the D7 migration is complete, very soon now) layering on the incremental workflow functionality that the community desires would be a much smaller body of work.

Removing the Testbot completely

Gaelan's picture

We could potentially completely remove the testbot as we know it and just use Travis CI (which I think will be dead easy to set up).

Not so fast

chx's picture

Travis CI docs: With our current timeouts, a build will be terminated if it's still running after 50 minutes

While we might actually be

jthorson's picture

While we might actually be able to squeeze our run down to 50 minutes, the travis-ci limit is actually 20 minutes. :)

Wait, what? According to

Gaelan's picture

Wait, what? According to http://about.travis-ci.org/docs/user/build-configuration/#Build-Timeouts, it is 50m total, or 10m with no output. Where'd you get the 20?

.

jthorson's picture

May just be a documentation issue ... http://about.travis-ci.org/docs/user/getting-started/ states "As a free community service, Travis CI limits build duration to about 20 minutes.".

OK. The official answer if

Gaelan's picture

OK. The official answer if 50min. https://github.com/travis-ci/travis-ci/issues/1362

Problems with the GitHub issue tracker

jhodgdon's picture

I have had the distinct displeasure recently of using GitHub on a group project (building a Drupal site). Although I agree that the code/pull stuff is fine, the issue tracker leaves a lot to be desired, and I think it would be a huge step backwards for us to switch from the issue tracker we currently have on drupal.org to using Github.

Here is a list of a few essential things we can do now with our issue tracker that we can't do on Github as far as I know:

a) project/issues/user (list of issues across all projects that you are following) - as far as I know this does not exist on Github.

b) BIG ONE: you cannot attach files to issues. Period. This is IMO a complete show stopper.

c) Meta-data fields such as component, severity, version, etc. You can file an issue and it's on a particular project, and you can tag it "needs review", "feature request", etc, but we have a much richer set of meta-data than what exists on Github. Fields I think we would need that are lacking:
- Branch/tag/version
- Severity
- Type (bug, feature request, etc.)
- Status (needs review, RTBC, etc.)

d) UI for listing comments lacks numbering and date/time. All it shows is "10 minutes ago" and "yesterday" etc. It makes it pretty difficult to say "In comment #3, Jane said...". You can make a permalink but who wants to click on that and jump to that spot?

Those are the big ones... Basically it's a very minimal issue tracker and I think it would be a huge mistake and a huge step backwards in functionality to switch to using it. I seriously think it would hamper my ability to contribute to the Drupal project, to use that issue tracker as I've seen it operating. Really horrible.

Builder of Drupal sites and modules, and Drupal tutor - http://poplarware.com
Drupal Core maintainer for documentation

Awesome, that's really

webchick's picture

Awesome, that's really helpful feedback on where exactly the shortfalls in Github's issue tracker are! Could you somehow add this information (or a pointer to it) in the issue summary?

Added to summary

jhodgdon's picture

I added a link to the summary. Although as a note, the proposal here says at one point that we don't want to use Github's issue tracker... I think it would be pretty difficult to host the code on github and not have at least some issue being hosted there, which could lead to fragmentation if we are trying not to use their issue tracker.

Builder of Drupal sites and modules, and Drupal tutor - http://poplarware.com
Drupal Core maintainer for documentation

I agree with jhodgdon on many

Mac_Weber's picture

I agree with jhodgdon on many points here.

Can we solve it hosting issues on d.o and using github's API?
I'm not familiar with their API, then I have no clue.

Images

Gaelan's picture

BIG ONE: you cannot attach files to issues. Period. This is IMO a complete show stopper.

Except for images. However, I don't get what we need other uploads for. Patches/interdiffs are replaced by PRs, and images are supported. In the rare case that we need some other upload, we can upload it somewhere (Dropbox?), then link it.

Off head...

chx's picture

we attach spreadsheets, text files, non-review patches...

non-review patches Do you

Gaelan's picture

non-review patches

Do you mean do-not-test patches, or something else.

I don't think this is that common, so an occasional inconvenience (upload to somewhere else) I think we can live with.

We've seen in the past that

greggles's picture

We've seen in the past that 3rd party services tend toward linkrot (e.g. skitch-hosted files uploaded to the issue queue from years ago are now missing) while we've kept attachments on our sites alive and well.

I don't think we can or should dismiss this feature so quickly.

I think the linkrot point is

webchick's picture

I think the linkrot point is a fair one, but OTOH I will also say that due to the lack of easy image inlining (which Github has), I used to use Skitch extensively before it became a flaming POS. So just because you can upload a file doesn't mean people do if there's an easier way around it.

I don't see Github migration

Hadi Farnoud's picture

I don't see Github migration happening without Github help.

what if we keep the issues on d.o and replicate it on Github with their API?

Also, given the support this idea is getting...

webchick's picture

Are there people out there who are willing to volunteer / fundraise / research / develop for this effort? If so, please add your info under the "Are additional resources available for discovery/implementation?" part of the wiki page.

Good point, Gitlab would motivate me

Elijah Lynn's picture

Integrating with Gitlab would motivate me much, much more than Github. Github is way too proprietary and I feel it goes against what Drupal stands for.

Here is a Gitlab podcast from November 4th that I am listening to right now and it is very promising!

http://pcasts.in/Ni32

It might be worth shooting GH

Gaelan's picture

It might be worth shooting GH a email pointing them to this issue; they might be able to say something like:

Oh, better issue statuses? We're releasing that tomorrow!

Or:

We don't have feature X, but you can replicate it with feature Y.

I'm not sure if they will care enough to add new features "just for us", because there is no financial incentive for them (unless we find something that we want to be private, maybe Bluecheese).

EDIT: I've gone ahead and done so.

Yes, let's talk to them

Crell's picture

Thanks, Gaelan. Let us know what they say.

It's certainly true that in my experience the GitHub issue/PR categorization/curation tools are weaker than Drupal.org's. However, Drupal is no small project. If we move to GitHub wholesale, we'd likely become the most active PHP project very quickly (eclipsing Symfony, the current PHP leader). That gives us some weight to throw around, and it would be a coup for GitHub for us to move to their platform. We absolutely should try to leverage that to see if we can encourage them to evolve to suit our needs. Ironically, that would probably be easier if we were not using all-free services from them, as it then becomes in GitHub's financial interest to improve in ways we want. :-)

I don't see why somebody

Gaelan's picture

I don't see why somebody would fork, then not send a PR. Sending a PR takes less than 5 minutes (plus however long it takes you to give it a title and description, which are pulled from the commit if there is only one commit).

Same reason people fork

joachim's picture

Same reason people fork downloaded modules and don't post patches?

It takes about 5 minutes to make a patch too.

Let's try it out now

klausi's picture

Thinking more about this: we don't have to wait until we make a decision as a whole community, we can start to experiment with Github now already as individuals.

Example: I have disabled the issue queue for https://drupal.org/project/drupalpractice and I'm going to develop it exclusively on Github. The only annoying thing is that I have to push my commits to 2 remote repositories now (because I want the dev snapshot on drupal.org). I'm going to write a simple script to auto-sync the repositories with a cronjob on a server.

I'm planning to move over some other Drupal modules too I maintain (only with consent of co-maintainers), then I'll see how well the issue queue works there for me.

set up two push targets on

sdboyer's picture

set up two push targets on the desired remote - it'll push to both d.o and github with one command.

GitHub's Response

Gaelan's picture

I emailed GitHub about this, and they said I could post the reply:

Hi Gaelan,

Thanks for the link. Because there aren't any GitHubbers who are active in the Drupal community, I think it'd be a bit odd for us to show up in their mailing list thread ourselves. However, if you or anyone from the Drupal organization has questions about using GitHub we're happy to field them here at support@github.com

Have a fantastic day,
Steven!

Get them in here!

Crell's picture

Just because they're not active now doesn't mean they're not welcome. :-) When we started adopting Symfony components one of the key reasons was that Fabien and Lukas came onto g.d.o to help answer questions. It wouldn't be odd for a GitHubber to show up in this thread to answer questions, it would be fantastic. :-)

(Feel free to quote that back to them.)

Quoted back to them.

Gaelan's picture

Quoted back to them.

I pinged Ben Lanvender who

scor's picture

I pinged Ben Lanvender who contributed to Drupal in the past and now works for Github, hopefully he will be able to jump in this thread. If you contact Github support, make sure to mention his name (@bhuga).

Very much in favor of this.

cweagans's picture

Very much in favor of this.

--
Cameron Eagans
http://cweagans.net

Some features are missing

chx's picture

I mentioned this in the issue summary but let me elaborate

So, the way github works, an issue is opened, you decide to work on it, create a branch in your fork, hack on it, PR it back.

This is a Perfectionist Pat workflow. This is a workflow that also works well when you work on a dayjob and you are assigned tickets and you are resolving them.

This is not a community-based workflow, however. If I post a patch to the issue queue, there are a large amount of people who have a chance to see it -- anyone subscribing to that issue, looking at the queue for that component, etc. If I push a commit to my own fork, that's more of a "If a tree falls in a forest and no one is around to hear it, does it make a sound?" kind of problem.

I have emailed github roughly outling a solution where one fork/branch from an issue and the created, related branch will automatically get a PR and that PR is linked from the issue. Now, visibility is back.

However, until the visibility issue is solved I will maintain that this is a hype-induced, not-well thought out idea which will, in one deft move, fragment any collaborative Drupal development beyond any repair.

That's up to the individual user

Crell's picture

Nothing stops someone from forking, branching, making one commit, and then filing a PR before pushing the next 12 commits. That's all down to the individual person filing the PR when they decide to do so. We have no idea how many people have local branches of core that they've done skunkwork stuff in and never bothered to share. This is really no different.

That is, Perfectionist Pat and Sloppy Sam are both able to use GitHub, just as they both use the current issue queues. Nothing much changes.

This is correct. Pull

Gaelan's picture

This is correct. Pull requests reference branches, not specific commits. GitHub allows people to add commits to a PR at any time. You might even argue that this is easier with GH then with the current system: All you need to do to update the PR is git push.

But then when the maintainer

joachim's picture

But then when the maintainer merges in the pull request, doesn't that mean they merge in a whole other branch? We use a single commit per issue -- it sounds to me like we either:

  • lose one commit per issue. This will make our history a lot harder to understand.
  • we have to enforce that the person issuing a pull request has to make their own merge commit, so all their proposed changes are in a single commit. Which means we have to add custom behaviour to GH.

More robust

Crell's picture

In the normal GitHub flow, you get a merge commit when you use the "Merge this now" button, even if a fast-forward merge is possible. That means the history accurately reflects the way the code evolved. One commit per issue is actually bad: It means you get 500 KB commits that are utterly useless because they change one API, implement it in 4 out of 12 places, fix 17 bugs, and correct 8 documentation issues "along the way". That belongs in 30 separate atomic commits, not one massive unreadable commit.

It also more accurately reflects the work that went into a patch; currently, "rewrote the entire forum module to have new features for the first time since before the US invaded Iraq" gets the same single mention as "added a missing period to a comment". That's not an accurate picture. :-)

You can also rebase or do squash merges if desired. I know Fabien often does that for Symfony commits, or asks people to rebase a branch to remove flailing about so the history is clean before he merges. We can discuss how we want to approach that.

Really, the "one issue per commit, totally linear development" is an artifact of a 12 year old CVS-enforced workflow. It's not an accurate representation of how Drupal works anyway.

There's a fix for that:

Gaelan's picture

There's a fix for that: https://github.com/jeresig/pulley

Nothing stops...

chx's picture

... someone from learning git either and yet it's something we forced on people with the git migration. I maintain that was a mistake: most contribs don't need more than a central repo and a checkout (or bound branch). Accidentally, that's the only feature git does not have.

... someone from learning all the abstracted stuff we added to Drupal 8. Yet, it's something most people will find at least hard to do.

... someone from learning even more just to contribute to core. Currently you have an issue with a comment form at the bottom and an attach button. You write a comment and post your patch. Now, you will need to create a new branch, figure out the art of writing good commit messages, create a PR, link it back to the issue. Also, the issue will need to link to the PRs. Nothing stop you from doing it, it's just a lot more work and more concepts to learn (compared to current core at least: branch, commit, pull request)

Moving to Git was

webchick's picture

Moving to Git was single-handedly the smartest thing we ever did. http://buytaert.net/drupal-contributor-statistics-2011 In fact, it's probably the only reason the community has largely stayed on the mothership this long.

Your point about more/different steps to learn is true. Though one nice thing is there is lots of documentation out there that we don't have to write and maintain about how to do it.

But all those "extra steps"

zechola's picture

But all those "extra steps" (which are really condensed to a git push) end up with cleaner lineage. All too often, patches end up on d.o and never get merged by the maintainer to dev branches, let alone releases.

simplytest.me

tsvenson's picture

I have absolutely fallen in love with simplytest.me. Just to be able to, withing minutes after a click, have a purpose built Drupal instance with the patch up and running is amazing and incredible productive for me as a user.

Maybe something similar can be done on GH, but will I then be able to follow the patches evolving as easy as I can today?

Or be able to, with the help of dreditor, also have a nice code reviewer a button away?

I have personally no problems setting up my own patch testing locally, but the massive UX and productivity improvements thanks to simplytest.me is bar none.

Plus it allows anyone that can install drupal to test patches with no code experience needed whatsoever. And that for me is the biggest win of them all!

--
/thomas
T: @tsvenson | S: tsvenson.com

It is very easy to fork and

Mac_Weber's picture

It is very easy to fork and it's questionable whether people would pull request back or just use their fork. But, people often have "forks" of Drupal modules locally currently. Would forks on Github change this one way or the other (i.e. are pull requests easier than patching?)

It may be happening TODAY in d.o, silently. Discussed here: https://groups.drupal.org/node/313068#comment-953538

With the forking model it's very confusing to see which one is the 'real deal' and which is a fork.

This is also not a problem. Releases will still be published in d.o with a link to the real repository. See: https://groups.drupal.org/node/313068#comment-953468

For core development, already on drupal.org we have sandboxes and the visibility is a huge issue. The issue queue - patch based development model makes everyone's work visible but when people fork, it's just not on the radar, you have no idea what's going on.

This is true. However, from my own experience, other developers only take a look on somebody else's sandbox when the owner is talking about it and gives a link for it. Sandboxes are NOT separated from full projects on user profiles (it should be changed). Moreover, nobody (maybe a few people) is watching other user profiles to find sandboxes that may have useful code, again it is found only when the owner gives a link to it.

Pull requests, done properly,

dasjo's picture

Pull requests, done properly, actually enforce letting others know about what you are developing for the module. A pull request can easily be updated by pushing commits to the branch, others get notified and we don't have to care about interdiffs.

What needs to get thought through is how multiple users contribute to a single pull request. i have linked before the lengthy discussion that already happened around that topic.

I'm generally against this

rgristroph's picture

On the other hand, I am enthusiasitically in favor of improving our current setup, perhaps by using gitlab as proposed elsewhere.

My objections generally run in line with the others, particularly on trusting a third party with something like this when we don't have to. I like github, I use it, and I think it is has been a huge boon to open source and software development in general. I loved the "Brewing Github" talk at DrupalCamp Austin, and the other talks githubbers give about their managerless culture, using git repos for company policy and other non-code stuff, etc. I'm really interested in various attempts to draft legislation via github, for example.

But one of the trends of the computer revolution, is that things that required a big centralized organization swiftly become codified and can be set up by anyone. This isn't to trivialize the task, as I suspect it could be similar in scope and disruption to The Great Git Migration, but it is certainly plausible for a community with our resources to run our own GitHub, including contributing back to gitlab where needed.

The fact that a significant number of the people who contribute to Drupal and use d.o a lot support this is very important. It indicates that we are failing in our infrastructure. However, I think there is also a broad class of people out there, who complain about the d.o patch based infrastructure, but they don't contribute to any github projects either.

Suppose it takes something like 3 hours to go from reading and issue, figuring it out and googling some stuff, making the code changes, and posting a patch. Perhaps 20 to 30 minutes of that is d.o infrastructure inefficiency, and github isn't going to eliminate all of it. Most people who aren't contributing aren't contributing because of the other 2.5 hours, not the 30 minutes.

So we're not going to unleash some hidden horde of contributors with this move. We are going to incurr costs similar to setting up gitlab ourselves just in moving and integrating. There would be definite immediate benefits, but I think the costs and risks are more long term and not immediately apparent, and in balance I'm against it.

There is going to be pain

highermath's picture

There is going to be pain associated with moving to GitHub, and timing will be tricky.

Do we move now, in the heat of d8 battle, or wait until d8 is released? If we push long enough, we might be in position of the couple who divorced after 75 years of marriage because they wanted to wait until the children sere dead.

I am sure that there are folks in the Drupal dev community that never touch GitHub, and I am sure that it is a small number. I don't know anyone in the community who doesn't use GitHub for something. I do know there are many projects that are dual-homed in d.o and GitHub, and this is a PITA for the folks who have to do this.

My experience is that d.o seems to be at its limits. Searching the issue queues can be very tedious, and sometimes fails entirely. We can't afford the resilience that GitHub offers.

While I hope that GitHub doesn't change its T&Cs to something we can't live with, there will always be other choices, if that comes to pass. The mere possibility does not seem like a blocker.

I think that we should cry havoc and let slip the dogs of GitHub.

Oh I have no doubt

chx's picture

I have no doubt that as git hype have moved us to git, the hype moving us to github is inevitable. It'll hurt the community a lot but the hurrah over the perceived easiness of fixing a single character typo (can be done from the browser! fantastic) will be much, much louder than the quiet of those who leave.

It'd help a lot to have

webchick's picture

It'd help a lot to have concrete, logical, objective reasons why you oppose this move. I know a lot of people do. I might even still be one of them; I'm pretty on the fence at this point.

But these overly vague, dramatic, emotional responses do not actually help the decision-makers here to understand what exactly the concerns are, which is dangerous because if the "pro" people are presenting non-emotional, rational responses it only serves to strengthen their argument.

Not a lot of good arguments

chx's picture

I have actually collected the perceived benefits into a comment and it's hard to find a compelling argument for github except for some nice features and a lot of unfounded hype that pull requests are "better". I put the problems in https://groups.drupal.org/node/313068#comment-953938 here.

Searching issues...

rooby's picture

My experience is that d.o seems to be at its limits. Searching the issue queues can be very tedious, and sometimes fails entirely. We can't afford the resilience that GitHub offers.

Searching issues queues can be hard, but not so bad when using google.
I think the difficulty in searching comes from the massive number of issues that are being searched and possibly not knowing exactly what you are searching for. These things are platform independent.

When you are comparing to searching other platforms, do they have the same quantity of issues?
If we have the same number of issues in github instead, is it really going to make searching any easier?

Workflows

Gaelan's picture

I may be an oddball, but when I contribute to core, this is my current workflow:

  1. Create a branch for the issue, if I haven't worked on this issue before. (I have exactly one copy of Drupal for dev; this might change soon)
  2. Apply and commit the current patch (or interdiff), if any.
  3. Hack
  4. Commit my changes.
  5. git diff HEAD^1 > interdiff.txt
  6. git diff 8.x > node.fix_stuff.12346677.5.patch
  7. Post to d.o.

Here is the github workflow, with hub (a backwards-compatible wrapper around git) installed:

  • First time I contribute:
    1. alias git=hub
    2. git clone drupal/drupal
    3. cd drupal
    4. git fork
    5. Hub asks for my GitHub login. It converts it to a set of API keys, and remembers them.
  • New PR:
    1. git checkout -b smash_poll_module
    2. Hack
    3. Git add/commit.
    4. git push --set-upstream Gaelan smash_poll_module
    5. git pull-request
    6. Hub asks me for PR description. If I include "fix #5" in my PR description, these things will happen:
    7. My PR be linked to in issue #5.
    8. #5 will be auto linked in my PR description.
    9. When my PR gets merged (one click operation for maintainer), #5 will automatically get closed.
  • Improve my PR
    1. git checkout smash_poll_module
    2. Hack
    3. Git add/commit
    4. Git push
  • Improve somebody else's PR (that they kindly gave me commit access to (yeah, that requirement sucks)).
    1. git checkout http://github.com/drupal/drupal/pull/123 (this gives me a branch for their PR; first time only)
    2. Hack
    3. Git add/commit
    4. Git push

Thanks for this! This is

webchick's picture

Thanks for this! This is really helpful for people like me who don't use Github extensively to understand the differences in collaboration techniques.

You're welcome!

Gaelan's picture

You're welcome!

Not having an option to down vote is a bit strange

rooby's picture

With all these ideas I find it a bit funny that you can't down vote.

We can see that 33 people think it is a good idea, but for all we know there are 300 that think it isn't. Or maybe there is only 1, we can't know for sure.

The only way we kind of know is by people commenting, but not all people with an opinion will comment and it is not so easy to compare comments against a number.

Is there a reason for this voting system?

To answer your question,

webchick's picture

To answer your question, removing the "vote down" ability was a decision made by the g.d.o leads when this became effectively a way to insult/silence people in debates like this. You can get the same info by viewing how many opposing comments are voted up, or how many votes that counter-proposals have.

Speaking for the D.o SWG, though, we're not really interested in counting votes as part of this process; they're a factor, certainly, as they help point out serious pain points that need to be factored into the roadmap, but this brainstorming process isn't a poll, and popularity is not going to win the day.

They were removed...

chx's picture

... because comments with a negative comment count became kind of hidden; without that they could've stayed. At least I would have the most negative votes!

Sure?

tsvenson's picture

I think you and I would be competing for that honor ;)

--
/thomas
T: @tsvenson | S: tsvenson.com

What are the benefits?

chx's picture
  1. Lower barrier for people to contribute - unproven hype.
  2. higher visibility of Drupal code in the Github social ecosystem - do we expect more contributors? I have doubts... I never find code in the "social ecosystem" I always find code though Google.
  3. more efficient contribution workflow - unproven hype at best, patently false, at worst.

Build-in features:

  1. better git viewer - sure
  2. integrated dreditor (comment on line level) - that is indeed a nice feature
  3. inline editing of files - needs a fork first, however, but it is a nice feature
  4. linking issues - at least this should be easy to implement.
  5. pull requests - sure that's a feature but that helps us how?
  6. private repos possible - we don't need that
  7. extensive rest api - sure

Aside from the inline editing what is the killer feature that would make such a move justified...?

private repos possible - we

Gaelan's picture

private repos possible - we don't need that

http://dgo.to/bluecheese

We have this solved, though.

inline editing of files - needs a fork first

If I go click edit on a repo I can't commit to, GitHub makes a fork for me. Also, forking is a one-click operation.

what is the killer feature that would make such a move justified...?

Ripping out a huge chunk of code (some of which will be replaced by code that generates project pages from the API) that we have to maintain/port to D7 and beyond.

EDIT: No killer feature, but GitHub's notification emails are better, for two reasons.
1. I don't get emailed for my own comments.
2. I can reply to a notification email I got, and it will be posted as a new comment.

Bluecheese is maintained in

tim.plunkett's picture

Bluecheese is maintained in bzr. Not git.

Yes, I know. But if the

Gaelan's picture

Yes, I know. But if the Bluecheese maintainers ever do want to move to git/github, they can.

What I find compelling about

webchick's picture

What I find compelling about this proposal (assuming it is a whole-sale "move to Github" that includes not only repositories but also issue tracking, wiki pages for "meta" issues, etc.) is three-fold:

a) It greatly reduces friction for people who already use Github day-to-day to contribute to Drupal. This is, as of April, 3.5 million users with 6 million repositories. Compared to 28.5K users and 26.7K projects on Drupal.org; it's no contest. Technically, we could've run our own "IRC" channel with a pile of PHP code in a Drupal module. Instead, we chose to partner with Freenode, which gives us much more seamless collaboration with other open source projects. I feel that this move would bring a lot of the same benefits.

b) The biggest message I'm getting from this brainstorming exercise so far (and, granted, we're only a couple of days in) is that developers are desperate for better tools. And unlike Drupal.org, Github has a long history of delivering features that developers love, and 100 million dollars to do it with. Compared to the Drupal Association, which has more like a $150K/year budget for website improvements, and has to balance this not only with providing website features but also ongoing maintenance, improving the DrupalCon websites to allow us to fund development in the first place, etc.

c) Also coming out of that budget is upgrading the site to the new major Drupal versions from time to time, which always gets stalled for months (or in the case of Drupal 7, more than a year) on the pile of custom code (and technically contrib, but only used by Drupal.org code) that is our "dog food" repository and issue management software. It's a huge black eye to the project for our flagship website to be running a 5-year old version of Drupal, when we're about to ship a new major version most likely next year. Outsourcing that huge pile of code would eliminate this problem. (Granted, though, there are other options than Github for this; e.g. Gitlab, or splitting up the sites so "downloads" is on its own site.)

Now, there are a lot of very serious, fundamental communuty-shaking/shaping reasons not to do this, but I don't understand at all not seeing clear benefits of this proposal.

Nope

chx's picture

Developers are desperate for hype. If they would be desperate for better tools they would be breaking down the doors to contribute to drupal.org. We are talking of capable developers, right?

With that said, I already said (and in fact, I told you months ago in person): it's inevitable we move to github. The question is only how do we mitigate this inevitable hype-driven disaster community wise? I am actually serious. For example, the step-by-step guide tab right on project pages was a great way to mitigate the disaster git was.

We indeed should focus on this

chx's picture

So you said moving to git has probably kept people on the mothership. While that's likely true, I still do no think it was the best fit. But, fitness for a single task is apparently not enough when people use a tool day in and day out they don't care enough to use the best tool for the job. If Drupal did not force git on hobbyists, some other project, job etc would do.

Similarly, as the world moved to git and then to github, we will move to github. Not because any particular feature but because everyone else did, plain and simple. It's equally a shitty fit but that won't stop the tide. No risk me or jthorson listed will stop this tide either. Even if we could make the best tailor fitted tool for our workflow, noone will care except a few diehards. The community as a whole wants to learn one tool and that tool is github.

Acknowledging this is perhaps our best bet and then focus on how at this could actually happen and how can still keep the community together is probably the best way to spend our energies.

Similarly, as the world moved

Liam McDermott's picture

Similarly, as the world moved to git and then to github, we will move to github. Not because any particular feature but because everyone else did, plain and simple.

This is a possible risk that counters the maintenance advantage of moving to Github: if/when the next big thing comes along, does time have to be expended moving to that? At least with keeping everything on d.o the best features of each newly hyped system can be integrated into drupal.org's own.

Another risk: how stable are Github's APIs? Assuming tight integration between d.o's project tools (issue queue and such), would there be any cost savings if they're like Facebook and changing their APIs all the time?

Drupal developers should spend time on Drupal

Mac_Weber's picture

If they would be desperate for better tools they would be breaking down the doors to contribute to drupal.org. We are talking of capable developers, right?

This is not true. Most developers are willing to contribute, they just don't want to spend time reinventing the wheel.

Most developers want to help to make a better system. Spending time on implementing tools that are already available somewhere else is not helping directly on improving Drupal itself.

Yet you came with a good argument in your blog: http://drupal4hu.com/node/371
However centralized discussion also can happen in GH when a pull request is opened. Although I think this point should be improved in GH to be ideal.

@webchick: Regarding b and

tsvenson's picture

@webchick:

Regarding b and c:

Yes, budget is of course a big difference between github and d.o. However, I think it is also a matter of motivation.

Currently d.o runs on D6 and it was a massive effort to get it updated to that. The same with the current update to D7. Its not difficult to follow this work and get a feeling for the challenges those working on this are facing.

It is also quite easy to see why its difficult to motivate volunteers to this. Who want to fix D6 stuff when it will stop being supported in a few months? And then updating d.o to D7 when we are busy working on getting D8 ready.

Another problem with this is that it's quite easy to get a feeling that a many see d.o is just a burden that takes massive amount of efforts to drag along instead of the amazing resource and collaboration melting-pot it in reality is.

The arguments in this discussion that moving to github would free us from having to maintain "that stuff" partly stems from that.

Also, lets not underestimate the message we send to potential new users here. If we can't run our own "shit" on what we recommend them to use...

We also know from history that it takes 6-12 months after a new major is X.0 released until enough contrib modules are ported and users see it as becoming useful for building sites with. Also look at how long it took many big D6 based distributions to get D7 versions...

I believe we can rethink and change that scenario.

If we set the criteria that a new Drupal Core major can not be released until the d.o infrastructure runs on it, then it will change peoples look and motivation on all of the above. Including motivation to work on the contrib modules d.o uses.

It would also bring the whole of the community together around common goals, resulting in a lot of great synergy effects too. The project management and work on *.d.o, Drupal core and contribs will be able to co-operate in a whole new way. People will get motivated to volunteer working on d.o as they no longer see it as dragging it along and updating it to yesterdays major, but as a vital part of the development of the new Drupal Core major.

It will also bring in contrib projects into the core development process at a whole new level, especially when it comes to testing API's etc. But also make sure that a number of important, and needed, contrib projects are ported and ready for prime when the new Drupal Core is.

As an added bonus, *.d.o will face fewer needs for tweaking stuff to get it to work as it can be fixed at the source. Just using *.d.o as a test environment will, I'm sure, provide invaluable data and feedback to make each new major much better and ready.

Most important though, it will send a massive message to the market that we take things seriously, that we don't release a new major until we can run our own stuff on it.

Done right, I don't think this will actually delay the release of new majors. Since it will shift teams that now are busy working on D7 stuff to D8 it will actually add resources the community is already spending budget on.

I am also convinced this will at the same time make it much easier to raise funds for important community work too.

--
/thomas
T: @tsvenson | S: tsvenson.com

Major benefits after release too

tsvenson's picture

Oh, and lets not forget the benefits after a new Drupal core major is released too. All *.d.o work will continue on it and thus the vast majority of our limited budget will be used to both maintain *.d.o and at the same time the current and recommended Drupal Core!

--
/thomas
T: @tsvenson | S: tsvenson.com

If we set the criteria that a

Garrett Albright's picture

If we set the criteria that a new Drupal Core major can not be released until the d.o infrastructure runs on it, then it will change peoples look and motivation on all of the above. Including motivation to work on the contrib modules d.o uses.

That's kind of a crazy idea, but I like it. I think we would need to open up D.o repos and databases and such to more than a handful of developers in order for that to be feasible, though.

Thanks Garrett. Yes, I know

tsvenson's picture

Thanks Garrett. Yes, I know its a kinda crazy idea, but I am serious about thinking it can actually work and be greatly beneficial for us.

I've expanded a bit on it in a blog post I just published. Welcome to check it out on http://www.tsvenson.com/blog/2013/08/dogfooding-our-drupalorg-infrastruc...

--
/thomas
T: @tsvenson | S: tsvenson.com

We used to

Crell's picture

We used to do that up until Drupal 5, maybe Drupal 6. Then we went into a cycle of making much larger changes to core, which made upgrades more difficult, and Drupal.org started using more than just core+project*. A Drupal core update today takes way more work than it did 6-7 years ago. when we still did that.

If we did that today, Drupal 8 would not get actually released until 2016. No, we're not doing that.

If we did that today, Drupal

jthorson's picture

If we did that today, Drupal 8 would not get actually released until 2016.

If you're honestly saying that, with a dedicated effort, it would still take over two years to port Drupal.org to Drupal 8, I'd suggest that we've made some big mistakes with D8 core. Instead, I'll simply file this away under speculative exaggeration. ;)

If we did that today, Drupal

tsvenson's picture

If we did that today, Drupal 8 would not get actually released until 2016. No, we're not doing that.

Please elaborate on that. Today, as I explain in http://www.tsvenson.com/blog/2013/08/dogfooding-our-drupalorg-infrastruc..., the community is managing three major versions of Drupal Core. This change would effectively take that down to two and thus free up quite a bit of resources.

I didn't join the Drupal community until after D6 was out so I can't say much about how things worked before that. However, I am quite sure we have much more resources available today, as well as a lot more experience about project management at our hands.

Then factor in the transformation the Drupal Association has gone through only the last year. Including taking a much more active and resourceful role around drupal.org.

See for example https://association.drupal.org/node/17983 where a team lead by DA staff ported the project and issue queue modules to D7. The post says they rewrote roughly 90% of those two modules. Not a small task.

Instead of just flatly blank this idea, I think it is worth at least exploring it further. At least to better understand where our limited resources are used today, especially in regards to d.o and Drupal 6, the cost of upgrading it to Drupal 7 etc. Then look at how much of all those efforts that is helping to improve Drupal core and contrib projects.

Maybe we get surprised and find out that those resources can be used more smartly and at the same time help driving Drupal and the community forward. Or as I say in my post:

"This would in itself change focus from maintaining the past to pushing the future!"

--
/thomas
T: @tsvenson | S: tsvenson.com

Totally on board with this

haydeniv's picture

Totally on board with this just not for D8. Let's get that into the wild and get this structured properly with the next release cycle so everyone can be on board with it. I believe making d.o upgrade the same time as release will also result in a shift in priorities. For anybody that has ever tried to upgrade a site from one major Drupal to another knows it is VERY painful and costs a lot of money. We see that now with drupal.org as it is. Imagine what we are doing to our biggest Drupal users. I see this resulting in a much less painful upgrade path for all of our users and just make Drupal better. Right now upgrading core versions is the dirty little secret we hide from our clients because if they knew the actual costs going into it they may reconsider using Drupal for their platform.

+1.

joachim's picture

+1.

Another advantage of GitHub

danylevskyi's picture

Another advantage of GitHub is existence of mobile applications.

Don't know what you've got till it's gone ...

jthorson's picture

As a hobbyist (i.e. in my pre-"testbot cardiologist" days), I tended to float around from project to project, trying out new things one weekend to the next ... hacking on phpBB one week, menalto Gallery the next, and some sourceforge or github project the week after that. The problem is that I'd spend more time browsing through junk projects, obsolete forks, and half-completed code than I would actually hacking on the project I was working on. In the end Drupal's collaboration over competition ethos meant that there was probably only one or two modules I needed to look into, and in all likelyhood, they would actually work. And as an added bonus, they could all be found on a single site, and not mixed in with a whole litter of unrelated projects. I eventually quit wasting time sifting through google, sourceforge, and github, and have never looked back.

There is tremendous value in having everything located in one place, and the flexibility it provides ... as you shift components off to github, you dilute the overall value of drupal.org, and limit the community's ability to innovate on top of those tools. A move elsewhere adds complexity to issues like packaging, download/usage tracking, automated testing, update manager, d.o project browser, project management, security advisories, etc ... It's easy to imagine that moving to github can solve all our perceived workflow problems, but in practice, it will be magnitudes more work than the Drupal.org D7 migration ever was!

The most common argument I hear for a move to github is that we should be focusing on Drupal and not wasting resources developing/maintaining our own tools. I consider this a limited and very core-centric viewpoint, infering a presumption that everyone in the community has the same skill set and is well cut out for core development ... obviously this is not true. The community has a much higher number of drupal users and implementers than core developers; and while we do a great job of encouraging those implementors to get involved and try out core development, we actively discourage them from trying to help improve drupal.org ... through repeated disparaging comments about how slow, difficult, and impossible it is to get anything done on drupal.org, and sharing old horror stories, war wounds, and long-healed scars earned during past attempts. (Incidently, the infrastructure team has made tremendous progress in eliminating those earlier barriers to drupal.org development over the last 18 months!)

I'd argue instead that we, as a community, should be putting a much greater focus on our internal tools, using them to showcase the power and flexibility of the Drupal product ... perhaps even scheduling 'tool-upgrade' cycles in the months after a new core release and before work starts on the next major release branch. (Or maybe even before the core release ... which would guarantee that 50+ contrib projects in use on drupal.org would already be ported ahead and ready as of the core release date!)

Proponents of the github move will argue that this approach is a 'been there, tried that' dead end, as there are features we've wanted on drupal.org for years, which have never materialized ... I see these arguments incredibly ironic. For a community which prides itself on individual do-ocracy, and the ability to proactively re-invent Drupal 8 with a strong eye to the future, we also seem content to let ourselves dwell in the past when it comes to drupal.org, complaining that we've been waiting all this time, and "THEY" haven't built it yet.

In the end, the concept of ‘distributed versus centralized’ works in version control … but it does not work nearly as well in ‘community’. Distributed communities have a increased tendency towards fracture. Having drupal.org as the all-inclusive central hub is a large part of the glue which makes this community ... and product ... as strong as it is.

Project Pages

Gaelan's picture

The d.o project pages are here to stay. However, instead of being directly linked with code, people (who have passed the PA process) can click "Add Project", authorize drupal.org as an API client (so we can make sure people only add their own code, optional), see a list of their repos, select one, add some info (with defaults pulled from the project name and README, if possible), and click submit.

Thanks so much for your thoughts!

webchick's picture

In the end Drupal's collaboration over competition ethos meant that there was probably only one or two modules I needed to look into, and in all likelyhood, they would actually work. And as an added bonus, they could all be found on a single site, and not mixed in with a whole litter of unrelated projects.

Just want to point out quickly (and it's likely this needs clarification in the wiki page, because it keeps coming up), no one is advocating dropping centralized project listings on Drupal.org. They're advocating moving the git-related stuff to Github. I agree with you that this is one of the biggest advantages of Drupal, but it's also not under threat from this proposal.

The most common argument I hear for a move to github is that we should be focusing on Drupal and not wasting resources developing/maintaining our own tools. I consider this a limited and very core-centric viewpoint, infering a presumption that everyone in the community has the same skill set and is well cut out for core development ... obviously this is not true.

I've seen this argument as well, but I don't think it means "focus on core." It means "focus on more squarely Drupal-related things." If our testing tools "just worked" without ever having to futz with them (either because they actually worked, or because they were out of our hands to futz with as in the case of Github), you could be spending more of your time on sports-related modules, for example.

Proponents of the github move will argue that this approach is a 'been there, tried that' dead end, as there are features we've wanted on drupal.org for years, which have never materialized ... I see these arguments incredibly ironic. For a community which prides itself on individual do-ocracy, and the ability to proactively re-invent Drupal 8 with a strong eye to the future, we also seem content to let ourselves dwell in the past when it comes to drupal.org, complaining that we've been waiting all this time, and "THEY" haven't built it yet.

As someone who's probably spent the most time of anyone trying to encourage contributions to Drupal.org and documenting how to do it, I don't really resonate with the idea of casting blame onto volunteers for not volunteering hard enough and/or on the right things. Drupal.org is a top-1000 website, with hundreds of thousands of users, and is the central collaboration point of all Drupal-related projects. It should not (and realistically, cannot) fall to the shoulders of volunteers to ensure that sufficient progress is made on our tools.

A move elsewhere adds complexity to issues like packaging, download/usage tracking, automated testing, update manager, d.o project browser, project management, security advisories, etc ... It's easy to imagine that moving to github can solve all our perceived workflow problems, but in practice, it will be magnitudes more work than the Drupal.org D7 migration ever was!
...
In the end, the concept of ‘distributed versus centralized’ works in version control … but it does not work nearly as well in ‘community’. Distributed communities have a increased tendency towards fracture. Having drupal.org as the all-inclusive central hub is a large part of the glue which makes this community ... and product ... as strong as it is.

I think that's probably the most lucid example of someone stating some of the large risks with this proposal. Thanks for that. Can you make sure those work their way into the "Risks" section of the wiki page in some fashion?

.

jthorson's picture

... I don't really resonate with the idea of casting blame onto volunteers for not volunteering hard enough and/or on the right things.

Its not my intent to blame anyone for the lack of tools ... but as someone who's community niche has been working to improve those same tools (or keep them running, in the testbot case) so that other volunteers can focus on their priorities, I reject the argument that "it can't be done because it hasn't happened yet" as conjecture; and point to the recent governance initiatives, formation of the drupal.org working groups, and even this "2014 roadmap brainstorming" exercise itself as signs that there ARE people listening, and that they are both able and willing to influence change ...

... well, willing for now at least, as long as the implementers don't allow themselves to become demotivated by the community's constantly negative messaging regarding the problems that they are trying to solve; and mob-like momentum behind ideas which not only disregard their efforts to date, but also end up creating a greater body of work for them in the future.

ideas which not only

joachim's picture

ideas which not only disregard their efforts to date

If you mean that moving d.org from CVS to git a few years ago was a huge task, and the work that went into that is pretty much thrown away if we move to github, then I think you have a point.

Centralized Project Listings

Mac_Weber's picture

Just want to point out quickly (and it's likely this needs clarification in the wiki page, because it keeps coming up), no one is advocating dropping centralized project listings on Drupal.org.

It was already in the proposal. I've edited it making it more visible:
https://groups.drupal.org/node/313068/revisions/view/657148/657198

Organization

Crell's picture

The "all over the place" problem can be mitigated somewhat by having a Drupal organization on d.o, and only "official" projects can live in that organization. That helps with namespacing (you can still "own" a module/theme namespace just as you do now), and we can manage handing over a project to new maintainer(s) without the "Wait, who's fork is the right one these days?" problem. Also, only projects in that organization will have any Drupal.org integration (whatever integration we decide that is). That is not identical to what we have now, but can help mitigate the problem you describe.

On GitHub

Crell's picture

To be clear: I think there's a lot of benefits to moving to GitHub. I think there's a lot of risks, too. (The difficulty in bouncing a patch back and forth between people ad-hoc is my biggest concern, followed by the relatively poor issue categorization tools.) At the moment, I am not sure what the balance is and whether GitHub is the Right Thing(tm) for Drupal.

What I am sure of, however, is that it's very clear that the rest of the PHP community has adopted GitHub as the de facto platform for software collaboration, as have many other communities (Rails, Node.js, etc.).

What I am sure of is that moving to GitHub would vastly decrease the cost of adoption for already-PHP-skilled developers who want to contribute to Drupal, because they already know the toolchain.

What I am sure of is that we owe it to ourselves to give GitHub a good long hard look, and see if we can make it work. Maybe we can't. That's fine, but we need to dig into it and figure that out, not just dismiss it out of hand.

And if not GitHub, then we should move to some new infrastructure that involves vastly less code written exclusively for Drupal.org. Open Source applies to us, too. Let's make less code in the world.

The difficulty in bouncing a

joachim's picture

The difficulty in bouncing a patch back and forth between people ad-hoc is my biggest concern

And that is how the majority of issues are fixed: patches are bounced around.

I've just done a quick scan down the top of the core 8.x log, and most commits have more than one name in the credits. This is our main workflow, so we'd better be sure we're either improving it or at least not making it harder.

Druplicon sees the world

Gaelan's picture

How about this: we create a bot (I'll call it Druplicon for now). Then the workflow would be like this:
1. Contributor Caitlin posts a PR, gives Druplicon write/admin access to her forked repo.
2. Reviewer Rodney sees a few minor typos, and decides to fix them. He posts a comment with some sort of special syntax (@access maybe?).
3. Druplicon "reads" Rodney's comment and grants him commit access to the fork.
4. Rodney fixes/commits the typo, possibly via GH's web UI.
5. Rodney's new commit shows up on the PR.

Hm. Not crazy about this...

webchick's picture

IMO anything that moves us away from the standard Github development workflows only creates "Drupalisms" that then make Drupal harder to contribute to and develop for, and end up negating a lot of the original benefits of this proposal. :(

Is there a way to address this concern without resorting to our own custom "process hacks"?

Another problem with github's

Gaelan's picture

Another problem with github's issues is that issue attributes (title, body, tags, assignment) can only be edited by commiters (the issue creator can only set and edit title/body). Anyway, I think many of these problems could be solved with a bot sitting in the issue queue (with commit access so it could set attributes),

I remember when I first

Everett Zufelt's picture

I remember when I first became involved with the community. Things were confusing. This seems somewhat more confusing, and it may make it more difficult for people to get involved. I do like the idea of an evidence based approach, and I think that measuring something around increased contributions is a good idea.

I Am concerned from an accessibility point of view. Github's UI is not particularly accessible, and our community has no control over improving that today, or preventing it from getting worse tomorrow.

Accessibility Consultant & Web Developer - Zufelt.ca
@ezufelt on Twitter | LinkedIn profile

I'm not saying Github will

LewisNyman's picture

I'm not saying Github will solve all our problems but I find the current patch-centric workflow difficult. If you work with an issue that is touched by a lot of people it becomes very difficult to track what has changed, where, and why, even if everyone is experienced enough and being very thorough and creating interdiffs.

There's much more to it

tsvenson's picture

After reading through this whole thread I clearly get the impression that the proposal stems from issues with the current code production and collaboration workflow. While that is a very important part of development, there are a lot more that needs to be taken into count.

One of the aspect of the Drupal community I love the most, is that it is fairly easy for almost anyone, including with no coding skills, to start participate through the issue queue.

@webchick had good arguments, https://groups.drupal.org/node/313068#comment-953633, that moving to github isn't about picking parts we like there and keep the rest on existing features on d.o - "In other words, people who use Github use the whole enchilada"

That would mean moving the issue queue too. What features would we then lose? Take moving issues between projects as an example. It is a feature that greatly improves the collaboration UX between developers and projects on d.o. Someone pointed out that is not equally easy on github.

@chx pointed to the patch workflow, https://groups.drupal.org/node/313068#comment-953888, and specifically that the current one is community based and that will probably be lost on github.

For me being able to quickly spin up a Drupal instance with a patch on simplytest.me has been the best productivity improvements since I first joined the community. Its also the best improvement ever been added for non-coders to help testing things incredible easy.

From my point of view, it isn't the repository that drives development of Drupal - it is the issue queue!

The IQ is also where the strength of our community lies, its where users of all roles can meet, discuss and collaborate on things.

Michael Keara, with a little help from me, presented a session at DrupalCon Portland about the "Love and hate in the Issue Queue garden". Or, as I called it "Our Drupal Workplace: The Issue Queue" http://www.tsvenson.com/blog/2013/05/our-drupal-workplace-the-issue-queue.

Maybe moving to github solves some short term problems for code production. But, from what I have read in this discussion I have a bad feeling a lot of the current cross role collaboration might be lost, resulting in the risk that developers will become more isolated from non developers.

Is that really what we want?

--
/thomas
T: @tsvenson | S: tsvenson.com

No, that's why we want to

klausi's picture

No, that's why we want to move our Git repositories to Github and integrate with the Github API in the issue queue. I'm not convinced that we want the whole enchilada :-)

And don't worry about simplytest.me: it will work regardless where the Git repositories are, it will still pull data from drupal.org to locate them.

If I understand things

tsvenson's picture

If I understand things correct, moving to github will make it easier to use forks instead of patches to work on improvements, and then push back those to the main project. Is that correct?

If so, how easy will it then be for users to keep track on what is going on and participate? How will I easily find these improvements and be able to test them?

Someone who use Drupal to build sites with, find a bug or suggest an improvement need to be able to easily follow and participate too. And it needs to be on a level that doesn't require them to learn tons of new skills that is very far from their actual roles.

While not optimal, nor perfect, d.o is accomplishing this really well. A lot better than most other open source projects I have got myself into.

--
/thomas
T: @tsvenson | S: tsvenson.com

.

mradcliffe's picture

Of course there is nothing stopping someone from forking and adding the github features like issue queue, wiki, etc... to their own fork.

I see that as a benefit and a trade-off of Github because on the one hand it makes it easy to turn on/off features, and the other hand it doesn't necessarily solve the core sandbox project vs core issue issues from D8 development cycle.

Wha?

Crell's picture

Moving Git storage to GitHub but otherwise leaving the issue queues as they are on Drupal.org offers exactly zero benefit, other than people trying to use PRs on GitHub and being confused and lost when we ignore them or tell them they need to use some other site.

The PHP-FIG group does that now, and we constantly get people filing PRs or discussing on PRs instead of the mailing list, like we keep telling people to do.

IF we move to GitHub, we move the issue queues as well. There's no point otherwise. It's not like the disk space of the repos is expensive.

So moving to GitHub means we

Garrett Albright's picture

So moving to GitHub means we have to use their issue queues (at least according to you and webchick). That presents a whole bunch of problems, but I think the worst one is that it takes us a major step away from eating our own dog food.

Maybe if we were just starting a new project today it would make more sense, but we have what we have now. In terms of using our own site-building system to build our site, we should be gaining more ground, not less.

Had a hunch that would be

tsvenson's picture

Had a hunch that would be needed.

The risk then is it will alienate a large portion of the community that will question having to create a github account for participating. Just creating the account is probably the smallest obstacle compared to having to learn a very developer centric platform that they have little, or no, use of in their own roles.

In my book that is a huge risk and one that will distance developers and users instead of improving collaboration.

--
/thomas
T: @tsvenson | S: tsvenson.com

This discussion coder heavy

tedbow's picture

I agree that we would alienate a large portion of the community by moving the issue queue. Just looking at this thread it seems that most weighing in are coders but my experience dealing with contrib issue queues is that a lot of people involved in filing issues are not coders.

I think Drupal is a special case in that we are trying to make a tool that allows people to make websites without coding. So we get a lot of non-coders using it and participating in our community. I don't know enough about the general Github community to know if this is common, my guess is not.

If we moved our code and our issue queue you would be demanding that all users of Drupal not just developers create Github accounts if they want to file support requests, feature requests, or bugs. I think this would be a large hurdle and disincentive to most people.

Effectively we would be saying yes you can search for projects, participate in groups.drupal.org discussions but if you want to actually weigh in or get help with a module you have to create this other account for the place we maintain the code.

Drupal Trainer and Developer
sixmiletech.com

I added this to the wiki node

Liam McDermott's picture

I added this to the wiki node at the top:

Forcing users who just want to report and get updates on issues onto a separate, unfamiliar site (where they will presumably have to log-in again).

Unless I'm being stupid, this (very valid) concern wasn't on there already.

The simplytest.me integration

webchick's picture

The simplytest.me integration argument feels like somewhat a red herring to me. You need to use a Greasemonkey script (Dreditor) in order to have simplytest.me integration in the current issue queue. Adding it to Github's issue queue would be the same.

@webchick I see your point,

tsvenson's picture

@webchick

I see your point, but what concerns me is @chx description of the workflow in https://groups.drupal.org/node/313068#comment-953888.

For me his description sounds like the changes, patches in the d.o IQ, wont be as visible and easy to find and follow on github. Now I can simply scroll through a specific issue and see the patch evolution in it. However, again if I understand @chx's comment correct, the same patches might very well be in forks and more hidden in commits and PR's.

If that is so, then a lot of users that today participate in our IQ's will have a much harder time doing so on github.

I think it is important that we really consider the balance between just how much a move would improve the UX for developers with how this would change the ability for less code oriented users to participate and collaborate.

--
/thomas
T: @tsvenson | S: tsvenson.com

Don't understand visibility concern w/ PRs...

webchick's picture

Yeah, I'm not totally clear on chx's concerns. PRs seem to show you the visibility of the evolution of discussion way better than our current issue queues.

Just showing https://github.com/FredrikNoren/ungit/pull/75 and https://github.com/BBC-News/wraith/pull/16 as a couple of random examples that I found:

1) Whether or not testbot is passing the latest "patch" is built into the "issue summary," rather than you having to go digging for it.
2) Visibility into who is following the issue, not just how many.
3) Images are inlined automatically. This is actually easier for non-technical users than hand-typing HTML tags.
4) Visibility into interim commits that happened between the original post and its comments. We currently get zero visibility into interim commits with Drupal.org sandboxes, just a huge patch and a merge at the end.
5) It's clearly noted when/if an actual merge happened, as opposed for looking very, very carefully for "Committed and pushed to 8.x" type strings.
6) You can add a "Line note" for code reviews (similar to Dreditor), but line notes about outdated "patches" get collapsed by default so they don't sit there cluttering the list! That means we could most likely drastically cut down on 100+ reply issues.
7) Seeing peoples' faces makes for a much friendlier place to discuss things than tiny blue nicknames everywhere.

All in all, I'm not clear at all where the visibility problems get introduced. I think what chx is saying is when you click the "fork" button, but before you click the "pull request" button, no one has any idea what you're working on. But that's true of code i have on my localhost, or code I'm tinkering with in a Drupal.org sandbox. At least with Github it seems like you'd have the option of viewing a list of them (https://github.com/symfony/HttpKernel/network) (although I can make absolutely no sense of Github's network graph :P~).

Visibility into who is

Gaelan's picture

Visibility into who is following the issue, not just how many.

That is not the followers; those are just the people who have commented. We can't even tell how may followers we have.

although I can make absolutely no sense of Github's network graph

Dots are commits. Horizontal lines are branches. Arrows represent parents (of commits).

Embed image is only a click

tsvenson's picture

Took a look at the github links you provided and my initial reflection is that they are both quite busy with lots of information and at the same time cryptic in the way that there things are as minimal as possible. The link to the commit is just "4e6f06a" for example.

I can understand that developers has no problem with this and that ripping out as much distraction as possible is a benefit.

But that also raises the bar for learning to use this stuff and thus makes it much harder for non developers to get motivation to participate. Especially as we need to understand that these non developers see little benefits of this in their own roles.

I know I repeat myself when I say that this is something that d.o is already much better on, that is including non developers. I strongly feel the risk of this going the wrong direction is high by moving.

On d.o we have the ability to find a good balance about improving the UX, productivity and collaboration for all roles based on our needs. I don't think we will have the same on github.

3) Images are inlined automatically. This is actually easier for non-technical users than hand-typing HTML tags.

Once you have attached the image there is a handy [embed] button that does the job for you.

--
/thomas
T: @tsvenson | S: tsvenson.com

I know I repeat myself when I

tedbow's picture

I know I repeat myself when I say that this is something that d.o is already much better on, that is including non developers. I strongly feel the risk of this going the wrong direction is high by moving.

I think is would be even more of a problem in the future using github. I think we can assume that github will keep trying to make there issue queue better going forward for the developer's experience. I don't think this is necessarily going to improve the useability for the majority of the current participants in our issue queues.

Drupal Trainer and Developer
sixmiletech.com

.

Gaelan's picture

Once you have attached the image there is a handy [embed] button that does the job for you.

That's a dreditor thing.

Ahh, didn't know that. Its

tsvenson's picture

Ahh, didn't know that. Its definitely a thing that should be moved to the IQ module...

--
/thomas
T: @tsvenson | S: tsvenson.com

The first has very little

neclimdul's picture

The first has very little discussion. This is admittedly a less painful process in GitHub when you can just commit accept changes. I can't think of a single core issue I've been involved in that's happened on in my 6 years in Drupal development. We barely even do that in 1 line doc changes.

The second PR you link wasn't even merged. The other was without any apparently discussion or related issue or review or feedback. This is better?

I can't think of a single

Crell's picture

I can't think of a single core issue I've been involved in that's happened on in my 6 years in Drupal development. We barely even do that in 1 line doc changes.

Perhaps we should do that more often? Less talk, more code?

Disclaimer: I'm not aware of

greggles's picture

Disclaimer: I'm not aware of all the technologies that Github's ecosystem could provide use. E.g. maybe composer could help with packaging. So, I'm posting these as a comment first. They can be moved into the summary or merged once they've been vetted/responded to.

A big reason to move to github is that "someone will maintain stuff for us" but the below list is stuff that Github either definitely or probably won't maintain for us. So we would still be responsible to maintain things that require a large amount of effort to upgrade them between upgrades of Drupal.org (e.g. D6 to D7).

  • Testing: Travis' free level has <a href="https://groups.drupal.org/node/313068#comment-954168'>time limits that we would fail. We should make our tests more efficient. We could pay Travis.
  • Packaging/Licensing: Projects would still need to get packaged (including distributions and 3rd party libraries). We currently do this and it's a lot of work (both the original build and the ongoing maintenance). Would GitHub make this easier? Would GitHub respect limitations on 3rd party libraries to keep our licensing proper?
  • We would lose at least some data such as download counts
  • The "update" xml (which powers the update module and our usage statistics and notifications of available module updates) is woven into our packaging process. We would still need to maintain the code related to this xml.
  • The GH Issue queue lacks features we take for granted (updating certain metadata, issue statuses) - there are hacks to get around these (delegating to bots, tags) but those are weird and lack the formalized structure provided by the Drupal.org issue queue. Do we really need these features that we currently take for granted? Conventions around using the issue queue are fundamental to the way we collaborate on Drupal. Are these conventions around the issue queue something we need?
  • It's not possible to move issues between queues on Github
  • Canonical repositories/security team coverage: project pages could stay on d.o and reference the GitHub repository and the security team policy of facilitating SAs for projects that have stable releases on d.o would stay as is. However, I do fear people would be confused in trying to find the canonical repository which is a risk to getting the right (secure) repository.
  • There are some benefits to being strongly centralized in a place that has control from reliable parties. This research about WP themes shows how having things spread out on the internet makes it easy for malicious users to insert unwanted code into themes that are top ranked for popular search terms.

I would propose that d.o

mikey_p's picture

I would propose that d.o still maintain a package system of sorts along with updates.durpal.org etc. Look at examples such as Ruby Gems or Node's NPM system for a good example of how a VCS agnostic package system could work. It still allows tools like drush (or gem/bundler, npm) to search, install, and lookup dependencies (which d.o doesn't support yet).

Finding the canonical repo is a piece of cake as well, when you can check the package manager and see where it points.

Yes, but my point is to make

greggles's picture

Yes, but my point is to make it really clear there are things that we would still need to maintain.

AFAICT drush can lookup and resolve dependencies. Not sure what you mean about that.

I don't think "check the package manager" is a "piece of cake" for the average person who has downloaded Drupal. Are you saying we should concede that we're a geek only tool? My concern is basically around the case that someone searches on a non-drupal.org search engine and finds a random repository on github and thinks it's the real one when in fact it isn't.

webchick's picture

I'm dumb, but... could this be resolved by http://github.com/drupal/X being the canonical repos? For example, it's really obvious that https://github.com/symfony/HttpKernel is the canonical version and https://github.com/chadfennell/HttpKernel is not.

Yes,

klausi's picture

Yes, https://github.com/drupal?tab=members is already controlled by chx and mikl.

They could add some administrators like https://groups.drupal.org/node/142454 to that github group who would create the repositories and assign access to applicants.

Shall we move to GH for code

mongolito404's picture

Shall we move to GH for code repository, Drupal.org should still be the central place to look for and find contribs. Look how the Node.js (npm), Ruby (gems), jQuery (plugins.jquery.com) or part of the PHP (packagist.org) communities handle having code on GitHub (or other place) while still providing a central registry.

Discerning the "canonical" package of a module and ensuring uniqueness is handled at the registry level (ie. drupal.org in our case), not the repository one (ie. GitHub in the context of this discussion).

Having http://github.com/drupal/phingdrushtask would not much make sense, I'm the maintainer of this tiny project, not the people managing the Drupal oranisation on GH. There is no need to involve anybody else when it comes to managing the code repository. So if we move to GH, the code should be in https://github.com/pbuyle/phingdrushtask. Off course, https://drupal.org/project/phingdrushtask will still exist for me to manage releases, project description, etc.

Drush can only resolve

mikey_p's picture

Drush can only resolve dependencies after downloading each package. For instance http://updates.drupal.org/release-history/views/7.x does not list CTools as a dependency.

In contrast the Ruby Gems API actually has a feature that provides a gems dependencies via the API, for example: https://rubygems.org/api/v1/gems/rails.xml. Or something like http://registry.npmjs.org/express for NPM. I can't tell much about Packagist and composer at this point but I bet they offer similar options, since the dependencies are listed on each package page.

As far as determining the canonical URL of a package, I don't really see how that will be any different than it is now. I personally have several forks of Drupal.org projects that I made years ago for packaging purposes and no one has ever complained about those. Nothing is currently stopping me from starting my own Drupal Views module on Github now either, we just trust users to download their modules from Drupal.org and rely on Drupal.org's pagerank to ensure they end up in the right place.

This package system sounds

webchick's picture

This package system sounds like it might belong in its own proposal, since we could do that regardless of moving to Github or not, correct?

We already have a package

mikey_p's picture

We already have a package system, adding a few more small (non-critical) features is hardly worth a new proposal.

Also, personally I'm trying to avoid proposing anything I'm not willing to work on myself.

Keep release tarballs on d.o

Gábor Hojtsy's picture

Localize.drupal.org is currently dependent on the release tarball system to extract and parse out translatable text. Decentralising that would be an issue, but sounds like that is not the proposal here. Just wanted to make that point so its considered for similar ideas.

Centralization of releases

Mac_Weber's picture

Centralization of releases will be kept at d.o

Oh?

Just because github can make

greggles's picture

Just because github can make tarballs doesn't make it a good idea. There are some things we need to do (update xml for usage and security reasons being the biggest, download statistics as a moderate reason) that make tarballs on github a downside to this proposal, IMO.

How's the API integration for releases?

webchick's picture

Could we trigger these things to happen on Drupal.org using Github's REST API?

Not sure

Crell's picture

http://developer.github.com/

I don't see releases mentioned there, but if we ask for an API for them I doubt they'd be against adding it. It's probably on their internal todo list anyway.

If we're making releases on Drupal.org instead of letting Github do them (although I don't know why at that point), then we'd just be detecting a push of a new tag and taking action based on that. That's totally doable:

http://developer.github.com/v3/git/tags/
http://developer.github.com/v3/repos/hooks/

GitHub's API is fairly well designed, I think, so if it doesn't exist yet writing a Guzzle Service Description for it so that we can build logic around it should be dead simple:

http://guzzlephp.org/webservice-client/guzzle-service-descriptions.html

I just wanted to point out

cweagans's picture

I just wanted to point out that Github has the capability of creating releases, and if we can use it, that's great. If it doesn't make sense, that's fine too.

--
Cameron Eagans
http://cweagans.net

Just out of

webchick's picture

Just out of curiosity...

Let's say that we moved wholesale to Github. Including release management/tarballs. This would undoubtedly cause all kinds of work to have to happen in release-using projects such as localize.drupal.org (also Drush PM-* tools, Update Status, Simplytest.me, probably some others I'm not thinking of), because obviously you'd be talking to a completely different system than you do now.

But, if that work were to be done, and assuming it was done on open sourced Drupal modules like it is today, would that then mean that general Drupal websites could set up their own Drupal-powered translation UIs (and update checkers, and demo frameworks, etc.) that talk to their custom projects stored in Github? If so, that could potentially be a place where we could get much more people collaborating on those tools than we currently do, because they'd be of general usage beyond just Drupal.org.

update status, localize.drupal.org

Gábor Hojtsy's picture

Update status can already handle projects from different sources (eg. feature servers outside of drupal.org), the only requirement is that the update summary data is available in the expected format on that server.

The localization server (module powering Localize.drupal.org) can also be used on independent servers set up to translate whatever projects. There is a per project download URL for translations in l10n_update module (and built into Drupal 8 core), so you can get projects from different sources and still get translations supplied, again so long as the provided information on the 3rd party server is in the expected format.

I don't know about simplytest.me or drush pm-*.

It is extremely easy to see

Gaelan's picture

It is extremely easy to see what is a fork:

I think they are talking

webchick's picture

I think they are talking about the opposite problem... if I'm the "spif13/spif13-vim" project, how can I tell when someone spins up a new fork to work on something so I have some visibility into it?

Also, if I'm a member of the usability, accessibility, multilingual, documentation, "architecture," etc. teams, how do I easily figure out what all the pending changes are so I can provide early feedback?

https://github.com/spf13/spf1

Gaelan's picture

https://github.com/spf13/spf13-vim/network

I don't actually show up there, because my forked changes have long since been merged. :)

Some common d.org issue queue

joachim's picture

Some common d.org issue queue workflows:

  • user files an issue core, but it's a contrib problem. Maintainer moves the issue to the right queue.
  • what appears to be a code bug is in fact missing or erroneous documentation. Maintainer or reporter reassigns the issue's component.

How do we do things like this on github?

"user files an issue core,

dasjo's picture

"user files an issue core, but it's a contrib problem. Maintainer moves the issue to the right queue."
-> i believe the maintainer would ask close the core issue and ask to create the issue within the contrib module

"what appears to be a code bug is in fact missing or erroneous documentation. Maintainer or reporter reassigns the issue's component."
-> the github issue queue allows to tag issues. by default, the following tags are provided:
bug, duplicate, enhancement, invalid, question, wontfix
the available tags can be customized by the project maintainer(s).

i think those are valid options.

still, it seems like the tagging issues option on github is limited to project collaborators

It doesn't solve the problem

Mac_Weber's picture

It doesn't solve the problem raised by @joachim

Not everybody has the time/energy to report issues. Telling someone to close and reopen in another place is in no way good as just changing the value of a field.

There are many cases which someone opens an issue and even the maintainer thinks that the issue really belongs to his project, later researching he later moves it to the appropriate one.

Closing and reopening would lose this history in the fist opened issue.

-> i believe the maintainer

joachim's picture

-> i believe the maintainer would ask close the core issue and ask to create the issue within the contrib module

So that's quite a big regression from our current workflow, where the issue can just be moved to a different project.

-> the github issue queue allows to tag issues. by default, the following tags are provided:
bug, duplicate, enhancement, invalid, question, wontfix
the available tags can be customized by the project maintainer(s).

That's issue status. Do GH issue have a concept of components?

The tags are generic; they

Gaelan's picture

The tags are generic; they can be used for anything. The defaults include both status and category. You could easily add more tags to represent components.

Status and component are

joachim's picture

Status and component are orthogonal. Mixing them up in a single vocabulary seems messy to me.

Can we enforce a single component? A single status?

No, but only committers can

Gaelan's picture

No, but only committers can edit the tags, and I think we can trust Dries not to make a patch "closed (fixed)" and "closed (won't fix)" at the same time. :)

What about all the contrib

joachim's picture

What about all the contrib maintainers?

Also, if only committed can edit tags, how can a user reopen an issue? Or mark it 'works as designed' when they realize they've made a mistake?

This puts more work on maintainers.

Access!

Gaelan's picture

Here is how the access works (yes = can edit, no = can not)

Attribute Committer Creator
Title/Body Yes Yes
Open/Closed Yes Yes
Tags Yes No
Milestone Yes No
Assigned Yes No

so random passerbys are a no-go

Gábor Hojtsy's picture

So random passerbys are absolutely a no-go in terms of issue management. Eg. I cannot go and mark someone else's issue needs tests or even reopen it.

Yes, and this sucks. See my

Gaelan's picture

Yes, and this sucks. See my comment about a bot.

crowdsourced issue management

Gábor Hojtsy's picture

Currently issue management is crowdsourced. Anybody can tag an issue "Needs work" or mark it for backport, or figure out it needs an upgrade path. These are currently expressed with statuses and tags. If only maintainers can do these things, then issue triage as a thing will not be able to exist anymore and the crowdsourcing of issue management to initiative teams, etc. will not work...

We could have a bot with

Gaelan's picture

We could have a bot with commit access that obeyed everybody, but that's an ugly hack.

And very busy modules (such

joachim's picture

And very busy modules (such as Views when it was in contrib) rely on issue queue maintainers who are not committers to help with issue queue management.

Documenting Drupal.org vs. Github features

webchick's picture

Seems like discussion has now shifted to the level of individual features. This is great, but I can't keep up with all the discussion here, and there's not really space for it in the issue summary.

I made this spreadsheet https://docs.google.com/spreadsheet/ccc?key=0AusehVccVSq2dEtuWm1JQ1ZSV1Z... to capture this information. I made an attempt at filling out some things, but it's open to all to edit, so please help fill it out!

I'll add this to the proposal summary too.

The spreadsheet appears to

Gaelan's picture

The spreadsheet appears to have been spammed. We should revert it.

We should add GitLab to this.

Gaelan's picture

We should add GitLab to this.

Gitlab

attiks's picture

I added GitLab to the spreadsheet as well, so we can compare

Mail From Github

Gaelan's picture

Mail from GitHub:

Gaelan,

I haven't heard of anyone doing quite what you say here [referring to my "Druplicon sees the world" idea], but something that definitely does happen all the time are pull requests from fork to fork. So rather than just editing someone else's pull, you just make a pull request from your fork to their branch. Unfortunately, it requires a little more know-how to target a network branch than to just update a d.org issue as they are today, but this is absolutely something people do.

Something else to consider, and what I'd probably suggest, is that you can make a pull request from a branch to master in the same repository. Many, many open-source projects work this way. The core maintainers work on branches in the main repository day-to-day, and occasional or drive-by contributions are handled with forks.

With this setup, you get all of the same pull request goodness without any of the fork friction. The only thing that would be a real change from d.org's current workflow is that it would be harder for a core contrib to jump in and fix a new contributor's pull request without something like what you've said. The usual core contributors could all edit each other's branches as they can now.

People do run into this, and what usually happens is that the core contributor will check out the pull request, fix it up, and push to a branch on the core repo and merge that.

Your idea of an API that grants permission is possible. It's easy to imagine an oauth application that contributors give access to which will grant core contributors permissions on repositories named `drupal-*` or something. I'm not sure we'd exactly *recommend* it but people integrate with project management systems in lots of ways.

Hope this helps,
Ben

I started to simulate a d.o

Gaelan's picture

I started to simulate a d.o issue (#287292) in github. Will finish tomorrow/later today (it's 1AM). https://github.com/Gaelan/test/issues/1

.

mradcliffe's picture

I found it somewhat confusing to jump between the comments on the issue and the comments on the pull requests to do code review.

An equivalent of an issue summary would help.

100% in agreement with moving to GitHub

jrowny's picture

I personally think "dogfooding" is rarely a good idea. I've always been very unhappy with the workflow on D.O for contributing. I just don't like the "patch" workflow, and I learned that workflow before I learned the Git/GitHub workflow. For me, it's so much easier to fork, edit, and make a pull request. I can still pull in changes from master meaning I can take my sweet time working on a large pull request.

Sometimes we have to accept that dedicated tools (i.e. GitHub) are just really good at what they do. A huge part of Drupal contributions is integrating things which work really well at what they do... like Disqus for commenting, Recurly for subscription management, etc. I think we should look at this as a strength of our community.

Back to the original analogy, I think moving to a GitHub workflow is like eating our own "drupal flavored" dog food. The Drupal part comes with how we would integrate with GitHub and that shows Drupal's strength and helps the community contribute.

General observations

Crell's picture

Something I'm seeing here, which you're free to correct me on, is that it almost feels like GitHub would work better for contrib than core. That is, GitHub works really well for "small team of first-tier people with commit, plus assorted drive-by contributors". That's the typical contrib process, and I think it would work extremely well.

Core is the odd ball (and a few very high-tier modules like Views, when it was still in contrib). We have an extremely small number of committers, which on GitHub means they're the only ones that can do meaningful issue farming. We rely very heavily on issue categorization and curation and shuffling and tagging and so on, and on crowdsourcing that. We have an extra tier of people that get extra management/curation powers but don't get commit. We have issues that tend to come from lots of people rather than just one.

Also, I've seen cases where GitHub has a lot of trouble when a project has a bazillion forks. It becomes hard to setup PRs to the right repo if it's not to the main one. That's an issue that would affect, mostly, core.

Which is a really odd conclusion to come to. :-) Contrib would benefit from GitHub, but GitHub's issue management can't really scale to core.

To which my question then is 2 fold:

1) Could we vary our core process to still work with that tool? Given that many other large projects work quite well there I don't think this is an un-achieveable goal.

2) Could we pressure GitHub to improve the issue management tools to meet our needs? Given that we'd be the largest PHP project there by far, I don't think that's unreasonable to ask.

The issue tracker has been

cweagans's picture

The issue tracker has been the main pain point that I've heard about. I don't think it's unreasonable to ask Github to add some additional goodies to their issue tracker, but since they don't really have managers, what timeframe would that happen on? Is it a matter of convincing some specific person that these changes would be beneficial? (scor noted that there's a previous drupal contributor that now works at Github, so maybe he'd be a good person to ask: https://groups.drupal.org/node/313068#comment-953993)

The other thing is that we could potentially build a more full-featured issue tracker on top of Github, a la http://www.bitbucketcards.com/ (I realize that's for Bitbucket, but you get the idea - it's a 3rd party service that enhances the existing functionality of Bitbucket's issue tracker)

--
Cameron Eagans
http://cweagans.net

There's something very

Gaelan's picture

There's something very similar (also trello-like) for GitHub.

we could potentially build a

joachim's picture

we could potentially build a more full-featured issue tracker on top of Github

We could also extend our use of git to have pull requests on d.org.

Hard to say, but which would be more work?

I'm pretty -1 on github for

pwolanin's picture

I'm pretty -1 on github for Drupal. We use it for our work repos, and it's "ok" for that, but not much better than "ok".

PR's a great for one-off small contributions, but I don't think they are nearly as useful when several people are trying to contribute to a change.

While the grass may look greener, I would expect massive community disruption and fragmentation and I don't have any sense that it's worth the aggravation and loss of control we'd have.

Greener indeed

rooby's picture

I really do think this could be a greener on the other side situation.

As has been said elsewhere, our issue queue is fairly unique, so maybe what supposedly works for everyone else isn't a good fit for us.

What caught my eye here was "I would expect massive community disruption" and I'm not sure if even massive is strong enough a word.

This is changing our whole development workflow, which can be hard with a team of 4 let alone a team of thousands.

I know change is not something to be scared of but it is a huge and risky change.

Having to sign up for a new account because it is external to drupal.org and then learn a new tool is big.
I think because this is a developer focused discussion it might be underestimated how many people on drupal.org do not have github accounts and might be put out by the extra hurdles.

"Familiarity to new contributors who are used to github" has been mentioned, however that goes both ways. What about people who don't use github? They exist.

If issue queues go as well there are very large problems regarding people not being able to modify details of an issue, or move an issue to a different queue, or upload files, etc.

Maybe we can cross our fingers and hope github will change the way things work for us but it is not good to rely on external factors out of our control and it is very likely that not all of our requirements, if any, will be things they want in github.

I don't think it would be a matter of us off loading all our custom version control code to an external service, we're still going to have custom code somewhere if we are keeping ANY parts of our own, like project pages with release tarballs, issue queues, etc. and we would also probably be creating new custom code to integrate github to the parts of ours that we are keeping.

It seems that a lot of the benefits of github are helpful addon features, which should be implementable in our current system, possibly even with equal or less work overall.

Seems like it might be easier to implement what we are lacking on our end than to get github to implement what they are lacking (pure speculation as github is an unknown in regards to getting features added/changed).

These are not new points but they are concerning.

I'm with you about the

tsvenson's picture

I'm with you about the concerns you have, they maps well with my own.

As has been said elsewhere, our issue queue is fairly unique

The uniqueness of our issue queue is our own creation really. An issue queue is nothing complex, its actually very straightforward in the way it works.

Over the years we have tweaked it, we have taken many shortcuts in the implementations. Shortcuts instead of looking at the bigger picture of how an issue queue should work. The result is an IQ that quite frankly only is usable on d.o and for the development needs.

We use the issue queue for many non developing things too, but those queues are often hard to find, make any sense of and finally use.

We can change this though. We just need to change out attitude from looking at it as something that is not working for us to something that once again can help us push forward and drive development.

--
/thomas
T: @tsvenson | S: tsvenson.com

maybe .. maybe not

lsmith77's picture

Symfony works really well but there are a few observations:
1) all discussions happen inside tickets/PRs which are quite simplistic and we do not create bigger design documents there. note the github wiki is also very simplistic. that being said, discussions close to the code and inside the diff tend to keep discussions more technical and has severe reduced bikeshedding
2) there are only a handful of people with push rights to the main repository and only these people can add tags, close/reopen tickets etc. to get more granularity you need split things up into sub-repositories
3) it might be possible to layer a UI on top of github using their API to handle the deficiencies while still benefitting from their infrastructure, tools and 3rd party tools (like travis-ci.org, prose.io etc). Doctrine has also created some integration with Github and Jira.
4) I am sure you could contact github and discuss them whatever you feel is missing and whatever advise they have to offer given your very special existing setup.

Thanks for the insight to how

tsvenson's picture

Thanks for the insight to how Github works for the Symfony community Lukas.

What would be interesting to know in addition is what kind of users it is that participate in the Symfony issue queues. Here on d.o for example there are just under 30k users that has configured git access, but there are almost one million active accounts. Thus we have a lot of users with other main experiences and roles than developing that are participating in the issue queues.

Yes, our issue queues are often quite noisy as a result and sometimes the discussion are, shall we say, quite passionate about things. And sure there is bikeshedding going on too.

For me that is a price well worth paying for the added value all these users, me included I hope, brings to the community. Particularly as Drupal over the years have moved to a more UI-driven sitebuilding platform feedback from and testing by users are invaluable.

There are also many users, again me included, that have discovered they also wanted to give coding a go thanks to this. While my own skills are still very basic and at novice lever, there are many others who now are respected contrib developers and project maintainers.

Thus far in this discussion I haven't yet been convinced that will be possible to retain after a move to Github. I think the risk is quite huge we will lose most of them in the move.

I notice you have been a member here on d.o for 2+ years so maybe you can shed some light into this for me. Would be greatly appreciated.

--
/thomas
T: @tsvenson | S: tsvenson.com

I haven't really participated

lsmith77's picture

I haven't really participated in many discussions on here and usually it was just someone asking me to post my insights on a particular topic. So it was more hit and run.

One thing to note: Git was build for Linux and Linux has an onion development principle. ie. you have layers around the core and many people only ever send code to an outer layer from which those changes get to a layer closer to the core.

webchick's picture

First, thanks so much to everyone who has participated here, and has helped flesh out Ye Olde Spreadsheet of DOoOoOoOm. I feel like we have made a ton of progress here in identifying the actual pain points on Drupal.org, what makes Github so compelling, and what concerns are about moving our tools off-site.

Here's my current read of the discussion/research so far. Please help correct this where it is wrong; the D.o SWG has a meeting tonight and I'd like to provide an update, and I'd like to ensure that update is accurate.

Pro-Github

Github has a few key things going for it that are driving votes to this proposal. They are:

  • Ubiquitousness. With a community of 3.5 million users and 6 million repositories, used by many major open source projects, etc. there's almost no developer out there who hasn't been exposed to Github in some fashion, and most use it for their day-to-day work. Switching to Github thus reduces a pretty major "Drupalism," which reduces the barrier of contribution for a critical mass of developers.
  • Pull requests and amazing code review tools. Because patches are sooooo 2003. :P Pull requests integrate into the development process: Write code, run command, poof, record is there. Push more commits, poof, available to all. No need to code, diff, figure out interdiff, post, upload.

    Github also offers awesome code review tools; highlight a line, comment on it in-place, comments on outdated diffs get automatically collapsed from view, thus cutting down dramatically on 100+ reply issues. Drupal.org has Dreditor, a browser add-on which tries very hard to replicate some of this, but it's not really nearly the same.

  • In-browser editing of files. This is a huge killer feature, particularly for non- or less-technical contributors. See a typo in a comment somewhere? Click 3 buttons and you've got a fix submitted. Want to change the markup of that function? Copy/paste code directly from Firebug into the Github window and submit it directly to the maintainer in a couple of clicks. For a huge class of fixes, contributors wouldn't need to know anything at all about Git in order to help! Versus right now, they need an entire ladder of tutorials in order to learn how to do this type of contribution effectively.

    The other thing I really like about this is we'd no longer need to mark things "needs work" for silly minutia like whitespace or grammar; the reviewer could just fix it up as they go. This would be a huge shot in the arm, morale-wise to core developers, and probably others.

  • Not invented here. It could also kill off several thousand lines of "contrib but actually custom" code that our community is responsible for maintaining and paying for. Time/money spent on this huge chunk of code slows down our ability to keep our website up to date on the latest major Drupal releases, and would free up more resources for tackling problems that more directly relate to making Drupal a better CMS with better add-ons with a better website supporting it.

Pro-Drupal.org

While Drupal.org can feel old and crufty at times, there actually are a number of important benefits that it offers us, namely:

  • Centralization / Canonicalism. Two really critical features that Drupal.org offers are: 1) one place to look for all Drupal-related code (on Github it could be literally anywhere), and 2) an easy way to denote the canonical project (http://drupal.org/project/views is obviously "the views module" and http://drupal.org/sandbox/webchick/387493 is obviously not, even if I called it "the Views module." These are absolute requirements, so that means that even if we switch development/collaboration tools to Github, Project module (the thing powering the download pages) is here to stay. (Though, potentially not also Project Issue Tracking, Project Release, Version Control API, Version Control API Git, Drupal.org Git, Drupal.org Project, Git instructions, etc. etc.)
  • "Swarming" vs. Forking. Read Crell's nice comment for more details. This particularly happens in the core queue, but there's potential for it in any issue. The Drupal.org collaboration model is that a problem exists in one central place (issue), and people come around to solve it, working iteratively on the same chunk of code until it meets the proper standards. There is visibility by everyone interested in the fix at every step in the process, and all steps are in one place to go back and reference or learn from.

    By contrast, the Github collaboration model is built around individuals working in their own sandboxes, and only notifying people there are things going on when it's ready to merge. Lack of visibility into changes coming "down the pipe," as well as the risk of duplicate work happening is more likely here. It's also my understanding that people who've started PRs need to add co-collaborators one by one, which greatly works against "swarming" efforts. Because Drupal.org is great at...

  • Empowering the masses. Drupal.org-style "swarm" collaboration largely happens because anyone at all can change anything at all about an issue. Fix a title, add a tag, post a patch, mark it "needs work," update the summary. This provides a lot of opportunities for lesser-known (or not known at all) community members to help with "issue queue triage" and leave the tough problems for maintainers. The "Community Tagging" feature that Drupal.org offers, in particular, is huge in empowering small groups to self-organize around particular issues and knocking them out at sprints, etc.

    My understanding is that on Github, there is no concept of "community tagging," and rather all of these types of "issue metadata" need to be managed by the repo maintainers themselves. And, because there's no way to delegate people with permissions to only update issue statuses and not also the code like there is on Drupal.org, doing so requires limiting the list of helpers to those who you trust not to commit wanton code to the repo without review (either purposely or accidentally).

  • Structured issue queue metadata. We make rampant use of issue "components" (a pre-defined list of categories defined by the project maintainer to help find issues), as well as "stauses" (needs work, postponed (maintainer needs more info), reviewed and tested by the community) that put issues through a given workflow. Github offers non-structured tagging that would be a poor substitute for this, and as mentioned above, switching any of these statuses on any issue needs to be done by über-busy maintainers.

As a result in the short-fall of Github's issue queue/workflow features, this original proposal was written around the idea of only using Github for the code management, and not the issue queue. But doing this seems to negate a lot of the benefits of the proposal in the first place. :\ This is something we can likely work with Github on (especially if we become a paying customer of theirs), but it's worthy of note.

I know there are other many, many other pieces of feedback that are not reflected here, but I'm trying really hard to "bubble up" the biggest ones. Do you feel this captures them? If not, let me know!

webchick's picture

...but I'm going to continue editing that comment for accuracy, so feel free to add a mail rule to ignore me. :D

One correction

Crell's picture

That looks pretty good overall. There's lots of other factors in the spreadsheet but I think these are the big points of contention.

One correction I'd make is the swarming-forking. There's nothing at all that stops me from working in my own little sandbox until a branch is perfect, then posting a 200 K patch right now. Sometimes I do that, sometimes I don't. There's also nothing at all that says you can't file a PR on github with in-progress work. People do that all the time to get early reviews. There is no difference in visibility between the two tools; it's all down to how people decide to use them.

What is different is that the process for 3-4 people to collaborate on a single issue is different with GitHub. On Drupal.org if I post a patch, leave it for 2 days, it gets stale (fast moving core), dawehner can come by and reroll it for me without my involvement at all. (He does that for me frequently, because he's awesome.) On GitHub, if I file a PR, let it sit for 2 days, and then dawehner comes by to update it, he has to file a PR against the branch in my sandbox that I have to merge, after which it becomes part of the PR against core. If I am taking 3 weeks off of core for some reason, the issue is therefore stalled unless dawehner takes my branch and files a NEW PR against core, closing the old one.

PRs, in their current form, tend to favor a single "champion" for an issue with 0 or more people backing him up via nested PRs. That works well when one person is really driving an issue, which is often but not always.

If you have a lot of collaborative back and forth, I'd need to give dawehner access to my repo so we could just both push to the same branch and be done with it. (This is actually what we're doing now for the HtmlPage issue, since I'm very liberal with commit access to the WSCCI sandbox.)

D.o issues, in their current form, tend to favor "serial drive-by" issues. That is, I'll do one patch, dawehner will do a second, timplunkett will do a 3rd, etc. until it is done. That works well when you don't have a lot of back-and-forth between people, which is often but not always.

If you have a lot of collaborative back and forth, unless you manually setup a shared sandbox branch (as you'd do with GitHub) you basically start over with each patch that's posted, and never maintain any history, ever. I have a lot of issues where I've had 3, 4, or 5 local branches that contain just one commit for the latest patch from an issue plus one commit for my changes. Manually incorporating interdiffs to keep a meaningful branch history is not worth the effort at all.

I don't think either one wins in this case. Both models have places where they break down severely. Note also "in their current form" in both cases, as both could probably be improved or worked around in some way.

Awesome!

webchick's picture

That really helps clarify the workflow differences between the two. I think I'll just add a link to this comment since I can't really see a way to condense it without losing info.

This was the biggest point I

Dave Reid's picture

This was the biggest point I was trying to make in the Google Doc but Larry more eloquently explains it.

Senior Drupal Developer for Lullabot | www.davereid.net | Gittip me!

default, perception and the like

chx's picture

There's also nothing at all that says you can't file a PR on github with in-progress work.

Sure, but also there's nothing that makes you do that. Your code already is on github and anyone might see it -- if they bother so, which they will not.

Currently the default is to share work in progress in form of patches. That's not the default with PRs, however.

In my suggestion to github, the workflow would be: when you start working on issue, github creates a branch in your fork (creating the fork as well if there is none) AND files a linked PR immediately so that all your changes are visible -- by default.

Default is important.

Cultural

Crell's picture

Nothing makes you post a patch now rather than keeping it local or pushing to a Drupal.org sandbox. The tools are no different here, either way. It's all down to how we encourage people to work. There is no affordance on d.o that "makes" people post work early, and there is no affordance on GitHub that "makes" people post work late. This is a complete red herring.

I've got a couple of branches locally on my computer for core now that I never released, because they ended up not being worth posting. Again, nothing changes.

I agree with Larry.

webchick's picture

I agree with Larry. If we're going to throw "Perfectionist Pat" in as a reason not to move to Github, please note that that blog post was written back when Github was essentially unknown, and was firmly written for a Drupal, patch-using, audience!

However, one thing I think may contribute to more "Perfectionist Pat" syndrome is the fact that there doesn't appear to be a way for a contributor to easily mark their own PR as "I'm just messing about right now, don't bother looking at this quite yet" (aka "needs work"). This does indeed incentivize working in your own little corner because for Github project maintainers, there doesn't seem to be a way to segment "stuff still in progress" vs. "stuff that's ready for you to look at and consider" due to the lack of issue statuses. So once it's proposed as a PR, as opposed to just being a fork, I think there's an underlying assumption there that it's worth bugging someone to look at it.

In Symfony2 we prefix titles

lsmith77's picture

In Symfony2 we prefix titles of PRs with [WIP] (work in progress), [POC] (proof of concept), [WCM] (waiting for code commit. Filtering on the prefixes isn't so nicely possible though.

Pull Requests vs Patches

RobLoach's picture

@Crell

PRs, in their current form, tend to favor a single "champion" for an issue with 0 or more people backing him up via nested PRs. That works well when one person is really driving an issue, which is often but not always.

Two different ways around this:

  1. Nothing is stopping me from forking your PR and submitting a new PR off of my own repository. The issue-to-pull request relationship is one-to-many, since there could be different ways to solve the same problem. Which is the "better" solution is up to the project maintainer and for the two contributors. The issue-to-patch relationship is one-to-one, meaning one problem has one solution. We've had troubles with that past, but that's another argument.
  2. Another thing I could do is fork your fork, and submit a PR to your fork. This allows me to submit improvements to your solution. Whether you bring those changes in is up to you. Otherwise, see #1

Again, it's all about communication, and working together. We run into problems with patches, and we run into problems with pull requests. Different workflows, it all just comes down to working together to make the world a better place.

Allow both

attiks's picture

Why not allow both, if people want to upload a patch, they can do it. If people want to fork and create a PR they can as well.

I think both patches and PR have their merits, so we should aim to make it contributors as easy as possible.

This gets gnarly because then

webchick's picture

This gets gnarly because then patch reviewers then have to learn two ways of getting their patches downloaded and onto their computers for testing. Since patch reviewers are both our most precious and our scarcest resource, I'm not eager to throw barriers in front of them.

Swarming is possible with PRs

dkinzer's picture

Swarming is possible with PRs as long as the issue is associated to a branch any any authenticated user can pull and push to.

timplunkett linked to this

dkinzer's picture

timplunkett linked to this post as the most convincing of the arguments for why the GitHub move/workflow is a bad idea.

I want to point out that in the case were several devs have write access to the same repo then the work flow does not have to be so different than it already is now on Drupal.org.

For example: say you notice something so you create a branch issue-foo-bar-baz.... You commit a couple of things and you are kind of happy with it so you push it up as a new branch.

Next you create a pull request on this branch (The rule is you can't pull in your own PRs... that's just an agreement everyone follows.)

Say you go on vacation along comes another trusted dev... They think you almost have it but not quite so. So they pull down your branch make a change, push that branch back up. Their commits now become part of the pull request along with yours (history is maintained). They have to follow the rules, so they don't merge the PR yet.

Then you or another trusted dev comes back, sees the changes, and merge the pull request to the main branch.

So the type of back and forth that you are accustomed to does not have to disappear as long as the main contributors to a project all have write access to the same repository.

This is a completely flawed

tim.plunkett's picture

This is a completely flawed premise for one major reason:

Say you go on vacation along comes another trusted dev..

There are no trusted devs. I have to go about 100+ names down http://ericduran.github.io/drupalcores to find someone I don't know well enough to give write access to from the start. And new contributors would be completely shut out from that, preventing them from participating in exciting or major issues.

If I hadn't been able to jump into an issue with a small tweak or improvement or addition at any time without someone's permission, I wouldn't be the developer I am today, and I especially wouldn't be involved in core.

Yeah, but new contributors

dkinzer's picture

Yeah, but new contributors are already pretty much shut out from the process anyway (it's a culture thing). Those contributors can just do a regular pull request and it will still be an improvement over the current status.

I thought this whole argument was about getting the core devs happy with a process. If you're really interested in getting those new contributors up and running, then GitHub wins hands down. But that's not what I'm hearing from the anti GitHub move proponents.

Thanks Angie!

neclimdul's picture

Thanks Angie for that. That's some useful information and there's a lot of knowledge in that spreadsheet.

First I hope we're careful to not "cut off the nose to spite the face" as the saying goes. Especially as we're dealing with a lot of frustration going into release and people are just starting to hit their strides understanding git and working within the tools we have. Timing could be pretty important here as swapping that between now and D8 IMHO opinion could be disastrous.

On thing I'm not sure is visible in this breakdown is something I heard in several comments offline was that d.o is actually more powerful then people give it credit for. Also many of the the con's have discussions and proposed solutions dating back to the git migration.

We've got aging tools, but we've got an aging site. There is a lot of site building talent in the community that's should be motivated to help improve our site. So as a first step it seems reasonable to look at things we can do to make our lives better on parts of the infrastructure that aren't going away. We should see what we can do about lowering the barrier to contributing and along the was look at things we can do to empower people to use their own workflows since that's the real power of git.

Along the way, maybe things like hooks for maintaining you project in GitHub or other external git repository. Pull requests from any external repository too. Things like that really give us more power and really are the power of git. In that vein looking at only GitHub is actually near sighted and limiting on our part.

Timing could be pretty

webchick's picture

Timing could be pretty important here as swapping that between now and D8 IMHO opinion could be disastrous.

We haven't yet managed to launch the Drupal.org D7 migration, despite starting it over a year ago, so fret not: I don't think we're in any danger of doing any major undertaking to our dev tools until D8 is out. :D

There is a lot of site building talent in the community that's should be motivated to help improve our site.

Should, but if past experience is any indication, aren't. But your point about making it easier to work on the site should they so choose is definitely heard, and will be a big priority for us in 2013/2014.

Along the way, maybe things like hooks for maintaining you project in GitHub or other external git repository. Pull requests from any external repository too. Things like that really give us more power and really are the power of git. In that vein looking at only GitHub is actually near sighted and limiting on our part.

This is something I've been thinking about a lot, too, the past couple of days. Some kind of "hybrid" interim solution that could allow for people to work wherever they're most comfortable, while we wait and see on Github's issue queue / "swarming" tools and how/if they mature over time. This would also allow us to track how popular Github really is among our contributors, so we were basing this decision on data rather than gut feeling/interpretation of (at times) emotional arguments.

yes, awesome, and yes. Some

neclimdul's picture

yes, awesome, and yes.

Some kind of "hybrid" interim solution...

We should probably work under the assumption that it isn't temporary and build a solution we're happy with and can maintain. Especially, if as you say, project.module isn't going anywhere. Otherwise we're boxing us into the same problem in the future. :)

We haven't yet managed to

jthorson's picture

We haven't yet managed to launch the Drupal.org D7 migration, despite starting it over a year ago ...

Ironically, part of that delay is that fact that D.O D7 development was frozen for multiple months so that people could evaluate other options ... such as 'moving to github'. Now, a large majority of of the arguments for a migration are driven by frustration with outdated tools and workflows on drupal.org, and "look how hard it is to port/maintain drupal.org today" is one of the frequently quoted justifications ... even though i) the delays were just as much political as technical, and ii) as you pointed out, even if we did move to github, Project* isn't going anywhere!

Another aspect of those delays was a timing-induced resource conflict between the D.O upgrade and D8 feature freeze. No, that's not suggesting that all core devs should be voluntold to work on drupal.org ... but there were individuals with a vested interest in both priorities, who may otherwise have made larger contributions to the upgrade project.

Unfortunately, these angles are missed when waving the "Drupal.org is too much work, move to github!" flag. ;)

Ironically, part of that

webchick's picture

Ironically, part of that delay is that fact that D.O D7 development was frozen for multiple months so that people could evaluate other options ... such as 'moving to github'.

AFAIK it was only one month, though possibly two, because the board was told we spent the entire year's D.o budget on not delivering a D7 upgrade and said essentially, "Woah, back the hell up for a minute." As the DA board has fiduciary responsibility over the community's money and how it gets spent, I see this way more as "performing due diligence" than mere "politics."

Another aspect of those delays was a timing-induced resource conflict between the D.O upgrade and D8 feature freeze. No, that's not suggesting that all core devs should be voluntold to work on drupal.org ... but there were individuals with a vested interest in both priorities, who may otherwise have made larger contributions to the upgrade project.

I'm going to disagree here, too. I think what you saw briefly was core developers who were completely burnt out during the D7 release cycle decided to work on D.o tools for awhile because they were so pissed off with the tools limitations that they couldn't stand it anymore. Once they got their itches scratched and some of their mojo back, they returned to what they came here to volunteer for in the first place: working on Drupal core.

If Drupal.org tools only evolve when core developers take time away from working on core, this is not a remotely sustainable situation. We have much to do, both process-wise and governance-wise, to ensure that a motivated volunteer has a much easier time of contributing to Drupal.org, and we are and will be actively working on that. But the DA owes it to the community to ensure that their development platform can evolve apace with the needs of the community without requiring volunteers to "double-dip."

Timing could be pretty

jthorson's picture

Timing could be pretty important here as swapping that between now and D8 IMHO opinion could be disastrous.

Timing is a critical consideration, but even waiting until after the D8 launch doesn't help things in this case. With all the concern that the scope of D8 changes risk alienating a portion of the existing community of users/contributers, this proposal also suggests a full-scale changout of familiar and long-ingrained workflows at the same time. Even if there was a strong focused (people) change management plan in place, things will seem very foreign very fast; and that double-whammy of disruption is going to be very difficult for the 'non core-developer' Drupal community to overcome.

Yep, this is definitely a

webchick's picture

Yep, this is definitely a good point. One of the big advantages of the issue queues is they look/feel a lot like the forums, which look/feel a lot like the download pages, etc. It makes crossing the boundary between "user" and "developer" and "documenter" and so on quite seamless, and culturally has brought us many advantages, like people who are mere "users" actually contributing very salient points on developer discussions. Using Github for developer collaboration would create a very strong barrier there, both physically (in terms of requiring another login, look and feel being totally different, etc.) and also mentally ("oh, that's where the developers hang out. That's not for me.")

@webchick great points

tedbow's picture

@webchick great points here.

I think this is the biggest drawback of moving the Issue queues to Github which seems would most likely to happen if we moved the Git repos.

It makes crossing the boundary between "user" and "developer" and "documenter" and so on quite seamless, and culturally has brought us many advantages, like people who are mere "users" actually contributing very salient points on developer discussions.

Our biggest risk of making this move is that "crossing the boundary" will be become less seamless and fewer people will choose to do so.

If you view Github's about page they clearly state:

GitHub is the best place to share code

and there mission is to

simplify sharing code

This is a great goal and mission. I haven't used Github enough to know whether they are achieving this but from the feedback I hear from people who are using it extensively they are probably succeeding in their mission.

I think the question we have to ask is Github's mission the same as the mission or goals we have for our Issue queues. I would say that one of goals and part of the success of the Drupal project/community is the seamless boundary mentioned above. Our issue queues also serve as an application support system for those who don't consider themselves coders at all and maybe have never touched a line of php, html or css.

Even though as a huge software project we can hope to have some influence on Github, we should also expect them to continue to strive to tailor their site and issue queues to their own site mission may at times be in conflict with ours.

Drupal Trainer and Developer
sixmiletech.com

Not a support system

Crell's picture

Our issue queues also serve as an application support system for those who don't consider themselves coders at all and maybe have never touched a line of php, html or css.

Yes, but that's also rather controversial. I've seen more than one call for the "support request" category to be removed, as it's a distraction. And... it is. Of course, people use it because it's better than the forums, which I've barely touched in 7 years because I find them unusable. (They were weak by forum standards 7 years ago and have not much evolved; and forum software is almost universally bad to begin with.) There's another discussion on here somewhere about dropping those in favor of StackExchange or StackOverflow or something, too, I believe.

I am going to challenge the "d.o issue queues put everyone together!" claim, and the "and that's great!" claim. Both of those have an implicit "GitHub discourages all but the True Coders(tm) and that's bad" claim.

I don't think we can main such claims without hard data, especially on the GitHub side. https://www.healthcare.gov/ is currently hosted on GitHub (https://github.com/CMSgov/healthcare.gov), accepting random updates and edits from the community as Pull Requests. A ton of other Jekyll sites use GitHub/PRs as their CMS.

The "GitHub is just for elite code monkeys" claim doesn't hold water. Meanwhile, we hear complaints about how the d.o issue queues are so bad for "Designers" about once a month (usually accompanied by "because they're built for developers, those mean developers!", which is also BS but I digress). So really, I would hardly say that the d.o issue queues are a paragon of welcoming-ness to all comers either.

So the "GitHub would encourage a white-tower developers-only enclave away from everyone else" claim I don't think has anything to substantiate it.

And even if it did, we should also consider if letting developers focus on code with a better signal to noise ratio and do meta-discussion elsewhere (drupal.org?) wouldn't be a good thing after all. Our general signal to noise ratio has been getting worse over the years.

Khan Academy does something

Gaelan's picture

Khan Academy does something cool (that's probably lots of work) where it has a "Report an Issue" like on their site which then posts a GH issue with the info there. The issue is posted by a "bot" user so reporters don't need GitHub accounts.

@crell, good points. I

tedbow's picture

@crell, good points.

I probably painted too rosy a picture of our current issue queue system. I do think that there needs to be some sort "application support system" even if we don't rely on the same system as we do for bug tracking.

Like I said I have hardly any experience with Github except getting code from it.

https://github.com/CMSgov/healthcare.gov doesn't seem to be very good example though of use of the Github issue queue as it seems there only 9 total issues.

"GitHub is just for elite code monkeys"

"GitHub would encourage a white-tower developers-only enclave away from everyone else"

I don't think this what I said. I was just pointing out the stated goal of GitHub is as a code sharing site. Like it or not our issue queues are now our default application support system.

If we are going to move our issue queues to Github along with the repo hosting it would seem to make sense to discuss how the issue queues on Github would function for what we currently use our issue queues and if we need to change what we use issue queue for.

If we are going to change what we use issue queues for we should probably figure before hand what will replace their current functions.

Related issues.
Redirect Drupal support forums to Stackexchange Drupal Answers

https://groups.drupal.org/node/312853

Drupal Trainer and Developer
sixmiletech.com

It appears that all of the

Gaelan's picture

It appears that all of the people who have reported issues on https://github.com/CMSgov/healthcare.gov are already GitHub users (they all have code associated with their account), so everybody who contributed appears to be a geek (including someone who looks like a Drupaller).

Good summary ... but I think

jthorson's picture

Good summary ... but I think the "kill off several thousand lines of 'contrib but actually custom' code" argument in favor of the move is a bit of misdirection, especially considering the 'Project* is here to stay' statement which follows.

The foundation of the custom code argument is the frequently quoted 'pain of the D7 port' statement; when in fact, that pain was caused by the refactoring of Project* (which we don't get rid of in moving to Github), specifically to remove the several thousand lines of custom magic and instead rebuild it based on fields and contrib (making it much easier to maintain over the long run) ... maybe I'm just naive, but I haven't heard many complaints about the maintainability of our repository-related codebase.

Note: "Project" not "Project*" :)

webchick's picture

Note: "Project" not "Project* " :) If we moved issue queues, release management, repos themselves, etc. wholesale over to Github, we'd just need the download listings, not the whole of Project* and Version Control API* and so on. That's still several thousand lines of "contrib but actually custom" code we'd be able to shift off our plates. I edited my comment to make that more clear.

We've yet to see if the D7 code base modernization of Project* results in more outside contributors. I'd like to hope that it could (that was, after all, the main justification in doing the refactor as part of the upgrade), but since the upgrade was so delayed due to that refactoring, all those people who claimed they were eager to work on these tools "if only they were on D7," all work on D8 now, so I worry that D7 is the new D6 in terms of "ewww, yuck." :\

In-browser editing

jbrauer's picture

I'm not sure in-browser editing is the killer feature it may appear to be at first.

In my experience this is much more a fantastic theory than it is actually the case. Others have already addressed a bit of the difference in flow that gitHub brings but it becomes even more evident with the 'in-browser editing".

A touch of background. I use gitHub enterprise and private repos a lot. And in some cases in-browser editing is the cat's meow. Need to update a password in settings.php ... nothing could be easier than logging in to your repo on the web, make a quick change and presto bamo press the deploy button on your deploy management system and things are great.

With that experience in hand I recently decided to use the in-browser editor to submit a patch to drush. You can see my first attempt here. But before I could get there I had to fork the repository and go edit the file. Making a commit for each file. If there are more than one files to be edited each is a separate commit. Or if your commit isn't quite right it takes another commit to fix it.

In the end this pull-request is abandoned because of what may ultimately have been a search for the single canonical patch to "commit". Of course GitHub doesn't provide this so it's up to the maintainer to take my branch, squash all the commits and make a single commit out of it or ask me to do the same.

After some confusion about whether it's possible to create a branch in the web browser, which it is, I created a new branch and made edits there. Now this was only possible because the file I was trying to edit was checked in. Had it been a pull-request from another user I could not have made a quick edit, fixed a typo, or done any of that. I could have left a line-level comment (which is WAYYY cool in terms of reviews).

Compared to the many times I've downloaded a patch from Drupal.org and made a few edits and posted the patch back to the issue. It's a very different flow and as with many other of the pros here it seems really to be a difference rather than a strong plus.

From my perspective in-browser editing, pull requests etc are fantastic features on sites I've built and do great things for teams. They are very different from the collaboration tools that have built our community. In browser editing is great if you own the repo and a quick change makes sense. If you have to fork a repo and fix a typo in a couple places it quickly becomes non-trivial. While it might not take a real knowledge of git you need to know at least several things about GitHub's system for git including forking, pull request, branching... in and of themselves not trivial...

no more licensing issues ?

mortendk's picture

one thing that would make me happy is just the idea of being able to get around the licensing issues & beeing able to use CC 3.0 graphics in our themes.

This might be a small thing but it drives me nuts that i cant use cc3.0 opensource icon libraries in a theme cause of the goddamn GPL2 on d.o

... on the other hand thats a policy issue - argh my brain is fried after reading all of these comments

/morten.dk king of rock
morten.dk | geek Royale

It's a distribution issue.

redndahead's picture

It's a distribution issue. Since the project would be packaged and distributed on d.o. anyway the licensing issues stay unfortunately.

so hard to stay off my soap box

Yep, this is correct, IMO. It

webchick's picture

Yep, this is correct, IMO. It also doesn't get us away from the project approval process.

so hard to stay off my soap box

I like this idea, but it

Mac_Weber's picture

I like this idea, but it seems to be a complete new proposal.

If Drupal is oriented around

kylebrowning's picture

If Drupal is oriented around not re-inventing the wheel(i.e., adding symphony) then why are we doing just that with git.drupalcode.org....

git.drupalcode.org is not a drupalism

fgm's picture

It appear to actually be basically the old gitweb package ( https://git.wiki.kernel.org/index.php/Gitweb ) with some tweaking.

So true

bayousoft's picture

"Switching to Github thus reduces a pretty major "Drupalism," which reduces the barrier of contribution for a critical mass of developers."

Would this require drupal.org and gitbub.com user accounts

tedbow's picture

It seems like this would require anyone you wanted even to post a comment on issue thread or post support request to make a github.com account (if we move the issue queues also).

Is that true? Would there be ways around this?

Drupal Trainer and Developer
sixmiletech.com

AFAIK, yes that's true, and

webchick's picture

AFAIK, yes that's true, and no there isn't. Unless we were to implement a Github single sign-on solution on Drupal.org, but it'd only work in the Github -> Drupal.org direction.

If it happens we can ask them

Mac_Weber's picture

If it happens we can ask them to get bi-directional oauth integration, or something similar.

micheas's picture

While it seems accurate, it feels a bit skewed towards github and a bit of the grass is always greener on the other side syndrome.

The drupal security team spends a lot of time making sure things like this:

http://vel.joomla.org/articles/844-spotting-spam-code-in-malicious-exten...

don't happen

tl;dr;

Was there a way to prevent this happening?

The answer to this is 'no'. The extensions as originally submitted to the JED were fine, there was no way to pick this up. Even if they had contained the bad code it is unrealistic to expect the JED team (who are a small group of volunteers) to conduct a thorough security scan of every extension in the directory.

Drupal spends a lot of time trying to make this happen.

github is nice for developers that are targeting developers, when you are targeting people that are not developers that cannot be expected to test all the code they download, it falls woefully short.

One note is that the linux kernel does not accept patches generated by github as the tools encourage bad patches.

One other thought I just had

greggles's picture

One other thought I just had - this would mean that places like infrastructure, community working group, content working group, etc. would all move their issues to github. Right? Somehow that feels harder to achieve than moving a contrib.

Possibly, though another

webchick's picture

Possibly, though another option would be to choose a different "project management" solution that's better for non-technical projects. Since Github doesn't support "re-projecting" issues either way, there's not much difference.

Just a reminder that creating

Dustin@PI's picture

Just a reminder that creating a package management system on top of Github is probably not trivial. NPM has a large code base, and the jquery plugin site took a year and half to build on top of github and it doesn't even come close to Project*'s functionality.

I find the idea of integrating with GitLab or one of the other gitb based "project management" systems a pretty appealing idea: in that scenario the new tools could point to existing repositories during transition etc. etc.

Pull Requests Workflow for Drupal.org

RobLoach's picture

As a completely unrelated note, it is possible to use Github Pull Requests on Drupal.org's patch workflow.

  1. Fork drupal/drupal: http://github.com/robloach/drupal
  2. Create a new branch for the Drupal.org issue # in your repository: http://github.com/robloach/drupal/tree/2038135
  3. Commit your changes to that branch: commit
  4. Compare the branch to Drupal's original branch: drupal:8x...2038135
  5. Append .patch to the URL and upload that .patch to the issue: 2038135.patch or 2038135.diff

Github is

rfay's picture

Github is awesome.

Maintaining our own git infra is way too much work and we're making way too little progress. People who say we shouldn't offload this but who haven't been involved in making this work should step back a step.

Yes, GH's issue queue is inadequate, and we'd have to figure something out. The issue queue is also the thing that would be most at risk if we moved to GH and it somehow changed. So there's another good reason to figure out how to control it ourselves.

Ouch

mradcliffe's picture

I appreciate all of awesome documentation that you put into Git, and helping people out on IRC. But I do not think that should stop anyone from expressing their opinion or concerns.

I agree that we need to find an alternative to Github's issue queue, but I am concerned that at this point the solution is to find workarounds for Github's inadequacies.

Drupal is unique

chx's picture

Our issue queue using populace is fairly unique in its size and breadth in experience, in goals and so on. This is not easy to cover. There are saner ways in keeping our issue and yet improving the experience.

Trying to file an upstream

pwolanin's picture

Trying to file an upstream issue against symfony with Daniel. I'm further against github now.

The connection between pull requests and a specific issue is not tight and there is not real coherent view of the history and activity.

Also, my comments on the PR were erased since Daniel force-pushed changes to those lines. As much as I'd sometimes like to erase a review's comments, I don't think that's in line with our community values.

Terms of Service

mgifford's picture

I do love the GitHub interface. I just think they got the UI right.

However, there are problems with moving to an external service, and one of them is the Terms of Service.

Worth looking at the review of GitHub in http://tosdr.org/

Do we just exclude those folks who aren't comfortable signing their ToS?

Important point!

dddave's picture

This needs to be part of the discussion. In case the switch happens we need a fallback strategy BEFORE we move.

greater visibility outside of the Drupal community

blairski's picture

One of the benefits of moving to Github is to increase the visibility of Drupal outside the current community. Most web developers are used to using Github and many leading open source projects are already there. Being where those developers are might just spike their interest in Drupal. And using a tool they already use might mean they are more likely to play around with Drupal and eventually contribute.

Why all or nothing?

Brian Altenhofel's picture

Why should the only two options for the repositories be moving to GitHub or keeping them in D.O? (I'm including changing to self-hosted GitLab and similar ideas hosted on Drupal.org in the latter group.)

Why not allow project maintainers to choose where their repository is hosted and their preferred workflow for their project? For the project page, they'd just need to tell Drupal.org where the repository is hosted (and perhaps even what branch applies to what Drupal major version).

Consistency

rooby's picture

I think it's already annoying enough for users to have to switch systems.

Making it so that users have to be across two systems and switch between them depending on what project they want to contribute to would be more confusing and annoying to users who are trying to help out.

We already switch between systems...

Brian Altenhofel's picture

Making it so that users have to be across two systems and switch between them depending on what project they want to contribute to

We're already switching between GitHub and D.O for contributions and issues on several projects. Some projects just push to D.O as a secondary remote.

I don't think that changes anything

rooby's picture

Just because we're already doing something, doesn't mean it's a good idea.

I actually really dislike going to a module page and seeing that the code is elsewhere, like github.

I can see that it might make sense for non-module things like dreditor and drush, but for modules I find it very annoying.

Even worse are the ones that don't even have releases on the drupal.org project page.
I find it gives a very incohesive experience.

I think it is much easier if everything is in the same place, wherever that place may be.

Backdrop's issue queue

DamienMcKenna's picture

The new D8 fork by quicksketch, Backdrop, uses github. You'll note that they had to set up duplicate projects to manage the project's issues "because of Github restrictions on the use of labels and issue assignment". Food for thought.

I think it would behoove the

kevinquillen's picture

I think it would behoove the overall objective to improve the issue-queue / patch etc experience than to just shift off to GitHub - it will create more problems than it solves.

Death to the NIH Syndrom

eigentor's picture

Free the energy of the poor people working on the project module, and better put that into integrating Github with what should be kept on d.o.

As this has a lot of votes, it might become a reality. Incredible...

Ah, yes: moving a project as big as Drupal mostly to github should allow to negotiate some favorable terms with the github team as to restrictions that would hinder our work. Leverage our project size to our advantage }:-]<

Ah, yeah, and if it turns out Github is just not right: replace Github with [other external service that is better]
To get the discussion going is the important point.

Life is a journey, not a destination

Everything that I read seems

cyberswat's picture

Everything that I read seems to indicate that github could take us 80-90% (rough numbers ... put the pitchforks away) towards a workable solution leaving some serious, but not unsolvable, issues to work on. I keep getting stuck on that thought.

So I called github to see what they thought about this issue and the associated google doc. Here's a rough summary of their thoughts because this was an informal conversation.

  • They feel that their platform is a perfect companion to the work we are doing. (naturally)
  • They feel that the project would benefit the most, in regards to some of the user management issues, from using their public facing site vs some type of an enterprise partnership where we ran versions of their software.
  • They reviewed the spreadsheet and concerns and feel that the majority of the issues are related to workflow and are solvable.
  • They are willing to provide direct contacts between the projects (they can't give us a dedicated team but can connect us to individuals).
  • They are willing to help us develop a migration plan.
  • They are willing to do a webex with us to discuss as a group.
  • They are willing to help us with messaging regarding workflow changes.
  • They are willing to send a group to talk with us in person.
  • They are willing to escalate their support efforts if we migrate to help us as a community.

I think that we could save a lot of infrastructure cost, maintenance, and contributor time with a partnership like this and that it could, in theory, help pave the way for future relationships that might be able to help us. It seems like the next logical phone call would be to the Travis-CI folks. I'd love to tell them "Hey, we have tests that are so good that we can't use your existing system and want to help you create modification X, Y and Z."

I'd like to clarify that this

cyberswat's picture

I'd like to clarify that this conversation happened from the perspective that we maintain our own issue queues, etc. This is purely focusing on what would happen if github hosted the repos and we were to begin tighter integration with their API.

Drupal.org 2014 roadmap brainstorming

Group organizers

Group categories

Difficulty to implement

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: