Move Git repositories to Github

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Posted by klausi on August 24, 2013 at 10:21am
Last updated by tvn on Thu, 2015-11-05 13:31

Know something about Github or Drupal.org? Please help flesh out the Drupal.org vs. Github feature list!

August 27: Here's an attempt at summarizing the high-points of discussion/research so far.

What's your idea?

Github has superior features such as Pull requests, forking projects, inline code editing, inline code commenting etc. that are far easier to understand for new contributors and more efficient for developers working with open source code daily. We should use drupal.org for things that it is good at: project releases, centralised repository of projects, the issue queue etc.

We should not host our own Git repositories and rather integrate our issue queue with Github pull requests and their API.

There is a similar proposal: Implement Pull requests on Drupal.org

What are the benefits?

Less time spent on building/maintaining our own Git infrastructure. Less code to upgrade during major version upgrades of Drupal.org. Lower barrier for people to contribute, higher visibility of Drupal code in the Github social ecosystem, more efficient contribution workflow.

Built-in features:

better git viewer
integrated some dreditor features (comment on line level)
inline editing of files
linking issues
pull requests
private repos possible (we only use this for bluecheese)
extensive rest api (how would we actually use this?)

What are the risks?

Relying on a commercial entity that can change whatever they want without our consent. However, Github has earned the trust of many open source projects already. In the worst case scenario we have to move back our Github repositories back to drupal.org or choose another provider (Gitorious, Bitbucket, ...)
Registering an account on *.drupal.org makes sense when you are a member of the Drupal community. Registering on Github and give away all your data (account data, activity, etc.) to an entity you do not particularly identify with might not.
The Github Pull request workflow may not work so well for us because multiple people working on one pull request have to be granted access to the forked repository. Another solution is to send a PR to a fork.
It is very easy to fork and it's questionable whether people would pull request back or just use their fork. But, people often have "forks" of Drupal modules locally currently. Would forks on Github change this one way or the other (i.e. are pull requests easier than patching?). May be already happening silently in d.o, see comment: https://groups.drupal.org/node/313068#comment-953538 and also: https://groups.drupal.org/node/313068#comment-953898
~~With the forking model it's very confusing to see which one is the 'real deal' and which is a fork.~~ Solution: Keep d.o centralized project listing, as stated in the proposal. Publish in d.o project's page the link to the real repository. It is the model already used today. See comments: https://groups.drupal.org/node/313068#comment-954488 https://groups.drupal.org/node/313068#comment-953468 Counter: Solution does not address repository ownership transfer (via forking), and updating actual user sites to the new fork.
For core development, already on drupal.org we have sandboxes and finding sandboxes is a huge issue. The issue queue - patch based development model makes everyone's work visible but when people fork, it's just not on the radar, you have no idea what's going on.
Impact on long-term project involvement (Historically, we have seen less longevity and participation in any 'off-drupal.org' initiatives than those on drupal.org). It's unclear if Github would follow that same trend.
Impact on code quality/moderation/spam control in the repositories (loss of 'git vetted user' permission check)
Concept that 'all forks are created equal' will cause pain for new users and make it harder to manage projects on drupal.org. (i.e. transfer of repository ownership via a new fork, and challenge in updating drupal.org references/pointers; not to mention pointers for existing user sites which are not redirected).
Adds complexity to packaging, distributions, whitelisted libraries,
download/usage tracking, automated testing, update manager, d.o project browser, project management, security advisories, etc.
Significant development efforts required to break drupal.org repository ties and refactor to leverage github APIs, and potential burnout as this would be the same resources who have been involved in the D7 drupal.org migration.
Github is not as accessible as drupal.org and we can't do anything to fix it
Additional risks related to moving issue queues to github (which some have argued for in the comments -- a summary is in this comment: https://groups.drupal.org/node/313068#comment-954428):
- Impact on ability to host private security issues (security.drupal.org uses the project issue module now - we would need a similar system or to reinvent or flow for managing those issues)
- Inability to differentiate between bug/support/feature request issues (we can use tags, but that feels like a hack and might not be done consistently between queues)
- Inability to easily move issues between queues
- Impact on ability to moderate issues (e.g. blocking a troll account)
- Inability to upload files other than images to issues (don't make me query for how often this happens...) would result in hosting elsewhere and then linkrot when elsewhere goes offline.
- Forcing users who just want to report and get updates on issues onto a separate, unfamiliar site (where they will presumably have to log-in again).

How can we measure the impact of this idea? (metrics)

By the average time it takes to write a patch and get it commited. By the number of people contributing to Drupal core and Drupal projects. By the amount of time that would be saved maintaining our own Git infrastructure. By the amount of time that would be saved during major version upgrades of Drupal.org.

Who directly benefits from / will use this improvement? (target audiences)

Developers.

Are additional resources available for discovery/implementation? (volunteer effort, financial backing, etc.)

In Gaelan's experience, GH has great support. Gaelan is trying to get them to weight in on this discussion.

Related Wiki Pages

Use gitlab for all projects

Comments

indeed

Posted by chx on October 28, 2015 at 10:57am

You know, this is a big problem. Yes, it would make creating such a patch easier, indeed. But as with all software, creating it is one thing and maintaining it is another. Our problem is: what if someone else wants to roll the next patch for the same issue?

How to use Github

Posted by johnslegers on October 28, 2015 at 12:10pm

A typical way to deal with issues on Github :

User X finds a bug.
User X posts a description of his error on the issue tracker
If the issue is indeed a bug that needs to be fixed, core developer Y is assigned to fix this issue
Developer Y updates his own repository and fixes the bug.
Developer Y merges his changes with the main repository.

Another typical way to deal with issues on Github :

User X finds a bug.
User X does some debugging and finds the bug in the source code
User X updates creates his own repository and fixes the bug.
User X sends a pull request to have his his changes merged with the main repository.
Developer Y approves the changes and merges them with the main repository.

At any time, the repositories of user X or developer Y can be cloned by a third user who wants to continue their work. Multiple users can collaborate on a single branch of a project without interfering with the main branch. Milestones and labels can be used to organize issues. etc.

Git is extremely flexible, and you can set it up in many different ways. However people have very strong opinions on what are good conventions to adhere to in using git. However it is setup requires some coördination, but certain ways can require more or less. The general term to search for is "git workflow" to find discussions about this.

Note that every git checkout is a repository, so in some trivial sense every developer must indeed have their own repository. It is quite possible to only have one "shared" repository on github, either with multiple keys, or one key. This requires more coördination up front -- you have to agree on where the development is headed. With multiple "public" repositories, you can delay this. People can look at each other's work and discuss whether it should get merged into a blessed repository.

(You can also do this with multiple branches on one repository, and you can also do discussions via sending around patch sets, it's just matter of what's easier.)

source

There are two main workflows. Both assume each developer has set up a github account which is free and sensible:

Add other developers as collaborators to your main repository so they can push their changes to it.

Let other developers fork your repository and issue pull requests to integrate changes

Collaborators are preferable if a group of you are making or maintaining a software product. There is one repository for the project on github and each developer will have a local repository for their development work.

The forking is useful if there are other teams or developers using their own version of your product. It gives them a way (pull requests) to offer their changes back to enhance your product. Each fork is generally considered as a distinct viable variant of your project.

Forking is also be useful in an open source environment while you are 'coming to trust' a potential collaborator. If you find yourself routinely accepting their pull requests, you may 'promote' them to being a collaborator.

Branches are important and very useful. Its hard to do without them on the remote repo if more than one developer needs to work together on an experimental feature or significant extension to your project.

source

Git was designed to be used with a repository for each developer. Make an account for each person, then designate one as the maintainer of the master branch. Everyone else will fork the master, and they can work on whatever they want on their own. Once they finish something they will send you a pull request and you can pull their changes into the master branch. Then everyone else can pull from the master as often as they like (once per day, twice per day, etc).
Managing multiple branches may sound difficult, but as long as you communicate effectively it shouldn't be an issue. Once a developer finishes a feature it's important that they send you a pull request and they don't just sit on the commits and nobody knows about them.

A possible good policy for developers to follow before sending a merge request is to have them pull from master and ensure that there are no conflicts.

If you really want to use one account, you don't have to share the same key. Github allows you to upload as many keys as you want. However, if you want something that works like SVN you should use SVN because Git isn't designed to follow the same workflow as SVN.

source

+1 for github

Posted by orlissenberg on September 26, 2014 at 5:14am

lock it.

@orlissenberg - please read the whole thread

Posted by pwolanin on September 26, 2014 at 12:01am

@orlissenberg - please read the entire thread of comments if you want to meaningfully add to the discussion. This is not a simple issue, and it doesn't seem like you've read e.g. the one by jthorson from May: https://groups.drupal.org/node/313068#comment-1037488 and webchick from August of 2013: https://groups.drupal.org/node/313068#comment-955593

we should probably lock the thread at this point?

Time for Github to move on

Posted by johnslegers on October 28, 2015 at 12:46pm

Almost every open source project is hosted on Github these days. I find it odd that the Drupal core team refuses to change its ways and prefers to stick to their old ways instead of joining everyone else at Github.

Sure, Github may require some workflows to change here and there, but Github is what most open source devs are accustomed to anyway. Thus, the move would make it significantly easier for newcomers to contribute to Drupal.

Best of both worlds

Posted by johnslegers on October 28, 2015 at 1:08pm

I've been thinking about how to get the best of both Github and Drupal.org.

The approach that would make most sense to me, is this :

People host their project on Github.
People register their Github repository on Drupal.org as a "theme", a "module", etc.
The Drupal community leaders can add flags to these projects to make them "official", "recommended", "deprecated" or whatever.
Drupal.org polls for changes to these projects, triggering new review workflows when changes are detected.

That way, you truly get the best of both worlds. You get :

Drupal.org as a centralized, curated, end-user-oriented system that allows you to easily search for modules or themes, and
Github as a decentralized, developer-oriented system for easily maintaining and improving own open source projects.

jQuery plugins are managed this way.

Gitlab

Posted by attiks on October 28, 2015 at 12:51pm

The best of both worlds can easily accomplished using gitlab, and allows Drupal_infra full control over the service.

Please look at the title of

Posted by pwolanin on October 28, 2015 at 2:28pm

Please look at the title of this group. 2014. Please read the specific comments I linked before:

the one by jthorson from May: https://groups.drupal.org/node/313068#comment-1037488 and webchick from August of 2013: https://groups.drupal.org/node/313068#comment-955593