Now that Drupal is making the move to Git, I imagine lots of Drupal shops will be too. Ours is definitely among them.
There appears to be a lot of marks in the 'Win" column when it comes to Git, but as Randy's latest blog post points out: the road can be a dangerous one (http://www.randyfay.com/node/89). This seems to be especially true when it comes to 'casual committers'. These type of people feel much more at home in a UI than on the command line (I skirt somewhere in the middle). Tortoise Git, SmartGit, Eclipse, and others are where 'casual committers' make their changes and interact with their VCS of choice.
So I'm posting this with the hope that those of you who have been enjoying a Git work-flow at your company for some time, can talk a little about what works, and what doesn't. Where are you hosting your code? What web-gui (if any) do you use? What is your process for maintaining a company-wide stable branch? Do you keep the web-server document roots themselves under version control? How has Git improved or changed your workflow (as compared with SVN or CVS)? Do you make backups of your repositories? How/Where?
I look forward to the community's insight and a future of easy branching, tagging, and all the other great things about Git.
Comments
I'm interested in this as
I'm interested in this as well. We're contemplating the following:
Each site has a repo, which contains custom modules and site specific files.
Each site repo has submodules (repo in repo) which includes Drupal core, our set of standard modules, our set of standard base themes (in different subrepos)
All developers work locally (MAMP or similar) with local files and local DB
Work is locally commited and pushed when needed to our repo on GitHub
When needed, developers pull from master (Github) to update
Updates are handled through Features + Ctools export (strongarm etc.). Everything is in code to the maximum extent.
Staging server pulls regularly from GitHub to ensure latest version is there.
Actual pre-release content is added on staging server, and Devel-generated content is added in local dev environments.
Local servers: Used for development. Developers may not see the exact end result due to lack of "real" content
Staging: Solely used to "stage" content. I.e. it should look and act as the production server.
Production is, for us, always different from client to client and we don't always have terminal access so we often FTP files. Ideally we would use Git here too - to pull a safe release from the repo at github when we've tested it on the staging server.
Problems:
Themers have a hard time theming with dummy-content. CSS is fast paced and not as suitable for version control as code is. We're fearing that themers will have to push/pull very often in order to avoid clashes or working on same areas.
I've read a bunch about submodules (repo in repo) but haven't figured out yet if it's a good approach or not
DB changes that aren't exportable will have to be manually applied. If they are important for development this can become a huge problem if all local environments have to manually apply changes in config.
Staging server will have to pull from the repo continuosly in order to keep up.
/Adam Gerthel - Projektledare, Odd Hill
How has GitHub been working
How has GitHub been working for your organization? How many devs and projects do you host. Do you use their issue tracker? Is the limit on # of repos a problem?
local sites can use shared remote dev db
@insats
One way to keep content & configurations on a Drupal site synched is to literally use the same database remotely, even while each developer works on code changes locally between commits.
We do this with multisites, vhosts and symlinks - matching links between each local site, and the shared development site.
multisites
Here's a handbook section on multisites for anyone who doesn't know how to do this: http://drupal.org/documentation/install/advanced-multisite
We use the .local convention for any local site that points to a remote db (to help us keep track which is which). For example, if your dev url is http://dev-url.com, our local equivalent would be http://dev-url.local.
In our local sites directory, we create a settings.php file, and modify that file to connect with our remote shared dev server. No need to install Drupal or use a DB dump.
vhosts
For our local vhosts we use the dev-url.local convention for any local site that points to a remote db (to help us keep track which is which). For example, if your dev url is http://dev-url.com, our local equivalent would be http://dev-url.local.
To do that, each developer creates the hosts & vhosts file entries to run as a local multi-site. For anyone who doesn't know, basic info on how to do this on a mac can be found here: http://foundationphp.com/tutorials/vhosts_leopard.php. On PCs it's the same basic idea, with slightly different details. Instructions for setting up XAMPP on windows here: http://drupal.org/node/161975 (apply the same instructions as in the first link, except for the location of the hosts & httpd-vhost.conf files. You can use this link to also set up XAMPP itself so you can run a webserver on your PC if that's not already set up).
symlinks
Once the multisite is set up both locally & remotely, we add symlinks to keep the menu system from freaking out when we drush cc between local & the dev server. Corresponding symlinks go in the sites directory of each live/shared environment.
On the dev server:
On each local site:
Hmm .. same db. That sounds
Hmm .. same db. That sounds like it would work in most situations. Even if a dev is making schema changes, they are not likely to effect other devs since two devs probably aren't even working on the same module concurrently.
One habit (good?bad?) I have gotten into is truncating watchdog a lot and this would be bad in that situation.
Some cool ideas there. I'll
Some cool ideas there. I'll admit I know nothing about submodules, but it sounds really useful. If the 'standardized' code (Core, etc.) is able to be in the same repo, but pulled from different sources that sounds great. For example, when a new core tag comes out ... it can be pulled in (on the staging sever first, of course) .. and the update can be tested.
One idea we had instead of developers working locally, was having a sandbox server with sites folders for each developer and each works with their own db prefix. All developers would be working from the same core and sites/all. The unified sandbox would be sites/default and stable custom modules/themes would be pushed/pulled from there. This seems to make it easier for devs to collaborate or just get some quick advice (even if there are not in the same building). Also, if the unified sandbox is kept clean with good dummy content ... new development branches can be cloned from that db.
It also seems to allow all the devs to be working in something closer to the 'real' environment. For example, most our sites are varnished, and it is useful to be able to log out and make sure things are being cached appropriately. Other server technologies such as Solr/Sphinx or things like Wim's CDN daemon would be easier to coordinate as well.
Non-code updates will always be tricky, but as you say improvements to exportability are making this less and less required.
@scottrigby Won't you
@scottrigby
Won't you experience problems regarding content files (images etc.) and modules that the DB references but which aren't in your local dev? For example if one developer downloads and activates a module. Another developer sharing the same DB would have a module activated, but would receive errors because ha hasn't got that module code on his local dev. The same would go for cck images, files etc. as well.
How do you handle that?
@te-brian
We've only just started with GitHub so I'm not sure yet. We don't want the hassle of having to run the gitserver and backup ourselves so I think it's worth it so far.
/Adam Gerthel - Projektledare, Odd Hill
rsync
Yeah, there are def other steps to keep in mind for each developer on the team...
We always keep a terminal tab open for both the local server, and the remote dev server. And also keep a browser tab open for both URLs as well.
Yep - that's true for modules for sure. For contrib modules we use the same drush command to dl both locally and remotely (so we know versions are in sync etc). For new custom modules - we always commit once we make important changes that the Drupal DB would care about (like, it needs an info & module file - if there are schema changes that will blow up the site for other devs who don't have the same code etc). And since this is a rapid dev setup we're talking about - we all communicate major changes on IRC as needed ("hey guys, svn up " - or git pull etc).
For cck files, uploads etc - we just use the actual dev url to do those configurations. That way everyone has them - and the .local url will also see those changes.
If we're doing work that doesn't require the shared db, we use a different local multi-site (in the same codebase) instead -- either using a DB dump from dev or just a clean install (so we can isolate an issue in a single module or combo of modules, in a theme, etc), without forcing the other developers to do without those modules, switch themes and all that.
The .local site is just a way to be able to do what you're talking about - allow devs (front end or back end) to make code changes on current content - without having to commit/pull every second to see those changes on a shared dev site :)
I'm currently trying out
I'm currently trying out Features + Git to see how well a small team can work on a Drupal project using local databases using only code to synchronize installations.
I tried the following:
I started an installation of D7, downloaded a couple of contrib modules (Features, views and a few more). I built a dummy feature containing a content type with some fields and a view. I commited the installation to my new fresh remote repo using git. I then cloned the remote repo to another folder, and started that installation with a fresh database. I now had two different drupal installations using different databases but with the same codebase. I then changed the Feature I created on both installations by adding different fields to the content type included in the feature, and commited these to the remote master branch. I solved the conflict that arose. I then did a pull on both local repositories, and their code bases are identical. However, on one of the installations the feature is "overridden" and on the other installation, the state is "default", and I can't revert the "overridden" the feature either. Nothing happens.
If this problem occurs on such a small change then I'm having problems visualizing a code-based development flow. What do you guys think?
/Adam Gerthel - Projektledare, Odd Hill
Try considering feature versioning and cache
I have had better luck when being careful to bump version numbers when updating features and also clearing cache when deploying.
Give it a go.
I'm also curious about what merge conflicts you would have with a simple feature change. You really shouldn't be getting any.
I actually did bump the
I actually did bump the version number on the feature on one of the installations, but not on the other one - to simulate a real world scenario where one developer might consider it being worth a bump. Do you mean that they should be bumped for each change or that they shouldn't be bumped ever?
The merge conflict happened because I had changed the feature on both Drupal installations individually - making different changes to it.
Install 1 -- Created feature "X" ---- commit+push -------- added cck imagefield to "X" ------ commit+push ----------------
Install 2 ------------------------------------------ pull from repo -------------added ckk textfield field to "X" ----commit+push
/Adam Gerthel - Projektledare, Odd Hill
How about drush?
Two things to try then - cache clearing and if that doesn't work have you used the "drush feature-revert" command rather than using the UI?
Also in the real world we
Also in the real world we almost never have developers working on the same exported component configuration - perhaps the same feature but not the same component (e.g. same view or both editing fields).
Even if you are doing pure code (custom modules and update functions etc.) rather than exported config you still don't want developers constantly in the same files - even with git.
I was having some interesting
I was having some interesting struggles with the 'overridden' status when I played around with Features a while back on D6. At that time I didn't really take the time to research what was going on. The concept of features is really enticing though.
overridden status
@te-brian: this doc page had a decent (and quick) description of the common feature states and how to deal with them: http://drupal.org/node/582680.
More in depth explanation in features' API.txt (under "Component states"): http://drupalcode.org/viewvc/drupal/contributions/modules/features/API.t...
I personally find these states to be super-helpful. But yeah, there are some interesting exceptions - depending on a variety of factors...
Tip: Install Diff module so
Tip: Install Diff module so you can see exactly what is overridden with a feature. A common problem is when you have a generic cck field that is shared between two features. If something on this field is changed, then you'll need to recreate all features that use this field, or them will be "overridden" alternately.
I though I'd give an update
I though I'd give an update on our progress.
We're now working with Git locally, using Features to export as much as possible. We have also made our own installation profile that among other things activates our standard settings, through Features modules. We're going to use custom modules that use hook_update_N to control things that Features can't handle - and some things we're just accepting that we won't be able to keep in code such as Ubercart classes and attributes.
We're trying out with one branch for staging, one for production, and local branches that vary from developer to developer. Local branches are merged with the staging branch and the pushed to the staging server through Beanstalk deployment. When OK on staging, it's then deployed to the production server, also through Beanstalk.
/Adam Gerthel - Projektledare, Odd Hill
My turn for an update
After a few weeks using Git, I thought I'd post how we are currently working.
http://nvie.com/posts/a-successful-git-branching-model/
Above is a very visually pleasing article that describes one possible workflow for organizing branches. We're following it exactly (for the most part) and so far it has been successful.
Everyone works locally in topic branches. They may back up said topic branches to the remote (GitHub) for sanity. When the feature is stable it is rebased against the current state of a "develop" branch, and then merged into it. So essentially, "develop" is a shared branch that contains the latest "stable (sorta) but not fully tested" code. "develop" is tested and then a release branch is created. The release branch can then be deployed to a staging server (cloned from the live site) where we can run through all the db updates, test stuff, and hotfix any bugs. Once the release branch is stable it is merged into "master" and a tag is created for the release. ONLY stable tags based on master are deployed to the live site.
The most immediate gain from Git (and this workflow) is that is soooo much easier to juggle multiple features being developed at once. With SVN, often a developer would commit unstable code just for the sake of backing it up. Then we have a code-base that cannot be pushed live until it is made stable. With all development going on independently (in their own branches), features that become stable can gradually be folded into the master code base.
As was hinted at above, we are using a rebase workflow as opposed to a merge workflow. The general rules follow:
Type 1: topic branch
Use a topic branch if your changes will likely span multiple commits, or if they will take more than an hour, or require a review process, etc.
Step 1: you have a task (or “topic”). Start out creating a new local branch to work out the implementation.
git checkout develop
git pull
git checkout -b cr/my_topic (where “cr” is my initials)
Step 2: Code
Make changes in your local branch, committing as you work, with short descriptive log messages.
git add [files to commit...]
git commit -m ‘Move login logic to its own module.’
Changes in develop may affect how you write your branch. Avoid conflicts down the line: to “reroll” your branch on top of the current develop HEAD, use a rebase (make sure you have committed first):
git checkout develop
git pull
git checkout cr/my_topic
git rebase develop
If there are conflicts, fix them and do git rebase --continue
Step 2.5 (optional): Back up your code by pushing it to the server.
NOTE: THIS IS THE ONLY CASE WHERE --force IS APPROPRIATE TO USE! IT WILL DELETE OTHER PEOPLES’ COMMITS IF THEY HAVE BEEN PUSHING TO THE REMOTE BRANCH!
git push --force origin cr/my_topic
Step 3: When you’re done, squash or remove superfluous commits to clean up your history.
git rebase -i develop
Comment out commits that aren’t necessary, and put “squash” in front of commits that don’t need to be a separate commit. :wq to continue and finalize the commit message.
Step 4: Merge your branch back into develop, and push to the server.
git checkout develop
git pull
If you have only a single commit (after step 3) to merge,
git merge cr/my_topic
Or, if your topic branch is complex (2 or more commits)
git merge --no-ff cr/my_topic
git push origin develop
Step 5: Clean up (delete the topic branch)
git branch -d cr/my_topic
git push origin :cr/my_topic (if you did step 2.5)
Type 2: direct commit
If you have simple changes, such as deleting a folder or making a text change, you can commit directly to develop. We use pull --rebase here to avoid causing unnecessary merge commits in develop. Instead of merging, our local commits are merely moved to the tip of origin/develop.
Step 1: Make sure your copy of develop is current.
git checkout develop
git pull
Step 2: Do your changes
Step 3: Make sure you’re up-to-date with origin/develop.
git pull --rebase
Step 4: Push
git push origin develop
BTW, we are using GitHub and could not be more happy, although I am curious about Beanstalk.
We are beginning to explore better automation of deployment, but I'll leave that for another post.
Gitolite
Good to hear how other people are using git.
Once the Drupal on Git (dog) project matures, I think there will be more interest in this space.
We're following a model similar to te-brian, but we have our own install of gitolite instead of using github.
I've proposed a talk on gitolite for Drupalcon London [please support!]
We're not using rebase, and ask developers to push to the origin all the time.
This makes it easier for other people to jump in and help out on an issue, or to review code.
But we're adjusting and tweaking the process all the time,
and it's very interesting to see other people's variations.
We have a relatively small team, 3 full time developers, with as many contractors who help out on various projects.
The thing I like about
The thing I like about rebasing is that it follows the same philosophy as Drupal core development. If you want to add some code to the global space, it is your job to make sure the code cleanly applies to the current 'head' (rebase in git, re-roll patches in issue queue).