SVN Structure

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
cityemp's picture

Hi,

I am setting up a new Drupal site. As far as SVN goes I am looking at the best practices. how existing SVN structures have been modeled. I am thinking of just having the sites folder in the SVN but am open to suggestions. Also how did you create the branches - Split between two thoughts - One to have each release as a branch and the other create a separate branch for module / theme / scripts / files.

I am looking at various options and looking forward for some advice.

Thanks!

Comments

GIT > SVN

afreeman's picture

I don't typically include the contents of the files directory in my repositories. With a few oddball exceptions I've had good results creating branches for releases. I think you'll find that's easier to deal with than having separate branches for each section of your codebase.

Speaking of ease of use if you're starting a new project I highly recommend taking a look at GIT before committing to SVN. I've been using GIT for a little less than a month and I wouldn't willingly switch back to SVN. It's insane how easy branching and merging are in GIT.

I concur with afreeman. I

daveparrish's picture

I concur with afreeman. I use git and highly recommend it. What I'll usually do when I create a Drupal project is check everything in under the httpdocs folder and I'll put my .gitignore file on the same level as the httpdocs folder. The only files I ignore are the httpdocs/sites/default/settings.php file and the httpdocs/sites/default/files directory.

I have seen others only check in their theme and custom module files and then symlink those files into their httpdocs directory.

I create branches for development and production but that is about it usually, unless I have a complicated feature where I need to work in a separate branch for awhile.

That's what we do. We will

jroberts's picture

That's what we do. We will set up a local Drupal install using drush, then in a separate directory we have a git repository that contains any custom written modules and the site's theme.

We then symlink those folders into the local Drupal install.

The benefits:

  • you can update Drupal core and any contrib modules without it affecting your repo
  • your repo is focused on just the custom work you've done on that particular site
  • your repo is smaller as it doesn't contain the Drupal core files, contrib modules, or the sites/default/files directory
  • it's simple to maintain and get running on a new development machine

We switched from svn to git this year and haven't looked back.

Drupal and Versioning on the same server

cityemp's picture

Thanks for the reply. Has anyone installed Drupal on the SVN / GIT server? The reason I am asking is so that every induvidal sees his instance on Drupal at the same time does not mess others instances.

Thanks!

Some thoughts on this

cgmonroe's picture

(Sorry... trouble getting formatting right...)

I think the first thing you need to do is to define exactly what you want to use your code repository for... E.g.,

  1. You want to keep "snap shots" of your active site for quick restore or making debug sites
  2. You need to maintain multiple sites and want to keep modules in sync across them
  3. You do custom coding for sites that is shared across mulitple sites.
  4. You apply patches to contributed modules that you develop or get from issues
  5. You have multiple developers/themers and need to share / track changes / etc
  6. You have "template" sites that need to merge with your core site (e.g. used in a multisite shared code base setup)
  7. You develop code using an IDE of some sort (e.g. EclipsePDT).

If you want to just do #1, then the Backup_Migrate (and it's Files add-on) might be a good fit. You could store the output of this in a repository... but there would be no "delta" versions for the compressed output.

If you're doing #2, (with #1), then you might want to think about creating a full site master SVN repository. I.e., something that has Drupal Core plus all your standard modules but sans the specific site info (e.g. sites/default or sites/www.my.org directories). Need a new standard site, just export the master site. Need to update the site image, check it out, update, commit. Need to update an existing site, export the master, copy the sites directories, backup existing, delete existing, copy/move in new, run update.

Suggestion here... you might want to consider putting site specific modules under the specific site directory (e.g. sites/default/modules ). This keeps the core separate from site.

If you are doing #3, then you will probably want to define a structure for maintaining just the module code. E.g., in our SVN, we have a directory structure of /modules//trunk (and tag and branches). for our modules.

4 has a bunch of special challenges... a simple way to do this is to use the #2 solution and update the master. A more complex solution below...

The other items start raising the complexity as you try to map workflow into code management. In my situation, I had to solve all these for our organisation, here's a synopsis of what I did... (Full Disclosure: I also come for a more formal code development world (Java mostly) where it's easier to keep the parts version controlled, separate, and bring them together in a version control larger product, so this guided some of my "have to haves". It was also partly based on a blog entry I found about this.)

In our situation, we decided to track Drupal Core and all our "stock" contributed modules, 3rd Party libraries, themes, etc separately in SVN. So, we set up a structure like:

"company" (all things we "own")
  libraries (Non-drupal stuff we add to the all/sites/libraries area of sites)
  modules (Modules we develop)
  projects (basically template sites / could be client sites)
  servers (Sites for each "production" server we maintain, e.g. staging, demo, production servers)
vendor  (all things we don't "own")
  drupal-contrib (all the d.o/project modules we use)
    modules
      "module name"  (each core module gets a main directory, e.g. node_gallery)
        branches (only added if we create branches (e.g. patched code)
        tags (tracks revisions, e.g. 6.x-1.1)
        trunk (the current "production approved" version of the module.
    themes
      "theme name"  
        branches (only added if we create branches (e.g. patched code)
        tags (tracks revisions, e.g. 6.x-1.1)
        trunk (the current "production approved" version of the theme).
  drupal-core
    drupal-x
      tags
      trunk
  third-party-libraries
    "library name"  (e.g. tinymce, htmlpurify, and the like)
      tags
      trunk
  third-party-modules (there are a few modules not on d.o that we use)

This was a pain to set up.. so I wrote some scripts to make it easier (drush dl is your friend!).

So, how does this all come together? Via the magic of the SVN externals property. SVN has the ability to link together multiple project under a "master" project. Basically, we maintain a list of our "production approved" modules in a form that can be used as the value of the svn external property. This defines the what parts of SVN to pull in and where based from the "webroot" directory.

Here's a sample of what this looks like:

.htaccess-default https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/.htaccess
CHANGELOG.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/CHANGELO...
COPYRIGHT.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/COPYRIGH...
INSTALL.mysql.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/INSTALL....
INSTALL.pgsql.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/INSTALL....
INSTALL.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/INSTALL.txt
LICENSE.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/LICENSE.txt
MAINTAINERS.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/MAINTAIN...
UPGRADE.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/UPGRADE.txt
cron.php https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/cron.php
includes https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/includes
index.php https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/index.php
install.php https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/install.php
misc https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/misc
modules https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/modules
profiles https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/profiles
robots.txt https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/robots.txt
scripts https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/scripts
sites/_default https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/sites/de...
themes https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/themes
update.php https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/update.php
xmlrpc.php https://svn-server/svn/drupal/vendor/drupal-core/drupal-6/trunk/xmlrpc.php
sites/all/modules/_local/emfield https://svn-server/svn/drupal/vendor/drupal-contrib/modules/emfield/bran...
sites/all/modules/_local/jquery_media https://svn-server/svn/drupal/vendor/drupal-contrib/modules/jquery_media...
sites/all/modules/_local/realname https://svn-server/svn/drupal/vendor/drupal-contrib/modules/realname/bra...
sites/all/modules/_local/secure_site https://svn-server/svn/drupal/dukece/modules/secure_site/trunk
sites/all/modules/_local/webfm https://svn-server/svn/drupal/vendor/drupal-contrib/modules/webfm/branch...
sites/all/modules/_local/views_attach https://svn-server/svn/drupal/vendor/drupal-contrib/modules/views_attach...
sites/all/libraries/dukece-multimedia https://svn-server/svn/drupal/dukece/libraries/dukece-multimedia/trunk
sites/all/modules/acl https://svn-server/svn/drupal/vendor/drupal-contrib/modules/acl/trunk
sites/all/modules/admin https://durham-sqlstage.dukece.com/svn/drupal/vendor/drupal-contrib/modules/admin/trunk

The lines up to xmlrpc.php, define the 6.x core as externals. The lines with sites/all/modules/_local define company specific and patched modules (as indicated by the branch/local SVN location). The last two lines are just the latest standard versions of modules. (use trunk as SVN location). Note that if for some reason you need to "freeze" a site to a specific module version, you could use the tag/"version" SVN location with this scheme.

Here's an example of how we use this with EclipsePDT.

First, create a PhP project with a structure like:

"project"
   webroot (set as a src directory)
      sites
        all  (only empty dirs because the project needs to "own" the root dirs.)
          modules
            _local
          libraries
          themes
        default ( site / project specific items)
           modules
           themes
  resources (non-code stuff that goes into the site, e.g. Photoshop images files, initial database, and the like)
  • Create an SVN project for this (e.g. "company"/projects/my-project/trunk )
  • Import (Eclipse Team/Share/SVN) the full structure into SVN (e.g. from the "my-project" directory).
  • IMPORTANT: Update the project from SVN.
  • Set the svnexternals property on the webroot directory (E.g. Team/Set Properties)
  • Commit the property change
  • Update the project again

After the final update, you will see all the core and sites/all directories become populated with the current versions.

Notes:

When you commit, all changes will go back to their respective SVN locations. E.g. if you change a file in the sites/all/acl module... it will change the contrib project. Be careful that you are committing what you want and not changing default code. (Team Synchronize is your friend). FYI - there are ways of making some SVN projects "R/O".

If someone has updated modules in the SVN to the latest from d.o/projects, just do "Update" your project... and you will get the latest code. (Make sure to run Update.php on your dev site!).

If there is a new core module, check in latest in your project, update, change your svnexternals to include the new modules, commit, update, update.php and you're developing against your production set again.

We use variations on this to manage our production servers, etc.

Some Golden Rules for your Developers to live by:

  • Thou shalt not modify anything in core or all/*... unless it ends up in a local branch and is moved to the _local section.
  • Thou shalt inform everyone that you are creating a new branch.. and why.. and what they should test on their projects.
  • Thou shalt add modules and themes need by your project only to the sites/default area.
  • Thou shalt look closely at any new recommended module updates to verify impact on others (e.g. v2.x to V3.x updates that require major changes)... and talk to others about them.

Unfortunately, I don't thing GIT can support something like this right now... but then I haven't really explored it fully yet.

git versus svn

Screenack's picture

While Cgmonroe gives an impressively comprehensive answer; I'd quickly add that I use git for ALL my drupal projects. I find git quick and "cheaper" to use over svn. (I'm a single person shop) It's much easier and "cheaper" compared to svn (what I was using)

I do not track external files, only those that I modify, such as subtheming, libraries or custom modules.

You can do all that branching business, but since I've left svn, I no longer need it. Of course YMMV.