Creating a Drupal development environment - achieving development, stage and production

Events happening in the community are now at Drupal community events on www.drupal.org.
jerodfritz's picture

What is the best you've found to setup a Drupal development environment when working with a team to achieve typical dev, stage and prod environments? These are my experiences:

How I've done it in the past:

  • Create an SVN repository and do an initial import of the drupal core and add all of Drupal core files leaving settings.php and the root .htaccess file unversioned bc they vary across dev, stage and prod.
  • Checkout three working copies of the repository for each environment
  • Create a new database user/account for each environment
  • Modify .htaccess, apache directives, and settings.php to reference the different environments
  • Install Drupal on one of the environments and perform db export/import into the other three environments db's
  • I now have three Drupal environments that use a single repository for the file system

Migrating changes from one environment to another:

  • Run a mysqldump to save the current environment's db to /database/databse_$environment.sql
  • Run a mysqldump to save the destination environment's db to /database/databse_$environment.sql
  • Add both files to the repository and commit them for safe keeping.
  • Do a diff of the two .sql files and then resolve the conflicts into what should represent the new fully migrated database *tedious step
  • Drop the destination environments db tables entirely to ready it for import of the migrated .sql file.
  • Import the migrated .sql file to the destination environment's db
  • NOTE: I typically create a script for each of the environments to automate the dump and import and drop steps for that environment's db so I don't have to use phpMyAdmin or some other interface

Problems and Limitations:

  • This is not single click migration. Even after all of the initial environment setup, I've found that the migration is best handled by the person who configured the environments
  • Resolving the merging conflicts in the .sql files requires a lot of manual inspection and requires you to make decisions based on intimate knowledge of the previous db states
  • Each of the environments is tied to an svn account so this process requires a single account holder to be responsible for
  • I notice that permissions on folders like /sited/default/files get a screwed up sometimes using svn so environment directory writeability gets corrupted on svn updates sometimes

Moving Forward:

  • GIT over SVN. My most recent project uses GIT and I'll never go back
    • GIT doesn't tie a single user to a working copy. Any user can migrate their file system changes to another environment. SVN required this to be done only be the user who performed the initial checkout
    • You can start a working copy from a git repo from a compressed file rather than needed direct access to the repo server. (This has been useful when working with an outside contractor who's changes you would like to keep versioned with little effort)
    • No permissions issues experienced like svn as of yet
    • Users can make local commits to their environment without committing to the repository.
  • Looking for a better solution than running mysql commands for migrating the databases. DBscripts (http://drupal.org/project/DBscripts) looks promising. Perhaps Drush's (drupal.org/project/drush) sync command or Ægir (http://groups.drupal.org/aegir-hosting-system) may have some solutions as well.

Thanks,
Jerod Fritz - jerodfritz@gmail.com

Comments

Environments

jerodfritz's picture

New module called Environments has been released earlier this week http://drupal.org/project/envts . Anyone tried this out yet?

excerpt:

This module was created to fill the need for *two-dimensional* environment/host management.
* What's an environment? An environment is a development or server location where a full Drupal build (possibly including multiple hosts) is set up. In our work, we tend to have
1) a primary 'dev' environment (site.dev.ourwork.com),
2) local environments for each developer (site.dev.local),
3) a staging environment (staging.site.com),

I looked into the project

jerodfritz's picture

I looked into the project and I like the idea, but there is no way to specify different settings.php or .htaccess file settings which imo is the only way to facilitate true isolated environments.

Thanks jerodfritz for the

thebuckst0p's picture

Thanks jerodfritz for the comments, I replied to your similar thread here: http://drupal.org/node/529942#comment-1848584

gitosis and capistrano

jerodfritz's picture

I stumbled on a pretty nice walkthrough for setting up stage and prod environments using gitosis and capistrano. There is no mention of database isolation but a little tweaking to the capistrano deployment could fill in the gap.

A nice feature is a distinct branch for drupal core

Check it out at http://www.forwardthinkingdesign.com/blog/manage-and-deploy-drupal-code-securely-git-gitosis-and-capistrano

Deploy Module

jerodfritz's picture

The deployment framework is a series of modules which are designed to allow developers to easily stage Drupal data from one site to another. http://drupal.org/project/deploy . I would be cautious marrying yourself to this process though because I wouldn't put much stock in this being ported to D7 ever.

D7 version of Deploy

gdd's picture

I fully plan to port Deploy to D7, as stated explicitly on my project page, although I did not sign up for D7CX because I'm unsure of how extensive the rewrite will have to be and I'm focused on getting Services ported first. As in all such situations, this could move forward more quickly dependent on community interest.

correction

jerodfritz's picture

My oversight and apologies, thanks for the clarification. I'm a ridiculous fan of Services and Deploy and I thank you for all of your contributions!

Another practice I have

cmcintosh's picture

Another practice I have gotten into habit is to setup three stage setup:

  1. Development Server

- This is where all of my brainstorming / fooling around with new modules and scenarios takes place. I am not concerned here with content ever being synced with the Production Server.

  1. Staging Server

- This server has MYSQL synced with the production server. All of its primary key auto increment fields for all tables are modified to use +2 increments. For my purposes I set the staging server content to be on EVEN only numbers. This is also where I build the custom module that would be used to port over final changes into production. All settings, custom blocks, etc are placed in the module and hook_update_NN is used to update anything that is needed as far as database changes goes. Once things are ready. I do a final import of Users, Content, Permissions into the staging server. Once the content is brought in and passes SimpleTests / QA I place the both the Staging Server and Production server into Maintenance Mode, update url alias, then bring the sites back live. In case there is a need to quickly revert back I can still do so.

  1. Production

- the live server is never directly developed on with new modules or functionality without going through the vetting process.

All of the code for the modules is kept in SVN system, and runs thorugh DreamWeaver for check-in/out of files. Before committing if i am working with multiple developers on single project i just follow standard merge procedures before I commit any changes. So Far every thing has worked out great when it comes to this.

How-to's?

gtkakega's picture

Glad I discovered this thread. I'm also looking for a 2 stage environment solution:

  • running Dell Poweredge 2650 w/ 18 GB (BOOT) RAID 1 and 18 GB (DATA) RAID 1 configuration
  • installed WAMP framework on BOOT partition using Windows Server 2003
  • using HTDOCS (Drupal 6.x instance) and MYSQL/DATA on DATA partition
  • configured virtual hosts for development (/htdocs) and production sites (/htdocs2)

I was originally going to do version control manually, but I have second thoughts. I'm not familiar with SVN. Can someone write up step-by-step how-to instructions on how to implement 2 stage development and production solution using SVN?

another way to do dev/prod

toddgee's picture

I do a staging/prod environment w/in my managed host.

See toddgee.com SLASH node SLASH 58
(sorry for the obtuse way of posting this; but the spam filter seems to hate me.)

This is based on my drupal scripts which you can see on the above server (toddgee dot com) slash drupalScripts

I'll be doing a presentation at the April CDMUG meeting (on the 29th) on why I think my Drupal Script Library is better than drush (O! the hubris)

My little rules:

Bußmeyer's picture

My little rules: Configuration goes up, database and files goes down between dev, stage and live. We never migrate database dumps. We try to separate content and configuration.

Use the features (http://www.drupal.org/project/features) and strongarm (http://www.drupal.org/project/strongarm) modules to do this. Keep all config in cvs/svn/git and do the deployment with a e.g. 'svn up'.

Read this: http://www.lullabot.com/articles/site-development-workflow-keep-it-code

Chicago

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: