Posted by morisy on January 27, 2010 at 4:27pm
It's gotten to the point I've realized I should stop mucking around with features on our live site and create a stage or development site.
This seems tricky with a newspaper, though, because if you work on the stage for a week and then just move it live, you're wiping out a week's worth of uploaded content.
What are you using for moving features, etc from stage to production?

Comments
Could you expand on "move it to live" please?
Hi morisy,
I'm curious how you've been migrating changes from your development site to your live site. My guess from the symptom you're trying to avoid is that
Is this accurate, or do you use a different process?
I would suggest that, while you're making changes on the dev site, keep track of the actions you're taking, modules you're adding, configuration settings you change, etc. When you get to the final setup for that particular change, document the final configuration. Then, manually make those changes on the production site. This method takes extra time as you'll be duplicating some work, but has the benefit of not needing to monkey around with databases (and messing things up on production if your dev SQL export includes changes to tables production relies on before you've upgraded or enabled particular modules). Further, you'll not be losing any content from prod.
There's other things you could do, including only exporting tables with changes, or setting up the dev server to share the content tables in the production site, but these both run the risk that, when you do SQL export/imports, you may overwrite the wrong data or miss exporting something and end up breaking your prod. site.
The safest way I've seen it done is to do the development on dev., write out a set of upgrade instructions for yourself, and then execute the upgrade instructions on prod. (and skip the SQL exporting altogether unless explicitly needed).
Thanks! Right now, I don't
Thanks!
Right now, I don't have a production site, only the live one. We just try and keep the "under construction" bits out of public view, but I think what you're saying jibes with a lot of the concerns I've seen elsewhere regarding sharing or porting SQL tables back and forth.
Thanks!
Web guy, SpareChangeNews.net
Twitter: @morisy / @sparechangenews
Sorry about the jargon, I
Sorry about the jargon, I tend to interchange "production" and "live". But yeah, if you're only running a live instance I'd suggest setting up either a separate codebase-and-database (a completely autonomous drupal installation for testing) or a multisite setup that uses the same codebase but a separate database (again, not sharing any tables or w/e). If you've got a hosting solution that's not stingy on disk space, I'd go with the two installations setup.
And yeah, transferring changes between databases is a beast. You should be able to make all the changes needed through theme files (which shouldn't be dangerous to the site to transfer around) or through core & contributed module settings. If you do run into an instance where you just can't get something done with configuration or the theme, it's better to build a custom module to make the needed changes than directly modifying the database. Easier to turn the changes on/off, and you can actually find them again if you need to modify further (cause they're right there in your custom_settings.module file).
Honestly, we gave up on
Honestly, we gave up on trying to track changes to the database. We push as much as we can into the modules and then make any changes that hit the DB manually on a QA site test them, then make the same changes again on production.
Tracking Drupal db changes keeps psychologists in business
Tracking changes to the db is really a pita, especially if you've got lots of customization or lots of modules.
I'm still a bit foggy as to why you'd be losing site content. Can you expand on how you "make the same changes"?
Sure, there are two basic
Sure, there are two basic ways to do this, but both require (at least) two servers -- production server and a similar test server.
If you can survive with some downtime on the production site, the safest way to make the changes is to put the production server in maintenance mode (so nothing changes on it), copy its database to the test server, make you changes on test server, and then copy the database back to production.
Alternatively, you can copy the database from production to test, make the changes on test and see that they work, and then make them again on production. The advantage is less down time, but more work.
You should be using SVN or another version control system for the source code, and I would also strongly recommend taking the time to write some scripts that automatically backup/restore the database and can automatically copy it between production and test with a single command or click.
Sorry about the reply above,
Sorry about the reply above, I had though you were a part of morisey's dev team. I agree with the need for developing on as recent a copy of the prod. site as possible. Downtime is anathema to any news site, and I've always been wary of importing database dumps to the production server.
And yeah, major support for backups. If the site's worth anything at all, it needs to be regularly backed up.
Backup and Migrate!
Have you considered the Backup and Migrate module?
http://drupal.org/project/backup_migrate
You might also want to add SVN (version control) to your work flow to have an added safety net in place for when your moving your changes around.
PS - No need for two servers...
Just for the record, there's no need for two physical servers but rather two installs of Drupal with their own databases running in a single mySQL install. The development site would run off of subdomain like dev.yoursite.com with updates being sent to the production site when ready for release. The golden rule is that development should never be done on the production site unless you have nerves of steel and few years to loose off your life when everything goes "boom".
Thanks for all the responses!
Thanks for all the responses! Sorry, Gmail was sending all my groups.drupal alerts to spam.
It's all really helpful. I've played with Backup and Migrate, but haven't actually used it for the "migrate" functionality, just daily backups of the 4 sites I'm running.
Web guy, SpareChangeNews.net
Twitter: @morisy / @sparechangenews
Database Sync Module
I've been thinking about a way to resolve this problem. Although this thread appears to be before the time of Features Module, which might solve some/most of these database update issues, I think a module that exists simply to "sync" database from staging/development to live might be useful. The question is, would it work? Is it possible to effectively track all database changes made on a development version of a Drupal site, and then sync those changes into the live Drupal site?
I think there are database hooks we could use to track every database operation, which could then be stored, prepared for syncing. The problem is that a collision will occur on some tables, such a node table, where the same nid may be in use on both the staging and live site and each refer to different content. I have an idea for how to resolve that. It involves assigning a globally unique Sync ID to each database query stored by the sync module. When the sync is run, it would check for conflicting primary keys, and it would then resolve that conflict by using the next available primary key. It would also need to roll that change out to any subsequent sync operations that involve that altered row.
My view of this issue is to basically look at database rows as roughly equivalent to files in a version control system. With this approach we need to insert/update/delete rows in the live database based on the change history in the staging/development database since the last update. I'm only thinking about upstream syncing at this point... I figure if we can solve how to sync the staging server with the live server, after the sync you could use Backup/Migrate module to copy back the updated database into the staging server, at which point a new cycle of development would start with new tracking in the sync module.
Exporting Content
If it's actual content you're losing in the form of nodes, the Node Export module might help.
It uses unique IDs via the UUID module to distinguish pieces of content and allow them to be migrated across sites.
You can export in bulk via the Find Content page, and I've also used this module to export specific content for backup and restore during tests or updates.
A word of caution: I have experienced errors when enabling export and import of images, which may result from images not available on the target/import site.