I'm working with the drupal redesign administration team and volunteered to help with content migration. Posting this to the group to collect some feedback. Apologies in advance is this has already been considered/discussed.
As content is modifed and added to pages on the staging and local development sites, we'll need a plan for migrating back to live when the redesign is launched.
The easiest thing to do would be to create the pages on live and push them to staging regularly for the redesign theming, but it seems that db dumps are very slow in coming. The last one was 5 months ago. This is due to the size and complexity of the database.
Given that it's likely that at least some content will be created on staging and local development sites that doesn't exist on live it seems like we'll need to consider the following:
- Identifying modified content to be reviewed for migration
- Tracking the status of content
- Migrating the approved content from staging to live in an unpublished state
Here are my initial thoughts:
- Identifying content could be the responsibility of the local developers.
- Tracking could be done through issue requests. (Is there a process for reviewing content? http://groups.drupal.org/node/45844 mentions it, but I'm not sure what's in place already.)
- Migration could be done by assigning the issues to implementers on the doc or redesign teams.
I'd welcome any ideas, thanks.
Comments
Good thinking
This seems like a good wrapup to me.
The one thing that's missing is that configuration needs to be considered as well. Are there new content types? New views? We should get that stuff into code. There might be other non-exportable configuration that needs to be recreated.
No new content types unless
No new content types unless there is a very very good reason.
Hi Ezra, Thanks for thinking
Hi Ezra,
Thanks for thinking about content migration from the server side of things. No doubt you've already seen this document Content Migration Guidelines started by stevenc: http://groups.drupal.org/node/45844
One of the questions was how the content would be pushed.
After the admin team has figured out the best approach, let Steven know and he can update the Content Migration Guidelines document with your input/decisions and that doc remains the primary reference for people helping with the constant migration.
Thanks :)
lisa
==================================
http://about.me/lisarex
Agree with rfay. The
Agree with rfay. The configuration updates should be pushed to the drupal.org database by a robot. The actual configuration of the site is controlled by a bakers dozen of config tables. Many of these tables we cloud DROP/CREATE/INSERT the new configurations.
It gets more complicated though when part of that configuration contains references to specific nodes. Obviously, each dev install has its own sequence of nodes. So we can't push those unique identifiers. There may be few enough that it makes sense to create the nodes themselves as part of the update, and then setting those identifiers.
Unfortunately, we don't even have a latest build for the redesign's configuration, and a scary amount of content is fake place holder content, not real, so we're not even sure of what kind of content and settings we'll be working with in a lot of cases. In my mind we need to take the following steps:
1. Consolidate all settings, content, and configuration that's been set in the 9 staging sites onto one master staging site. This is frankly going to be a messy process, and i'm not sure if anyone actually knows what's important on each site. rfay and I have thunk up a plot to get a good birds eye perspective of differences between all the staging sites.
2. Develop a method for updating a staging sites configuration and content for syncing up with the master staging site.
3. Expand on work we've been doing to streamline the drupal.org database -> dev site database workflow to automatically include the method we build in step 2.
4. Create an easy, standard process that developers use to push these changes to our consolidated site.
Gotta get back to work, I'll make some diagrams later tonight to better illustrate what i'm thinking.
"We are all worms. But I believe that I am a glow-worm." - Winston Churchill
work: http://www.chapterthree.com
blog: http://www.nicklewis.org
a *rough* flowchart
Here's a visualization of the system that exists in my little head. My ponderings inline with the flow. D
Red arrows = Upstream movement: e.g. pushing out new content and configuration to drupal.org or our latest development build.
Green arrows = Downstream movement: e.g. syncing our development build with the latest drupal.org database.
Blue Boxes = A critical process in need of development/improvement
"We are all worms. But I believe that I am a glow-worm." - Winston Churchill
work: http://www.chapterthree.com
blog: http://www.nicklewis.org
Staging site for content?
Hi guys
I'm interested in picking this conversation up again. I was chatting w/ Bojhan and we think there will masses of content to revise (the About section is 3- 4 levels deep and has some pretty outdated info.
I'm concerned about the content from an editorial and accuracy POV. It could take awhile to get it right. :)
We know we need a single staging site to review the new d.o. altogether, but what if there was also a staging site for content?
Ideally we can have the revised content in a publicly-visible staging site so that pages/sections of book content can be reviewed in issue queues. That won't be possible if we've got unpublished pages on d.o.
==================================
http://about.me/lisarex
I know there are a lot of
I know there are a lot of modules that claim to streamline stuff like this. But I haven't used them enough to recommend them.... At the moment I can only imagine two systems for this:
1. Drupal.org direct updates (no ability to stage - easy for docs team to use - drupal.org immediately benefits from updates)
2. Store new/changed content in text files that update.php uses to "push" changes (can be staged, reviewed - but potentially hard on docs team - and someone has to build it).
I lean towards #1. #2 can't be started today.
Neil may have some input on this.
"We are all worms. But I believe that I am a glow-worm." - Winston Churchill
work: http://www.chapterthree.com
blog: http://www.nicklewis.org
Yes, update drupal.org
Yes, update drupal.org directly as much as possible. This gets things completely done.
Some complex pages are implemented as .tpl.php files, like http://drupalcode.org/viewvc/drupal/contributions/modules/drupalorg/drup.... Which gets pages in version control and staged. And allows you to use a real text editor.
Unfortunately this does make content a bit disjointed - simple subpages existing without the new complex landing page, but hopefully we can deal with that on a case-by-case basis.
Updates via update.php are certainly possible, but I would like to avoid that for content if we can.