LSD CSI Technical Architectural Document

The Content Staging Initiative (hereafter referred to as CSI) aims to meet the requirements of the Large Scale Drupal (hereafter referred to as LSD) group in conjunction with the Drupal Community to solve the issues of Workflow, Collection System, and Site Preview for Enterprise Drupal Clients.

This project would build a system for managing the publication life-cycle and staging of content on a Drupal site. This system would equate to a “time machine” for Drupal allowing you to see the state of your site at any given time in the past or future. This would be accomplished via publication “states” that are date-tagged and can be moved through a publication life-cycle or workflow. In essence, a content editor would be able to package a series of content revisions for publication and view that content in the context of the overall site at any given point in the past or future based upon the selected state. For more information see the CSI home page.

This document will lay out the proposed solution along with a detailed technical implementation plan.

Table of Contents

• Solution Overview
• Site Preview System (SPS)
  • Content
    • Nodes
    • Blocks
    • Other Entities (e.g. Drupal Commerce products)
  • Layouts
    • Block Reference
    • Panels
    • Context
    • Notes about reference fields
• Collections
  • Collection Field
  • Collection Preview Condition Plugin
• Workflow
  • Requirement Details
  • Schedule Transitions
  • Workflow Banner/Review Mode
• Workflow Integration with CSI
  • Content Revision Administration
  • Content Revision Actions
• Drupal 8
• Conclusion
• Appendices
  • Definitions
  • System Walk-through
  • SPS Technical Implementation
  • Plugin definitions
  • Plugin Structure
  • Pseudo Code

Solution Overview

For workflow, this solution document will propose a recommended practice along with how any current Drupal workflow modules can integrate with the larger preview system.

The Site Preview System (hereafter referred to as SPS) will layout a methodology to accomplish a full site preview given certain conditions are applied. The primary site conditions that will be referred to in this document are a Collection and a state in time.

The Collection System will create the ability to have groups of content revisions that can be packaged together. Actions can occur on these groups such as workflow transitions or scheduled publishing.

Site Preview System (SPS)

The SPS is the cornerstone to previewing the site as close to actual as possible without exposing the draft content to unsecure visitors. The system will be a framework that can be used to control what version of a piece of content is shown to a person based upon certain conditions. For example, viewing all of the active node revisions for 7/12/2012 at 12:24pm. Access control and content loading all flow through this system so that at any given point you can add, change or deny access to content.

The SPS is a set of conditions and reactions that are active for a given authenticated person. For example, a user could select to look at a site with the "Spring 2012" content collection conditions active. Then the revision of the about us for “Spring 2012" would be show to the reviewer.

Content

The primary concept we are using to build the SPS is based upon revisionable content. Revisionable content means that the same piece of content (a node) can have multiple version of itself at the same time (node revisions) while having only a single version of that content active or live (published node revision). This flexibility allows the system the most flexibility with the least amount of editorial work. The revisioning concept is built into Drupal nodes and is very familiar to most Drupal developers and site builders.

Content will work with SPS on a revision level. The content will be required to manage its own revisions. SPS will provide an interface of integrated content revision with the preview functionality. The selection and loading of the correct revision will be controlled by plugins to SPS.

Nodes

Nodes will work with the SPS with very minimal effort. They already provide a revisioning system along with a UI to manage it. Work to improve the UI along with the revision moderation will be covered in the Workflow section of this document.

Blocks

The block system in core is not and can not be revisionable without contributed modules. The two primary options for handling revisions of blocks are the Bean module and various modules that incorporate nodes as blocks (http://groups.drupal.org/node/93499).

The bean module allows the creation of blocks types and instances of blocks. Each instance of a block type is an entity. Using the entity system extends core Drupal concepts into the block system. Included in this project will be making the Bean module fully revisionable.

Views blocks themselves can not be revisionable in the UI but the parameters passed to the view can be revisionable by using Bean or Panelizer (if you are using panels). For a site currently using views blocks, the configuration of the block will not be versioned but the content that is displayed inside of the block will be versioned..

Other Entities (e.g. Drupal Commerce products)

The Entity API module provides an API for each entity functionality, and is a dependency of many of the more advanced Drupal 7 modules. This module provides the base to several entities in Drupal contributed modules. Due to the wide acceptance of this module and that several of the concepts will be in Drual 8, SPS will support modules that use the Entity API module.

There is currently a patch to provide revision support for any module to implement (http://drupal.org/node/996696). If the patch has not been committed by the time this project starts, it will need to be worked on as part of this project. Then any module which implements this functionality will be supported in SPS.

Layouts

How you choose to control the laying out a page in Drupal is one of the most important technical decisions in a Drupal project. The selection drives the user experience, what modules you will use and how the site will function. The SPS will not enforce any single layout modules. Instead, integration will be provided for the most common use cases with an API available to integrate any custom solutions.

Revisionable layouts are obtainable if layout is tied to a revisionable piece of content. The commonly achieved by using the Drupal node system. Storing the layout information with the content will allow the layout to take full advantage of SPS and workflow via the content revisions. Currently the primary method to attach layouts to revisions is via the node. The actual implementation of the layout is a per project basis but the following are the recommended practices.

Block Reference

Using the Block Reference module provides a field similar to node reference that allows the association between the node revision and a block. A field would be created for each of the regions on the node add/edit form (for example, fields for “Right sidebar", “Ad region") that are filled in as part of the content, and the content author would select a block to reference in each field. If the Bean module was being utilized then the Entity Reference or Relation module could be used to relate the bean entity to the node entity.

Panels

Using Panels to control your layout is supported by SPS as long as the Panelizer module is used. Panelizer associates the panel to a node revision and allows for a per node revision layout. Full revision support is currently in the works and would need to be completed: http://drupal.org/node/515518#comment-5880078. A panel can reference a bean, block, view or Fieldable panel pane. Fieldable panel panes create entities that are used as panel panes. The Panel stores the reference and any parameters passed to the reference.

Context

The Context module currently does not support revisions. The Context Field module provides a way to associate a context with a node. The module will be updated to support revisioning as part of the CSI project, in order to support sites that utilize Context for layouts.

Notes about reference fields

All of the modules relate a node revision to something (bean, block, box, fieldable panel pane, etc). These modules store either just the reference or the reference along with parameters. Only the stored items are revisioned against the node. The item they are referring to is revisioned separately but will still integrate with SPS by the use of collections or time.

Let’s say there is a page node for the about us page. The node has 3 block reference fields referring to different blocks. The system is using the bean module to manage block instances. One of the referenced beans contains a list of the most recent blog posts. The bean itself is storing that the view should show 5 posts. If you want to change the number of posts, you are changing the bean and not the page node. You would create a new revision of the bean which would be unpublished. To preview that bean change in context, you would add a collection to the bean revision then preview the site with the collection Preview Condition set. The page would then represent the new block settings.

Collections

Collections are groups of content revisions. The revisions are grouped together in a logical way to allow for mass actions on the revisions. The collection system shall provide an API and UI to manage the collections. Different content will implement the API to add collection support.

Collections will have the ability to integrate with node revision actions.

Associated Wireframes

• Collection Admin
• Collection Add/Edit

Collection Field

The initial integration with the Collection API will be via a field. The field can then be attached to any entity type to provide the collection functionality to the entity’s revision.

Collection Preview Condition Plugin

A plugin shall be created for the SPS that adds a collection filter to the Site Preview Form. This plugin will also be mapped to the Views Preview Setter plugin to provide integration with the View that creates the revision overrides.

Workflow

The workflow flowcharts are recommend patterns to use. The system will not require any particular workflow methods or particular modules. Instead, a set of workflow guidelines will be prepared. The guidelines will be implemented for Workbench and State Machine as part of this project.

Requirement Details

The following requirements will be necessary for a module to integrate with the SPS. The exact details of how they will integrate (hooks they will call) will be a deliverable for phase 1.

• Integrate with content at a revision level.
• Associate the revision of content with a state.
• Control the current published revision of content.

Schedule Transitions

Workflow modules can independently control the scheduling of their transitions. The collection system will use Entity Revision Scheduling to change its status to published.

Scheduling currently exists for Revisioning, Workbench Moderation, ERS and State Machine. This means that all of those modules will work with the scheduling of transitions.

Workflow Banner/Review Mode

The workflow banner shows up on the page when a user is viewing the page in review mode. Review mode will show a content revision in a real life preview. This lets the reviewer and author see exactly what the article will look like before performing an action on the content revision. On the banner will be information about the revision along with an easy way to transition the revision to a different workflow state. Any workflow module will be able to integrate with the workflow banner as long as they expose their states and transitions to SPS.

To handle the viewing of the node revision, the rest of the SPS will be leveraged. A query string will be passed in the URL indicating the revision to use as the override. The query string will only active if the user has permission to view the content revision. All of the necessary plugins will be developed to create the integration with the SPS. The main advantage is any module that provide integration with SPS will work with the real review mode.

Workflow Integration with CSI

For a workflow module to integrate with the Workflow Banner/Review Mode, it will need to expose the states and transitions used. The states and transitions information will be used by the Workflow Banner/Review Mode to expose them to the user in a unified manner. This will allow for different workflow engines to be used with a more consistent user interface.

The Workflow Banner/Review Mode will expose hooks that workflow modules can use to integrate. Initial integration between State Machine and Workbench will be created.

Content Revision Administration

The Content Revision Administration page will be a page similar to the current node administration page. The difference will be that every revision that fits the filter criteria will be shown. The results will also show the status from the workflow and not the standard core published flag. The page will also provide a way to perform bulk operations on the content revisions.

The initial filters will be Collection and workflow state. These filters will then be used to create a list of content to perform actions on. For example, the user would select “Spring 2012" collection filter then publish all of the content in that collection.

Content Revision Actions

To provide the ability to perform mass actions on content revision, a revision action system will be created. The system will provide an API similar to the current node action system in Drupal core.

Support for Node revision and Entity API based revision will be part of phase 1. The initial supported actions will be workflow transitions.

Drupal 8

To see suggestions on how content preview system can be made much easier to do in Drupal 8, see here.

Conclusion

The Content Staging Initiative has some very high but achievable goals. We are excited to be able to be part of bring this functionality to the Drupal community.

In summary, the following areas are vital important for the success of the project.

Revision support
This is a prerequisite in order to support additional, non-node content within the site preview system. The tasks to accomplish this include:

• Add revision support to the Bean module to support revisionable blocks.
• Add revision support to Entity API module to support revisions in other entity types such as Drupal Commerce products.
• Add full revision support to Page Manager to allow Panels-based layouts to be previewable.
• Add revision support to Context module, to support Context-based block visibility.

Drupal 8 improvements

In order to ensure the viability of the Content Staging solution beyond the lifetime of Drupal 7, we also want to include work on refactoring for Drupal 8, including:

• Make all entities in core revisionable, as well as metadata such as URL aliases and menu links.
• Add basic site preview functionality as an out-of-the-box capability.
• Add underlying APIs to support content staging (e.g. UUIDs) across multiple web servers.
• Make various under-the-hood improvements to the revision system to make it more robust and flexible.

Appendices

---

Definitions

Content

Content refers to anything an editor or non-site administrator would change in the UI. This could be nodes, settings, layouts, etc

Configuration

Configuration is managed in code. The process could be features, install update hooks, etc. Anything not allowed to be changed in the UI by non site administrators.

Content Staging

In the context of the LSD CSI, Content Staging refers to workflow and site preview on a single site. CSI is split into two sections: Workflow and Site Preview.

Workflow

How content is published. This could be an approval process or a single step publishing via a scheduled event.

Site Preview

The ability to view the site with certain Site Conditions applied.

Site Preview System Definitions

Site State

A Site State object defines a target that can be previewed. The Site State is how the system will know when it's in preview mode and what conditions should be applied (e.g. showing content on a future date). All other parts of the system interact with the Site State object. The Site State Object will be the storage container interface for the preview system. The Site State Object is unique per session. A user can only have one active Site State at a time but different users can use different Site States at the same time.

Site State Key

The Site State Key is the unique identifier for the active Site State. The key will be unique across all users. This key is how the SPS will keep track of which user will has an active Site State and which do not. The key should be stored in the user’s session information.

Preview Condition

Preview Conditions define what filters can be used to set the preview mode. Preview Conditions will expose the input form for authors to select the criteria (e.g. a date widget for selecting preview dates), and interact with the Site State Object to store and retrieve the conditions for the preview mode. The Preview Condition will not store the data themselves, only interface with the data.

Revision Overrides

A revision collection is a collection of overrides and extras that are used when a site is loading. Overrides instruct the system to load a different revision of content. Extras are extra content that wasn’t in the original list that should also be used. An example of an extra is a node that isn’t published yet. The original node will not haveThis will be represented by a cache table “revision_override" that will be filled when the Preview Setters are running.

Preview Mode

Preview Mode denotes if the site is currently being viewed with an active Site State. When the site is in Preview Mode, the Preview Reaction objects will control the loading of entities and altering of queries.

Preview Reactions

A preview reaction knows how to interface with the Site State to override data that was set by the Preview Setters. The two primary Preview Reactions will be a query alter and a entity load overrider.

Preview Setter

Preview Setters are fired when the user submits the Preview Selection Form with the Preview Conditions set. A Preview Setter will know how to interface with the user submitted data in the Preview Condition to create a list of overrides. The list will contain an id, revision id and type. For example, the node Preview Setter will return nid, vid and “node".

System Walk-through

The following is a walk-through of what a user does to set the site preview mode and how SPS reacts. You can also refer to the Preview Flowcharts for a high level visual representation.

Step 1: Entering Preview Selection Form

• A content author will ask to view the site preview form. This request could be triggered by a link in the admin toolbar. Behind the scenes:
  1. SPS will look for all of the active Preview Conditions that have been defined (e.g. Date, Collection).
  2. SPS will ask the active Preview Conditions for the form they would like to display to the user; for example, the “Collection" Preview Condition will display a drop-down list or autocomplete text field of collections that are on the site, depending on how it’s configured.
  3. SPS will gather all of the forms and display them in a modal to the content author.

Step 2: Select Preview Criteria

• The author will select the criteria with which to preview the site and submit the form. For example, they select the "Spring 2012" collection and press the "Set Preview Mode" button. Behind the scenes:

Step 3: Preview Setters at work

• SPS will create a list of Revision Overrides for the loaded content, by using the Preview Setters. There are several types of Preview Setters; supported out of the box will be Node (to swap out an individual piece of content based on revision), Views (to handle content listings powered by the Views module), and Query (for other, non-Views powered listings, such as core’s “Recent content" block). The following is an example of how a Preview Setter for Views would react:
  1. SPS instantiates the Site State Object
  2. A unique Site State Key is generated and stored to the session
  3. SPS loads up all of the active Preview Setter Objects via the Site State Object
  4. The Site State Object sends the Preview Conditions to each of the Preview Setters.
  5. The Preview Setters loads a View, passing the filters set by the Preview Conditions. The view is selected based upon it’s reference to the Views Preview Setter. The View Preview Setter knows how to take the information in the Preview Conditions and map it to a filter in the View.
  6. The view runs a query and returns a list of nids and vids for the nodes that are in the Spring 2012 collection.
  7. The Preview Setter sends the results from the View to the Site State Object
  8. The Site State objects takes the results, adds a reference to the Preview Setter Type and the Site State object then saves the information to a database table
  9. The Site State Object then goes to the other Preview Setters to get the rest of the lists of overrides
  10. The Site State Object is stored in a persistent cache.
  11. SPS will reload the page with the active Site State when completed

Step 4: Entering Preview Mode

• The user views the site in Preview Mode with the "Spring 2012" collection active. A toolbar will appear on the screen letting them know what Preview Conditions are active with a button to change the current conditions.
• The site being in Preview Mode will invoke Preview Reactions, based on the affected content in from the Site State Object.

Step 4: Loading a Page

• A node is being Loaded (Also see http://groups.drupal.org/files/entity_load_reaction.pdf)
  1. The Node Preview Reaction will override the loading of a node.
  2. The Node Preview Reaction will check if there was already a specified revision ID to load. If so it will load that and ignore the Site Preview Mode. This is important when dealing with references that are already pointing to a specific revision. For example, if there was a node reference field that was refering to a vid instead of a nid, the correct vid would be loaded.
  3. If no revision ID is specified, the Node Preview Reaction will check the SPS if we are in Site Preview Mode
  4. SPS will check the user’s session for a Site State Key
  5. SPS will load the Site State Object from the persistent cache based upon the session key
  6. SPS will send the Site State Object back to Node Preview Reactions
  7. The Node Preview Reaction will ask the Site State object if there is an override for the node it's loading
  8. The Site State object will query the Revision Override table if there is a revision for the node for the active site state
  9. The Site State object will return the revision ID that should be loaded,
  10. The Node Preview Object will set the revision ID for the revision it was given by the Site State Object
  11. The node system will load the given node ID/revision ID combination

  A query is run (http://groups.drupal.org/files/query_alter_reaction.pdf)

  1. The Node Query Preview Reaction hooks into the Drupal query process by using hook_query_alter().
  2. The Node Query Preview Reaction will check the SPS if we are in Site Preview Mode
  3. SPS will check the user’s session for a Site State Key
  4. SPS will load the Site State Object from the persistent cache based upon the session key
  5. SPS will send the Site State Object back to Node Query Preview Reactions
  6. The Node Query Preview Reaction will alter the query to pull in the correct revision
  7. The Drupal DB system will run the query and return results

Step 5: The Preview

• The page will be shown to the user with the content replaced, and an indication on the parts of the page that are currently respecting the preview mode and which parts are not.

SPS Technical Implementation

Plugin definitions

PreviewSystemManager

The primary factory for the SPS. This is a singleton and will control the loading and caching of the objects

SiteStateController

Provides functionality for the SiteState. This is also how it interacts with other parts of the SPS

SiteState

The Site State data object. The SiteState is composed of a PreviewConditionCollection and a SiteStateController

PreviewCondition

Stores the values for the condition. It's composed of a PreviewConditionController.

PreviewConditionCollection

Provides functionality on a group of Preview Conditions

CollectionPreviewCondition

Integrates with the Revision Collection System

TimePreviewCondition

Provides the Time based preview conditions

PreviewSetter

The main purpose is to build a list of overrides. It also consists of Classes that know how to use the PreviewCondition objects by using PreviewSetterFilters.

ViewPreviewSetter

Provides an interface to use Views to build the list of overrides

PreviewSetterCollection

Manages a collection of PreviewSetter objects

PreviewConditionSetterMapping

This is what a PreviewSetter uses to map the PreviewCondition to different parts of the Setter used to build the filters. Separating this out will allow the mapping to occur independently of the Setter itself.

Plugin Structure

PreviewSystemManager

• getSiteState() - Loads the SiteState object. If the current user has an active SiteState then it's loaded
• getAllConditions() - Loads all of the condition. Returns a PreviewConditionCollection
• getAllSetters() - Loads all of the Preview Setter plugins. Returns a PreviewSetterCollection

SiteStateController

• getPreviewForm(SiteState $sitestate, $form, $form_state) - Builds the FormAPI array for the site preview from. Delegates to PreviewConditionCollection.
• validatePreviewForm(SiteState $sitestate, $form, $form_state) - Validates the Site Preview Form. Delegates to the PreviewConditionCollection
• extractValues(SiteState $sitestate, $values) - Extracts the values for each condition from the FormAPI values array. Delegates to the PreviewConditionCollection. Returns an array('PreviewConditionPlugin' => $values)
• save(SiteState $sitestate) - Saves the instance of the SiteState to the ctools object cache.
• setActive(SiteState $sitestate) - Sets the SiteState as active for the user and unsets any other site states. Also notifies any observers about new active SiteState.
• buildOverrides(SiteState $sitestate) - builds a list of overrides for the SiteState. Delegates to the PreviewSetterCollection

SiteState

• __construct(SiteStateController)
• getPreviewForm($form, $form_state) - Loads the FormAPI array for the site preview form. Delegates to SiteStateController.
• validatePreviewForm($form, $form_state) - Validates the preview form. Delegates to SiteStateController
• save($values) - Saves the SiteState. Delegates to the SiteStateController
• setActive() - Sets this site state as active. Delegates to the SiteStateController

PreviewCondition (Interface)

• __construct(PreviewConditionController)
• getPreviewForm($form, $form_state) - Get the FormAPI preview form for the condition. Return an array
• extractValues($values) - Extracts the values for the conditions from the FormAPI array.
• validatePreviewForm($form, $form_state) - Validates the preview form.

PreviewCondition (Implementations)

• CollectionPreviewCondition
  • getPreviewForm($form, $form_state) - The preview form will be the list of the collections
  • extractValues($values) - Get the collection ID form the select element.
  • validatePreviewForm($form, $form_state) - Validates that the collection is set.
• TimePreviewCondition
  • getPreviewForm($form, $form_state) - Date/Time Element
  • extractValues($values) - The Unix timestamp in UTC from the Date/Time element.
  • validatePreviewForm($form, $form_state) - Validates that a Date/Time is set.

PreviewConditionCollection

• getPreviewForm($form, $form_state) - Gets the FormAPI for the preview form. Delegates to each PreviewCondition in the collection
• validatePreviewForm($form, $form_state) - Validates the preview form. Delegates to each PreviewCondition
• extractValues($values) - Extracts the values for the conditions from the FormAPI array. Delegates to each PreviewCondition

PreviewSetter (Interface)

• buildOverrides(SiteState $sitestate) - builds a list of overrides for the given SiteState. Then the PreviewConditionSetterMapping is used to pull the compatible PreivewConditions for the PreivewSetter. Finally, the PreviewSetter will return an array containing the id, revision id and type.

PreviewSetter (Implementations)

• ViewPreviewSetter
• buildOverrides(SiteState $sitestate) - The view uses the PreviewSetterFilters for setting the filters of a View. Then the View is used to build the list of overrides which is passed back to the SiteStateController

PreviewConditionSetterMapping (Interface)

• This probably won't be an actual class but more a hook with an alter allowing for changes to occur in the mapping

PreviewConditionSetterMapping (Implementation)

• ViewPreviewConditionSetterMapping
• Maps the condition plugins to the filter on the view.

Pseudo Code

Loading the site preview form

Example Code for the Node Setter Controller

Query Alter Setter

This example assumes that the override data is stored in a table called "revision_overrides"

Fields

• id
• entity_type
• entity_id
• entity_vid
• site_state_key
• timestamp

Before Query

After Query