Blocks & Layouts (SCOTCH) Post-Feature Freeze Overview

sdboyer's picture

The big picture

Drupal 8 had very high goals for Blocks and Layouts. We were planning on user interfaces for landing pages, "node type overrides," and something akin to the Context module, and Spark had laid out plans for a responsive layout builder. It's now clear most of these interfaces will not make it into D8 core - they are each complex interfaces requiring considerable effort, experimentation, and gestation, and as Dries explained in his post-18th blog post, the time for such things is now past. However, this may not be such a bad thing.

While having such user interfaces in core is a good eventual goal, the real challenge in this problem space - the one inhabited by core blocks, Context, Panels & CTools Page Manager, Display Suite, and others - has never really been the UI. Rather, the difficulty has been in the lack of fundamental API concepts for defining what a "page" is. Without such APIs, contrib has been forced to create its own notions that are a) mutually incompatible and b) inherently limited by virtue of being in contrib. A unified underlying API is the only problem that we truly need to solve in core - and we are very close to that right now.

But we are not giving up on user interfaces completely. At the very least, kicking everything to contrib would be unwise because an API without implementations is doomed to inadequacy, at best. Also, it would be a significant regression: whatever its flaws, the existing blocks administration system (at admin/structure/block) serves an important basic function, and we need to replicate it. And so, that is probably our final, major goal for SCOTCH: make a new block administration page that more or less matches existing functionality, but is built atop the new first-class page model. By using these new tools that are designed for extension, the new administrative user interface should cease to be a dead-end that contrib must struggle with or hack around, and will instead be a natural foundation from which a healthy, more interoperable contrib ecosystem can arise.

How?

It's easiest to explain the new system by in the context of changes to the admin/structure/block page. In previous versions of Drupal, blocks were configured per theme; there was one tab atop admin/structure/block for each enabled theme. To support this, themes had to provide a list of regions in their .info file or a hook, and a corresponding page.tpl.php that would display those regions. Those regions would be used everywhere the theme was in use, giving rise to the notion that "Drupal has one layout that bends a lot." (courtesy merlinofchaos)

In Drupal 8, the the page template (page.tpl.php) is changing drastically: its job is being taken over by a layout* - inspired by, but an advancement over, the Panels plugin of the same name. Each layout plugin has its own regions and corresponding Twig template. Core will provide at least a few layouts out of the box; modules or themes will be able to provide more, and themes will (of course) be able to swap in their own Twig templates to replace the core ones if desired. So, no more "bendy" layouts - we can build them for specific purposes, keeping them terse, tight, and beautiful. The layouts system and API have been in core since last September/October, and while it still needs a few tweaks, all we really lack now are layout implementations.

The new block administration user interface will be the one place core allows users to place blocks into these layouts. The basic purpose of the interface will be the same: to allow block placements that apply sitewide. But block placement per theme no longer makes any sense - layouts will provide the region list and page template. Nor does it makes sense to have a tab per layout - the goal of layouts is to provide choice, not to force more configuration. Plus, when would each of the different layouts be used?

Enter displays*. Displays are configuration entities that record the arrangement of blocks within a particular layout. This information is analogous to the block configuration we used to store per-theme - block visibility, ordering within regions, etc. - displays just make it portable. What’s different about them is that they are a complete picture of what is shown on the page. Whereas in Drupal 7 we had mechanisms like hook_page_alter() that any module could use to inject markup onto the page (and consequently, no way of predicting what would be shown on any given page) in Drupal 8, markup only makes it in if the display has a record of it. Making the display a canonical record gives us a predictable page model; a predictable page model enables reliable "metaprogramming" or middleware-like patterns, including (but hardly limited to) things like caching. And displays are what we save to at the replacement for admin/structure/block.

...almost. There are actually a few different flavors of displays, and what the new blocks administration will save to is called a "master display." Master displays are rather like master slides in presentation software: they define shared elements (blocks), then normal displays inherit from the masters, but can make their own local additions and changes. When the master is updated, every normal display inheriting from that master is immediately updated, as well.

Our plan is to ship core with two master displays: "frontend" and "admin", alongside two identically named normal displays that inherit from their respective master. These displays will be bound to “routes”(which are roughly like hook_menu() items of old) such that they coincide exactly with the default and admin themes (which we intend to retain): the admin master display will be used for all admin paths, and the frontend master for all other paths. This simple and intuitive default configuration should be a good starting point for users, helping ease them into a complex system.

Master display inheritance is a major reason that we believe this system will foster a healthy, interoperable contrib ecosystem. It bridges the gap between the two rival philosophies of page building: the site-wide or "global" orientation of Context module and blocks in the past, vs. the more page, entity, or "local" orientation of Panels and, to an extent, Display Suite. Master displays provide the global outlook, but flow seamlessly into the localized control of an individual display. Our hope is that, by providing the base API and an interface for managing master displays, contrib will be able to focus on building somewhat more use-case-targeted interfaces and applications:

  • Landing pages: conceptually, these need not be any more than a custom entity type with a display coupled tightly to it.
  • Customizable dashboards & profiles: a display per user, placed at a special private URL. In most use scenarios, the site builder would probably want to preselect a master display to be used by all dashboards. This system could easily accommodate the dashboard on Drupal.org, for example.
  • Per-node-type visuals: the primary use case of Display Suite and a major use case in Panels, this would entail a display that's bound to a single node type.

There are plenty of other cases. But every one of these systems could save everything they need to as a display, and follow standardized patterns for binding those displays to routes. And all would benefit from having a master display to contain shared elements, like a "site logo" block, the primary links menu, or the toolbar, so that they need not be manually re-added locally on each and every display.

These use cases are complex enough that it merits a contrib cycle for us to experiment on top of the new API, figuring out how to make them play nicely together and how to get the interfaces really, really right for the different use cases.

* Note: we intend to rejigger names: "layout" is planned to become "template", and “display" to become "layout". Probably.

What else?

The preceding ideas represent a consistent and attainable overall goal for visual site-building tools in Drupal 8. It's most of the picture for SCOTCH, but not quite all. This system also needs to be able to accommodate sophisticated caching mechanisms like ESI and hInclude; significantly improving cacheability was a major goal justifying the creation of WSCCI and SCOTCH in the first place.

There's also a fair bit of replumbing that needs to happen for all these pieces to work together: we need to move html preprocess logic elsewhere; we need to change the way CSS and Javascript assets are declared and managed.

Additionally, we still have follow-up issues from the original blocks-as-plugins patch, committed back in early January. Dozens of them, in fact. Some are resolved by the architectural switch to displays, but most still need addressing.

How to help

As we are now in feature freeze, the committers have suggested that we shift work over to a sandbox repository; the goal is to keep core “releaseable at any time,” which means less incremental, more big-chunk work. We’ll be using the SCOTCH repository. We hope this will be a good thing for ongoing SCOTCH work, as many of our patches have suffered from the incremental approach as reviewers were unable to see the benefits of functionality in small patches in isolation from the rest of the (in-progress) systems that complement them.

Here's our current high-level hit-list. Jump in on the following issues, or see the Blocks-Layouts tag for an up-to-date list:

Come visit us in #drupal-scotch, and ping sdboyer or EclipseGc. We're there almost all the time.

Comments

the post needs links

podarok's picture

the post needs links fix
11111"


Andriy Podanenko
web: http://druler.com

yeah, it seems those happened

sdboyer's picture

yeah, it seems those happened in a couple places...sorry. bizarre copy/paste error. i'll fix them up, thanks.

Missing sentence

kim.pepper's picture

This doesn't make sense:

"a predictable page model means introspection, and introspection makes it easier to And displays are what we save to at the replacement for admin/structure/block."

Did some of the sentence get chopped?

Kim

yikes, it did, sorry. this

sdboyer's picture

yikes, it did, sorry. this got rewritten MANY times :)

fixed it.

Wonderful intention - Dangerous Byproduct

sonicthoughts's picture

I applaud the effort to unify and lead in defining what a "page" is and helping to provide leadership and direction to address the fragmentation of display approaches (ie. Context, Panels & CTools Page Manager, Display Suite...) but in practice I think this is actually making matters worse. At this point, a where should a new developer turn for direction. So many approaches have been abandoned in the past. I asked Merlinofchaos for some direction and here is the thread: http://drupal.org/node/1960336#comment-7321404. I almost feel like there is a bureaucracy / bourgeois intellectual discussion trying to make everything "right" but all this well intentioned idealism creates uncertainty, inefficiency and abandonment.
If panels were the future but may take a new form then I'd make it work. If Display suite were the approach then I'd make that work. I've heard many people who tried to develop in Drupal abandon it because of these issues. How should a newbie approach this?

Moving Targets

cpelham's picture

Generally speaking, there are likely to be big changes rolled out in each Drupal release for the foreseeable future. If one needs to build a web site now, use the tools that work best for your project at this moment...with an eye on what the next release is going to look like. In Drupal 7, one has a lot of flexibility to choose among various solutions.

Drupal 8 will provide a lot more ease of use out of the box and more decoupling of content and presentation and that will likely allow for some of the existing contrib tools to become even better even as they will have to change substantially, and will probably inspire some new contrib options as well.

It is often quite difficult and painful to upgrade sites from one major Drupal release to another. It is often easier to export one's data, build a new site architecture and theme, and then import the data (nodes, fields, users, etc.) in and then finish building out the pages etc.

But making new sites is almost always much better in each new version of Drupal.

I think this community has always, and rightfully so, made the choice to make new releases as great as possible regardless of how much this breaks old sites. I think this is necessary to ensure the continued use of Drupal.


Christopher Pelham
Director
CRS (Center for Remembering & Sharing)
http://www.crsny.org

CRS (Center for Remembering & Sharing) is an arts & healing center located just south of Union Square in Manhattan.

Could we have a new big

Jeff Burnz's picture

Could we have a new big picture update by any chance, a lot has changed since April, esp Layout module now removed from core and other big issues pushed to D9.

Cheers, following the issues as much as I can, but hard for me to really get a grasp on where this is going and is likely to get into D8.

Seems (almost?) none of this

Antti J. Salminen's picture

Seems (almost?) none of this got into D8 which is unfortunate.

SCOTCH in Drupal8

cpelham's picture

Which parts made it into core? Which parts did not but should be ready to be usable via contrib? And which parts don't look to be ready any time soon at all?


Christopher Pelham
Director
CRS (Center for Remembering & Sharing)
http://www.crsny.org

CRS (Center for Remembering & Sharing) is an arts & healing center located just south of Union Square in Manhattan.

Update?

groovedork's picture

Is this still the vision? Did this make it in? Sounds pretty good to me.

A lot has changed since this post... but not for the worse!

dsnopek's picture

Unfortunately, a lot has changed since this post... but not necessarily for the worse!

Check out our "State of the Panels Union" presentation from Drupalcon Los Angeles:

https://www.youtube.com/watch?v=wzgyOgvNe84

The "Future" section (starting at 39:45) talks about all the things that got into Drupal 8 - which is many of the low-level APIs necessary to build Panels and similar systems.

Unfortunately, all of the layout stuff that was merged into core was later reverted, so we're still going to have Panels, Display Suite, etc in contrib (for Drupal 8.0.0, at least).

But, we also created a new module called Layout Plugin so that we can all share the underlying concept of what a "layout" is:

https://www.drupal.org/project/layout_plugin

Currently, it's being used by both Panels and Display Suite. The goal is to try and get it merged into core in Drupal 8.1.0.

And we're going to continue to try to get parts of the Panels eco-system merged into core in later 8.X.0 releases.

This issue is a great way to stay on top of the effort:

https://www.drupal.org/node/2169319

It has like 1.5 years worth of SCOTCH meeting notes. :-)

I hope that answers your questions!