Butler developments in Copenhagen

Crell's picture

I am substantially behind in getting this posted, but the following is an update and notes from various meetings and discussions about the Butler project at DrupalCon Copenhagen.

In short: A lot of good discussion and some really cool new ideas here.

Drupal: TNG presentation

I gave a session on Tuesday (day 1) entitled Drupal: The Next Generation which was, in essence, a sales pitch and status update for Butler. It broke the general problem down into five categories:

  1. Context: We have no contact system, just a thousand different competing hacks.
  2. Plugins: We have no standard mechanism for swappable parts. Hooks solve a different problem.
  3. Layout sucks: The block system in core is crap. Context.module is fancy block visibility. Panels is the most robust approach in contrib.
  4. Everything is a page: Non-page responses are a hack, still. Even Services has to hack around core.
  5. Performance: We're getting slower every version. :-(

Which had four complementary solutions, along the lines we've been discussing in this group:

  1. Butler context module in Drupal 7 contrib, which we then slide into core for Drupal 8.
  2. Unfied plugin module in Drupal 7 contrib, based on context, inspired by and conceptually borrowing heavily from ctools plugins. We then slide that into core for Drupal 8.
  3. Rebuild core page handling around a panels-like layout-centric pull-based model, with Blocks as simply a "renderable plugin". That unifies a lot of other things, too. (Think: Dashboards, AHAH, etc.)
  4. Replace the current incarnation of hook_menu with something more Services.module-like, which can handle routing to different Response Controllers that can return HTML pages, HTML fragments, JSON objects, do form processing, etc. Different configurations thereof become different layouts, et al. All built on Butler context.

The responses to the presentation were overall quite good. I've uploaded the slides on my blog since they're too big to attach here, although as of yet there is no video online that I know of. :-(


Tuesday night I had dinner with various people from Drupal and Microsoft, particularly Damien Tournoud. We eventually started ignoring the Microsoft folks and talking Butler. One of the points that we've been struggling with is context mocking, where we create a new context object wrapping a previous context object but with some override data and then pass that along to a sub-block or sub-plugin or whatever. We need that butler object to be immutable to prevent lower-level actors from modifying the context of higher-level information. However, we also need to make the process of overriding a context object and setting up the mock to be easy for developers, something that requires the context object being writeable.

Damien suggested having the object be "lockable". That is, when you create a new context object it starts off mutable. You can set whatever properties on it you want and/or set up context handlers. Then you call $context->lock() (or something like that) and the object is locked against future changes; it goes into read-only mode, essentially.

I suspect that is probably the best we'll be able to do. It means we don't need to fold all overrides into a single static array, which greatly simplifies development. (Making life easier for developers is one of the key goals here, da?)

Butler BoF

On Wednesday, we held a two-session BoF to further hammer on the architecture of the Butler object and Plugins. The biggest remaining question is how we associate a given context handler with a context key, so that when code asks for "hey, what's my node" the butler object knows what other routine to pass that request off to, and how do to it in a performant way.

Attending were:

Larry Garfield (Crell)
Bart Feenstra (Xano)
Hugo Wettenberg
Pelle Wessman (voxpelli)
Joakim Stai (ximo)
Alex Barth (alex_b)
Fabian Sorqvist (fabsor)
chx (chx)

(If anyone else was there but left before I wrote down names I apologize; please post a comment.)

One idea that was kicked around was to make context handlers BE plugins. Arguably we don't need to know the context in order to declare a context handler, only to instantiate one, since plugins are declared via a normal info hook. The concern there is that we get very very close to creating a circular dependency. To combat that, it was proposed to have plugin types declare whether or not they are context-sensitive. That is, some will need a context object passed to them and some will not, and context handlers would simply be the non-context type. (Sam Boyer, in separate conversations later, also implied that he didn't like plugins being built atop context.)

I'm honestly still not wild about this approach; there are enough types of plugins that will be context sensitive that I am very worried about the potential for circular dependencies and a complete implosion of the system.

The question of what the syntax should be like for setting up mocked context objects was also discussed. Someone suggested making it chainable:

= $ctx->mock()->change('a', 'b')->lock();

mock() would actually return a factory object, not a context object. You'd then set overrides on that, and then Factory::lock() would return a new, locked context object ready for usage.

Without the intermediary object, but using the ArrayAccess interface of SPL, it would look more like this:

= $context>mock();
$ctx['a'] = 'b';

Internally those could all be wrappers around the last syntax that we discussed:

list($ctx, $t) = $context->mock(array(
'node' => $node,
'language' => 'es',

// After we're done with the mocked context...

No final decision was made.

Alex brought up the question of the current Context.module. Presently, it primarily tracks lots of boolean values. Blocks can then base their visibility on boolean rules of those values. (E.g., show this block if context_a && (contetx_b || context_c)). How would that be handled in the new system? We discussed that a bit and the best we could come up with is that Context module's "contexts" would turn into just context properties that happened to be boolean. Not a great solution, but a possible solution.

Context dependency was another topic that came up. For instance, "current organic group" would, most likely, be based on the current node. However, if you override the current node does that implicitly also override the current organic group? Should it? About the only thing we were sure of here is that the behavior should probably not depend on whether or not that piece of context has already been requested, as that makes the behavior unpredictable.

Hugo Wettenberg went quiet for a few minutes and then showed off some sample code he threw together that, I think, will blow open a lot of possibilities. Specifically, he proposed a colon-delimited format for context keys, e.g. $context['http:get:q']. Then you can "register" a handler for a given layer in the key, or for several layers.

Personally I really dig this idea, and would even take it a step further. Allow for automatic fallback within the string. That is, if you register a handler for "http:get", then "http:get:q" will get passed to the handler for "http:get". That handler then gets "q" as a parameter and returns $_GET['q']. However, you can also add an override (or handler?) at "http:get:q" itself, which in turn would handle the request before the original handler does.

It's essentially the same pattern that the menu system uses now, only without support for infix variables (which is what makes path matching complicated).

I think there's a lot of potential to this approach.

Discuss. :-)


Mobile Web

mikeytown2's picture

Right now I'm thinking about ways to integrate different caches based off the user agent. I have confirmation that these rules work in htaccess, now I need to leverage them in boost.

How is butler going to handle generating different versions of the website based off of the user agent?

Just more context

Crell's picture

The user agent is available as an HTTP header, so it is available as context. The goal for part 4 is that there is some as-yet-architected routine that examines the context object and loads the appropriate Response Controller; that could be "HTML Page configured with these blocks", "HTML page configured with these other blocks", "JSON object based on the context", "SOAP response based on the context", "Engage form processing on POST data and then redirect to this other path", etc. The browser information is just one more piece of information in the context which that routine can leverage for its routing.

How that routing object will work, we don't know yet.

i attended the bof but i have

dasjo's picture

i attended the bof but i have to admit, that i'm not too familiar with butler details yet.

how about providing 3 concrete use-cases in order to visualize how butler would ease the work of module development. let's say a dashboard, a services-request and an organic-groups/multilingual related use-case.

Good idea

Letharion's picture

I find the butler ideas very interesting, but I'm not very familiar with them either. (Though I did read everything in this group).
I would like to read the use-cases, they could help in putting the contexts in context, no pun intended.

easy and hard use cases

Stalski's picture

a more difficult use case: get friends of a user. (I posted the use case here: http://groups.drupal.org/node/67583#comment-287589)

This is a rather drastic and difficult one, since it involves lots of difficulties the butler could/should handle. A bit the same as the thing Crell posted about organic groups: should the context object be immutable or not. As the "current organic group" exists, what would happen if you reset the node. Does the behavior change globally or did the current group change as well?
The same thing matters to "get friends" for a user. User being the "currently logged-in user" from the context? Suddenly i viewing a post of a user, and really nice would be to fetch his/her friends.

In my opinion, we should indeed create a couple of easy and difficult use cases so we can see immediately whether all of those cases can be achieved (through testing or mocking data).

"User" stories

Crell's picture

Yeah, it's still very hand-wavy. Some explicit "user stories" would probably help here to make sure we are covering all bases. I'll try to put one or two together and then we can have other people add more for us to discuss.

Handlers for given layers

dmitrig01's picture

I think that having handlers for given layers is an unnecessary complexity. But I do think having extra optional parameters might be useful.

For example,

->get('http_get', 'q');
// Or
$context->get('arg' /* . 'gggghhhhhhh' */, 1);

Does this make (more) sense?

Subscribing. (Kudos for all

Xano's picture


(Kudos for all the creative ideas that were proposed during the BoF btw)