We had a call today regarding the Services module in Drupal 7. Greg Dunlap (heyrocker), the maintainer, wants to use the context concept for Services as part of the D7 port in order to simplify a lot of code, which makes Services a great guinea pig to start implementing stuff and see what happens. These are the nodts from the call.
On the call:
- Larry Garfield
- Greg Dunlap
- Sam Boyer
- Jeff Micollis
Background
Services is about 2/3 of the way through a major upgrade to 6.x-3.x, with the plan to port just the 3.x version to Drupal 7 shortly afterward. Given the nature of the system it does a lot of work with the super-globals ($_GET, $_POST, etc.), which of course makes the code incredibly hard to unit test in any meaningful way. It also makes Services integration with Drush essentially impossible. Greg wants to fix that.
Game plan
We chewed over exactly how far to go at this point, and came to the following conclusions:
- There will be a Request class, which is the "level 0" context; that is, it wraps the super globals, HTTP headers, etc.
- The RequestInterface will be modeled on an HTTP environment, but because it's an object with separate interface it won't be bound to that. Drush, for instance, can provide its own implementation of the same interface that "fakes" an HTTP environment.
- The Context object will compose a Request object.
- Services probably doesn't need really robust mocking yet, so for now we'll skip it.
- Services only needs basic "extended context" support, so for now it will implement a simple version of it. If it turns out we need to throw that approach out and try again with something more robust later, Services will adapt as needed. It is, fortunately, a small-enough ecosystem that the hassle from doing so will be reasonably contained.
- There is a good case to be made for services having its own user object context separate from the normal Drupal one. Greg and company will experiment with that idea.
Currently Services works by taking over one menu callback and doing its own thing from there, which actually works out well for our purposes. The over-simplified pseudo-code for what we discussed would be:
<?php
interface RequestInterface {}
interface ContextInterface {}
class RequestHttp implements RequestInterface {}
class RequestDrush implements RequestInterface {}
class ContextHttp implements ContextInterface {}
class ContextDrush implements ContextInterface {}
function services_menu_callback() {
$request = new RequestHttp();
$context = new ContextHttp($request);
return services_do_stuff($context);
}
function services_drush_command() {
$request = new RequestDrush();
$context = new ContextDrush($request);
return services_do_stuff($context);
}
?>And then services will pass $context around to the constructor of its various objects to use instead of super-globals or globals. What those interfaces will look like the Services folks are going to work on for us at a code sprint in Sweden on 3 June. We'll see what they come up with. :-)
Development
For now, development will happen in the Services github repository: http://github.com/heyrocker/services
Assuming it all works out, the context system will move to a separate module that can be required by Services and anything else that wants to use it. Since the Context namespace is already taken, we discussed where it should live. Going into the existing Context module is one option, but we weren't wild about that. Another possibility is a "Drupal Context" module that serves as a pre-backport from Drupal 8 (since the goal is for this system to become the cornerstone of Drupal 8). While we were talking Sam claimed the Butler namespace for the same purpose. He's silly. :-)
In after-hours trading, Dave Reid indicated that he may be interested in making the xmlsitemap module the second guinea pig, depending how it goes with Services.
As always, if I goofed on the summary anywhere please feel free to smack me for doing so. Just do it gently.
Comments
Chicken or Egg
I am looking at this from a relative outsider's point of view and it is not that important beyond logical consistency but I was having difficulty reasoning about this code and here is what I think is my problem with it:
It strikes me as strange that a Service does Stuff based on a Context
services_do_stuff($context);I would expect services to do stuff based on a request that was performed within a specific context. I find it strange that:
Context is there - it does not depend on what the Request is.
In other words context is the environment, the world as we know it, and a request comes in. At this point the environment is about to change based on the state of $context, and the series of events the $request will cause to take place.
Of course one could argue that context includes the fact that there is now a request to do something within the environment but that's a but too philosophical and not necessarily useful :-)
So a service would take these two things in ($context and $request) and do stuff with them.
function services_drush_command() {$request = new RequestDrush();
$context = new ContextDrush();
return services_do_stuff($context, $request);
}
It is up to the do_stuff_function to interpret $context and the $request and decide what to do.
Of course I am probably missing something here or interpreting too literally, etc but just thought I'd share.
The Rooster
On the contrary, that's at the crux of the matter. All context originates from the request, or is derived from the request, or is derived from something derived from the request, etc. This is actually the first time we've discussed splitting context from request at all, and that's purely an implementation convenience. "Information about the environment in which I am running" is the context, and that includes the GET and POST parameters and such. The request really is "part of" the context.
Additionally, one of the crucial cornerstones of this approach is that it becomes possible to mock up or simulate context information. That includes information like GET or POST parameters, which we need to be able to simulate just as much as fancier context like "current language", "current organic group", "node that I should care about", etc. By keeping the request information part of the context object, we need only one override/mocking/simulation mechanism. If they were two separate objects, we would need to fake out two objects, pass around two objects (remember, they could get passed pretty far down), track the simulation of two objects, etc. It doubles the amount of work we have to do, and is twice as many moving parts for module developers to keep track of.
Interesting
I get how this is useful from an implementation point of view - you lump everything together (request and the context within which the request came) into one "thing"
But there are still two separate things that are being fused here.
If:
then context is not really context if context = stuff to help determine how to respond to a request or how to interpret a request).
For example, what user I am, what role I have, what language I prefer using, how long I've been logged in, or what state the site is in (e.g. how many other users are logged in, etc.) has little to do with a specific request of (e.g. give me all nodes from the taxonomy term "Philosophy").
You suggest to then add the request (give me the nodes) to the rest into one single object called context and pass that around. This does reduce the moving parts but I am not convinced it makes things simpler.
Say I want to simulate how the same request would behave under different contexts. By lumping everything together it would be harder for me to do so conceptually.
Maybe something along the lines of
$context->request
$context->state
would provide a clearer separation of concerns whilst keeping the number of "moving" parts down. If I want to keep the state and see how it would react to a different request I just switch that out and vice-versa.
It would also force us to think about what is a request "Please do something for me" and what is state "How was the world just before I was asked to do something and what parts of that world are relevant to what I am about to do next".
It might also help module developers think a bit more about what is actually going on...
Getting philosophical for a second you can actually make a pretty strong argument that a request is not part of the environment but simply an intent to change the environment - as such it should really be modeled separately.
I apologize but years of agent-based artificial intelligence brainwash you to think like this :-)
The point of this group
Nothing to apologize for. Getting input from a variety of different backgrounds and perspectives is exactly what we should be doing right now.
Things like "what is the active user" will usually not affect a listing of nodes in a given taxonomy term... but if you're using a node access module then it likely will.
As far as SuperBlocks go, though, "do something for me and "how is the world" are the same question, sort of. A block's generic logic flow becomes "here's a context object, here's a configuration object, now render yourself and gimmie a string." And the block is not allowed to vary its output on anything else (usually). That way, a block becomes idempotent about its context, which means we can move the block around, call it from different places at different times, cache it safely, and so all kinds of other really cool stuff reliably.
To the block, whether the "node I should care about" comes from the request or is simulated is irrelevant. It just asks the $context for "primaryNode" (or "node", or whatever we end up making that API). In that case, the node is part of that block's context. Whether or not it has anything to do with the request (eg, arg()) is irrelevant, and in fact the block shouldn't care and shouldn't even be allowed to care.
It also keeps the implementation simple, which is critical if we want people to actually be able to use this system. :-)
What about agent-based AI would suggest that this won't work?
Getting philosophical for a
I like this point. I actually don't think it's far from what we've been talking about, but I like it. But also I think it misses some of the point. Whether we use
$context->requestand$context->state, or some other datastructure, isn't really that important. The goal of the context system is to provide an interface for code executed during the lifetime of a Drupal request to be able to 'ask questions' about the environment in which it's being executed - which sorta makes the question about how we store/represent environment vs. request moot, or at least secondary to how we architect that interface. What really matters, as I think you pointed out in another reply here, is what creates the most useful, intuitive DX.It will work wonderfully
There is nothing, as far as I can tell, that is inherently wrong with the approach. I guess bells went off for me because of the mention of Services (which gives Context a much wider use case).
In agent-based software engineering, independent programs communicate with each other by exchanging messages (i.e. they are services). A typical cycle is:
a. beliefs about the current state of the environment (Context)
b. beliefs about who originated the query (Requestor)
c. what it wants the world to eventually become based on goals and motivations
Although Request, Context and Requestor can all be lumped into a wider concept of Context from a practical implementation point of view it makes sense to tackle each one on its own as each one brings a full set of issues to the table in terms of what an agent needs to consider. Hence my raising the issue that request and context could be treated separately.
Now, if we bring this back down to SuperBlocks it all sounds like a huge overkill (as most things in AI tend to be...) and you are right that "do something" and "how is the world" are pretty similar things in a simpler world.
So I guess a possible conclusion would be that what Services needs to do is not necessarily what SuperBlocks needs to do. SuperBlocks is a subset of what Services would need.
Although the starting point is the same services may need to go a bit further to eventually accommodate more sophisticated types of interactions between services. This may mean having a more granular concept of what context is.
First, just to note - dude,
First, just to note - dude, g.d.o has threading! use the 'reply' link at the base of the comment you're responding to, it makes it easier for all of us to follow!
...anyway:
Sure, on some levels anyway. Services is probably going to need to interrogate the incoming request data far more than a normal page render (using SuperBlocks) will; however, it may have much less of a need for complex chains of derived 'state'-type information. We bring them together because, IMO, HTTP is a stateless protocol, which means we need to bootstrap everything on every request in order to even figure out what family of response mechanisms to use. Since that all has to happen at the very beginning to give us maximum flexibility and control of our critical path flow, bundling them together is a natural approach. But since the resulting response mechanism may actually need only a small subset of the available functionality (your observation about SuperBlocks requiring a subset of what Services does is an example of that), the remainder of the exercise is to make the response mechanism as light and lazy-loaded as possible.
I hope not. In fact, I very much hope not. This is the value of context object as 'Butler': what we define as 'context' in core doesn't say anything about the granularity of information it can handle. It just defines a system by which modules can add questions that they're able to answer. Which means it's up to contrib, not core, to be able to answer those questions of increasing granularity.
picking up on this thread.... ;-)
Great - as long as it does not limit the granularity of information it can handle and it does not loose info along the way then contrib can pick up the responsibility to decide to do more.
Overall I really think this approach is fantastic and in the bigger picture of things brings Drupal much closer to a being able to create a proper dynamic Hierarchical Presentation - Abstraction - Control architecture and what is even cooler it will do this through unified set of concepts that deals both with the interface in the form of SuperBlocks (where PAC originated) and in dealing with outside services through Services. That as far as I know and kudos to all of you driving it forward, is a first.
Generic non-html rendering
I haven't kept up with all the discussion on the Butler group, but I saw this discussion on D7 services, so I thought it made sense to link to a new topic I posted on the Services group: http://groups.drupal.org/node/93644. @Crell: we've discussed this before, and I haven't done anything with it since we discussed it last, but figured it's time to put out a call in the hopes people out there have the time and inclination to drive it forward.