PHP-free front-ends to any Drupal page for D7 services

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
effulgentsia's picture

Hi, I'm not sure if this has already been discussed in this group, but to the extent that D7 services is underway, I wanted to get this discussion started if it hasn't been already. If it has been, please redirect me to the appropriate thread.

Services is great, but as I understand it, due to tight coupling between page callbacks and rendering in D6, you can't have a non-HTML front-end request an arbitrary Drupal page/form and get back unrendered data. Instead, you can only communicate with some Services request handler function written in PHP and running on the Drupal site. Of course, there's a rich library of those for things like viewing and saving nodes and users, and you can write your own pretty easily, but consider the use-case of someone wanting to build a front-end to Drupal functionality for which there isn't a Services handler, and you want to do this to a site that you can't add PHP code to.

D7 almost lets you do this, because all page callbacks (should) return unrendered render arrays, and you can plug your own delivery callback for any request. What's missing, however, is a generic renderer that can take $page, and instead of calling drupal_render_page() on it, as drupal_deliver_html_page() does, to instead call some other function on it that can convert the $page array into consumable and useful data, but without exposing data that the client shouldn't have access to (you wouldn't want to, for example, return the full #user property of some arbitrary element deep within $page). That's where http://drupal.org/node/145551 is currently stuck, but I think D7 Services is the place to solve that problem, and perhaps http://groups.drupal.org/butler is where to solve it for D8.

If we can solve this, it will allow non-HTML front-ends to be built for any arbitrary functionality that already exists on a Drupal website, without needing access to add additional PHP code to the website, and I think that would be very very cool.

Thoughts?

Comments

If I understand you

moshe weitzman's picture

If I understand you correctly, you are saying that D7 services should mostly be a matter of changing using hook_page_delivery_callback_alter() to switch regular menu items to using own delivery callback handlers that render $page to JSON, XML, etc. If we had hook_menu_active_handler_alter they could also change 'access callback' to a function that checked for a token on the url or some OAuth based access check.

Quick reply late in the

voxpelli's picture

Quick reply late in the evening so excuse any stupidity - I just wanted to reply with something in case I forget later on:

I don't think all pages should automatically be exposed through Services in D7 because I think that most pages was designed with that in mind which can make it very frustrating and even insecure to work with - as you say yourself.

I do however think that there are some interesting possibilities in D7 that should be investigated - would it for example be possible to allow the REST Server to "overload" menu callbacks with the services interface for menu callback's resource?

The goal of magically accomplishing a complete Drupal API without any PHP code isn't really feasible enough to be desirable yet though I think and I'm unsure if it will ever be. Some kind of PHP code will always be needed - it's all a question if Drupal will become rewritten in a more resource oriented way or not (which Butler is somewhat dealing with).

Why not feasible?

effulgentsia's picture

No worries about replying during late hours, and I'm very interested in a discussion on this, so I'm happy with any reply (g.d.o. is all about brainstorming, no such thing as stupidity, when a comment is meant sincerely).

I don't think all pages should automatically be exposed through Services in D7 because I think that most pages was designed with that in mind which can make it very frustrating and even insecure to work with - as you say yourself.

Why not? Every menu router item already has an access callback. Via a browser, you can cause any page callback to run (subject to access callback restrictions) and get back an HTML rendered representation of the $page array that that page callback returns. Why is a non-HTML client allowed any less than this? The only difference is a non-HTML client doesn't want to screen-scrape HTML. It should be able to get back $page in a format that's easier to parse. We already have the ability to implement hook_page_delivery_callback_alter() to inspect the request, determine what format the client wants the page content in, and cause Drupal to route $page via some function other than drupal_deliver_html_page(). The only thing we don't have is a replacement to drupal_render(). I.e., instead of having drupal_render() run recursively through $page, calling #theme and #theme_wrappers, we could have a services_render() recursively run through $page, and output a sanitized array, containing only information that should be available to the client, in the same way that #theme functions convert elements to sanitized html fragments. We would need to figure out how to do this. Perhaps introduce a #services_sanitize property for elements that would be a function that could output a sanitized version of the element. The Services module could provide a default implementation for each core element type via hook_element_info_alter(). Contrib modules implementing hook_element_info() to define new element types could implement a default sanitize function for the element types it defines. For any element without a sanitize function, we output nothing for its properties, but continue the recursion to child elements. Sound familiar? It's exactly the same model we already have with how drupal_render() and #theme interact.

Surely, there are many details here to figure out, like should there be the equivalent of preprocess functions, or are the #pre_render and #post_render functions (or Services-specific equivalents) sufficient? And it will require time and attention to take it from concept to reality. But other than the API details, and then testing this in the wild, I don't think there's anything fundamentally hard to do here that can't be done in D7 contrib, and then moved to D8 core once put through the paces in real applications.

Once we have a sanitized $page array, returning it in XML, JSON, or some other format is trivial.

I'm unsure whether this

fago's picture

I'm unsure whether this really helps the "Services" use case. Rendering to something different than HTML might be useful in other case though, however there are always some already pre-rendered HTML elements in there, so I don't think that approach takes off for that.

For the Services use case a XML or JSON representation of the page doesn't help us much as the output would change rather quickly as modules or site builders do some small changes - odd for anyone that relies upon such a service. Furthermore a service needs more operations than just GET, but you cannot update the data presented in $page as it doesn't match the internal data structure. Altering the deliver callback for services surely makes sense, I'd not go with the render system though. Usually you don't want to render anything but expose a API to the web.
For proper RESTful services you need knowledge about the data in order to be able to properly map CRUD operations and to apply proper access level restrictions. So I'm all for metadata based services, what would even enable us to generate service docs or service descriptions (WSDL,..).

It remains the use-case of not having to screen-scrape, but we have already a solution for that in core: RDFa.

Clarified thread title

effulgentsia's picture

The goal of magically accomplishing a complete Drupal API without any PHP code isn't really feasible enough to be desirable yet though I think and I'm unsure if it will ever be.

I realize the old thread title was misleading, so I changed it. I'm not saying we should expose all Drupal API functions to client-side code. Just anything for which there's a menu router item page callback.

Services

Group organizers

Group categories

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: