Create published standards for Drupal REST response formats using services

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
techninja's picture

After using Services for a month and a half, I’ve been amazed at the incredible foundation and modularity set forth, only now to find it lacking a little bit in follow-through. I wish to bring together the movers and shakers of the services module community (and simple services module users) who demand more consistency from the services they provide and support, to help do something about it.

Imagine for a moment you’re a Drupal engineer tasked with using services to provide data to and receive data from a third party.

How comfortable would you feel answering these questions from their integration team?

  • Should we expect error 406 for a call to a non-existing user at user/[uid], or error 404?
  • Is there machine parsable data included in the response properly describing the problem if there was an error?
  • Why do some working calls return data, but others return “true”, “1”, or "null"?
  • If an exposed REST service is available on one of our Drupal 6 sites, how sure are you that a POST for a node will go through even if the node->body has possibly been replaced with a required cck text field?

The answers to these questions vary, but it’s almost guaranteed they’re not going to be happy answers. The root of the problems comes down to standards. Without a public resource that applies to at least a large default subset of imaginable implementations, modules that provide service endpoint resources have little to fit to, and this leads to the real world inconsistency only partially touched on by the questions above.

I propose two things:

  1. Create a reference document to suggest standards for use in every returned success and failure response, so that no matter what the data to be returned some default levels of success or failure can be parsed programmatically without having to understand the data specifics.
    • This would mean parsing form errors into objects that describe missing or invalid fields, data types and valid ranges
    • Also including general standardization on use of HTTP error codes for particular use cases, E.G.- 200 success, 406 for incorrect or incomplete arguments, 404 for missing (given but not found locally) URI path based arguments, etc.
  2. Allow for aliasing (and clean defaults) for all fields handled during any services transaction, built into services (standardized and exportable!), and hooking node type specific forms for field management.
    • UI configurable and exportable, no longer will your external services need to reference internal Drupal content type architecture like exact field names, or whether you’re updating the [url] or the [0][value].
    • Allows for field renaming, moving, and deletion on the Drupal side without having to worry about versioning your service API (Aliased are simply remapped)

Some parts of this proposal may be a bit overwhelming, while others could be (at least partially) accomplished in a weekend code sprint. Not to mention, some of these ideas may just be the wrong way to go about it, so I bring it here to the group. Lets discuss!

EDIT 11/18/11: Added this discussion to the WSCCI group as well, welcome D8 Devs!

Comments

WSCCI relevant

Crell's picture

This is definitely relevant for the WSCCI initiative, so I'm glad you pointed me to it. Can you edit this post to file it to the WSCCI group, too, to get more exposure?

Better standardizing our services handling is part and parcel of becoming a better REST application server, so I fully support starting that discussion now. It's also relevant to discuss the full spectrum of HTTP responses, since all are available to us. Not just the response code, but consistent response content-types. (As in HTTP mime content types, not Drupal node types.)

I am not as wild about part 2; aliasing and persisting and carrying along field names introduces a frightening amount of complexity. There's enough work going on with the Config Management Initiative and revamping the Entity API that I think it's best to table that for the time being. I'm also not even sure that directly accessing fields via a REST call is wise. That said... HTTP has this very nice "301 Moved Permanently" response command, which could be used instead, and done entirely at the application layer on a site-by-site basis without having to touch the Entity API at all. Would that work? I don't know, but it seems (at 1 am when I type this) like a cleaner level to do that sort of mapping at. Let's get back to that in 6 months. :-)

Can you take a quick stab at a recommended standards doc for what type of response is appropriate when that we could dissect and debate about?

Aliasing is probably wrong

techninja's picture

As has been said in later comments, you're probably right about aliasing. And really, as long as there are definitions for content in a machine readable formats (RDF or the like) then aliasing doesn't matter at all. An initial (and probably cacheable) request is made to query for a specific entity's full "schema", telling the guest basically how to both understand it's read data, and write to it. As opposed to aliasing, wherein the arguments could possibly be assumed or simplified, this would provide enough data to allow any data consumer to configure their requests appropriately. The overhead becomes that custom fields may not have a given RDF description available and would need to provide one (How does D7 do this?)

Book recommendation

mparker17's picture

I recently picked up the book RESTful Web Services by Leonard Richardson from amazon.ca. I would recommend reading it... it contains both step-by-step instructions on building RESTful clients and servers; as well as extensive reference tables for quick reference (one of it's appendices contains a list of HTTP response codes and their meanings with respect to web services; another contains a list of HTTP headers and their meanings WRT web services).

I suspect it would be quite useful for the writers of the standards document proposed in part 1.

Ordering it...

techninja's picture

And with your Drupal association membership, you get a hefty discount! ;)

Another Book

kvanderw's picture

REST API Design Rulebook by Mark Masse
isbn: 9781449310493

I just finished this as my intro to REST. While it does not go into the specifics of building clients and servers, it does dig pretty deep into the rules around each.

Kurt Vanderwater
Drupal made Easy

subscribing

bsenftner's picture

subscribing

Automatic Defaults & Services for CU Operations

Grayside's picture

(Coworker of Techninja speaking)

More than just documentation, Services resources could be defaulted to one of 2-3 responses if a well-formed response is not given.

For example, if every resource is supposed to respond with an array containing a full response, something sane could be done with TRUE, FALSE, NULL, etc. should it happen to show up. I can imagine object-oriented structure helping to support this, especially when you add in the idea of supporting override and extension of default services with something tweaked for use case.

As for the aliasing, I'm not sure about that. I do think that Services as currently structured requires insight not only into Drupal's array structures, but into the specific workflows of a site. Are images all nodes? How should node references be wrangled? This is crappy if the goal of the service includes abstracting content and behaviors in the Drupal REST server to generic web stuff. In other words, of Services CRUD, Read and Delete are easy, Create and Update are ridiculous. (I'm thinking about the place where Services and Feeds meet.)

I don't really have an answer to this. I have a budding idea... there are countless XML and RDF schemas out there. Supporting one (or in pluggable fashion) multiple schemas, and providing the means by which a default sort of mapping can happen, overridable for specific site information structure... that would allow someone to say "Create Content" and have everything defined generically, but submit it to any Drupal site supporting the standard. This is a much grander idea, the more I think about it, the more it seems like a separate conversion. Possibly this includes separating Feeds into the "Creating Cool Stuff From External Data" module and the "Configuring data sources, and pulling their data in periodically" module.

VIE?

Crell's picture

Possibly VIE compatible?

(Note: I haven't looked in VIE in detail yet, so I may be talking out of my backside, but having an existing or budding standard to work from gets a general +1 from me.)

That looks interesting. It

Grayside's picture

That looks interesting. It and PHPCR look to be a standardized layer for content creation that might sit between Services Resources and Entity API, and Form API Submit handlers and Entity API.

The other half of this is having an example content schema, such as Dublin Core or perhaps a tighter binding with RDF efforts.

PHPCR is of course

lsmith77's picture

PHPCR is of course interesting, but its really at a different layer as it sits behind the server side business logic.
More relevant is indeed VIE and more specifically create.js see also https://github.com/bergie/create/issues/2

Hey guys, Services

kylebrowning's picture

Hey guys, Services co-maintainer here.

Yeah, its a little inconsistent. :(

This is definitely a big discussion. Theres a lot of questions, Do you return 404 if a user has no rights to the endpoint or do you return 401 unauthorized, some use cases would need a 404, like non public APIS, etc.

It kind of all comes down to how the developer(or client if you will) wants to handle the situations when it makes a call to the site and this is different for every single website.

In services there is a specific reason certain response codes are used and they follow the HTTP 1.1 status code specifications. Id almost expect this to be dissected up and down on the WSCII front so because this is what the status codes were made for and what everyone understands them to be.

For reference: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

Now, as for responses from the actual methods it again depends on what you want to do with the information. Do you just want a true false returned on login with a cookie returned, or do you want the full user information returned as well in the same call.

Its very hard to guess at what the developers use case will be and some of these are edge cases.

I may be rambling here, but I believe this is a hard problem to define for everyone so it will need to be extremely extensible, it may mean, offering the ability to attach contexts on when a method is called.

example: I set a setting in the interface that says return the user context whenever the user login method is done as well as the default return for the method. So we can more pieces of information added to our return, much like views relationships works.

I don't know if thats the right way to handle but that would be pretty flexible if you asked me.

On the services.module front, we havnt figured out a way to do it, but hopefully our 4.x solves this issue. At the very least method responses will be consistent.

If nothing has changed on the WSCII front, Services in Drupal 8 will seek to be more of a REST Server provider than defining endpoints and contexts of information for which to return.

Configuration vs. Defaults

Crell's picture

True, there are lots of different use cases. I don't know if we want an actual configuration toggle for that sort of thing, but certainly we should make it possible for someone to change. Eg, if the default responder at /user/login just returns a true/false and session cookie, it should be possible to swap that out with another responder entirely that returns a user object if appropriate.

Still, we should standardize what the default, typical behavior is. Eg, are "user not found" for login and "node not found" for a node load operation both 404s? Is the former a 403 or 406 instead, and if so why? What's the guideline we use for determining that?

I don't know yet. :-)

Services 4 is a non-starter

Grayside's picture

Services 4 is a non-starter for us (D6/next few months deadline). Community standards to work off of would be compelling on their own.

I don't know if responses need full content objects, but for Create and Update operations they certainly need Drupal's ID for them, since Drupal can't accept an externally provided ID.

Create Node Service → Submit Node Form → Node Save → Return 200/NID
Update Node Service → Submit New Information with NID → Node Save → Return 200/NID
Request Node Object Service → Submit NID → Return 200/Node Object

Much more than that is likely added value, though I could imagine wanting to return the full object or checksum for content validation on the client side.

201 Created

kim.pepper's picture

201 Created with a location header pointing to the new node url is the most appropriate response according the RESTFul Web Services book.

See also: http://restpatterns.org/HTTP_Status_Codes/201_-_Created

Kim

Just want to point out a few

voxpelli's picture

Just want to point out a few things - not really active in Services development anymore, but couldn't really ignore this post when I saw it.

Most standardized/default responses shouldn't be defined by Services core or even the REST server but by interchangeable modules and mostly the modules defining the resources. As I see it Services should be able to model most API:s since otherwise Services can't be used as a framework to implement standardized API:s like the Atom API or a Twitter compatible API.

So - resources should take care of standardization. Then http://drupal.org/node/1060362 becomes interesting - that Services core perhaps shouldn't take care of anything but example resources. Then it's up to a resource project to define a good standardized response.

An interesting start for such a resource project could be the Services port of RestWS which I did as a proof of concept a long time ago: https://github.com/voxpelli/restws/tree/services (Related issue: http://drupal.org/node/1042512)

Another could be FCC's Content API module: http://nodeone.se/blogg/fcc-launches-content-api-using-services-module and http://drupal.org/project/contentapi

As a final note - eg. authorization modules also sometimes needs to have specific kinds of responses to map to standards - eg. the OAuth authorization module for Services must conform to the OAuth specification on it's responses.

No, probably not...

techninja's picture

You might be right in that the services module itself probably can't go on supporting its resources with "global" standards for returns, seeing as there are so many possible needs (though looking at [#1060362], I don't get the feeling that splitting those "core" resources out is a popular subject).

As long as services provides some very strong standardized default returns (if anything, just for examples) and along with that a clean single hook API for altering the exact output (all headers, data object to be converted to output format, etc.) then all bases should be covered. If you're looking for "true" instead of response:{ status: true, count: 15, operation: "updated", entity: "user" }, then you're a single hook implementation away.

I recently ran across another

christianchristensen's picture

I recently ran across another flow diagram for HTTP return codes - it was off a stack-overflow question: "HTTP status code for update and delete":

http://i.stack.imgur.com/whhD1.png

REST API design

sun's picture
  1. The HTTP Accept header is used to negotiate the response format.
  2. You have to differentiate between the protocol layer and application layer response.
  3. On protocol layer, every response uses the appropriate HTTP status code defined in the HTTP protocol standard. Nothing to discuss here, the HTTP spec is very concise.
  4. Every non-HTML response contains reliable application layer parameters. These elements are most commonly named status and message, located on the top-level.
  5. The status response element (integer) value is identical to the HTTP status code, if that sufficiently and appropriately denotes the application layer status (most commonly the case for 200, 302, 304, 401, 403, 404, 50x). Custom application-specific status codes are possible, but need to be carefully and explicitly defined.
  6. The response contains the requested resource in a top-level response element, whose name is derived from the resource URL. E.g., GET /node/123 returns (with Accept: application/xml)

    200 OK
    <response>
    <status>200</status>
    <message />
    <node>
      <nid>123</nid>
      ...
    </node>
    </response>
    
  7. All server responses contain status and message. There is no single server response that returns a unspecified/unknown value directly.

  8. Only HTTP GET and POST methods are used.
    1. The HTTP spec precisely defines the PUT method and it is not supposed to do what many mistakenly believe. Requests are supposed to send the (full) resource as envelope (in the specified Content-Type) in order to replace it on the server. Mainly designed and suitable for file operations only.
    2. HTTP DELETE does not allow for a post body to be sent with the request, which makes it insufficient and unsuitable for most use-cases.
    3. Neither PUT nor DELETE are supported in HTML forms/browsers. There's little point in trying to support them, as it only means to make the system more complex and code paths ambiguous.
  9. Resource paths are standardized:

    List:   GET /name
    Read:   GET /name/[id]
    Create: POST /name
    Update: POST /name/[id]
    Delete: POST /name/[id]/delete
    
  10. Read, create, and update always return the full, current representation of the resource. ("current" means post-update in case of update.)

  11. The update operation allows for partial updating of resource properties. I.e., only properties contained in the request body are changed.

Daniel F. Kudwien
netzstrategen

Sun don't mess around...

techninja's picture

No intro, just straight to it. I like it!

Some of these decisions completely change the foundation of the Services module, though are already planned as part of WSCCI. Should Services 4.x become more WSCCI compliant to ensure that users
have a better jumping off point from D7? I'm not sure, but it's great that we're talking about it.

Off the top of the head responses to the above^^^:

  1. Use the HTTP Accept Header: Perfect, it's already there, we just have to start using it.
  2. Differentiate between protocol and application layer: Agreed, though I don't believe this has been clearly defined
  3. Protocol layer, use given status codes: I agree that the HTTP spec is canonical and should be followed, but does it account for non-public api standards of using 404 for unauthorized contectually (or other oddball rule-breakers)?
  4. Every response includes application parameter elements: Wonderful! Though an obvious bit of overhead, it really standardizes requests beyond the occasional "null" and "true".
  5. status element includes HTTP status code: I'll buy it, though it seems to be a bit of duplication if the status code is already in the returned header
  6. Resource included in response with a key matching url element: Another win, though if you want to be really pedantic, maybe a "content" key could list the associated returned resource keys? eh.
  7. All responses contain status and message: Basically the same as #4, but without mention of HTML returns. I'd imagine if the expected response was to return themed html (EG, for a block), it could be included as the returned resource. GET /block/<id>/themed (I'm probably totally off-base here)
  8. Only use GET and POST (not PUT, DELETE, etc): What?! Really? Well.. sheesh. Here I was getting all worked up about how cool it was that we're using these never used methods, and now we're ditching them? This flys a bit in the face of current protocol on Services module (though I can't speak for current WSCCI initiatives on the subject). Is it really so bad that there's no post body for DELETE? Usually all the information you need to pass along is in the URL. If anything I agree that more complexity to solve the same problems is never a win, but if it's more syntactically obvious, pre existing terminology (and most of our requests won't be made from HTML forms anyways {XHTTPRequest via JS and serverside cURL}), why not?
  9. Standardize resource paths: This sounds perfect, and is somewhat already there, and kind of alludes to a non-endpoint specific solution. I'm pretty sure WSCCI is taking this directly to heart as it will be core, but the Services module is endpoint centric (Not that it doesn't do a good job of standardizing path structure on it's own inside each endpoint)
  10. C.R.U. always return the latest version of the resource: Yep, totally agreed. Again, this may seem like a great bit of overhead, but in certain use-cases it can halve the number of requests needed
  11. Allow partial resource updating: Yep, again a full win. Why pass duplicate information if it's already there. This opens the door to super-simplified content publish/unpublish requests, edit in place immediate content update, etc.

I do notice though that there's no mention of something we've been musing about above, and that's the need for third party machine readable entity definitions that basically describe to another service how to successfully submit a specific type of node, or create a user (required fields and all). What thoughts do you have on this?

.3. ... using 404 for

sun's picture

.3. ... using 404 for unauthorized

Makes no sense. The HTTP spec leaves no room for interpretation. Unauthorized means either 401 Unauthorized (operation requires authorization) or 403 Forbidden (you are who you are, but you're not going to play with this).

404 Not found means 404 Not found. Nothing else. And you can only attempt to find things if you're authorized to find things in the first place.

.5. ... duplication if the status code is already in the returned header

The protocol (communication or transfer) layer is a completely different, concisely defined and designed thing. It must not be mixed with application-level logic. Taking over status code values and their meaning into the application layer is merely done for simplicity. You could as well define completely custom status codes for these basic response status. But why?

.7. ... if the expected response was to return themed html

If you want HTML, you request HTML. Accept: text/html

.8. ... how cool it was that we're using these never used methods, and now we're ditching them?

Boils down to reading and understanding the HTTP specification.

I've to admit I was confused at first, too. Especially after reading lots of articles that actively promoted them. But fact is, the HTTP spec doesn't leave much room for interpretation on them either. And unless that spec is going to change, any soft or custom interpretation is just simply invalid, wrong, and bogus. There are very good reasons for why the best known and best documented REST web service APIs on the web don't use them. They only use GET and POST.

So, in short: Not cool. Cool people can read.

need for third party machine readable entity definitions that basically describe to another service how to successfully submit a specific type of node, or create a user (required fields and all). What thoughts do you have on this?

@see http://www.w3.org/TR/wsdl

Daniel F. Kudwien
netzstrategen

Mmm... details

Crell's picture

1-3: Fully agreed.

4-7: I am not fully agreed, on the grounds that it's unclear what the word "all" means here. Drupal will (or rather should) be able to return all kinds of response formats, most of which have their own semantics. If we're returning RSS, or Atom, or an OAuth token, or whatnot, those application layers already have their own messsage and error reporting semantics. Those should be followed, and wrapping our own stuff into there is not cool.

However, if you're talking in particular about a Drupal-specific non-HTML responses, like an AHAH response that's part of FAPI, sure. Having a standard envelope format makes sense (I think we have one now, it's just so undocumented as to not be worth having), and reusing the HTTP status codes seems like a nice shortcut.

However again, there's a good argument to be made that we should not have Drupal-specific non-HTML responses, or at least we should minimize them. For instance, if we're sending a node in "raw form" from one Drupal site to another, why not use Atom as a wrapper format for it? There's a good argument to be made there, and if so then Atom already has its own semantics that we should follow. I actually did that on a previous project to much success.

Similarly, should an AHAH response really be wrapped in a custom envelope all the time? Why not simply push ESI-like functionality to the browser and use HTTP itself and be done with it? There are cases where we need more logic (eg, when we need to send new CSS or JS along with new HTML), but if we want a block to be available to a 3rd party, for instance, we shouldn't force them to support proprietary logic and formats without a good reason.

So on these, "maybe".

8: I wouldn't say "never" use non-GET/POST. We want the system to support any valid HTTP method. However, I also agree that we should adhere as closely as possible to the HTTP spec, not to cool-sounding behavior. If that means we use 99% just GET/POST, I'm cool with that, but contrib should be able to use whatever verb it wants.

9: Mostly agreed. I'm not 100% happy with putting verbs into the resource path, as that seems mixing identifiers and verbs. /name/id/delete seems like where a form should live, not the actual path you hit to delete something from a non-browser. I need to check the spec and see if there's a reason that DELETE name/id would not be a good thing to support. It seems like it should be. Otherwise I agree that most (if not all) objects (entities) should be exposed at [entity]/[id], potentially with /action. That means we may want to move /taxonomy/term.

10: I think this makes sense.

11: That would get interesting with field dependencies, wouldn't it? Validation could become trickier.

Also open question: Do we want to make our default service-form of entities JSON or XML? JSON is thinner and easier to process, but XML has far more existing tools and standards we can leverage (e.g. Atom, CMIS, etc.) Naturally all should be possible but we probably want to make one our primary target.

PUT, DELETE

sun's picture

You need a latest greatest state-of-the-art browser, or otherwise, browsers only understand form methods GET and POST.

Furthermore, by opening up PUT (and fwiw DELETE), you have to create a completely separate router, code, and logic paths for REST/web service operations, which is definitely not what we want in core.

HTTP Method Definitions clearly describes the protocol-layer expectations of the PUT method:

The PUT method requests that the enclosed entity be stored under the supplied Request-URI. [...] The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request

In short and other words: PUT is only valid to send/replace an entity in its entirety, and the entity is sent (enclosed) in the request body, using the specified content-type.

As a typical example, you can PUT a file to create or update it on the server. But in almost all other cases, it's impractical:

Incorrect:

PUT /node/123
nid=123&title=Whatever&status=1

Correct:

PUT /node/123
Content-Type: application/xml
<node>
  <nid>123</nid>
  <title>Whatever</title>
  <status>1</status>
  [...]
</node>

Note the [...] - the entity enclosed in the request is going to replace the stored entity on the server.

And lastly, partial updates require to use POST anyway, as they are exactly what the HTTP spec means with a "data-accepting process."

The DELETE method is practically unusable. The request body in DELETE requests is ignored and dropped by some server implementations and proxies. REST purists will argue that it's still semantically correct and those implementations should be fixed instead; or don't even attempt to send additional parameters in the request body in the first place. But for a generic multi-purpose implementation as in Drupal core, a stance like that is incompatible and unacceptable.

As already mentioned before, it would be naive to believe that the developers of the most commonly known and best documented REST APIs on the net wouldn't have spent plenty of time to design and develop their APIs, revising and fixing them after running into pitfalls and gaining experience over years.

Daniel F. Kudwien
netzstrategen

First of all APIs are not

Hugo Wetterberg's picture

First of all APIs are not only for browsers. They are widely used by other servers and applications to communicate with sites. So whether browsers support the full set of http verbs isn't the argument to end all arguments. The usual fall-back method is to use POST requests with a query parameter __method=PUT, this is how it's implemented in the REST server. The principle behind REST is to use the HTTP semantics as the API semantics for manipulating resources, what you describe is inspired by REST at best.

Yes, a PUT request should contain the resource in its entireity. The REST server doesn't enforce that, because it can't, so the developer implementing the actual callback is free to say: "Yeah. I know that's how it's supposed to be. But I don't wanna". What I've done in the REST server is that if you want to perform actions on a resource, which could be an partial update, you'll call POST http://example.com/resource_name/123/change_image. As we're using the full http semantics we can also do stuff like POST http://example.com/resource_name/bulk_delete to perform actions that aren't targeted on a specific instance of the resource type.

I don't really know why one would want to send a body with a DELETE request. And even if the spec doesn't forbid it the current consensus seems to be that it shouldn't be done. You're still free to add parameters to the uri query. So this wouldn't be a problem DELETE http://example.com/resource_name/123?cascade=yes&notify_author=yes.

I'm not really arguing that

Hugo Wetterberg's picture

I'm not really arguing that designing AN api the way you describe is a bad thing, or that it is a bad design. Designing an API through URL schemes and GET/POST is a perfectly good solution. But the point that I want to get across is that it is not a design that we can limit ourself to. Especially not if we want to have it labeled as REST.

I support using the verbs for

glennpratt's picture

I support using the verbs for resources because I think it encourages us to think beyond the expectations of the menu system and FAPI, which in my experience has been a bit of a struggle to graft web services onto.

I'm curious if you find issue with the pattern from Rails 4 as I've interpreted it:

  • GET /type/%id = read
  • POST /type = create
  • DELETE /type/%id = delete
  • PATCH /type/%id = partial update
  • PUT /type/%id = replace or create in place.

https://github.com/rails/rails/issues/348

Embedding the latter three methods in POST (PATCH as default) for compatibility, seems reasonable to me.

you have to create a completely separate router, code, and logic paths for REST/web service operations

It doesn't seem unreasonable to add this as a path part, an argument, or a new menu item type then reserving a $_POST variable for compatibility. Sure - it would be an API change.

The DELETE method is practically unusable.

Many APIs use DELETE, including the one we've built at my day job - I'd say it's useable; perhaps not for all things.

The request body in DELETE requests is ignored and dropped by some server implementations and proxies.

I don't see why DELETE requires a body? What would you put in it - if it becomes a data-driven process that might delete but just as easily unpublish or optionally alter collections of related content, then a POST does seem appropriate. It's OK to use 'POST' and redirects for a data-driven process - http://roy.gbiv.com/untangled/2009/it-is-okay-to-use-post

Google Docs is one of my favorite APIs, and seems fairly true to the tenants of REST as I understand it... it's not just web services. They use DELETE and it doesn't seem to require a body. They use standard headers to control conflicts.
https://developers.google.com/google-apps/spreadsheets/#deleting_a_list_row

(They do mis-use PUT for update, as do many.)

REST purists will argue

I would think usage should comply with their HTTP definition if used, but using or not using doesn't seem to matter to REST purity. It would be wise evaluate whether what we design (or even want) is RESTful. http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

the most commonly known and best documented REST APIs on the net

Please share some your list would include? Examples of what people think of as good APIs really helps make things more concrete.

Another relevant post

Crell's picture

http://martinfowler.com/articles/richardsonMaturityModel.html

The Level 3 concept of sending back the "next steps" commands appeals to me from a discovery and DX perspective, as well as making BC easier. (At least to a degree.)

Crazy idea: Should we consider standardizing on Atom as the "wrapper format" for Intra-Drupal actions? Or a JSON equivalent? (Any time we don't need to develop our own spec, it is a good thing.)

consider standardizing on

sun's picture

consider standardizing on Atom...

Before we can consider that, we need to collect and evaluate the available options.

Atom ... Or a JSON equivalent?

I don't see an OR there. It would have to be an AND, since they are different content-types, and in particular ATOM over JSON doesn't seem to be a widely adopted, nor really standardized at this point.

And of course, the other question is whether we actually want or need hypermedia controls in the first place.

Personally - though I'm happy to convinced otherwise - they don't really map to other content-types than HTML+RDFa for me. In other formats, you expect to read and write resources, without meta data in between (only exception being HTTP headers).

I.e., as an API consumer talking XML, YAML, or JSON, I'd rather refer to a global specification and description of resources and methods à la WSDL or similar.

In other words:

Correct assumption:

<?php
$garbage
= drupal_http_request('node/123', array(
 
'headers' => array('Accept' => 'text/html'),
));
$whats_related = $garbage->xpath('//link[@rel]');
// We've lots of stuff to show, but nothing to do,
// and obviously no raw resource/data to work with.
die();
?>

Correct assumption:

<?php
$result
= drupal_http_request('node/123', array(
 
'headers' => array('Accept' => 'application/json'),
));
// Let's assume for a moment that d_h_r() was smart and parsed what it Accepts... ;)
$node = $result->data['node'];
$node['status'] = 1;
$result = drupal_http_request('node/123', array(
 
'method' => 'POST',
 
'headers' => array('Accept' => 'application/json'),
 
'data' => $node,
));
$updated_node = $result->data['node'];
?>

Daniel F. Kudwien
netzstrategen

without meta data in between

glennpratt's picture

without meta data in between (only exception being HTTP headers).

Hyperlinks could be in the headers - agnostic of data format.

http://tools.ietf.org/html/rfc5988

My concern with relying on a global specification is managing change. Being out-of-band it will most likely be un-done or at best, out-of-date.

Working with Google Docs API has been an eye opener on links in data.

https://developers.google.com/google-apps/spreadsheets/#working_with_wor...

To be clear, the reason I

glennpratt's picture

To be clear, the reason I found the Google Docs API to be an eye opener is because I hardly read that documentation. I read enough to get the documents feed (entry point), then worked my way in based on the metadata (links).

AtomPub or just a

glennpratt's picture

AtomPub or just a format?

Maybe take a look at http://atomserver.codehaus.org/ based on http://abdera.apache.org/ which has extensions for other formats including JSON.

http://www.ibm.com/developerworks/library/x-atom2json/index.html

GData is AtomPub based and also provides JSON representations of Atom.

http://code.google.com/apis/gdata/

I haven't used the JSON in either of these...

Great discussion going on...

techninja's picture

Just wanted to come in and mention a great blog post from this summer I just ran across, mentions an interesting concept I had somehow missed, ensuring you're never passing along meaningless machine ids, but actually the REST URLs for that referenced resource! Quite cool, though possibly more obvious to others than myself.

http://www.stereoplex.com/blog/mobile-api-design-thinking-beyond-rest

Also is mentioned the standard PUT/POST/DELETE, though it's explained in a different fashion than I'd seen before and seems to make a lot of sense. Also highly recommend reading the comments for useful data from the peanut gallery :)

Should have time later this coming week to join back into the fray and see if we're ready to start jotting any of some kind of consensus we're forming here down somewhere official. Not that we're finding the "right" way so easily, such is the way with complex multi-faceted issues. Carry on...

Good PUT/POST REST Clarification Post

ethanw's picture

I'd recommend http://jcalcote.wordpress.com/2008/10/16/put-or-post-the-rest-of-the-story/ for a good unpacking of the cases for using PUT and POST. Generally in line w/ what has been said here, but to recap:

  • Create = PUT iff you are sending the full content of the specified resource (URL).
  • Create = POST if you are sending a command to the server to create a subordinate of the specified resource, using some server-side algorithm.
  • Retrieve = GET.
  • Update = PUT iff you are updating the full content of the specified resource.
  • Update = POST if you are requesting the server to update one or more subordinates of the specified resource.
  • Delete = DELETE.

I'm particularly interested in this conversation as it pertains to integration with REST JavaScript frameworks, especially Backbone. We've had some issues with some of the existing REST options for D7 having different implementations of the HTTP verbs that can cause issues integrating external libs. For Backbone JSON is definitely the way to go, and I think that's true for most JavaScript libraries (for obvious reasons). Having JSON as a standard would allow for a lot of great lightweight integrations, and should also suffice for enterprise apps, etc.

techninja's picture

Thanks for injecting a few drops of life back in the convo, though I get this feeling everyone has agreed to disagree and moved on with harder problems. Not to mention the intersection of D8 Symphony core communication and current gen D6/D7 sites wanting to make real live production things with it, gets pretty tricky to pin one thing down.

Allowing for multiple data return types via "accepts" header allows for JSON and other outputs, even though it doesn't look like XML. As long as the most verbose breakout of data existed, less "wordy" versions like JSON could be rendered during the request, even though they might contain the same basic structure or breakout for request info (status code, status description) and request data (array of node objects, updated user object, file path reference, etc).

Nothing new to report really, this whole thread should probably just be split up into two parts.. one to solve the here and now for services, and the other to take what was learned in that conversation, and apply it (somehow) to D8.

Hard? Yes. Impossible? ... We'll see ;)

I'll be at any D8/REST/Services related session I can manage at DC Denver, seek me out if you want to talk! (Tweet me during @techninja42) Maybe I can organize a quick BOF of some sort.

Another consideration should

christianchristensen's picture

Another consideration should be how PATCH may actually play into this (specifically on the recent decision of Rails supporting PATCH for updates instead of PUT): https://github.com/rails/rails/issues/348

AFAIK: Currently services.module uses POST for creation and PUT for updates to a resource; turns out there is actually a fairly nice diagram for this mechanism:

    X = CRUD
    A = Action
    T = Targeted action
    R = Relationship request

    COUNT |0|1|2|3|4|N|
    -------------------
    GET   |X|X|R|R|R|R|
    -------------------
    POST  |X|A|T|T|T|T|
    -------------------
    PUT   | |X| | | | |
    -------------------
    DELETE| |X| | | | |
    -------------------

    Create:   POST /services/rest/node + body data
    Retrieve: GET /services/rest/node/123
    Update:   PUT /services/rest/node/123 + body data
    Delete:   DELETE /services/rest/node/123

Services

Group organizers

Group categories

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week