Proposal for RESTful entity API

The other week, I posted a call for feedback about the viability of using PUT for Drupal entities as part of a REST implementation. We had a number of concerns, relating to how hooks and revisioning impact the idempotence requirements of PUT.

We had a fair bit of feedback and some good discussion, and even some input from the co-editor of the HTTP 2.0 specification, Julian Reschke. In short, hooks are not a problem, revisions are not a problem, forward-revisions are a problem we can work around.

Based on that discussion, ongoing discussions elsewhere, and a few offline conversations, I propose the following strategy for handling REST operations on entities in Drupal 8.

Disclaimer

For the purposes of this example, I will be talking about nodes. The same logic applies to any other entity that is revisionable (or not), but it helps to have a concrete example. Also, from a pure hypermedia perspective all URIs are irrelevant opaque strings that are subject to change, so paths don't matter. Humans care about paths, though, and I need something for examples, so for the sake of discussion please just ignore that detail for now. It doesn't affect the discussion at hand. (I'm not certain we'll be able to pull off fully buzzword-complaint hypermedia in this round anyway, although I'd like to.)

I will also reference "in JSON" below a lot, by which I mean "JSON-LD in the format we're developing for Drupal entities, or any other appropriate format, including XML-based, that people decide to support other than HTML, because HTML pages and REST just don't mix." (See why "in JSON" is easier to type?)

Easy details

First off, some conclusions based on the above-referenced conversations that make life easier:

1. Logging and other "incidentals" do not break the idempotence of PUT.
2. Hooks that modify "something else" do not break the idempotence of PUT.
3. Hooks that modify the object being PUT are out of our control, so if some contrib module decides to break the contract of PUT that's not our problem.
4. Recording the last-updated time of an entity counts as an "incidental". We're good there.
5. Per the HTTP spec, incidental changes that cause some other IRI to change are OK. (This is very important.)
6. The creation of an old revision of an entity, with its own IRI, does not break idempotence, provided that following a PUT to that IRI with a GET will return the same object that was just PUT. (That is, forward revisons are tricky but resolved below.)

The plan

All entities have a canonical path, in the form /node/{node} where {node} is a site-specific ID. That canonical path is the entity's RESTful ID, aka URI.

Another concept that has been kicked around a few times is offering more than one forward revision line of work. Effectively, it should be conceptually possible to do "branching" of an entity, with different people working on different, alternate versions of an entity. Think git-style branching, where each revision (vid) of a node equates to a Git commit object, and a "branch" is simply the label of a given revision. Each revision tracks its parent revision, which is a simple ID. Poof, you have multiple forward branches. I have discussed this concept with Workbench Moderation maintainer Steve Persch (stevector) at length this year, and we've concluded that it would be fairly straightforward to implement braching but merging would be very difficult, perhaps impossible.

I am not proposing that we adopt branching forward revisions right now. That is almost certainly a Drupal 9 question (or a really cool Drupal 8 contrib). However, I mention it because we can and should design our REST structure to allow for easy extention to such a concept. It also helps to clarify how revisions work in this model, as our current single-line-of-revisions is then simply a degenerate case and it makes it easier to explain.

Additionally, many many people suggested it's time we bite the bullet and just get rid of the concept of toggling revisions on and off. There is always a revision. Period. That is, switch to CRAP, not CRUD. I completely agree. For that reason, this is modeled on a CRAP design. CRUD is trivial to implement as a special case of CRAP by following "Create revision" operation with a "Delete old revision" operation. Even if we don't switch Drupal 8 proper to that model, the REST API should be designed around it for simplicity and future-proofing.

Legend

(Some of these are out of order because I didn't want to bother reordering them yet. That will get done later.)

X

Explicitly disallowed

1

The most recent revision on that branch, in JSON.

2

Create new revision (which implies a new /node/{node}/revision/{vid} gets created), and set /node/{node}/{branch} to refer to that revision. That means a subsequent GET will return the just-created revision. This is also almost identical to the mental model of a git branch. If that branch does not exist at the time of the PUT, the branch gets created on-the-fly branching off of the current active revision (subject to access control checks, of course). Returns HTTP 204 No Content.

3

Delete branch. This may or may not also imply deleting the revisions of that branch. I think for now we should not allow this operation, since core will have only a single branch, but we can leave this open for contrib elaboration.

4

Identical to PUT, except that the newly created revision is the combination of the previous revision specified by this branch and the body of the PATCH. It is very likely core won't support this, but contrib may do so. We're just defining what the full API would look like if someone wrote it. Returns HTTP 200 with the just-created revision in JSON.

5

HTTP 204 No Content if this revision is currently the active revision (aka, would be return by GET /node/{node}). HTTP 404 Not Found otherwise. Effectively this means the "active" subpath exists only if that is the active revision.

6

Sets that revision as the active revision. Returns HTTP 204 No Content. Also implies that /node/{node} now returns the same response as this IRI. Also implies that all other /active IRIs for the same node go away and now return HTTP 404 Not Found.

7

Make this revision not active. IF this was the active revision, returns HTTP 204 No Content. By implication it means that /node/{node} now returns 404 Not Found, or maybe 403 Access Denied. IF this was not the active revision, returns 404 Not Found.

8

Identical to point 5, for whichever revision this branch refers to.

9

Identical to point 6, for whichever revision this branch refers to.

10

Identical to point 7, for whichever revision this branch refers to.

11

Create a new revision and set it to be the active revision. Effectively identical to PUT /node/{node}/branch/draft followed by PUT /node/{node}/revision/{vid}/active. If forward-revisions are enabled, this will return a 403.

12

Identical to point 11 (PUT), except the saved revision is the combination of the previous revision active revision and the body of the PATCH. Returns the just-created revision in JSON. Again, unlikely to have core support but that's fine.

Explanation

Important to note here is that there is no CRUD support. There is node revision CRUD support, which by implication means CRAP for nodes. That is very much by design. If you really want to you can emulate it by adding a delete-old-revision hook to the revision-save operation. But really, let's just make the REST API "revisions are enabled only".

Also, I will say again that I am not proposing that we support git-style revision branching in Drupal 8 core. I am suggesting that we consider single-line forward revisions as a degenerate case of a more robust system, which add more fanciness in contrib at our leisure.

There is also, of course, distinct access control on each one of these possible operations. That can be sorted out later. Several people have also asked what happens if field-permissions are in play. After some discussion off-thread, the general consensus came down to "if you enable field permissions, PUT stops working and returns 403s, and you can deal with the consequences." Really, that's all we can do without changing the semantics of PUT, which we can't do.

Also of note, I don't anticipate PATCH to be supported in core at all. We're just carving out how it should work, and leaving it to contrib to implement.

For full HATEOAS/hypermedia support, I recommend following Lin's suggestion and not putting links in the body of the JSON-LD (or Atom, or whatever other format) but rather putting them in HTTP headers. That allows us to be consistent across different data formats, and also means we don't need to deal with links in the content itself.

Not addressed here is resource collections; vis, resources that are by definition just a list of other resources, like a node list. IMO, there should be no global list. Rather, creating context-specific useful lists should be the responsiblity of Views, which can put a properly-referencing contextually meaningful collection at whatever the heck IRI it wants. That's just a matter of providing the appropriate Views plugins, and then pushing buttons.

Although I'm talking about nodes here, any data entity can support this same model. Those that don't have revisions would instead implement PUT on their main IRI as above, and ignore the rest. That includes even config entities, which introduces all kinds of exciting possibilities.

Since we're heavy into development and on a short-timeline, I'd like to timebox this discussion to Friday 2 November. We can tweak until then, but after that we start coding.