What features of JSON-LD would we use in Drupal 8?

We should review which features of JSON-LD we are likely to use in Drupal 8 and what their status is, as some are still under development.

Since many contributors do not have a grounding in linked data and haven't been involved in the JSON-LD selection process, I start with the basics, which are pretty stable. I then then move on to the parts of the spec that are fluctuating.

1. Identifying things with URIs
2. Identifying properties with URIs
3. Explicit type handling
4. Language handling
5. Language version handling
6. Easy API to retrieve specific data

What's available with regular JSON?

JSON uses an entity-property-value data structure to communicate information. The entity-property-value way of modeling things should look familiar... it maps well to Drupal's Field system.

An example of entity-property-value:

And this can be represented with regular JSON.

For example:

The second two values in the table above (the article and the person) are actually their own entities which could have their own properties.

To represent this additional information about other things, we can turn the values into objects and add properties to those objects. This way the information is connected to the thing, rather than being represented as a property of Larry.

What more do we need?

The structure above is a nested set of objects, also called a tree. This becomes a problem when two or more parent objects have the same object as a property, as would be the case if two nodes had entity references to the same entity.

For example, Larry and I might co-author an article. In the JSON above, there's no concept of the article as an entity independent of its tree, so it is hard to refer to it from another place. We would need to duplicate the info about the article, once under each of us.

Modeling these kinds of graph relationships between objects, particularly between objects on different web pages and sites, is the kind of use case that Linked Data is really tailored for.

Identifying things with URIs

JSON-LD uses universal identifiers to make it easy to merge objects together. Specifically, the kind of identifiers it uses are links, aka URIs.

Entity URIs:

• Larry = http://www.garfieldtech.com/user/1
• The Kernel has landed article = http://www.garfieldtech.com/blog/wscci-kernel-merge
• Lin = http://lin-clark.com/#me
• Microdata article = http://lin-clark.com/blog/microdata-drupal-early-preview

(To the SemWeb pedants, yes, I am aware that the hashless URIs above return a 200 response, and thus as identifiers of NIRs are not compliant with httpRange-14...)

Because we use URIs for entities, it is easy to take two objects and merge them based on the entities' IDs.

This allows us to separate our data into sensible chunks. The object representing Larry could have the info that Larry knows me. The object representing me could contain the info that I wrote the microdata article. Since I'm indentified with the same URI in both objects, the the final graph would show that Larry knows the person who wrote the microdata article.

Here's what it looks like with Larry and I each handled within our own objects, using URIs so that the info can easily be merged.

Identifying properties with URIs

Similarly, because we are using URIs for properties (and using external, widely-used vocabularies to provide those URIs), it's easy to know that my 'field_name' provides the same kind of information as Larry's 'field_fullname'.

Property URIs:

• field_fullname, field_title, field_name = http://schema.org/name
• field_authored = http://xmlns.com/foaf/0.1/made (I use FOAF here because Schema.org only has the inverse relationship, $this->author)
• field_about = http://schema.org/about
• field_knows = http://schema.org/knows

Here is the full object. Note that the context has aliases for the property URIs. This means that we can use short attribute names as we do in regular JSON, but that we still have access to the property's full URI if we need to merge data from different sites.

ISSUE

We actually need to be able to use multiple property URIs for each term.

The primary use case for the WSCCI initiative does not require merging data from different sites, but rather is about communicating server-to-server within one site (the deployment use case) or server-to-client on the same site, or with custom built apps. We want developers to be able to do this without having to figure out which external vocabulary to use, so we will need to automatically generate a vocabulary for each site. This vocabulary simply exposes the data as it is modeled in Drupal.

However, for those who do want to share data with an independent service or a network of sites using a shared vocabulary (such as Schema.org), we need to allow the facility to map additional property URIs to the same term.

Mapping one JSON attribute name to multiple property URIs is not currenlty possible in JSON-LD, but I have raised an issue and it is likely to be resolved soon.

Explicit type handling

Sometimes it makes sense to explicitly give a datatype for a literal value. For example, if you have an Event entity type with a start_time field, you might use xsd:dateTime. However, one thing to be aware of is that many datatypes expect specific formats. For example, xsd:dateTime expects a variant of ISO 8601.

These datatypes can be assigned at a property level, so any value for this field will automatically be typed with the datatype.

For example:

You can also use types for objects themselves. For example, http://schema.org/Event can be used to indicate that the object is an event.

We would also create types in the site generated vocabulary which would correlate to the bundle (something like http://example.org/schema/types/node/article). If we want to, we could provide machine readable definitions of types at that URL, which would make it easy to reuse another site's content type definition.

Language handling

Language handling for strings is fairly straightforward. We can add the language in the @context attribute, and the language will be applied to all string values in the tree. For example:

Note that child objects can override the parent context. For example, the following snippet sets the language to German for a particular article:

Language version handling

A more complex part of Drupal's multilingual capability is the fact that a node can have different entity references based on which language is active.

For example, the German version of a node might be tagged with Term 1 and Term 2. The English version might also be tagged with Term 1, but not tagged with term 2. Another example of this is the reference to the node's author. If we have an article about the multilingual intiative, Wolfgang might write the German version and Gábor might write the Hungarian version. We want to be able to maintain that distinction.

Because language handling in JSON-LD only applies to strings, we can't depend on the language that is defined in the context. We have to find a different way to separate the English version of an entity reference field from the German version.

The approach that I would recommend based on the current spec is named graphs. Named graphs allow you to add a 'source' column to your entity-property-value relationships. For example:

Using named graphs, this would look like:

ISSUE

Named Graphs were only added to JSON-LD a few months ago, and some tools don't have support for them yet. It is also possible (though unlikely) that named graphs will be removed from the final spec. In addition, some parts of the spec have not been reworked to function with named graphs yet (specifically framing, mentioned below).

Named graphs also start moving us into linked data deep magic, and Drupal would depend on the (very few) necromancers who could debug, support, and maintain it. While necromancer status is flattering, it isn't healthy for the project.

Another potential approach is language maps, a recent proposal being developed to support the Wikidata project. This matches Drupal's internal data structure more closely, as the language switching happens at the field level instead of the entity level.

This is a proposal in flux, and the latest version of the proposal doesn't fully cover our use case, so I've done a bit of extrapolation to come up with this example:

ISSUE

The language maps proposal is still in heavy development, so it's unclear whether the final version will work for us or not, and also unclear what the timeframe is for standardization and library support.

Easy API to retrieve specific data

To provide a better developer experience, we want to have functions that make it easy to manipulate the data. This is particularly important if we have to use named graphs, which have a complicated tree structure that is unfamiliar to most devs. It is less important if we can use language maps.

The JSON-LD API's approach to this is still evolving.

• The first proposal had a toProjection function.
• The currently spec-ed approach is framing.
• An active proposal, which seems likely to either work with framing or supplant it, is objectify.

ISSUE

We need to consider how important this part of the API is to us and whether these approaches are workable for us.

Conclusion

I believe that we need to answer three questions before we move forward with JSON-LD.

Can we use multiple vocabulary terms for a field?

As mentioned above, I have filed an issue for this and it seems likely that it will be accepted. It is on the agenda for the JSON-LD community group's weekly telecon tomorrow, so we should know shortly.

How will we represent language versions?

Named graphs are currently speced, but the rest of the spec still hasn't been changed to accomodate their use. Plus, they are a pretty complicated concept. The current language maps proposal seems to fit our use case well, but it's unclear what the final proposal will look like.

How do we want to use the API?

The API for retrieving a specific subset of the data is still fluctuating. How important is this part of the API to us? Do we like the way the current proposals work?