Posted by Remon on December 28, 2010 at 7:34pm
Hi, yesterday I've released VARQL (SPARQL Views Query Backend), the module is intended to help in generating SPARQL queries using our Views lovely and intuitive UI.
VARQL utilizes Drupal 7 built-in rdf mapping in building SPARQL queries; for example if you want to select a couple of recored from DBedia, namely, name and homepage, varql will map these fields to foaf:name and foaf:homepage respectively to build a query
Any reviews or feedback would be greatly appreciated (especially regarding RDF mapping and views query plugin).
Comments
I like the idea behind the
I like the idea behind the module. It provides quick access to endpoint data, also for people less involved with the semantic web. I would allow an admin to delegate generating views to other editors and programmatically provide mappings for them so they do not need to care about writing SPARQL queries.
Two problems I came across when testing the module:
- Providing a possibility to mark fields optional would provide better results.
- When fields have more than one mapping assigned the module currently only takes the first one.
I still see a problem with the management of the mappings for different endpoints. Maybe for this module an endpoint specific mapping table would provide better results. I have always seen the rdf mappings in cck as rather output than input oriented.
Why did you decide to put these features into a separate module? The possibility to edit the auto-generated queries would be absolutely fantastic!
Thanks Dominik very much for
Thanks Dominik very much for your interest in varql :). Regarding your points:-
Providing a possibility to mark fields optional would provide better results.
Sure I'll do it, it is on the roadmap.
When fields have more than one mapping assigned the module currently only takes the first one.
It's a bit dirty for now, I'm planning to infer the exact predicate to use depending on the content type rdf mapping, e.g. If I'm querying fields of "Profile" content type (which is mapped to foaf:Person), then I'll use the pridicate that has rdf:domain Foaf:Person. Not sure if this is the best approach, please give me your feedback on this.
I still see a problem with the management of the mappings for different endpoints. Maybe for this module an endpoint specific mapping table would provide better results. I have always seen the rdf mappings in cck as rather output than input oriented.
Why did you decide to put these features into a separate module?
Actually, Drupal Entity <-> RDF is handled by RDF module, not VARQL, and SPARQL Endpoints module are meant to be a registry of endpoints to be used anywhere else (e.g. RDF, Sparql_views) or any other sparql based module.
P.S. please don't hesitate to create feature requests or bug reports on modules issue tracker :) thanks.
Overlap with SPARQL views?
From looking at the description of the VARQL module, there seems to be some overlap with SPARQL Views. I'll let Lin confirm this, but when Lin started to work on SPARQL views, she initially started on D6 because the Views API for D7 was very unstable, but with the intention to port it to D7 as soon as Views would be more stable. It would be unfortunate to duplicate some efforts here. Is there any way for VARQL to build on the SPARQL Views functionality? Maybe VARQL could focus on providing the RDF mappings and the UI for building the queries, and leave the Views integration for SPARQL Views?
This is interesting. I hadn't
This is interesting. I hadn't thought of this as a way to help people in forming the query. It could possibly be a really good way to help users develop the conceptual graph pattern without building the query themselves, though there may also be some use cases that it makes more difficult. I would be happy to discuss the pros and cons of different approaches more, it would be a great discussion to have for anyone developing SPARQL modules in Drupal. Also, adding an external SPARQL Endpoint registry is a good idea, though may not require it's own module (on a side note, I think calling it sparql_endpoint will cause confusion with the sparql_ep module and potentially cause namespace conflicts in the future).
I am concerned that having two SPARQL query backends for Views will confuse users. If there are two modules and the user starts using one, they might discover later that they need the other for some queries. I think the community will be better served if we work together unless there are major architectural differences. Since you are using hook_views_data_alter to add the fields as options, do you think that this different approach requires a separate module, or is it possible that there could be multiple ways for fields to be added in one module? Can you explain some of the other differences between VARQL and SPARQL Views besides the way the user adds fields?
Two SPARQL views modules...
Hi Lin,
I guess it is interesting to experiment in parallel on two modules, but in the long run it would be nice to see the merge. I find the VARQL ideas interesting for simple queries, but I would nevertheless like to have the possibility to define my own SPARQL query if needed. (I've seen that you prefer drush to web-admin interface, so no need to explain further...)
This said, wouldn't a SPARQL feeds parser (SPARQL query => nodes, and then we can use standard views functionality) be more convenient?
Btw, I enjoyed your video presentation.
It is indeed confusing to
It is indeed confusing to have two SPARQL modules for drupal (maybe three), and sorry if seams like I'm taking advantage of sparql_views being for D6 at the moment :).
What made me create VARQL is that I wanted users to do SPARQL in a more drupaish way. It is not that users won't ever write SPARQL queries, but it is like the case of SQL, which is extremely flexible and gives you full control of your queries, but still most of the users would choose Views module to write their queries.
I think I was a little bit hasty when I created the module without discussing collaboration chances, but I thought that varql is a completely different approach of building SPARQL queries (I admit that sparql_views' visual query editor is sexy ;) ). Anyway the module is still in early stage, we still have sometime to make sure that it really fits or if it is just a redundancy.
Regarding sparql_endpoints naming.... actually I couldn't make any better names :D. so I named it like this, at least unless I grab it into a separate module on d.o.
From my point of view it
From my point of view it would make sense to have a well though through mapping of node/cck fields which allows a user to specify both standard input and output mappings. In some cases e.g. an aggregator I would want to merge data which has different annotations, but essentially the same content, e.g. a title. With the current cck-based approach it would make me add multiple classes to each field which then also occur in my sourcecode. This again would make navigating my knowledge base and selecting classes from it more difficult for other users. Actually the vocabulary should model this behaviour and results should be inferred not explicitly modeled.
I really like the idea of standard-mappings as endpoints are not likely to change a lot over the course of time, whereas there's a lot of variance across the different endpoints. Additionally, a standard mapping table for drupal node fields and metadata could provide catch-all input mappings, although this might lead to problems in some specific cases again. If mappings from views could feedback to the cck based mappings, i.e. if i could also select classes corresponding to a view field, one would spare the time in switching between the admin-pages when mappings aren't finished yet. So, I think there would be a lot potential a "sparql_endpoints" module (or whatever the name will be ;)) which works together with sparql views.
On another note: This is not so much different from the idea of a localization client. One could even think of provding an overview on the standard mappings used throughout the community via a "translation" server. It would be interesting to see which vocabulary classes are used on which field-types. Are there any modules planned which would allow such things? I imagine this could serve as a could tool to provide/improve class recommendations aside from the standard classes used.
So I have been considering
So I have been considering advantages and disadvantages of this field approach over the current handling in SPARQL Views. It is really attractive in a number of ways, this kind of approach could solve two really big problems I ran into while approaching SV. There are still some disadvantages that I think need to be considered, so any opinions would be great.
Advantages
The current, dynamic way of handling "fields" in SV is a hack, as I note in the comments, so being able to use fields that are defined through the Fields system would be good. It would also make the Views maintainers wince less.
netsensei pointed out that he had a use case where he was pulling in URIs for videos. Instead of displaying the URI, he wanted to use a formatter that actually displayed the video. I haven't really poked around to see how this would work, but it is probable that being able to use different types of fields which have their formatters predefined would give you the opportunity to do this. This would be a BIG advantage.
As you said, negates the requirement to interact with SPARQL at all, which is a good thing. SPARQL Views UI still needs work and isn't a fully fledged solution for creating the query at this point.
Disadvantages
Because there is no feedback from the endpoint when you are creating the content types, the user has to know which predicates are used and in what relation to each other.
However, this disadvantage might be overcome through some changes. I think that it might work to make the current SPARQL Views UI a way of exploring endpoints and creating the content model. It could then have a button that says "Create content types" that generates code to create content types that will match the query. SPARQL Views UI needs work anyways, so architectural changes like that aren't out of the question.
As Dominik mentions above, the predicates you use to consume aren't always the ones you want to use to publish.
Above in advantages, I listed the fact that it allows you to use formatters. However, because different field types store their data differently, it might require bridge code for any field formatter. I'd really have to dive into code to understand whether or not this is the case. It also wouldn't be a huge deal, but one thing to consider.
A lot of queries traverse multiple relationships. This would require creating a new content type for every different type of entity in the query, which makes it very difficult to experiment with the query and also could lead to having tens to hundreds of different content types, if I'm figuring correctly.
@lin, your thorough analysis
@lin, your thorough analysis is extremely helpful I have to say.
Users have to really understand the content model of the endpoint
The idea of "Create content type" is quite interesting, but I'm not sure if it fits very well in a Views Query module. What if it could be made as a separate utility module, or as an extra functionality to RDFUI?
Predicates for consuming aren't always the same for publishing
I'm not sure if I understand this correctly.
We might need to maintain formatter code
it's been on my mind from the beginning, and I'm aiming to re-use existing formatters as much as I can, but I'm still not sure how to handle Video/Image or generally file formatters, since they need a file in drupal's files directory, in other words a file record (fid).
ould clutter Drupal UI with unnecessary content types / hard to experiment
We don't have to have one-by-one Bundle <--> Ontology mapping, I think that a normalised content type (a Profile content type for example) could handle multiple ontologies (FOAF, DOAC ...etc).
Predicates for consuming aren't always the same...
My take is, a given Drupal field could be published as a particular RDF predicate, but when ingesting RDF data we could map a number of different predicates onto that particular Drupal field. For example, all the variants of a person's name in multiple ontologies could map onto the single field "name" if the website owner so wished but the field "name" could get mapped only to the FOAF predicate "name" for publishing.
I'm sure that the above is a bit crude, and not sure if that's what Lin meant, but I reckon it's always good to be real flexible when it comes to mapping fields to RDF and back again. Hmm... I could also envision a scenario where different Drupal fields need to be mapped onto the same RDF predicate for publishing. So, we are talking potentially one to many mappings in each case for publishing and ingesting.
Cheers Daniel
The idea of "Create content
When I was writing that out, I meant to say stand-alone, but I forgot to say it. I think it would be better to have it as it's own administration area, it doesn't really fit into Views UI and I have to hack Views UI to make it work, so would make sense to extract it into it's own utility module.
If you use the same content type for queries against multiple endpoints, you will need to use multiple different predicates to generate the query, because different endpoints model things in different ways. Or you may not agree with the way that a particular dataset models something, but still want access to the data.
For instance, an example that recently came up on the Pedantic Web mailing list was that identi.ca modeled relationships between users as foaf:knows, even though users could be organizations and foaf assumes that the subject of foaf:knows is a person and that both parties actually know each other. So many people who use the data from identi.ca would not want to replicate the inaccurate modeling.
One option would be to have a set of predicates that is only used for consuming. If these aren't set, you could use the predicates that are used for publishing.
This is the kind of case where it would need custom bridge code. Also, other field types that aren't part of core need to be considered, like link, and that gets tricky. But I think it could be worth it, because it does allow for cleaner field and formatter handling.
I'm sorry I didn't make myself clear, I didn't mean a one-to-one bundle to ontology (or vocabulary) mapping... most linked data models, including Drupal's default mappings used for nodes, use multiple vocabularies in describing one object.
What I mean is that if you have a query that requests all songs by bands who are playing shows within a 20 mile radius of your city, you would need to create a content type for bands, for shows, and for cities to get the information you need. And there are queries that contain many more relationships than that. These would be one-off content types that would probably never get used again.
However, I think I may have found a good solution to that issue when I was looking for a solution to another issue. I saw a project called Micro (don't try downloading it with CVS yet, the developer messed up the CVS stuff, but initial code is available at http://drupal.org/node/986014). This allows you to create micro bundles that can be attached to a node. There are a few bugs with the code, so I can't try it out yet, but I'm assuming that these wouldn't show up in the node list, but that you could still attach multiple micro bundles to a node. So it would basically be like node reference and field group in one.
I can't help but agree with
I can't help but agree with you in all aforementioned points :). Seems that "formatters bridge" will give me a hard time, I'm working on it right now.
somebody know solution for hide a field's view by code?
which code and function (hook_ I think) is necessary for to hide a field's view directly by code php?
this isn't pertinent to the
this isn't pertinent to the discussion at hand, you can ask in IRC
Want to install it but where is rdfx?
I installed the VARQL module on a vanilla install of Drupal 7. Also installed Views and Chaos tools which are required, but I can't find a Rdfx module on Drupal.org. VARQL can't be enabled without it. Where can I find the required Rdfx?
Already answered at
Already answered at http://drupal.org/node/1019138 :)
How can I limit the VARQL query?
Hi Remon,
I have been trying out the semantic web capabilities of Drupal for a while, and finally got to VARQL yesterday. The idea of automatically building a SPARQL query based on field mappings of a custom node has certain appeal, but I'm stuck with query returning too much data. Take this for example:
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX db-owl: <http://dbpedia.org/ontology/>
PREFIX db-prop:<http://dbpedia.org/property/>
PREFIX db-res:<http://dbpedia.org/resource/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?species ?label ?genus ?binomial ?abstract ?thumbnail ?depiction WHERE {
?feline db-owl:family db-res:Felidae;
db-owl:abstract ?abstract;
rdfs:label ?label;
db-prop:species ?species;
db-prop:genus ?genus;
db-prop:binomial ?binomial;
db-owl:thumbnail ?thumbnail.
FILTER (
langMatches(lang(?abstract), 'en') &&
langMatches(lang(?label), 'en')
).
OPTIONAL {?feline foaf:depiction ?depiction}
}
run against DBpedia SPARQL endpoint, this returns a couple of information on felines. Works quite nicely with lins' module, but I don't see how to restrict the query to members of the Felidae family using VARQLE (?feline db-owl:family db-res:Felidae; part). I don't see how to handle multilingual fields either (FILTER part).
Actually, VARQL is not quite
Actually, VARQL is not quite useful until filters are implemented :S
OK, got the message
Thx, I thought that I may have missed the functionality. "Not implemented" is clear enough. :-)
This is great work, thanks.
Hi, new to drupal.org and just learning my way around views and tried out sparql_views and varql, now I'm learning both SPARQL and views :)
How do I create nodes for the views data, is this good practice ?
Yes it is good practice.
Yes it is good practice. Views data can be nodes (the basic unit of Drupal content), users, taxonomy terms, or other entities, including custom entities created using the contributed Entity API. Best work through some basic Drupal documentation or sign up for tutorials on drupaliz.me, buildamodule.com etc to get into the basics of Drupal before moving on to Views & SPARQL, then add Views knowledge on top of that.