Drupal Push, Or: How I Learned to Stop Worrying and Love the Rest of the Net

Events happening in the community are now at Drupal community events on www.drupal.org.
twistor's picture

Overview: Adding outbound methods of communication to Drupal. The services module exists to allow other services to communicate with Drupal but there is no easy way for Drupal to tell other services information about its own content.

Description: There are a number of services which would be beneficial for Drupal to implement. I think that building them as a set of actions would probably be best, as actions already provides a simple event mechanism for when things should be pushed out.

One example would be to implement different social network apis as actions. When someone posts a node, a link is automatically posted to their Facebook profile. This is not my favorite idea but one that's easy to imagine and probably implement.

More interestingly, having Drupal make (XML | JSON)RPC calls to another server. It would allow different servers to synchronize content without one having to poll one another. Providing a ui that allows an easy mapping of values and fields to ones that another server expects, as well as making sure that only the data that's intended to be sent is actually sent would be important. No data leakage.

The gdata api would be very interesting as well. Being able to aggregate your own Youtube feeds, having certain nodes be uploaded to Google docs or base, or syncing calendars with Drupal and Google Calendar would all be very useful.

The most AWESOME idea however, would be to provide STOMP support to Drupal. This would allow communication with other servers as well as direct page updates through Comet. There is a rather good Stomp client for php, one thing to research would be whether or not it would integrate well with Drupal or if a new one would have to be written. I know for a fact that it does 'work', I just haven't tested it thoroughly. The biggest dilemma in my opinion would be attempting to support the whole protocol, as there is a subscribe mechanism as well as a sending mechanism. I have written middle wear in the past that listens for Stomp messages from a message broker and then calls Drupal through the Services module, but that was tedious and more code to maintain. Trying to make these services abstract enough so that they'd be useful for a wide number of use cases would also be difficult. The results, however would be astounding. Drupal would be the first major CMS to support automatic page updates without polling.

The current state of the technology is frustrating with many half working implementations using incompatible technology. Dojo is trying with their Cometd, but for some reason they have four or five different server implementations. The best route to go in my opinion would be Orbited. I have had success using their stack in the past on other projects and it works well. Web sockets are suppose to be coming in HTML5 anyway so getting Drupal ready ahead if time would seem like the best idea. Just imagine it though, automatically updating blocks, or views without having to poll the server. It's a thing of beauty.

I realize that there are a lot of ideas here and to implement one of them well would be more important than all of them half way. I look forward to getting feedback on which ideas would be most valued by the community. Also, there are probably modules in existence that do some of these things and integrating them would also be a major goal. The benefit of Drupal would be to gather all of these disparate services together under the same mechanism seeing as how they all provide similar data just through different protocols. The bottom line is that Drupal is such a powerful CMS, it should be easier to share all the data that Drupal sites contain with the rest of the net.

Mentors:

* None yet

Difficulty: Medium

Comments

This would appear to overlap

mitchell's picture

This would appear to overlap with messaging/notifications, however the difference, as far as I can tell, is that messaging uses pull and you're talking about using push.

@twistor: Have you looked through the Messaging Framework code base yet? I think it may help the evaluators if you describe how your approaches differ.

here's a simple example

twistor's picture

Haven't gotten any feedback yet so I thought I'd post a simple example of some of the stuff I'm referring to. Keep in mind that there is NO polling from the client. There is an actual two way connection from the server to the browser. The actual stomp protocol is implemented over TCPSocket, which as the name implies implements a socket in javascript. There are other protocols already implemented such as, IRC, telnet, and xmpp. AJAX posts and the STOMP replies or notifies. I set up a stupid example at http://slyngit.com. There is a wall that you can post on that sends live updates back to the browser. As well as some rotating kittens. Outside of the actual module code I use rules to perform the STOMP send action when a node is saved.

One of the major design goals would be to allow it so that any block, or page could be updated automatically simply by checking a box on the edit page. There should also be an api to allow other module developers to do the same thing. Views integration would also be a major goal.

It's rough code but I think it demonstrates a few of the possibilities.

some work done already

boris mann's picture

arto bendiken did some STOMP stuff with his crazy RDFbus stuff. See http://www.flickr.com/photos/bendiken/2119345198/ -- I would get in touch with him for more info, I don't know what the current state is.

And yes, you definitely want to do publish / subscribe. Think multiple Drupal sites talking to each other.

Schedule

twistor's picture

Schedule:

First Half:

* messaging protocol api
o examine the needs of various messaging protocols to determine the requirements of api

* set of configurable actions
o configure messaging protocol destinations (ip address, port, protocol, and authentication credentials)
o select the fields that should be sent

* example protocol implementations

1. drupal to stomp
2. drupal to xmpp, look into existing xmpp framework
3. drupal to facebook

Second Half:

* real-time drupal through comet
o configuration UI
+ per node
+ per node type
+ per block

* integrate with drupal authentication to ensure that permissions are respected
* views integration
o look and relationship vs display types for configuration

* voting api integration for real time voting stats
* charts integration

Look at

mlncn's picture

Look at http://drupal.org/project/rdfcck for the mapping of fields

benjamin, agaric

SoC 2009

Group categories

Admin Tags

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week