Views as Web Widget

rpfilomeno's picture

Moved to official ideas list at http://drupal.org/node/237626
GSOC 2008 Wiki site: http://groups.drupal.org/node/10984

Overview
Provide an Embeddable Widget option for Views aside from Page and Block. This will enable the view to be embedded to any website as Javascript, IFrame/FBML (Facebook support and similar), Google Gadget, or even Flash.

Version
Draft Soc 2008 (rev 2.1), 18/03/2008
* Added support for Service.module

Version
Draft Soc 2008 (rev 2), 18/03/2008
* Added support for OpenSocial SPI

History
Draft Soc 2008 (rev 1), 18/03/2008
* Formal Draft

Discussion of Ideas
http://groups.drupal.org/node/9633

The Emerging Platforms
FaceBook is probably the leading social application available but it only allows application building within its platform and it server side codes has remained unavailable to the open source community. Developers must also host their server side codes while communicating to Facebook via REST API and FBML to render the application view,.

Ning is close in second with more open approach compared to FaceBook by allowing developers to modify the underlying code written on PHP and provides hosting as well. Still most of the server side technology to provide data storage is kept under wraps therefore the code must still reside on the Ning servers for it to work.

OpenSocial is probably the most promising framework being developed but it still a long way to go and currently the efforts being done is focused on client side (the applications) and not much on the server side (the platform implementation to run the application). Right now, Orkut is the only sand box that runs OpenSocial applications and it's less than stable.

Views as Widgets will try to fill this gap for the mean time but the development plan will include the eventual integration with OpenSocial.

Finally, however enticing FaceBook and OpenSocial platform is, not all websites had this API nor has the site owners will always have the ability to build a platform to support them.

What we propose
Most of Drupal's usage is for presentation of content and aggregation. Before we syndicate contents as pure data using RSS/ATOM but with introductions of FaceBook applications, Google Gadgets and the likes has opened the concept of syndicated logic with data. What this means is site owners can now distribute how their data is presented as well as the how users will interact with their data.

Now think of Drupal modules as application; with Views as Widgets we are not just allowing syndication of content (data, including media type contents) but the modules (logic) themselves. In short any of the modules that has views or can be rendered as views will have the ability to be embedded on any site.

Why code embedding and not another platform.

If we look at what we learned in web history, the fastest way to syndicate application to another site is via cut-and-paste javascript codes -- remember when everyone use to have Tripod and Geocities websites with lots of cut-and-paste javascript menus.

For now we have to keep it simple while providing an easy way via API to extend/transform the widget to support platforms such as FaceBook and Google Gadgets.

We believe this project will live-up to the goal of Drupal to "eliminate the developer" in terms on application syndication.

Why join this project
Aaron made an awesome blog post to explain why this project is probably one of the coolest idea there is. Check out: Drupal will Explode your Site into a Million Pieces, and Why You Want That.

Proposed output
* API for building modules to integrate with existing modules such as Themes/Color, Stats, etc.
* Allows Widget builder modules, developers can create a module that output the widget code for specific social network. Example: FBML Widget for Facebook and XML for Google Gadget.
* Embed Shortcuts. Example: an "Add to Facebook" button or "Add to Myspace" button on Views that has Web Widgets is enabled.
* Customizable themes using templates and color schemes
* Provide automatic Javascript rewriting to resolve namespace collisions for sites using the widgets from multiple sites. This will also provide absolute location URL for contents, style sheets and scripts.
* Initial supported modules: Facebook module and Google Gadgets and Service.module
* API support for OpenSocial SPI for hosting OpenSocial Apps.

References
* http://en.wikipedia.org/wiki/Web_widget - refinition reference.
* http://drupal.org/handbook/modules/viewss - reference for extending view module.
* http://drupal.org/project/component - reference for providing theme-able widgets.
* http://drupal.org/handbook/modules/color - reference for extending color module.
* http://geekswithblogs.net/aguest/articles/4213.aspx - reference idea for javascript namespace collision
* http://www.dustindiaz.com/namespace-your-javascript/ - another reference idea for javascript namespace
* http://developers.facebook.com/documentation.php - FaceBook API
* http://code.google.com/apis/gadgets/ - Google Gadgets API
* http://code.google.com/apis/opensocial/docs/0.7/spec.html - OpenSocial API specification.
* http://drupal.org/project/services - reference for modifying services to support other platforms such as OpenSocial.

Mentors
Roger Filomeno
Aaron Winborn
Alex Urevick-Ackelsberg

Difficulty Level
Medium-Hard

Fun Rating

AWESOME!

Comments

Initial thoughts

rpfilomeno's picture

Initially when i had this idea is will it go against the efforts for OpenSocial?

We'll in my opinion, OpenSocial has still a long way to go before releasing RC1 and currently the efforts being done is focused on client side (the applications) and not much on the server side (the platform implementation to run the application). Right now, Orkut is the only sand box that runs OpenSocial applications and it less than near beta.

Views as Widgets will try to fill this gap for the mean time but the development plan will include the eventual integration with OpenSocial.

What's proposed that was never been done?

  1. API for building modules to integrate with existing modules such as Themes/Color, Stats, etc.
  2. Allows Widget builder modules, developers can create a module that output the widget code for specific social network. Example: FBML Widget for Facebook and XML for Google Gadget.
  3. Embed Shortcuts. Example: Add to Facebook button or Add to Myspace button under Views that Web Widgets enabled.

Please feel free to comment and discuss.

Great Idea!

Alex UA's picture

The only additional requirement that I would love to see added to this is integration with the Facebook Module so that you could serve up your widget either on your site or on facebook. I can pretty much guarantee that this would lead a lot more non-profits, and probably a lot of for-profits, to adopt Drupal, as it's one of the bigger requests that I receive when I build a site for orgs or candidates who are running campaigns or doing outreach aimed at a younger crowd. This also might be a good first step towards open social integration...

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

That's the idea

rpfilomeno's picture

Precisely, as most of Drupal's usage is for presentation of content, aggregation. Before we syndicate contents as pure data but with introductions of FB applications and Google Gadgets, logic is also syndicated.

Now think of Drupal modules as application, then with Views as Widgets we are not just allowing syndication of content but the modules themselves.

Finally, however enticing FaceBook and OpenSocial platform is, not all websites had this API nor has the ability to build a platform.

If we look at what we learned in web history, the fastest way to syndicate application to another site is via cut-and-paste javascript codes -- remember when everyone use to have Tripod and Geocities website with lots of cut-and-paste javascript menus.

Anyway, I believe this project will live-up to the goal of Drupal to "eliminate the developer" in terms on application syndication.

You got my vote...

Alex UA's picture

I would definitely be down to help mentor this, though I'd have to make sure my mentoring "partner", Aaron, was down.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Component

agentrickard's picture

I think the Component module already provides this framework. And I have seen other work in this area.

Some student research is needed.

--
http://ken.therickards.com/
http://savannahnow.com/user/2
http://blufftontoday.com/user/3

Might be a good starting point.

rpfilomeno's picture

This might be a good starting point.

However the entire theming feature will be needed to be included on the widget. Let say your site is using Garland theme with color scheme 'Ash' then the Widgets created will be is 'Ash' color scheme -- the issue if the widgets should follow the site theme or use color.module by its own is up for discussion with students on what is realistic to be accomplished.

Actually I'm more concerned with views containing dynamic contents like with media.module or voting.module. Most of the module's javascripts cannot be included into the widget directly since it may cause conflict with the aggregating site's javascript (namespace collision).

Example: both are Drupal sites with voting module, one is serving a widget while the other aggregates but the same time has its own local content using voting.module.

Actually this is a very common problem with widgets; even with Google :) -- you cannot have 2 adsense script with 2 different publisher-id

So the project might need to handle also re-writing javascript namespaces X_X

Anyway well cross the bridge when we get there -- we don't want to scare the students early-on, do we? :D

Are you writing up a formal proposal?

Alex UA's picture

I certainly hope so! Also- I talked to Aaron and he's definitely interested as well, so if this gets accepted there will be some mentors ready to go!

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Great!

rpfilomeno's picture

Yup I'm starting a formal proposal, even if it doesn't get into GSOC im planning to build it anyway -- its too good to pass. I'm preparing a sandbox at the moment.

We'll just open up a wiki page for the formal proposal as soon as we get a go signal; I'm happy to have you guys along as mentors!

Thanks for the support!

Count me In!

Time for a chat?

rpfilomeno's picture

Hi Guys,

Scratch my last email use my GTalk address rpfilomeno-at-g-mail-dot-com instead (the other one I sent privately gets spammed frequently), we can have some preliminary discussions.

What's the best time for you a chat? My timezone is +8:00 (Philippines), but I'm usually available through the night here from 3PM to 3AM
:D

I got the mentor invite

rpfilomeno's picture

Hi Alex and AAron, I just got the invite . I also revised the original draft proposal with a more complete one according to the discussions. Also named you both as co-mentors.

Do you think this draft is good or did I miss anything?

Regards.

OpenSocial SPI

rpfilomeno's picture

Ive been experimenting with Shindig (http://incubator.apache.org/shindig/) and i think its possible to fit OpenSocial apps hosting into the project scope with just the basics implementation to pull user information.

So the measure of success for the project at the end of SOC is it should be able to transform a view into a GoogleGadget XML, Iframe HTML code, Javascript code, and OpenSocial XML (which is pretty much like GoogleGadget XML).

See XML sample:
http://hosting.gmodules.com/ig/gadgets/file/117247905274371511495/Social...

Then I realized Drupal Views has export features where we can set exporting a view will also export its Widgets -- and this is what OpenSocial really does!

More stuff on Shindig at: http://trac.hyves-api.nl/hyves-api/wiki/ShindigStarted

Panels as Web Widget

ChrisBryant's picture

I'm not sure how you are planning to implement this, but I would recommend naming it and working with it on a higher level, possibly at the panels level. In panels you can put in any node, view or block so it would make sense to support panels rather than views specifically. Maybe the way you are planning to implement it this would be possible but I thought I would mention it as well. That would allow a user to embed any panel, panel pane or mini panel directly as a widget.

Also, for reference have a look at what Ted Serbinski did for the Mom Blog Network. I'm not sure if anything is released for that of if it applies but it's worth looking into:

http://tedserbinski.com/2008/02/13/mom-blog-networks-drupal-widget-system

--
Gravitek Labs

Roughly this is how it works.

rpfilomeno's picture

Panels may be too large to be embeddable, plus using panels takes away the capability of the embedding site to decide how to layout each block.

As for how it will be implemented:

For example we created a CCK content called "shops" with custom field "type" and enablea 5-star voting module. Then we created a tease type View for list of "shops" with exposed filter "type".

When Web Widget option is enabled it render an embed code as Iframe, FBML, Javascript, or Google Gadget / Open Social XML. It will also create a REST url using Service.module and a auto-"Path" as the javascript source path.

The auto-"Path" generates all the needed script for loading the View's html code into the embedding site, it will render the rewrite of the javascript for 5-star voting javascript to prevent namespace collision.

When the script is loaded it will call the REST service to fetch the items list to be loaded, it will also provide function for handling basic paging and exposed filters while the 5-star voting will also work normally.

However when you click an item to view the full node, it will redirect the user to the original site (just like rss).

As for FBML or Opensocial, you can install contrib modules to support them such that post-fixing the auto-"Path" with /facebook or /opensocial will change its output at behavior accordingly.

Of course this will also have to work with caching and themes -- hopefully we can fit that to the timeline.

I am interested in working

jtsnow's picture

I am interested in working on this project. I will be posting an application later today.

Awesome!

Alex UA's picture

Looking forward to reviewing your app. Please post an update here so we know to go check it out.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

I have posted an application

jtsnow's picture

I have posted an application to the GSoc website. Any feedback will be appreciated. Great idea for a project! :)

Link to the Application

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Thank you to everyone who

jtsnow's picture

Thank you to everyone who has provided feedback so far. If anyone else has any more comments before the deadline, go ahead and post them!

Also, thank you for the excellent and detailed proposal.

What would be the best way

jtsnow's picture

What would be the best way to incorporate flash based widgets? I have seen a few Flex applications/widgets that are powered by Drupal. It doesn't seem too difficult to do using the Services module and AMFPHP module together. Any thoughts?

Synergy effects

alex_b's picture

What's the overlap with this proposal?

http://groups.drupal.org/node/10188 - can you use parts of what Allister is going to work on?

Its a bit Different

rpfilomeno's picture

Views as web widgets aims to provide a container (view views) and services query/data source (via services) while the other project wishes only to provide a data-source via views. So depending on what type of Widget is created it also needs to create the corresponding services.

If Allister's project were to allow publishing the "data source" via services module rather than as a view then Views as Web Widget can reuse that module.

I inquired to Allister if

rpfilomeno's picture

I inquired to Allister if the project he's working with could provide a support to services as well for a possibility of re-use.

See

Being Evil: SWFHttpRequest

rpfilomeno's picture

I found an easy way to address cross-domain Ajax requirements for this project using SWFHttpRequest. However due to the possible "evilness" of cross-domain Ajax maybe it will also need to implement dynamic/generator crossdomain.xml

Any thoughts?

Thought about NetVibes integration?

redndahead's picture

Netvibes, http://dev.netvibes.com/ , has a great widget platform that handles numerous services. It may be worth looking into as an output format. Hopefully I understand your project correctly.

I doubt this will happen this summer...

Alex UA's picture

...but we are stressing the need for an easy-to-use API as a central part of the application, so it should be relatively easy for you, or any other developer, to create new widget provider types that work with the project.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Amazon's picture

Take a look at Ringside networks platform for insight into how to make social network widgets work off platform.

http://www.ringsidenetworks.com/

Cheers,
Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

One of the problems you're

ebeyrent's picture

One of the problems you're going to run into when dealing with the flash widgets is the crossdomain.xml file. I ran into this one myself.

In order to get Flash to consume web services/views from any domain, you need a promiscuous crossdomain.xml file. This unfortunately exposes your entire site to flash applications to be able to read - personal user data, private messages, etc.

There seem to be two ways to deal with this: you can either do some mod_security trickery and only allow flash widgets to connect via ip address; or you have to split out access to your services to a subdomain, such as api.yourdomain.com. This is what most other sites have done.

The problem then becomes how to split out access to just /services/amfphp to the subdomain. I've explored using a "proxy" mechanism on the subdomain to only pass valid requests onto the main domain.

I'd be really curious to know how you plan to address these issues.

Its the other way around.

rpfilomeno's picture

Project aims to provide embeddable widgets, therefore flash versions of the widget will be loaded by the third party site and will consume the data from a Drupal site and not the other way around. The module can automatically create the crossdomain.xml (or with use of clean url rewriite this into index.php?crossdomain.xml request) for this purpose. For consuming third party data, the module must act as a proxy for the widget.

I guess it has been started

gausarts's picture

I guess it has been started => http://drupal.org/project/web_widgets

laught, light n laughter