Working on the improvement of the RSS/ATOM aggregation provided by Drupal.
Simple Pie Live
Has somebody out there played w/ SimplePie's Live service?
I would love to hear people's impressions.
Alex
Read moreFeed Element Mapper improvements
The Feed Element Mapper maps now not only standard RSS tags to taxonomy, but also any RSS tag to taxonomy or CCK fields.
Check out the screencast here:
http://www.developmentseed.org/blog/2008/jan/24/pick-it-feed-stick-it-no...
and a step by step guide how to build your own mapper:
http://www.developmentseed.org/blog/2008/jan/24/build-your-own-mapper-fe...
Read more
Spinn3r Crawler
I recently contacted spinn3r.com, and i will use their robot to spider the web, they provide a java client
I tried using the Feed URL with feedparser however the outcome was not great,
I want to try their java client
Does anyone know how to implement it
here are some links
http://spinn3r.com/documentation
http://code.google.com/p/spinn3r-client/
Read more
The Google AJAX Feed API for use with FeedAPI module
Has anybody bothered to take a look at the Google Ajax Feed API with the intention of using it for Drupal aggregation?
We've implemented client side aggregation for a project being released soon. This gives a similar experience to NetVibes.
Read more
Simple parsing to display story titles without saving info into nodes.
I need a vary basic functionality to display story titles from an external feed as a list.
Each user have their own "profile page" which is a custom content type created with CCK. The content type has a Blog Feed URL field. I would like to parse 10 newest items from the feed and display them on the profile page without saving them into nodes. The perfect way actually would be - having aggregation of the items into nodes optional. This way users could choose if they want their content to be aggregated here, or just shown as links on their profile pages to their own content on their sites.
Aggregation proof of concept site
Hello, all,
As part of some work we're doing, I put together a proof of concept leveraging the functionality within the FeedAPI and Views.
It's a pretty simple site -- all it does is bring in some selected feeds, import any existing tags, and add on some additional tags set on each feed as it is added to the site.
But it starts to get fun when you use views to organize the content based on author, taxonomy, etc --
I was also amazed at how quickly the site came together -- like I said, it's basic, but the basic config came together in about 6 hours.
Read more
SimpleFeed 1.0 ships!
Yes, it's finally out!
http://drupal.org/project/simplefeed
More details of what has changed over the past 7 months since I introduced it:
http://tedserbinski.com/2007/11/16/simplefeed-1-0-ships
Example site processing over 300+ feeds every 15 min: http://www.momblognetwork.com/
Enjoy!
Read moreFeed element mapper
Over the last weekend I started to develop the Feed Element Mapper - a Feed API compatible module that ultimately should allow you to map any item on a feed to any field on your feed item node. Currently there is an implementation for taxonomies.
Check out the screencast on our blog here:
http://www.developmentseed.org/blog/2007/oct/30/pick-it-feed-stick-it-node
and you can grab the module here:
http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/feed...
(I just upgraded it to FeedAPI 1.0 compatibility)
Read moreFeedAPI 1.0 beta 1 out now
I am happy to announce that after a week of cleaning up and improving the API and adding some critical features, yesterday FeedAPI 1.0 beta 1 version came out. These are the features:
Read more
Embedded Media Import
Embedded Media Field now ships with Embedded Media Import, for importing photosets, playlists, and the like. If you install that module, admins will have the ability to import photosets from Flickr as individual nodes. Just make sure the 'Allow import' checkbox on the content type (which must have a field defined with Embedded Media Field) is checked, proper permissions are given to editors, and look at the settings on the new tab at /admin/content/emfield/import.
Read moreParsing Enclosures
Since the gods of aggregation are all here, and I've exhausted all other options, here goes my question:
In our project, we are dealing with aggregated feeds that have media enclosures in them. Photos, the occasional video, etc. How can one parse these files and use them?
My related post:
http://drupal.org/node/180385
FeedAPI: release, documentation
Results:
- A release after the beta
- updated developer documentation (it changed a lot because of the concept changes)
- Screencast for users to make familiar with installing, setup and use the module
I have to admit, that i'm glad that the issue queue is full of feature requests, it seems that people want to use FeedAPI for various purposes. And yeah, their purposes are not impossible with FeedAPI (using the API, developers can write the proper modules)
Read moreFeedAPI developer's guide
Last updated by Chris Johnson on Tue, 2009-10-13 21:12
FeedAPI - there is something to install
Again, I got lots of valuable responses from my mentors and Morbus, they really kindly follow my work and tell the problems and the suggestions.
You can install the FeedAPI modules to any Drupal 5 site and you can do the following:
simplepie memory usage
I've noticed that on simplefeed and feedparser one runs out of memory if cron is set to parse a large number of feeds.
turns out this is due to a PHP bug http://bugs.php.net/bug.php?id=33595
SimplePie objects are full of circular and self references so basically no object gets destroyed until the end of the cron run, each feed you load and parse remains in memory until the end.
talking to developers @ #simplepie I don't think they'll be implementing any workarounds in the near future.
Read moreWhat are you using your aggregator for? A call for use cases.
It's a question that I wanted to ask long before. Kreynen's story of how the University of Nevada could use an aggregator for covering the Tahoe fire makes me finally post: What are you using an aggregator for?
Read more
Providing Coverage of the Tahoe Fire Without Reporters
As some of you know, I work for the University of Nevada's Reynolds School of Journalism. I’ve been working with their graduate program to develop OurTahoe.org as well as individual graduate student projects looking at the new ways "Web2.0" technologies can be used by journalists. Some of the projects resulted in Drupal modules like Promise.
Read moreDesign on SimpleFeed
I discussed the further way of the Aggregation API SoC project with my mentors and the best way is to use SimpleFeed module as a base. There is a page http://aggregation.novaak.net/?q=node/224, where I tried to match the requirements of the project against what SimpleFeed does now. In addition I tried to sketch how SimpleFeed works now at http://aggregation.novaak.net/?q=node/225 . I contacted with the author of SimpleFeed, m3avrck, we'll arrange the efforts on the module. Here I would like to make the detailed plans of the planned modifications on SimpleFeed.
Read moreAggregation API requirements - SoC project
Previously I compared some feed parsers and aggregation-purpose modules to provide an outline of the current solutions. It will be helpful for me at the start of the summer of code project. At the project website you can find the devlog and details of the process of the work.. At this page I would like to collect the features what an aggregation modules should know and decide what's worth to include in the API.
Read morePer User Feed Management
There's a possibility for some development sponsorship for FeedParser system to allow each user to add RSS/ATOM feeds for aggregation.
So to make sure any work done is useful for others too - I wanted to gather any input on the best way to approach the implementation.
I noticed there's a CCK field available to add a new feed to Drupals aggregation process. So this could be extended to support FeedParser's FeedManager with a feed-management API for adding, deleting and forcing an update of a specific feed.
Read more