RSS & Aggregation

I've just contributed a new module for FeedAPI, it's called FeedAPI Field Inherit (http://drupal.org/project/feedapi_field_inherit). Here the project description:

Generalizing what the Feed Element Mapper module does for taxonomy, this module allows to automatically set fields in the feed items individual nodes by inheriting field values from the parent node.

Hi all,

I've been working on an add-on module for Feed Element Mapper that extract content from raw HTML contained into syndication feeds. It is called Feed Scraper, here its project page: http://drupal.org/project/feedapi_scraper.

As you can see from the screenshot it uses xpath queries to scrape out the information you are interested in. It allows to plug in other parsers as well, a regular expression one is already available with the module.

After defining a set of scrapers you are then able to map them to CCK fields using Feed Element Mapper. Since I've just released the first dev version every feedback, comment or feature request is really welcome.

Looking for a freelance developer that loves clean code and wants to make some extra side income by helping with development for one of our web projects. The site is already up and running, what we need is someone that can carry out some additional work and upgrades. This is a fun project for the entertainment industry.

Must have have strong familiarity and experience with:
- Standards compliant code
- MySQL & PHP
- RSS
- Custom module development

If you're interested, please contact me at: bsamii@inmotionrealestate.com

I have created a new site called Planet Drupal TV that aggregates Drupal video feeds in a manner similar to how Planet Drupal aggregates Drupal blog posts. Videos can be viewed in-browser or downloaded. There is a "most popular" section that utilizes the fantastic Radioactivity module.

It utilizes my Blue Droplet Video module and its Feed API integration.

I've added all the Drupal video feeds I could find, although it does not work with YouTube feeds as they do not use the enclosure tag. Miro can acquire video from YouTube, so it should be possible in future. Really, blip.tv is much better than YouTube, or you could use Blue Droplet which outputs the source video in the enclosure tag.

I'm building a community blog aggregator using Drupal and have tried out several feed aggregation modules. May be I'm missing the logic of doing this but none of the modules seem to fit my purpose. What I'd want to do is have "Most recent posts", "Most voted posts" etc (i.e. feed items coming not from a single feed, but from multiple feeds and then have them mixed according to the criteria I need). On the front page, I'd want to list the posts of the listed blogs as an when they are published, ordered by the time of their posting.

Hi guys,

I would be very interested in knowing your opinion and shall appreciate a little help concerning some functionality of RSS news feeds.

The idea is the following: my website visitors should be able to create a single database of RSS news feeds. Simply that could be a single wiki page with many RSS links. Then obviously when the link database becomes big, there will be too many news ITEMS to read. To solve this problem users could set up their own personal keywords, so that the module saves only those news items which matches the keyword.

I've just posted step-by-step instructions on how to use the Drupal 6.x RDF module to upgrade your Drupal site's RSS feeds for compatibility with RDF and the Semantic Web:

http://ar.to/2009/03/rdfizing-drupal-rss-feeds

As I don't do comments on my site, please feel free to discuss here.

I tried writing a FeedAPI Processor module to remove the Username: string that preceeds each item on a Twitter feed. To this end I copied code from the example processor posted on this group and also took the preg_replace code from the Activitystream module. However, it doesn't seem to work even when feedapi_node is enabled and placed at a higher weight (0) than my own processor module (-1).

I'm not very good with PHP or Drupal (just someone that understands a little bit of programming knowledge and does a lot of copy/pasting), so it's probably a fault with the code. Would really appreciate if someone could look at it and point me in the right direction. :)

Code after break.

I have a multi-site Drupal installation. Each multi-site is a subdomain of example.com (like ms1.example.com, ms2.example.com, etc.)

Now, the homepage (http://example.com) is not run by Drupal, but I want to insert a feed, which consists of a mash-up of several selected feeds of the subsites. And this new feed shall not have the URL of a subsite, but the URL of the homepage (where no Drupal is installed).

Example:

the feeds

• sub1.example.com/new-nodes.rss
• sub2.example.com/feed.rss
• sub3.example.com/blog.rss

Hi guys,

I am a bit lost and need your advise. I want to be able to simply filter all feeds by my keywords.

In other words, if I subscribe to many many news feeds (let say all available RSS feeds from a many many websites), I do not really wanna read all that stuff. Instead I would just like to set up my own keywords and filter or delete the rest of the feeds downloaded from all these websites.

I am using drupal 6. I have check already built-in module "aggregator" and "aggregation" module - it is not possible, right?

Thank you for advise.

At DrupalCon DC next week, I'd love to get together with other aggregation heads to discuss ongoing and upcoming work in aggregation modules in Drupal. Who's interested, what issue would you like to talk about?

Read more and reply to the BoF post on the drupalcon web site: http://dc2009.drupalcon.org/node/3834

I'm wondering what would be the best way to generate a feed (xml file) for data associated with a node. I have a custom node type with a cck imagefield that allows multiple images to be attached. Think of the node as an album. What I would like to do when the node is created is to generate the appropriate feed file needed by piclens, but I'm unsure where is the appropriate place to do that.

FeedAPI Node lets you aggregate feed items into nodes. What if you need slightly different behavior but still want to retain the most of the functionality of FeedAPI Node (save feeds as nodes, check whether nodes already exist before importing them, deleting nodes when content expires, etc)? Just create an additional processor, weight it "lighter" than the FeedAPI Node processor and enable it. Note: the code below will only work if another processor is enabled because it does not implement its own saving or purging of feed items. Also, it must be given a lower weight so that it executes before the FeedAPI Node processor so that it can modify each feed item before FeedAPI Node processor saves it.

This example addresses the problem of parsing the date of an aggregated item directly from its URL. This should work for items that encode the date in the URL in the following format: YYYY/MM/DD

For over a week i'm looking for a way to integrate a RSS feed in my site. The RSS feed is a podcast. There's also an open dir which i could use.

I use Feedapi & Views to show the feed. The feed is only showing the link to the MP3, not a Mp3-field. I tried several players and methods, but none of them worked.

1) How can i convert in Views a link to a MP3 field to a player?
2) which modules do i need definitely?
3) what is the shortest way (with not too many modules) to accomplish this?

Hopefully, someone here with some experience (and maybe an up&running site) can help me

Just wanted to get some advice on a content aggregation project I'm working on.

I'll be sucking content from multiple XML feeds (Atom and custom XML feeds). I'm going to use either FeedAPI or SimpleFeed. I haven't decided that yet, but I'm sure either will work just fine for the basic requirements. I'm using Drupal 6.x.

Hi there,

I'm using Feed Api to aggregate a few blogs and want to grab the photos that accompany the posts as well.

I've taught myself the PHP to get the files, parse them and return the image URLs and have it all themed and looking nice.

But I need to store this data with the feed items at the time of "capture" rather than calling it everytime I reload the page and hotlinking the images. Not cool.

Feed API appears to have hooks that will let me create a processor that can add specific scraping functions like this to specific feeds.

Sorry if this has already been discussed.

I have a website that would like to display RSS feeds, but the feeds are actually audio and video. I would like to aggregate the data and display the video, so a user would come to the site and see all the video's or audio and be able to just start the video or audio.

I'm working on an Aggregation site. I am using FeedAPI to pulling the feeds. I have a vocabulary for "tags" and "category." I assign each feed at least on "category" when the feed is created. I control the create of the Feeds and the "category". The "category" vocabulary also drives a menu on top of my page. I am using Feed Element Mapper to add tags from the Feeds to the "tags" vocabulary so I have no control over what gets put in it.

I am also using Messaging/Notifications to allow the users to subscribe to feeds and/or "category" vocabulary.

Looking to see how people handle output of views blocks (or otherwise) generated by feedAPI that have HTML markup in the feed.

I have a blocks that display a feed title and the teaser. However, the feed has HTML that the teaserfield isn't parsing. Things like breaks 'br' font-sizes etc are displaying as pure text in the field. Has anyone managed to avoid this? Possibly by mapping a teaser field to a specific CCK field of sorts?

Feedback welcome.

Hi there,

With a friend, we develop our drupal (v5) site, and use the contemplate module to customize the output. Everything is ok !

Now, I would like to add a RSS feed for all the users wishlist (please check @ http://tinyurl.com/68fpe3 ).
Each item should contain the title, description, picture, and direct URL.

I tried to activate the "Affect RSS Output" in the content template, but it does not work.
I cannot find the generated feed anywhere.
Also if I look into the source, it seems this page does not contain any RSS feed "embedded".

I'm wondering if there is a comprehensive guide for configuring FeedAPI and Flickr to pull in Flickr feeds to create thumbnails that link to content/nodes. Thanks in advance!

I've been working on an extension module / processor for FeedAPI (similar to feedapi_node) that does node deletions based upon incoming feed items parsed by FeedAPI. So, you can bring in and create new nodes from feed items -- why not be able to delete them as well!

I am currently matching on guid or node title -- the match type is configurable. Guid is pretty safe and should be unique. Also, you are probably not going to bring in and process a "delete feed" from an untrusted source.

Hi.
I have this need: use drupal as a rss remixer. That means I want to suscribe to feed1, feed2, feed3 and give "tag A" as a tag for feed 1 and 2.

Then I make a view that is the taxonomy term view for "tag A".
I want the RSS feed of this view to redirect title links and "read more" links directly to the original article, and not to my site.
But I can't, since the RSS style in views is not changeable.
I tried with "Contemplate module", but it will only allow me to change the teaser and body of the rss feed.

I'm working on a project that involves refactoring a content site while upgrading from Drupal 4.x to 5.x. One of the pieces I thought would be somewhat easy—displaying recent posts from blog feeds—has turned into a roadblock.

The desired outcome is to have the latest post from each of several external (i.e., non-Drupal) blogs show up such that the title of the blog links to the blog and the title of the post links to the post.

Let me describe where we started, what we tried, and where we wound up.

I am looking to import RSS feed items into nodes. FeedAPI and Simple feed are currently being looked at but wondering what the group's thoughts were.

Celsius Technology Group is building a platform to create and deliver content globally for multiple verticals including newspapers, schools, and government.
We are looking for php/drupal experts to help us develop and roll out this project over the next 3 - 6 months.
If you consider yourself a pro with php, drupal, and mysql, contact us at jobs1@celsiustechnologygroup.com

Guys,

bear with me on this one please, as this is simething I need to understand but don't know where to start.

I install a default Drupal site, I create content and promote and voila I have an RSS icon at the bottom of the front page allowing a someone to subscribe to the feed.

Alternatively I create new content types and implement Views to display said content and again voila I have optionally an icon at the bottom of the list (or whatever) which allows someone to subscribe to that subset of my sites data.

hello
im new in drupal. i looking for help. is there any aggregator module which gives possibility to add my own custom image to aggregator feed?? image. if there isnt ,any hints how can i develope this by my self? - its very important to me.
tnx for advices

I've proposed a session for Memetracker for the upcoming Drupalcon at Szeged.

Any votes for the session would be appreciated :)
http://szeged2008.drupalcon.org/program/sessions/introducing-memetracker

Jose Reyero blogged this morning about how he used FeedAPI to aggregate mailing lists:

http://www.developmentseed.org/blog/2008/jul/29/surviving-information-ov...

As announced, we've begun the Media Code Sprint to put better media handling into Drupal core!

It's been a great time so far. In the morning, drewish compiled a file function guessing game (free beer!), and andreiashu took the bait and completed the first round with great feedback about how unusable the current state is.

Next, we've sat down and begun writing SimpleTests for the hook_file patch. This has been great fun for me personally; though drewish is an old hand at building tests, this is completely new for me. And to top it off, webchick, Queen of Drupal testing, dropped by our table and got involved!

Even the process of writing tests has been helpful; we found at least one case for validation that had been missed in the original patch, and drewish decided that file_scan_directory needed refactoring. (He's currently chasing other problems as well, and cursing about finding himself going down rabbit holes.)

There's more fun to be had for all! If you want a quick 15 minute task that everyone is qualified to do, go play the File API Function Guessing Game. If you want to do more, then ping me (aaronwinborn), drewish, or dopry at #drupal in IRC, or leave a comment here!

(Cross-posted at AaronWinborn.com.)

The Media Code Sprint is underway! Here's a cross-post from my blog detailing the goals of this sprint, which runs through Saturday. We need your help!

Andrew Morton (drewish), Darrel O'Pry (dopry, remotely), and I are heading up a Media Code Sprint in Portland this week! Come help, in person or remotely, if you're interested in multimedia and Drupal! It has now officially started, and as I've volunteered to help keep folks updated, here goes...

First the reasons.

Number One: Better Media Handling in Core

Dries conducted a survey prior to his State of Drupal presentation at Boston Drupalcon 2008, and number one on the top ten (or 11) list of what would make THE KILLER DRUPAL 7 Release was "Better media handling".

Let me repeat that. Better media handling.

People have done really amazing stuff in contrib, but it is difficult (if not impossible in many cases) for developers to coordinate the use of files, as there is no good means for file handling in the core of Drupal. Thus, we have several dozen (or more) media modules doing some small part, or even duplicating functionality, sometimes out of necessity.

We need (better) media and file handling in Drupal core. In particular, there has been a patch for a hook_file in the queue for over a year, which has been in the Patch Spotlight (for the second time, no less) since May! (And has been RTBC several times during that process...) Come on folks.

One of the powers of Drupal is its system of hooks. We have hooks to modify nodes, to notify changes to user objects, to alter nearly any data (such as forms and menus). Noticeably absent is a consistent handling for files or any sort of notification. We need hook_file.

So goal Number One: get media handling in core. The means? Add hook_file and make files into a 1st class Drupal object. We'll be creating a test suite for functionality in the hook_file patch to validate it and "grease the wheels" to get it committed.

The other goals of this sprint pale in comparison to the first in utility, but are still highly desirable and worthwhile.

Number Two: Refactor File Functionality in Core

I wrote a piece on our blog on why I think it is important that we unite efforts in aggregation land.

My intention was to make some noise around Aron's patch to core aggregator and get some of you guys to review ;-)

I just entered a proposal for the session that Aron Novak and I would like to hold in Szeged: a presentation of a new extensible aggregator in D7:

Proposal:
http://szeged2008.drupalcon.org/program/sessions/new-aggregator-drupal-7

New aggregator:
http://drupal.org/project/new_aggregator

I thought some of the folks here might be interested in a project we're working on - it's called Rootswire - http://www.rootswire.org - and is an aggregation of different political and place blogs. Our first big effort is to build a group site for all of the bloggers covering the Democratic convention. The site is up in beta right now (yeah, yeah, I know - it needs a lot of theming love) and we're currently working on the mechanical aspects of it, and getting bloggers on-board. It's all Drupal, and we're using feedapi and the calais module to import the feeds and automatically tag them.

Hi everyone,

I've been banging my head a little bit trying to get this sorted. Basically, I want members of my drupal site to have their own personal blogs and a section where they can read other members posts. This is obviously no problem if they're all using drupal blogs on my site.

However, many of the members already have their own blogs. It would be great if instead of submitting blog entries on my site, they could just add the url of their Blog feed and entries would automagically appear alongside other members entries.

I ran across a new memetracker built on Drupal/OpenCalais and custom code. It looks very cool. Check it out at Polymeme.com

They're tracking some 25,000 blogs and apparently running them all through OpenCalias as part of the process for finding "memes" along with some human editorial touch.

The creator of polymeme writes more about the site on his blog:
http://evgenymorozov.com/blog/?p=397

Have a look at PHP skeleton code for better understanding this outline.

Def:
Parser: Responsible for creating a feed data structure which is expected by the aggregator (hook_aggregator_parser). By design additional parsers could consume everything (ical, html pages, emails)
Processor: Responsible for accepting items and save them, show them to the user, etc (hook_aggregator_processor)

Planned modules:

* aggregator
* aggregator_node
* aggregator_light

I am aggregating Feeds from various site to aggregate at my site. Many of these feeds provide complete post instead of just a summary/teaser. I would like to aggregate only the teasers of these feeds (instead of the complete feed), and let the users go to the complete feed if required.
Is there a way for me to show summary only? At the moment, if the feed it aggregate is a full feed, it will show the full thing as well. I want to trim down the aggregated content, say to only 250 characters.

Thanks,
Victor

The major new feature of OpenPackage Video 5.x-3.8 is integration with Feed Element Mapper (feedapi_mapper).

The development of this functionality was sponsored by Hudson Street Media using our bounty system:
http://openpackage.biz/bounty/15
http://openpackage.biz/bounty/14

Feed Element Mapper is an add-on module for FeedAPI that maps a feed's elements such as tags or (in this case) videos to taxonomy or CCK fields on your site.

This enables your site to aggregate videos from other sites.

I have set up various video feeds on our site as a demonstration. Click on the 'Feed item' links to view the videos.

I am very excited to announce the first Daylife Developer Challenge for building applications using the Daylife News Aggregation APIs. Folks hosting Drupal based websites on http://ecproject.org and http://comminit.com have built drupal pages/blocks/modules serving syndicated content using the Daylife APIs earlier.

The prizes are: a $3500 first prize and two $750 runner up prizes. The contest ends midnight of July 25th

I'm just wondering how to import mediawiki XML and turning it into a wikitools drupal system. This should theoretically be simple, but I'm not sure where to begin.

Edit 07 August 2008 Edit 07 August 2008 This proposal has been passed along to Knight Foundation for consideration. http://drupal.org/node/292587

Introduction

Edit, July 10 This proposal has been passed along to the Knight Foundation for consideration: http://drupal.org/node/281048 End Edit

This project will create two complementary sites: a Local Publishing Platform paired with a Regional Aggregation Hub.

The Publishing Platform will support a range of activities, from a K12 school magazine, a community paper, a writing project (something like the National Writing Project or Youth Radio), a college paper, etc.

Here I summarize what you should expect from core aggregator for Drupal 7.

As maybe you heard, my application was accepted to write a new aggregator for Drupal core :) : http://code.google.com/soc/2008/drupal/appinfo.html?csaid=1222E090E875B3...
Here as a first step, I would like to collect lots of possible problems in aggregation area. Please share your opinion about these questions to help Drupal 7 to have as good aggregator as possible :) :

The items are not necessarily ordered by relevance.

Node creation

Hi,

There are many people, who ask a possibility to filter feed items based on keywords in order to exclude them from processing (or possibly based on some other criteria like on timestamp, etc.). This issue has been raised under most aggregator modules and such feature request is in the pipeline of FeedAPI project as well:

FeedAPI:
http://drupal.org/node/160692

The usual recommendation for such problem is to relay on standard syndication and use Views module afterwards for filtering.

TASKS:
WFM-IGP is a non-for-profit organization seeking a short term off-site Website Consultant to assist with the overhaul and recreation of a website, and the launching of a site under construction.

The Website Consultant will work with the WFM-IGP Website and Database Developer in ensuring and implementing a transfer of the website www.ReformtheUN.org off of its current Xaraya platform onto a new platform (Drupal, Joomla, etc.). The Consultant will be responsible for:

• ensuring the layout of the website/CMS is logical and accessible