HTML Scraping Libraries for RSS feed enhancement

Events happening in the community are now at Drupal community events on www.drupal.org.
tasumner's picture
Posted by tasumner on October 19, 2010 at 9:36pm

I am working on a project to pull in RSS feeds of events and provide additional information about them using HTML scraping. I haven't found any technologies that make the HTML scraping easier. Are there any smart libraries or modules I should be investigating?

Trevor

Categories:

Comments

QueryPath

Posted by coltrane on October 20, 2010 at 2:38pm
coltrane's picture

I like http://querypath.org/ for scraping HTML.

You point it at a DOM Document or string of HTML and can traverse it much like jQuery:

<?php
$qp = qp($html);
$table_cell_value = $qp('body')->find('td')->text();
?>

QueryPath

Posted by tasumner on October 21, 2010 at 2:55pm
tasumner's picture

Great suggestion. Thank you. We've installed it and are playing around with it now. Appreciate it!

T