PIRETS

Events happening in the community are now at Drupal community events on www.drupal.org.
Garrett Albright's picture

I first mentioned the Precision Intermedia RETS module in this thread. Basically, it's a module which can fetch real estate listings via RETS and import them into a site as standard Drupal nodes. Since Ferndale Real Estate has gone live, PIRETS gone through a great amount of refactoring and updating, so much so that I now call it PIRETS 3. Much of the updates had to do with making sure the module could scale to handle large amounts of listings (our largest client's site so far is managing well over 20,000 listings, about eighteen times as many as the original client has), but there's some sweet new features as well, including:

  • Everything is now a CCK field, including images. This means it's easy to do stuff like show everything using Views (including Views with exposed filters), enable faceted search on listings with Sphinx and the like, and use ImageCache to resize images. New feature: Locations can be stored in CCK Location fields, which makes it easier to do things like geolocation and adding maps to listings.
  • Batch API is now used when force-updating a category's listings. This can reduce the occurrence of PHP or browser timeouts or PHP out-of-memory errors when force-updating a category's listings, particularly for the first time when potentially several thousand new nodes need to be created. (However, it may still take a couple of tries the first time if you have a whole lot of listings in a category and no way to easily increase PHP's timeout or memory allocation config values. Update: Recent improvements with how the module uses the Batch API to do most tasks have almost eliminated these problems. Also, if you're using Pathauto which in turn uses Token, see this issue.)
  • Each property category can have a minimum "lifetime" before it is updated via a cron run. This means that you can have single-family properties update more frequently than commercial properties, for example. You can also easily disable fetching listings for properties you aren't interested in displaying.
  • If you're configuring PIRETS yourself, a "configuration checklist" helps to walk you through all the steps required to get PIRETS properly up and running and ready to fetch properties.

Unfortunately, none of the sites using PIRETS 3 with these new features are live yet, but when one goes live, you can bet I'll be bragging about it.

I have spoken to my boss about finally getting this up as a D.o project, but unfortunately he'd still like to recoup a bit more of the significant development cost before we do so. So won't you please consider allowing us at Precision Intermedia build you a slick Drupal-powered real estate site? Update: PIRETS is now available to the public! Check out PIRETS.info for more, well, PIRETS info.

Please feel free to post a reply or contact me directly if you have any questions - especially technical ones. I'm totally up for bragging about how I got all this to work.

Comments

It sounds pretty cool,

webavant's picture

It sounds pretty cool, unfortunately my budget is small, so I'll be sticking with Open-Realty for now. I hope PIRETS gets some sponsorship soon, gets the acclaim it deserves and gets a full GPL release in the spirit of all of the other wonderful Drupal!

is there a download for this

drupalninja99's picture

is there a download for this module? i was thinking of an mls module here: http://drupal.org/node/658662

Follow me on twitter: @drupalninja

Unfortunately, my boss has

Garrett Albright's picture

Unfortunately, my boss has not yet allowed me to make this module publicly available yet. I'd really like to some day, though.

Please contact us if you're interested in giving PIRETS a try.

this looks like what i wanted

drupalninja99's picture

this looks like what i wanted to create. you could solicit donations to help recoup some costs, if this does everything i want my proposed module to do http://drupal.org/node/658662 i dont want to waste time creating my own module. are you going to have a system for handling different implementations?

we are using TN realtracs right now and AR is switching at some point so I will want to support that as well, how are you handling that? Any sort of plugin system?

Follow me on twitter: @drupalninja

i think you could recoup

drupalninja99's picture

i think you could recoup costs farely quickly by soliciting donations for adding additional implementations and by offering support contracts for managed support, thats what i was planning to do w/my module idea.

Follow me on twitter: @drupalninja

The "support contract"

Garrett Albright's picture

The "support contract" approach is pretty much what we're doing at this point. These past few work days, I've been bouncing between three different clients, tweaking code, offering help, and tracking down bugs (thankfully a particularly nasty one recently turned out to be the fault of another module - yay, I can blame it on Somebody Else's Code!). Even though I'm the only developer on this so far, I've been using Mercurial (like Git, but sucks less </holywar>) to track changes and handle forks and such, which has been very helpful with regards to managing different tweaks and feature requests for different clients and then bringing all those changes back into a cohesive whole.

As for handling different implementations, all of the code which actually contacts the RETS server is in a single file, so theoretically it would be possible to "swap out" that file with one suited to talk to a particular server if need be. But the approach I've been taking now has been to try to drill down to a "common" RETS implementation that all servers can understand, and so far, I seem to have been successful in that regard; there's very few "exceptions" in the code to handle this bizarre implementation or that one (only one that I can recall at the moment for Rapattoni's implementation, and it's relatively painless). In fact, it's been the case that, often when I think a server is behaving exceptionally, it's actually just adhering to some aspect of the RETS standard which other servers (and therefore my code) previously ignored, and when I tweak my code to support it, it works just fine without upsetting the other servers - so the server is actually being exceptional in a good way.

However, this one-PIRETS-fits-all approach has meant some painful concessions - when PIRETS updates listings in a particular product class, it actually is re-downloading every listing in that class and then checking them against the listings in the database to see if they've changed (using a checksum approach so it's not that slow). It's theoretically possible in RETS to only retrieve listings that have changed since a particular time, but this approach was abandoned pretty quickly in PIRETS' initial development because so many servers don't seem to handle that correctly - they don't actually update the "updated" time when a listing is updated, for example. (With the download-all-listings approach, it's also easier to detect if a listing has been deleted or changed to a different sale status.)

looks like you released a

drupalninja99's picture

looks like you released a public version:

http://drupal.org/project/pirets

good for you i will have to check it out!

Follow me on twitter: @drupalninja

Real Estate

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: