Posted by MeadeRubenstein on March 6, 2008 at 3:47pm
Just joined this group.
I've been involved in the real estate business for about 10 years (various large real estate sites) and was looking for anyone that has worked on implementing MLS RETS/IDX integration into Drupal (or anyone that is interested in it) or has worked on real estate back office systems (Top Producer or Lucero - like). Having a site with listings and contacts is good, having integration and back office systems would make for a COMPLETE offering to Brokers......anyone here involved? interested? (any brokers/agents in this group?)
Comments
RE Broker - Yes
MeadeR -
I am a California real estate broker and though have not implemented MLS RETS/IDX, I am definitely interested! Let me know how I can participate/help.
http://tealeafmedia.com/
"Life is a fast strange trip, good thing we all have seats" (author unknown)
Where to start???
Being a project manager (yes, one of those), I'd like to put a plan together with some help - we can use a PM site maybe....but the min. we need is a broker/agent with an mls - without that there would be no way to test/develop. I've put a few in place, getting the mls info is not to difficult (perhaps a straight feed into the db), I'm relatively new to Drupal, so setting up a project group, coding, etc. would be a bit tough....
In addition to getting MLS info, I think we can add some neat functionality like the ability to load youtube videos for each listing, setting up newsletters with new area listings for subscribers, etc. (group think through the functionality).....have you worked on a module before?
About to build a RETS module
Hi,
I've recently been tasked to build a module to handle IDX and RETS MLS feeds. While I'm skilled with PHP/MySQL, I'm relatively new to Drupal, and I'm even more new RETS and IDX feeds! As such, I would love to collaborate. The only hang up is I need to get this done within the next month or two. How should we proceed?
I have started looking at an open source RETS client called VieleRETS (http://www.crt.realtors.org/projects/rets/viele/). Rather than reinvent the wheel and build a Drupal RETS client from the ground up, I've been thinking about either developing a wrapper for VieleRETS or somehow otherwise integrate it with Drupal. Do any of you have any experience with this? After poking around it quite a bit, it's pretty complex, but seems like a viable solution.
Viele
I haven't use Viele, but in the past I've use ezRets - http://www.crt.realtors.org/projects/rets/ezrets/ - same idea (for Windows based servers). The approach you are taking makes perfect sense, there's no need to reinvent the IDX/RETS interface. Just to be clear (some definitions):
MLS - Multiple Listing Service
IDX/RETS standard - not a real standard, each MLS implementation could (usually is) different
IDX - in most cases a mls will provide a for that you can iFrame into your site - allows searches, etc.
RETS - downloading the data directly
I highly recommend downloading the data into a database and then providing a front end to the data - there are a lot of fields in RETS and not all fields are used or needed, so you need to get the MLS's specific implementation info, see if they support Viele and what data fields they provide - you also need to know what fields your user wants and how they want to display them. Some MLS's require that you display the listing agent IN ADDITION to the broker/agent that you are developing the site for.
Next Steps:
Do you have the permission (and info) required from your user to access the MLS's data via IDX and/or RETS?
Have you spoken with the MLS to understand what they support (Viele?) and what display requirements they have?
What is your user (Broker? or Agent?) looking to do? simple listing display? feeds to other listing services (like Google? or Trulia?)
Could be that the tool you have will load the data for you, so you need to define the table structure and the module in Drupal would be focused on displaying and providing leads...
I could help build the tables and test the data feed while you develop the listing display with various search types (most important being location, price, beds and baths) - let me know
-meade
I actually dreamed
I actually dreamed (literally) up another solution last night that may or may not be a bit easier. I know a little bit of Ruby, and libRETS has native bindings for Ruby. So I could build a very small Ruby app (apparently the libRETS bindings make this very easy and straightforward) to handle making the connection to the server(s) and populate the DB, and have the Drupal module be the middle-man, fetching/displaying data from the database and allowing users to add new RETS-compliant access information.
My client is a green real estate company that is franchising, and I am building them a website that will allow their franchises to have sub-pages from their main site (primarily using organic groups and some custom stuff to meet their needs). Each franchise will likely be wanting to have their own local listings provided by some MLS local to them. They would like to be able to have searchable listings on the site for each franchise, and also be able to augment the MLS data with their own data - specifically "green" data. So I definately would like to capture info from the MLS maybe once a day, store it in our own database, and provide extra fields through Drupal for each listing where agents/brokers can add additional info.
I currently have access to one RETS-compliant MLS, and one IDX MLS (this is basically just an ftp service with zip files containing tab-separated data...). I would like to make it so that RETS compliant MLS feeds can be easily added by franchise or corporate managers through Drupal - the IDX feeds will probably need to be one-offs since they don't seem to adhere to any particular standards.
I was able to use VieleRETS to tap into the RETS-compliant MLS I have access to. Again, as far as the IDX one is concerned, that's a totally different ball game as far as I can tell.
Did this get off the ground?
Did this get off the ground? I am interested too. I have a reasonable level of php + sql experience an d could be interested in participating.
need a broker/agent
I think the biggest hurdle is getting an interested broker or agent - since we'll need them to gain access to the mls
test RETS servers available
I personally have access to some production-level RETS servers, although I think the nature of the licensing would permit me from sharing that info with other developers. That said, vieleRETS provides an open RETS server that anyone can connect to and play around with. There's nowhere near as many fields or as much data as would be on a real production server, but it's enough to play around and test with.
it's getting off the ground
Hi,
Sorry I sortta dropped off this group for a few weeks - got tied up with a bunch of other work. I've been mucking around with vieleRETS and started seeing how it might integrate with Drupal and am planning to start writing a module sometime in the next week. So here's my deal:
I have a client who is looking to franchise their real estate business nationwide, and wants a corporate site with sub-sites for the franchises that include MLS feeds. Users can search the feeds, save their searches and request that agents get in touch with them. They also want the ability to to add additional fields to the MLS data that come in from the feeds. We're looking to have this done by the end of May (!) and we're about 1/3 of the way done - our biggest hurdle being incorporating MLS feeds. vieleRETS seems to be the best pre-existing option written in PHP for interfacing with RETS-compliant MLS feeds (although my client also needs the ability to handle MLS feeds from idx/ftp).
My quick and dirty (idea for a) solution is to use vieleRETS to do all of the heavy lifting - adding sources, configuring targets and making the connections/auto-detecting server settings (my client and their franchises will not be needing to directly manipulate adding/modifying/configuring MLS feed connections, so I can keep it out of the Drupal picture for now) and then using a Drupal module to handle querying the RETS servers and sticking data into the database, creating nodes, etc. VieleRETS has a couple of files that handle the data downloading/updating (namely download.php, run_job.php and run_interactive.php) which look like I can include and use many of their functions, but will require some tweaking to work within the Drupal framework.
If anyone's at all familiar with vieleRETS (or wants to be!), it would awesome to fully wrap it's functionality into Drupal (as opposed to using my afforementioned Frankensteinian solution which I'm only doing in the interest of time). VieleRETS is EXCELLENTLY coded (and is beautifully OO) and can essentially serve as a library for negotiating a lot of the things needed to interface with a RETS server. So halb, if you're interested, I think that would be a wonderful place to help. Once I get down the road a bit with what I'm working on for my Frankenstein solution, I'll put it up on drupal.org and will very much welcome collaboration.
Cheers,
Arthur
interested in helping
I'm not familiar with vieleRETS, but I have 10+ years of experience with real estate sites and MLS's. Somethings I found to be issues with your approach:
.going nationwide will open you up to dealing with multi mls's, each having it's own dialect of the RETS standard
.MLS #'s can repeat - especially across MLS's (there's 900 mls's nationwide), occasionally within a MLS (data issues or resuse)
.some MLS's charge for the feed........
Let me know how I can help...
meade
meade.rubenstein@yahoo.com
Hi Meade, Believe me, I'm
Hi Meade,
Believe me, I'm keenly aware of the issues you raised ;) As to point 1, it's possible to pair two fields of what a feed returns in order to create a new unique field. In my database, I'm pairing the MLS's name with the MLS #. As to #2, I will be checking addresses for dupes (at display time). As to #3, it's true but not my concern - each of my client's franchises will be responsible for fees associated with a feed.
As far as vieleRETS is concerned, the folks at the RETS Organization have released a library of code called libRETS that is meant to aid (a lot!) in building RETS clients. The library has bindings for quite a few languages, but unfortunately not for PHP. I originally toyed with the idea of writing a client in Ruby or Python, but that seemed to be a silly thing to do when building something for Drupal. vieleRETS is a stand-alone application for fetching data from a feed. It stores the results of your queries in a variety of different formats, depending on what you want. The for vieleRETS is written in such a way that it can be used basically as an API - of course, there's not really much documentation, so it will be slow going. This is why for our first release I intend to only incorporate the data-fetching aspect of vieleRETS into Drupal. The rest of the work (negotiating connections, detecting server settings, etc.) will be left to vieleRETS.
Where would you be interested in helping? Do you have coding experience, or testing or general insight into real estate sites or...? As far as coding, I intend to just do the downloader part for Drupal. It would be great to get help (if anyone's interested) in incorporating the rest of vieleRETS into Drupal so there's a full Drupal solution for handling RETS-compliant MLS feeds.
Thanks much,
Arthur
help
I have some coding experience, but would probably be more helpful on the database side, testing and general real estate knowledge. Deduping by address could be a little tricky since you could have a multi unit condo with the same address (1 Main St - but various #'s). So, it sounds like you'll get the mls info and save into the Drupal database (or another - doesn't really matter) - and then link to any unique system data via the MLS name and MLS #? makes sense......
Let me know if you need help with the DB schema, testing, etc.
-meade
Interested in participating
I am interested in participating but can tell you from personal experience it will not be easy. I have played with Viele, EZRets, and others, and my MLS is a pain in the butt to deal with and have almost NO documentation or support.
I have access to my RETS feed, which is XML based RETS 1.7 provided through MLXChange (Marketlinx). I have been able to successfully query the feed for straight output thorugh searches but have not been successful in downloading to MySQL (my ultimate goal).
I am a real estate broker with a fair amount of knowledge of php, mysql, and drupal.
Damion Flynn
Damion Flynn
Real Estate listing options -
Hi Damion,
thank you for your reply. We are realizing that using a direct Drupal module or premade solution is not so solid of an option at this point. We are thinking on using open-realty.org with their IDX feed as a good starting point, although we are open to other suggestions- i see your brokerage uses http://www.netaspects.com/v4/ - how do you like this solution?
ALSO, we are looking for help in general with programming Drupal. Perhaps you know some programmers who really know their stuff. You being as familiar as you are, you are a great judge for quality programming.
thank you agian, and i look forward to your additional comments,
jeff
jeff@skysiteproperty.com
IDX is a good, solid option
IDX is a good, solid option and is easily configurable with Open Realty or Drupal as the basis for IDX are easily downloaded into a local database. IDX has been used for years and has plenty of tools available. I personally do not care for Open Realty as a broker with programming experience because I find it to be too limited for my needs, which is to build one of the most technologically advanced real estate brokerages in the country - I am already the top dog for technology in my MLS area and am the ONLY broker through my MLS with a RETS feed.
The thing that I like about RETS over IDX is that I have access to much more information and, in effect, could actually develop my own MLS system because I have access to much more information than a standard IDX feed. I think that you will be moving backwards if you develop a system in IDX instead of RETS. The upside is that IDX can be used with almost anything and easily plugged in to almost any system. I actually plugged in an IDX feed a few years ago to a WordPress blog - was quite easy and fairly well organized.
The current problem with RETS is that it is not really a standard per say, but more of an idea of a standard. It seems that every different MLS vendor has their own idea of what the standard should be and therefore makes it difficult to program universal solutions based on the technology.
As for my experience with NetAspects...
I was a franchise owner of SimplySOLD, which had several great ideas in the technology arena. Unfortunately, the corporate leadership lacked the experience and knowledge to push the company to what it really could have been and shut their doors a few weeks ago, leaving about a dozen franchise owners with very little direction and a lot of lost money spent into the vision. You can read a post about it at Tech Realty® (Opens in New Window).
Right before closing its doors, they set us up with NetAspects, I believe in preparation for closing. My experience with NetAspects has been very good. They are willing to go the extra mile to help you out, however, I think their backend is limited and could be vastly improved upon to really deliver functionality to the broker and agents. This is my ultimate goal. I also believe that their prices are too high for what they offer. This of course, is coming from someone who KNOWS that I can get most of the functions much cheaper in different systems and I believe the CRM could be improved upon greatly as well.
Other than that, for someone with little options, who wants a decent CRM, CMS, and MLS integration/lead management system, and does not mind paying the price tag, it works well for what it was designed for.
Damion Flynn
Damion Flynn
retracted comment
extra comment unnecessary
Short term fix
Thank you for your reply, Damion. Do you think using Open Realty with IDX is a good short term fix? we can get something decent up this week, it seems. I am most interested in your insights using RETS and some other platform that has more expansion possibilities over Open Realty. What do you suggest as a next step to considering a better solution?
I also appreciate your comment on the Deal Making site- i would think an integrated system between the Internet listing site along with the extranet deal making site would be the best of all.
Thoughts?
OR with IDX is a good
OR with IDX is a good temporary fix. If nothing else, it give the ability to have a slightly more custom feel than a straight iframe with a framed search url. OR does have some decent ability for customization (theming) - I just think the backend is a little limited and hard to work with when trying to add your own "modules" (for lack of a better word since OR does not work with modules).
I like the idea of integrating a RETS platform into Drupal - that IS the primary reason I joined this group, and the reason I found it (did a search for RETS + Drupal). I could easily code a search function and display pattern for listings, but where I am having a hard time is actually getting RETS data downloaded to a MySQL DB. After that it is just a matter of writing a few hundred lines of code.
What I would ultimately like to see is a RETS platform Drupal module with a "pick your RETS supplier" field that would automatically connect using the broker's RETS user/pass combination to download data locally. It could also be added to the cron page to be updated whenever the cron page is ran. Ultimately, having a "pick your supplier" tab would be the easiest way to go in terms of the end user, but also the most cumbersome to code. I cannot even figure out how to parse the XML data from my MLS, so I doubt that I would be able to do that aspect, but I would be able to get much of the other parts done.
Damion Flynn
Damion Flynn
is MLS integration key?
I found this site: http://www.retsbond.com/ - seems they have a simple/cheap solution to the MLS/RETS issue (getting data into the database). The more I'm thinking about it - the more I'm realizing that having the listings display is a small part of what a real estate site should be. Some ideas:
.listings search - yep...use IDX or another COTS - most people do not buy the house they find on the site
.local are info - it's all about relevance - if the buyer/seller can find your site and identify with you - you get a lead......
.honest feedback from clients - leads are 85% referrals
.newsletters/emails - stay in touch, keep it fresh, make it relevant
.responsiveness - more then 70% of leads are never followed up on...utilize some existing 3rd party tools to direct emails/calls around to the next available agent..and one that responds
I'm up for creating a site with the above focus for any broker/agent interested
RETS PHP class
I found this RETS PHP class that can be useful as a starting point. It's is posted on the RETS.org forums and the developer released it as open source.
http://forums.rets.org/showthread.php?t=1220
and
http://forums.rets.org/showthread.php?t=1208
It's also noteworthy that www.rets.org is a Drupal site.
cool! I am about to release a Drupal module
Very cool! I actually have a Drupal module that i will be releasing to the community at the end of may that serves as a wrapper for VieleRETS. essentially, you do all the configuration in Viele, and then run your batches from Drupal. The module creates each listing as a node and each associated image as a node, utilizing CCK. One of the notable things about the module over the original VieleRETS is that it allows you to store data from multiple feeds in the same table. Plus, you can easily augment the data with additional CCK fields, and customize how it all gets displayed (with views, for instance). I have the functionality complete and I will be adding some search functionality to the module. I am waiting until after the project I am working on launches (end of May) before I release this because I've got my hands too full to deal with setting this up on drupal.org, answering questions, soliciting help, etc. I will post again when the module is released!
Excited -
Please upload as soon as possible -- we are all ready to start using your module!!!
Module is UP! (but use at own risk ;)
Hi Everyone,
I pushed the module to Drupal last night, you can get it here: http://drupal.org/project/mls
Be forewarned - this is in ALPHA, meaning I wouldn't recommend it for production systems. I have been testing it for my own purposes, and am still not 100% convinced to make it live for my client - but it's very close. The module is admittedly clunky, but I needed to hack something together fairly quickly and this is what I've come up with ;)
Here's what it does:
* Integrates with the latest version of VieleRETS (1.1.6) to fetch RETS-based MLS feeds and turn individual listings and images into CCK nodes in your Drupal system
* Provides an API for IDX-based feeds with all (I think) of the functions you might need to turn IDX listings into CCK nodes, although you would have to write scripts to interact with your IDX system yourself (since there's not much of an IDX standard and every feed has to rely on one-off scripts, this seemed the best approach)
* Provides a search mechanism to search through available listings
* Includes a cron function to update individual feeds. Note that this cron function is independent of Drupal's cron mechanism (I had to do it this way to avoid weird memory issues I was encountering with Drupal's cron() function). In order to make use of this, you will need to create an additional script mimicking cron.php that calls the module's cron function rather than Drupal's cron call.
Here's what it doesn't do:
* Fully integrate with VieleRETS. You need to use VieleRETS to create your source, target and extract (for listings, map to cotnent_table_mls_listing, for images content_type_mls_image). You also will need to run your extract once through VieleRETS, telling it to download only one listing. Once it's done that, you will have a BCF created. Then, you will need to manually alter the BCF (in viele/batch_control_files/extract_name) to fetch the number of listings you actually want (0 means all of them). It would be great if anyone was interested in making this EASIER and fully integrated with VieleRETS (or found an easier way to tap into RETS servers w/o using VieleRETS that could be native to this module)
* Come with any guarantees
I could use a lot of help testing this, and if anyone knows PHP/MySQL/has experience working with Drupal modules, please help make this better and get it to a point of stability!
Let me know if you're interested at all in testing/coding.
Thanks,
Arthur
Any news from anyone?
I was wondering if anyone has contribute or tested this module. It seems that this one is one of the few idx/rets modules available on drupal. Please correct me if i am wrong, anyone has a list of different IDX or MLS or RETS Created in Drupal to use in an (obviously) real estate site?
Using an IDX feed
I am a broker associate looking for a Drupal solution. I see the discussion tends mostly toward RETS, but I can only get an IDX feed from my MLS.
Unfortunately, I am not enough of a programmer to decipher the data and map it. The feeds are in .zip files so what I imagine is they have to be unzipped then mapped. I think that would be handled by a CRON job, but again, I don't have those skills.
I have looked at one real estate module for Drupal, but documentation is basically non-existent.
It seems I would also have to get schooled in CCK. What are the chances of having a tutorial and a step by step process for implementation along with the module?
There are several reasons I want to use Drupal. One is to add a blog. One is to be able to add consumer information that would be branded to me. One is to have user login so I can capture leads. One is to be able to create my own search pages. I also think it would be great to give the capability for users to post comments.
Is there any benefit to you to have me aboard? I have been getting the IDX feeds, but so far they are useless to me since I haven't implemented them.
Hi Kyborn, At this point, I
Hi Kyborn,
At this point, I would not recommend this module in a production environment unless you /really/ know what you're doing. I've got this module working for a client of mine, but it's taken some serious work to get working right and it's pretty precarious. What would be really helpful for me is to have another active developer helping me out on this since my spare development time is almost 0 right now (hence no updates/improvements to the module, or documentation/tutorials/etc).
I could help you out to get this module working for you and provide you with training/support/etc., but I cannot do it for free. If you are interested in contracting me or want information on rates/estimates/etc, feel free to contact me via the contact form - http://groups.drupal.org/user/17140/contact
Thanks,
Arthur Richards
What's happening with the MLS module
I'm interested in picking this up and improving it. I'm currently working on an IDX system with CakePHP. I don't know Drupal, but have been meaning to get into it?
Is this module still actively developed, or is anyone else interested in working together on it?
Aaron Evans
one-shore.com
Aaron Evans
one-shore.com
The module is still listed
The module is still listed as under development but hasn't had a commit for quite a while and it isn't receiving much attention in the issue queue. You could always try it out and post patches in the issues for it. http://drupal.org/node/add/project-issue/mls
The other downer is there is
The other downer is there is no drupal 6 version, d5 is not an option for me and sooner than later we will be at d7, which I plan on moving to as soon as is possible. Apparently 34 weeks ago someone offered some money for a d6 version but nothing came of it.
I am going to be integrating the MLS possibly with RETS into our site, tho our mls also offers an FTP daily dump too .. im still not sure which would be the best, but AFAIK RETS is more timely with multiple updates daily, im leaning towards RETS so long as it allows my to fully customize the data display. I dont want those hideous mls searchs on my site.
Anyone get anywhere with this? Or want to team up?
Need some funding/dev help?
The PIRETS module for D6 is
The PIRETS module for D6 is still alive and well - I was working on it just as the notification for this message came in. I'm currently working on getting it up and running on another client's site. There's been some challenges; the differences in RETS implementations, of course, but also the fact that this client's RETS service is publishing over three times as many properties as our previous client's, so we're having some scaling issues - PHP timeouts when we're trying to do update runs and such. I'm currently poking at the code, trying to find ways to optimize the code and having some success, but unfortunately the real bottleneck is the RETS connection itself… I'll find a way to overcome the problems somehow.
Anyway, I'll hope you'll consider PIRETS an option.
The Boise Drupal Guy!
I was just about to post a
I was just about to post a job when you sent your message, how funny .. guess were operating at the same frequency today? lol
I dont see anything in the pirates namespace http://drupal.org/project/pirets
...is this going to be an open module or remain private? I'm cool either way.
Our local MLS uses InnoVia and has RETS, IDX and FTP deliveries.
I am certainly willing to consider jumping on the pirates ship, I'm not a fan of replication unless it produces a competitive advantage. If we can pool together perhaps we can obtain the best results for all.
Im going to send you an email.
Cheers!
It's open to the extent that
It's open to the extent that the GPL requires, but we haven't released it widely, no, mainly because the variance in the RETS "standard" from system to system means that there's a very low chance that the module will "just work" for any given MLS system. (And then the people that installed it will post in the issue queue asking for free help, but really the best way to diagnose problems is to have access to their server and RETS connection directly so I can run test queries and such, and assuming they actually are willing to give me that access, whose clock is all this sometimes difficult work going to be on, anyway?…) As I've adapted the code from our previous client to this one, I've tried to generalize the code so that it will require less adaptation in the future, but it will probably still require some.
Anyway, I forwarded your message on to my boss. Hopefully he'll get back to you soon.
The Boise Drupal Guy!
Makes good sense to
Makes good sense to me.
Thanks for sending the info to your boss, we are certainly willing to compensate as well as help wherever we can.
Im looking forward to hearing back.
Cheers!
Anyone got a Completed solution?
I am looking for a completed drupal design with an MLS feed that I can implement in a day not a month, or someone that can pull it togehter. From reading this forum, I am sure a number of people figured it out. As much as I like going through the design and implementation, I'm in the need for it rather rapidly. I need a prepackaged drupal install with a desent theme, or one that is easily customizable, all the modules and mls feed working or easily connected for South Florida. I need the design to also be user friendly for agents to have a virtual office, blog, etc. Of course best practice SEO. Anyone out there have this all put together or can do it fast? Please send me an e-mail at turismorealty@gmail.com
What do you have so far?
What do you have so far? Anyway can check it out? I would be willing to contribute. Do you have a drupal 6 version. I am not comfortable with 7 yet, as the contributing modules haven't caught up. Please e-mail me at turismorealty@gmail.com
What do you need
I am a developer and I have a real estate company. Can I check out what you have? and can you give me an idea of what you need to do to finish? Possibly i can contribute money and or work in exchange for you completeing and implementing the beta for my site. Let me know.
Imoria Please show me
Imora do you have a site up I can see? If so let me know. I would like to check it out. Please e-mail me at turismorealty@gmail.com