Architectural Brainstorming/Bikeshedding: Clustering on the data side or the display side?

Events happening in the community are now at Drupal community events on www.drupal.org.
Brandonian's picture

Hi all,

As Geofield draws closer to releasing an honest to goodness beta release in the near future, I'm starting to think about features for the next version of the module. One use case that I'd like to be able to handle with Geofield that it isn't really suited for at the moment is high-volume, highly dynamic display maps.

If you wanted to display a map with Geofield/Openlayers with thousands of potential points without custom code, you would probably end up with Geofield spitting out the data in a js array for the Openlayers library to handle the display. As you might guess, doing so is likely to melt the cpu of the client browser, since rendering the vectors of all your points/features is very costly. If you're smart, you'll turn on Openlayer's built-in clustering ability, which would display fewer points on the map, but would still be a huge burden on the client browser.

What I'd like to consider is a way for Geofield to handle clustering on the server side. I'm not 100% sure what that interaction would look like, but I imagine it could be done through either a pre-existing standard (WFS?) or some sort of re-rendered AJAX on page refreshes and whatnot.

I'm not 100% on what's been done, if anything, in this space in Drupal-land before, so I'd be interested in hearing other's thoughts, especially those with real-world experience on dealing with large datasets.

And... if you're interested in helping us push Geofield to beta and full release, please head over to the issue queue and test a patch or two!

Comments

quick thoughts

zzolo's picture

As far as the direction of handling good AJAX based communication between client and Drupal, this is key for sure: http://drupal.org/project/views_geojson Then it would be semi trival to hook up a way to pass the bounding box from client to Drupal views and only get the relevant data.

On the server side, well that is a harder question. There has been some work done, but I have never used it and dont know the details. Also I think its for D6 only.
https://github.com/ahtih/Geoclustering

Using WFS or similar is a much cleaner, more GIS centered approach.

Just some thoughts.

--
zzolo

Since Geofield already has a

Brandonian's picture

Since Geofield already has a way to format output as GeoJSON, it seems like the next step would be to add a component to views that handles the clustering algorithm, either ideally at the SQL level or processed at the PHP level.

Thanks for the Geoclustering link. I hadn't seen that module before, and I'm sure it'll be very informative towards adding that sort of functionality to Geofield.

wfs output, geoserver and leaflet

nottaken's picture

The WFS project is not supported, but might be worth looking at if you want to add that as a output. GeoServer REST interface is pretty nice. GeoWebCache as well, since it caches tiles and allows you to refresh just areas based on bbox and zoom levels when data is refreshed. Doing clustering at a high level, then client side as needed as you zoom. Leaflet seems to chew on geojson objects pretty quickly.

Location and Mapping

Group organizers

Group categories

Wiki type

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: