I've been recently inquired for a new project where a large number of geo-referentiated items should be placed on a map. In detail, there is a large dataset of a few hundred thousand equipments (around 0.5 million) that must be represented on a map and searchable on that same map. Each equipment is considered to be a full node with it's own data (manufacturer name, model, year built, ...).
These equipments can be grouped by some criteria, but it may happen that the users request the visibility of a huge quantity of them at the same time.
From my experience with OpenLayers we can use this "grouping" feature to organize them into several layers, but on the other hand every time I have a big number of elements on screen (a few hundred) it gets almost impossible to navigate, pan or zoom, as the system gets really slow. I can not imagine this working with such a huge amount of data points.
Has anyone had an experience with a similar project? and can anyone provide me with some guidance on what may be best approach (modules to use) in this situation?
Comments
Clustering
It sounds like the main thing you'll need is clustering. The GMap module comes with different types of clustering options. Might be worth taking a look at.
I'm working on something similar, but not quite as huge. What I want to do is implement clustering, but then have a view open up when the cluster is clicked that shows everything associated with the cluster. That is not something built in to the GMap module at present.
Hope that helps.
-Mike
Tilemill
Alternatively, you might want to "bake" the data into the tiles rather than attempting to display the data dynamically. See http://tilemill.com/manual.html#adding-layers for a very basic introduction to this. Eric also touched on this when he spoke about it at DrupalCon Chicago, the last 10 minutes of the video of this session: http://chicago2011.drupal.org/sessions/making-beautiful-maps. Definitely not an easy solution though.
http://www.thomasturnbull.com
http://twitter.com/thomasturnbull
I'll check both suggestions
Thank you both for the good tips. I'll have to check them both to evaluate the feasibility regarding this project. I'll give feedback as soon as something gets rocking...