Posted by adam_b on March 7, 2013 at 10:41am
I'm pulling together a site which needs to have maps that display a couple of thousand records. The current solution uses Getlocations/Views with Google Maps as the back end; it works fine up to about 100 records (see attached screenshot) but then slows down and falls over as numbers are increased.
Does anybody have useful experience in this area? I'm happy to consider consultancy fees; less happy to speak with agencies.
| Attachment | Size |
|---|---|
| sample_map.PNG | 171.1 KB |
Comments
You already use
You already use markerclusterer.js, right? One thing that always helps is to decrease amount of data that you request from the server via ajax and increase memory-limit of the server. Maybe you can split the request for the infoWindow-Content into two separate requests.
First, request all Markers with an id and lat/lon. Then on the infoWindow.open event request the url for the image and the title. This is just an idea. Starting with a preset country is of course another approach.
I hope that helps?
Did you give a try to some
Did you give a try to some server-side clustering modules? Examples are:
http://drupal.org/project/geocluster
http://drupal.org/project/views_geojson
The latter one states on its project page:
"Accepts bounding-box coordinates as an argument to filter the points returned."
But this module also requires you to use either Openlayers or Leaflet. I dont know if you have the luxury to change the mapping tool. I tried that module with OpenLayers and got some good results to some extend.
There is also a more complete solution: http://drupal.org/project/cartaro. This project includes PostGIS, Geoserver and GeoWebChange; all focusing on performance side.
Thanks for the feedback
@markaspot: Yes, we're already using markerclusterer.js. I understand your suggestion for splitting the info into separate requests, but I'm afraid that's beyond my ability.
@Sinan Erdem: I'd seen those before, but took another look. It seems that Cartaro is connected to Geops, who offer mapping as a service - so I'm talking to them and we'll see if they can come up with something.
It must be possible to map large datasets, because some websites manage it... but I haven't found one yet in Drupal.
There are a couple of options
First, are these points that need to be managed as Drupal entities? Are the lat/lon pairs changing frequently? If not, then I would recommend that you "bake" the points into map tiles with Tilemill and then host those tiles with MapBox. To help facilitate MapBox integration, my team just released http://drupal.org/project/mapboxjs.
If the lat/lon pairs need to be content managed as Drupal entities, you have a few options and issues to consider. Before looking into clustering options, I would first consider the overall performance of whatever javascript library you are using to render this vector data (ie, points drawn on the map).
In our experience, Leaflet is the most performant client-side mapping library. Check out http://drupal.org/project/leaflet. The following Leaflet implementation is loading approximately 1K points without too much of a performance hit:
http://theintertwine.org/explore?types[park]=park&types[trail]=trail&activities[8]=8&activities[90]=90&activities[73]=73&activities[71]=71&activities[72]=72&activities[11]=11&activities[17]=17&activities[70]=70&activities[16]=16&keys=
From there, consider optimizing the geodata that you're passing to that mapping library. Leaflet supports a number of data formats, each of which requires more/less processing client side. See: http://leafletjs.com/features.html.
Ultimately, this isn't really a "Drupal issue." The fact that Drupal doesn't support native geo extensions at the database layer is a weakness - but only in the querying of data and when doing advanced geospatial look-ups. Placing points on a map is as easy with Drupal as it is any other framework.
We have 3 use cases
We have 3 use cases:
- displaying the location of a record on its own node page - this is simple enough, and is working fine now
- displaying all records on a map on the front page - this doesn't change often and could be pre-generated
- displaying filtered results from a Solr search on a directory page - this needs to be generated on-demand (similar to theintertwine.org)
Thanks Seanberto - your suggestions look promising. And I wasn't implying that it was a specifically Drupal issue - it was just that I'd found other websites displaying large datasets on maps, but hadn't yet seen any built with Drupal.
You would need to tell us if
You would need to tell us if the problem is:
If its frontend performance, you already got many nice suggestions.
If its backend performance, I can tell you that Views 7 is unsuitable for displaying such many points, because it is loading all the entities fully via entity_load and such you have a problem in entity_field_attach and obviously memory.
A solution I came up with is called openlayers_quick_query (for OL), which just gets the results from the view and then does only display title and nid and does no other expensive DB load operations.
You can find it and the underlying idea in my sandbox here:
http://drupal.org/sandbox/Fabianx/1540306
Thanks!
Leaflet MarkerCluster + IP Geolocation Views and Maps
Sugest you use http://drupal.org/project/leaflet_markercluster in combination with http://drupal.org/project/ip_geoloc.
Works great for at least 5,000 markres.
See http://flink.com.au/tips-tricks/visitor-log-and-map (top image) -- note: current poor site performance is NOT due to Leaflet or Leaflet MarkerCluster!
ip_geoloc looks promising
I did a basic setup with 850 records and ip_geoloc using Gmaps, and it's performing well. Managed to get custom single markers (I haven't tackled leaflet_markercluster yet) but am stuck on how to apply the map styles (http://drupal.org/node/1940756).
as fabianx states, for large
as fabianx states, for large numbers of items, the server-side gets quite slow because views loads all the entities. my server-side clustering implementation for the Geocluster module provides clustering up to 100,000 based on MySQL aggregation and beyond 1,000,000 items using a Apache Solr implementation of the Geohash-basd clustering algorithm.
http://drupal.org/project/geocluster
filtering is easier than server-side clustering
anyways the setup is quite complex and i'd suggest working out more efficient methods by limiting the viewport of the map if you use case allows to do so. there is a variety of filter options provided by Geofield (proximity), Views GeoJSON (bounding box), Search API Location (support solr filters but is quite experimental, see [#1798168]) and possibly ip_geoloc (haven't use this module).
so in the end i'd recommend