How to display 400,000 or more points in Openlayers 3 using less than 200MB of memory?
I created a separate map to test this for myself. I took a snapshot of the heap using Chrome Developer Tools, of the page on load and found that it was using 882MB of memory. I'm looking for a graph that contains lightning weather data and I would like the user to be able to interact with it, so Openlayers makes sense here. However, it takes up a ton of memory and needs a solution that is much more memory efficient.
Below is the code I used to do this:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<meta name="viewport" content="initial-scale=1.0, user-scalable=no, width=device-width">
<script src="https://cdnjs.cloudflare.com/ajax/libs/ol3/3.6.0/ol.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/ol3/3.6.0/ol.css">
</head>
<body>
<div class="container">
<div class="row">
<div class="md-12">
<div id="map" class="map"></div>
</div>
</div>
<div id="span12">
</div>
</div>
<script>
var iconInfo = [{
points: 4,
radius: 3,
radius2: 0,
angle: 0
}, {
points: 4,
radius: 3,
radius2: 0,
angle: 0
}, {
points: 4,
radius: 3,
radius2: 0,
angle: 0
}, {
points: 4,
radius: 3,
radius2: 0,
angle: 0
}];
var i;
var iconCount = iconInfo.length;
var icons = new Array(iconCount);
for (i = 0; i < iconCount; ++i) {
var info = iconInfo[i];
icons[i] = new ol.style.RegularShape({
points: info.points,
radius: info.radius,
radius2: info.radius2,
angle: info.angle,
fill: new ol.style.Fill({color: 'rgba(0, 0, 0, 0.9)'}),
stroke: new ol.style.Stroke({width: 2, color: 'rgba(0, 0, 0, 0.9)'}),
});
}
var featureCount = 350000;
var features = new Array(featureCount);
var feature, geometry;
var e = 25000000;
for (i = 0; i < featureCount; ++i) {
geometry = new ol.geom.Point(
[2 * e * Math.random() - e, 2 * e * Math.random() - e]);
feature = new ol.Feature(geometry);
feature.setStyle(
new ol.style.Style({
image: icons[i % (iconCount - 1)]
})
);
features[i] = feature;
}
var vectorSource = new ol.source.Vector({
features: features
});
var vector = new ol.layer.Vector({
source: vectorSource
});
var map = new ol.Map({
layers: [vector],
target: document.getElementById('map'),
view: new ol.View({
center: [0, 0],
zoom: 5
})
});
var overlayFeatures = [];
for (i = 0; i < featureCount; i += 30) {
var clone = features[i].clone();
clone.setStyle(null);
overlayFeatures.push(clone);
}
var featureOverlay = new ol.layer.Vector({
map: map,
source: new ol.source.Vector({
features: overlayFeatures
}),
style: new ol.style.Style({
image: icons[iconCount - 1]
})
});
map.on('click', function(evt) {
var info = document.getElementById('info');
info.innerHTML =
'Hold on a second, while I catch those butterflies for you ...';
window.setTimeout(function() {
var features = [];
map.forEachFeatureAtPixel(evt.pixel, function(feature, layer) {
features.push(features);
return false;
});
if (features.length === 1) {
info.innerHTML = 'Got one butterfly';
} else if (features.length > 1) {
info.innerHTML = 'Got ' + features.length + ' butterflies';
} else {
info.innerHTML = 'Couldn\'t catch a single butterfly';
}
}, 1);
});
map.on('pointermove', function(evt) {
if (evt.dragging) {
return;
}
var pixel = map.getEventPixel(evt.originalEvent);
var hit = map.hasFeatureAtPixel(pixel);
map.getTarget().style.cursor = hit ? 'pointer' : '';
});
</script>
</body>
</html>
Any suggestions on how I could achieve better memory efficiency?
source to share
Short answer
OpenLayers 3 uses about 2KB per point (see below), so while there is some optimization, you need to keep the number of features. 400,000 functions will require about 800 MB of memory.
Load your functions dynamically or use MultiPoint geometry.
Move the style from geometry to layer.
Longer answer
Style
When I tested, removing the style from this function and replacing it with a simple property reduced the memory footprint by 290V per function. See http://jsfiddle.net/vkm2rg46/3/ :
var vector = new ol.layer.Vector({
source: vectorSource,
style: function (feature, resolution) {
var i = feature.getProperties().styleId;
return [new ol.style.Style({
image: icons[i]
})];
}
});
and help the style function:
feature.set('styleId', i % (iconCount - 1));
Spatial index
You can set useSpatialIndex to false for the vector source. The source maintains a spatial index to quickly retrieve features to a predetermined degree, which apparently requires about 200-250 bytes per feature. However, dropping an index can have bad performance implications with this number of features.
Reduce the number of functions ##
It is best to load fewer features. There are several solutions.
Download on demand
This is most often solved by letting the server take care of the data and dynamically loading it when needed. You probably don't want to display 400,000 points at lower zoom levels, and users won't skate all over the place.
This can be done with vector tiles or with a normal vector source using bbox or tiles.
This can also be done client-side by creating functions / geometries from your own dataset in a vector source loader function.
Multipoints
A MultiPoint geometry with 10 or 100 points hardly takes up more space than a single point geometry. If you are grouping lightning strikes in MultiPoint geometry, memory can be hassle-free. However, you will lose some semantics and the ability to attach metadata to every single point.
JsFiddle: http://jsfiddle.net/vkm2rg46/8/
Memory usage
I created http://jsfiddle.net/g7qduy1w/3/ to check the memory usage of geometry, functions and memory source. You can take a snapshot at different stages (in particular, event listening data grows when adding geometry to the function and function to the source). With simple point geometry added to the function with no additional properties and added to the source, the memory usage for each function is:
288 B geometry event listener
424 B rest of geometry data
752 B event listeners
184 B rest of function data
261 B (share of total memory with using 100,000 functions)
source to share
Take a look at this example with camptocamp
http://dev.camptocamp.com/files/ol3/webgl/examples/webgl-points.html
It displays 100k points very efficiently!
source to share