springmeyer / landspeed.js Goto Github PK

The test inmediately crashes with an EPSG projection error.

Error: Projection error during map.zoom_all: failed to initialize projection with: '+init=epsg:4326'

This is because (at least on Windows 10) mapnik cannot find the PROJ4 EPSG projections.

In order to fix this, I downloaded proj446_win32_bin from the PROJ4 website, unpacked the zip file and copied the nad folder to my PC (actually, only the epsg file seems to be required). Furthermore, I had to `SET PROJ_LIB=C:\mapnik-v2.2.0\share\nad' before it worked.

It would be appreciated if you could add these instructions to the documentation.

otherwise failures will look like really fast tile reponses when testing. This is imperitive because we're now seeing layer constructors throw when max_size is reached in the postgres pool

For the style in this we moved to using the +init syntax to ensure that srs strings match exactly what is passed in from the url request.

But, if we did the translation between +init=epsg:3857 and the exact mercator proj4 literal (that we choose to use in the style) in the server this would avoid proj4 (inside of mapnik's proj_transform) having to hit its /usr/share/proj/epsg file to get the proj4 definition. This lookup only should happen a couple of times max per request, so it may not be a measurable i/o burden, but could still be avoided as a small optimization.

We should think about a good name for this server after the benchmark and when we publicly announce.

Just to document for now (no action needed):

• request for seed went over size
• gdal crashed with async map loading
• 16 thread pool is what I've been using

diff --git a/renderer.js b/renderer.js
index 73c8b02..279be48 100644
--- a/renderer.js
+++ b/renderer.js
@@ -5,7 +5,7 @@ var util = require('util');
 var Pool = require('./pool');

 // Increase number of threads to 1.5x the number of logical CPUs.
-var threads = require('os').cpus().length + 4;
+var threads = 16;//require('os').cpus().length + 4;
 console.warn('Using %d threads', threads);
 require('eio').setMinParallel(threads);

@@ -19,7 +19,7 @@ module.exports = function(args) {
     var maps = new Pool(function() {
         var map = new mapnik.Map(256, 256);
         map.bufferSize = args.bufferSize;
-        map.load(args.stylesheet, {
+        /*map.load(args.stylesheet, {
             strict: false,
             base: path.dirname(args.stylesheet)
         }, function(err, map) {
@@ -29,14 +29,20 @@ module.exports = function(args) {
             util.print('\rCreating map objects (' + created + '/' + args.concurrency + ')...');
             maps.release(map);
         });
+        */
+        map.loadSync(args.stylesheet);
+        map.zoomAll();
+        created++;
+        util.print('\rCreating map objects (' + created + '/' + args.concurrency + ')...');
+        maps.release(map);
     }, args.concurrency);

     return function(query, callback) {
         query.width = +query.width || 256;
         query.height = +query.height || 256;
-        if (query.width < 1 || query.width > 2048 || query.height < 1 || query.height > 2048) {
+        /*if (query.width < 1 || query.width > 2048 || query.height < 1 || query.height > 2048) {
             return callback(new Error('Invalid size: ' + query.width + '×' + query.height));
-        }
+        }*/

         var bbox = query.bbox ? query.bbox.split(',') : [];
         if (bbox.length !== 4) return callback(new Error('Invalid bbox: ' + util.inspect(bbox)));

After implementing polyline clipping, we've jumped another 10-15 r/s (on top of ~20 r/s jump from deferred label rendering).

I assume that such high throughput is now triggering some race condition. Ultimately mapnik crashes, but it is in a different place each time so I think the problem is the initial node assertion, which is seen each time this happens (around ~150 r/s):

node: ../src/node_object_wrap.h:105: static void node::ObjectWrap::WeakCallback(v8::Persistent<v8::Value>, void*): Assertion `value == obj->handle_' failed.
*** glibc detected *** node: free(): invalid next size (fast): 0x00002aaab02728b0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x2acf0ba7d45f]
/lib64/libc.so.6(cfree+0x4b)[0x2acf0ba7d8bb]
/opt/mapnik/lib/libmapnik2.so.2.0(_ZN6mapnik12agg_rendererINS_8image_32EE7processERKNS_18polygon_symbolizerERKNS_7featureINS_8geometryINS_6vertexIdLi2EEENS_13vertex_vectorEEEN5boost10shared_ptrINS_6rasterEEEEERKNS_14proj_transformE+0xfc5)[0x2acf319307b5]

@kkaefer - did you have a sample palette you used for testing you could add to the repo?

The mapserver mapfile I think uses some shapefiles in wgs84. Early on we discussed with the mapserver team about ensuring these would be in mercator for the main test, and then their will likely be a fully reprojected test (where map request will be in wgs84).

So, just flagging an issue that we may need to change the style first thing next week when I can get more info on this.

shapefiles should be from: http://mapserver-utils.googlecode.com/svn/branches/imposm-branch/data/

Nginx http round robin should be fastest in front of N node processes. Could also give cluster a look. Need to document as part of the project the optimal setup.