mapzen / chef-mapzen_odes Goto Github PK

as described in #5

Users are seeing errors due to missing IMPOSM SHP output for Cardiff, Wales: https://mapzen.com/data/metro-extracts/metro/cardiff-newport-bristol-bath_england/ (error output due to missing upstream file).

Other Cardiff files are present on S3, just not that one. I checked with @sleepylemur and he suggested that an issue filed here assigned to @heffergm was the right step to fix this.

Right now, it appears as though empty IMPOSM output is not included in the zip file. For example, the Victoria extract is missing barrierways and barrierpoints downloads, even though they are defined in https://github.com/mapzen/chef-mapzen_odes/blob/master/files/default/mapping.json

If that's the correct mapping that the live extracts are using, can we get all shapefiles included even if they are empty? This will help the documentation be a bit clearer.

From a Twitter support request we're currently exporting too much precision in our GeoJSON files for Metro Extracts.

• https://twitter.com/jestferfun/status/895381713391542272
• https://twitter.com/jestferfun/status/895381844178354176

Reviewing the support request it looks like we export default of 15 coordinate precision now when 7 is plenty. This will result in smaller file sizes, a win for everyone – and the original OSM data is accurate to 7 decimal places so we're not loosing precision.

The 15 precision is coming from OGR's GeoJSON driver defaulting to 15. But it's possible to set that to 7 two different ways:

• http://www.gdal.org/drv_geojson.html

COORDINATE_PRECISION = int_number: (OGR >= 1.9.0) Maximum number of figures after decimal separator to write in coordinates. Default to 15 for GeoJSON 2008, and 7 for RFC 7946. "Smart" truncation will occur to remove trailing zeros.
SIGNIFICANT_FIGURES = int_number: (OGR >= 2.1) Maximum number of significant figures when writing floating-point numbers. Default to 17. If explicitly specified, and > COORDINATE_PRECISION is not, this will also apply to coordinates.
RFC7946=YES/NO. (OGR >= 2.2) Whether to use RFC 7946 standard. Otherwise GeoJSON 2008 initial version will be used. Default is NO (thus GeoJSON 2008)

If we're using a newer OGR >= 2.2 then we should just say RFC7946=YES. If not then set one of the other flags (I think COORDINATE_PRECISION=7 but verify).

The code would need to be changed here for OSM2PGSQL:

And here for IMPOSM:

@heffergm the osm2pgsql files are generating now but we're still missing the imposm-geojson.zip and imposm-shapefiles.zip.

From the IMPOSM spec, https://imposm.org/docs/imposm/latest/database_schema.html#generalized-tables

the layers gen0 (tolerance 200m) and gen1 (tolerance 50m) should be different, but have the same output for both shapefiles and geojsons.

To reproduce:

1. Choose an extract. I tested with the Prague pre-made popular extract.
2. Download either the geojson or the shapefile IMPOSM file.
3. I opened the files in QGIS and they both had the same geometries and attributes.

A user alerted us about this through email support.