Normally I'd go with a just start coding approach for defining layers, but for #2 and my presentation I need a well-defined set of layers to start with.

I'm inclined to the following as a starting place

water

Contents

Polygonal water bodies.

• zoom 0 to 4, natural earth ocean + lake
• zoom 5 to 9, osm simplified_ocean_polygons
• zoom 10+, osm ocean_polygons
• zoom 5+ OSM waterway=riverbank or landuse=reservoir or natural=water

Fields

• water: ocean, lake, river, reservoir, or value of OSM water tag

Ordering

By size

water_names

Contents

Points with water names
Sources as with water layer, except sources without name data can be ignored.

Fields

• name: name of water body
• water: ocean, lake, river, reservoir, or value of OSM water tag

Ordering

By size then name as text

transport

Contents

Transportation linear features
All from OSM data, with features appearing at the following zooms

• motorway: z6
• trunk: z7
• primary: z8
• tertiary: z9
• residential, unclassified: z12
• service, track, and non-motorized types: z14
  _link roads of any type not shown until z10

Rail features not included for now.

If the feature is oneway, then the direction of it is the direction of travel.

Fields

• name: Name of road
• ref: Reference of road
• class: value of highway tag, excluding _link
• link: True if the feature is a _link, empty otherwise
• oneway: True if oneway=true or reverse
• brunnel: tunnel or bridge for tunnels or bridges, with bridge taking priority

Ordering

By layer with highest layer first, ramp with ramps first, class with most significant first, then name

Options for cartography are

• CC0
• CC BY

Options for code are

• Same as cartography
• MIT/BSD/ISC

I'd like to get a partially transparent fill with a less transparent edge, similar to osm-carto and US NPS maps.

When trying fill: inlay I was getting odd results (tangrams/tangram#661) but there might be a better way to do this technique.

railway=

• rail
• narrow_gauge
• preserved
• funicular
• subway
• light_rail
• monorail
• tram

These will need grouping. For the WMF, I grouped rail, narrow_gauge, preserved, funicular together, and subway, light_rail, monorail, and tram together.

The scripts need Python 3, but tileserver uses python 2.7. This should be noted in the docs.

Tegola has some critical bugs that prevent a number of essential things in the style. This meta-issue is to track them

Critical bugs

These are serious enough to prevent release, and do not have acceptable workarounds.

SQL type sniffing

• Tegola uses LIMIT 1 to look at a row (go-spatial/tegola#320). This prevents ordering in the roads layer.
• function(!ZOOM!) and !ZOOM! < n cause syntax errors (go-spatial/tegola#343). This prevents filtering by area depending on zoom.

Output bugs

• Missing water polygon (go-spatial/tegola#348)

Major

These don't prohibit release, but cause major problems or have bad workarounds

Output bugs

• Geometry corrupted by simplification (go-spatial/tegola#341).
• Tiles missing data when multiple rows have same id (go-spatial/tegola#337) is part of the cause of the extra sub-select all the queries have.

Major enhancements

These make something which is currently difficult or messy significantly better.

• Merging multiple layers (go-spatial/tegola#127). This would eliminate most of the UNION ALLs
• Don't require ID field for layers (go-spatial/tegola#338)
• Best practices for deploying Tegola (go-spatial/tegola#356) is not blocking code release, but blocks any public demo.

Draft announcement

Bolder - Starting a new client-side OpenStreetMap style

I’ve started work on a new client-side style for OpenStreetMap data, and feel it’s reached the point where I can release it to the public. My goal is to make a style that shows a rich selection of the data OSM has, and to make use of most of the colour space, rather than a style designed for overlaying other data on top of.

As a new style, I’ve been able to approach a lot from scratch, looking at avoiding mistakes of previous projects, and using best practices while building on existing work. All the components are open-source, and no assumptions are made about using closed-source software or particular commercial solutions.

Technical overview

The style is rendered with Tangram, which allows for client-side rendering. Server-side rendering is possible but is a secondary target. Closely coupled with the client-side style is a set of vector tile definitions, handled by Tegola, a vector tile server. It pulls from an osm2pgsql database in the OpenStreetMap Carto schema, with additional data like ocean polygons loaded in by a script.

Cartographic target

The goal of Bolder is to be a general-purpose style, filling a target similar to OpenStreetMap Carto, while also being a better “default” for people wanting an OSM map. Being a client-side style, it’s easier to turn off classes of features like some POIs if a map with fewer features is needed.

The style should still be useful for mapper feedback, and some ways will become more useful. Vector tiles can associate OSM feature IDs with objects in many cases, helping debugging “where did that label come from”.

Setup

The style has two arts that are installed, one for the vector tiles, and the other for displaying the client-side style. The documentation for both of them has been tested by users who hadn’t seen it before, so it should be possible to set up for anyone reasonably experienced in style authoring.

Limitations

As a new project, Bolder has limitations. The biggest limitation is that only a small number of features are rendered, and many things have to be added. I’ve also been doing lots of new stuff with Tegola, and have uncovered a number of critical bugs, most of which should be fixed next Tegola release.

This is a style recently developed by @systemed - it's going to be marketed commercially by Thunderforest so not a direct basis for this project, but the sense of proportion and feature selection may be a good model to follow.

z10

z12

z14

z15

z17

running get-external-data.py i get:

Traceback (most recent call last):
  File "./get-external-data.py", line 225, in <module>
    main()
  File "./get-external-data.py", line 219, in main
    this_table.index()
  File "./get-external-data.py", line 81, in index
    '''.format(name=self._name, temp_schema=self._temp_schema))
psycopg2.InternalError: Geohash requires inputs in decimal degrees, got (-180 -85.0511, -180 -85).

This is due to the code using ST_GeoHash() for indexing and clustering which requires transforming coordinates back into geographic which the simplified water polygons are not suited for - see gravitystorm/openstreetmap-carto#2101 which is essentially due to the same problem.

You can work around this by clipping the data before or after the import into PostGIS but generally using ST_GeoHash() in this context for proximity ordering of geometries is a bad idea - because map rendering should not depend on transforming coordinates into geographic, in particular transforming a bounding box in projected coordinates into geographic cannot generally be expected to produce a meaningful geometry. And even if that is not a problem (unmodified OSM data, rendering in Mercator) ST_GeoHash() is not a good solution for this problem - see:

https://en.wikipedia.org/wiki/Geohash#Limitations_when_used_for_deciding_proximity

Using what ST_GeoHash() does but in projected coordinates (which PostGIS does not allow at the moment) would probably be a suitable generic method.

Yes, this is probably outside the scope here but wanted to mention it none the less.

This style needs a name, in order to not be called "Mapnik"

Ideas to base a name around are

• geometric puzzles
• street
• spirit
• not-mapnik

Scripts are needed to examine tile size data in the cache.

This should provide, per zoom,

• average tile size
• median tile size
• some percentiles, or a histogram

The four maintained browser-based client-side renderers are

• Leaflet.VectorGrid
• OpenLayers3 VectorTile
• Tangram
• Mapbox GL JS

VectorGrid and VectorTile both are styled by writing Javascript. There is work to allow VectorGrid to read Mapbox GL JSON, but it's not yet done. Javascript is not a style language, and we want one when dealing with something as complex as a basemap.

This leaves Tangram from Mapzen and Mapbox GL JS. They are similar in many ways:

• Both are essentially single-company projects with a low bus factor
• Both use WebGL and inherit a lot of their data models from it
• both read Mapbox Vector Tiles
• both have native versions for mobile apps which can also be run on the server to generate images

The big differences I've found are the styling languages, company practices, and multi-lingual names.

Styling languages

Mapbox GL JS is styled with Mapbox GL JSON, a well documented language. Because there aren't typically linebreaks in it, the merge conflicts of SQL in JSON won't occur here but it has no facility for comments. This means we'd need to do something like a yaml2json step that osm-carto used to have.

Mapbox GL JSON has no inheritance, unlike Carto**C*SS. This makes writing styles very verbose. It is also designed to primarily be edited with a closed-source SaaS editor, and not by hand. This is a problem for a complex multi-contributor style.

Tangram is styled with Tangram scene files, which are YAML. The documentation is reasonable, although the language is more rapidly evolving than Mapbox GL JSON. There is inheritance - or at least it doesn't feel like I have to duplicate everything many times like with Mapbox GL JSON.

Having written both, I find Tangram scene files a lot more fun to write. We know from the old Mapnik XML stylesheets that having a language that people like is a huge part of attracting contributors.

Non-latin language support

Neither Mapbox GL JS nor Tangram fully support placing a label of any language on a curve, but for different reasons. Mapbox GL uses a custom glyph placement engine, and only supports some languages. Other languages cannot be labelled at all. Tangram uses the browser's placement, but doesn't handle curving the labels for all languages. This is under work (tangrams/tangram#555) and most common languages are supported.

I see Tangram supporting all used languages before Mapbox GL JS.

Other factors are

• Mapbox GL JSON has more people using it, although probably not writing it directly
• Tangram is still evolving rapidly
• Mapbox's release management practices for rendering software have been difficult to work with in the past
• Mapbox GL is a bigger name

the 99th percentile tile is 2MB, and there's 7 tiles above 1M

Same as kartotherian/meddo@3b0a231

Creating the schema requires elevated permissions that get-external-data.py shouldn't really have.

https://github.com/tangrams/tangram/releases/tag/v0.16.0

A vector tile generator needs to be selected. The standard reference for options is http://paulnorman.ca/blog/2016/11/serving-vector-tiles/.

The sane options are tileserver, tegola, t-rex, and tilemaker. node-mapnik is because it is a pain to deploy and we're not using Mapnik elsewhere in the style. Tilestache is out because it relies in SQL in JSON, and we know from JSON MML files that is a bad idea.

tilemaker is the most unconventional of the options because it prohibits diff support.

tileserver contains all the features needed, because Mapzen needs the features for tilezen. It's a question of how complicated is to deploy use, because it's probably the most complicated of the options, and it's possible it's tied too much into Mapzen does stuff. This needs evaluating.

t-rex and tegola are both new and may be missing some features required for vector tiles for a basemap.

This can be done with an env var and tegola

If anything other than tilemaker is chosen for #2, the osm data needs to be in a postgis database.

The three options are

• something that abstracts away tagging like imposm or osm2pgsql ClearTables
• OpenStreetMap Carto's 4.x schema
• osm2pgsql default

Abstraction is normally an advantage, but when aiming to be a stylesheet fairly close to the data and tagging like osm-carto is, it's a disadvantage. It doesn't make sense repeat the problems of osm2pgsql default, so osm-carto's transforms are a good option. This also allows the same DB to power both styles, which is a huge advantage.

VACUUM does some important maintenance.

Road casings can either be thin or thick, and this has a big influence over the look of the map.

Some examples of thin casings are
OpenStreetMap Carto

Skobbler

Google

Some examples of thicker casings are
OpenCycleMap

TomTom Basic

New Wikimedia

Right now I have relatively thick casings

For the same reasons as https://phabricator.wikimedia.org/T190237 the script needs to have a way to specify the directory it uses to download data

in the readme

Detailed instructions can be found in the client README.

links to https://github.com/pnorman/bolder/blob/master/vector/README.md rather than the expected https://github.com/pnorman/bolder/blob/master/client/README.md

Hardcoded postgres connection name. Not willing to change my unix user name.

Default location for map is vancover

With vector tiles it's necessary to load external data sources like oceans into the database. I'd like to do this with a proper script that separates the declaration of what data is needed from the code, and handles tracking updates.

I have some code I can reuse for this.