openmaptiles / openmaptiles Goto Github PK

Just as example how one could do that. From discussion #1

housenumber_centroid.sql contains

UPDATE osm_housenumber_point SET geometry=topoint(geometry)
WHERE ST_GeometryType(geometry) <> 'ST_Point';

topoint is not defined anywhere.

When rendering roads and rails it's necessary to handle ordering of both together - i.e. place them relative to the layer tag, not always have roads before or after rails.

This is done in tilezen and is necessary for rendering with Mapnik. osm-carto does it with a big UNION ALL that gets both roads and rails and allows for objects which are both.

Positron contains labels for seas as well so sea labels should be included in water_name.

Setup a quick webserver to demo the tile.mbtiles in quickstart.sh. I plan on using the existing mapbox studio classic.

Preview:

I would like to create some draft layers ( for personal interest and for creating basic examples )

First is in my mind a wheelchair topic :

1 New layers + 1 New tileset definitions !

• ♿ poi_wheelchair layers
  • adding a wheelchair and a toilets:wheelchair key to the poi layers

In my mind - it is :

• ./layers/poi_wheelchair/* ( for codes )
• openweelchairtiles.yml ( for tileset definitions )

Can I add this repo ?

😄 What is a (future proof) suggested method ?

• for 20-30 tileset definitions and for 50-100 new layer definitions

Problems:

• Probably we need to add a test system for testing all layers in once
  • and in extreme case it will be working for 100 layers ?
  • If yes, We need to create a name suggestions for a PostgreSQL work tables
    • ( Avoiding table name collisions )

Create an analysis to check min, max, median of nodes and tilesize to ensure #31 will work well.

Produce graphs similar to
https://mapzen.com/blog/v1-vector-tile-service/

Hello :)

My ideal README will contains only 3 line to test a new opensource project ..

https://github.com/openmaptiles/openmaptiles.git
cd openmaptiles
make quickstart

( or just a make start or make bootstrap or ... )

make quickstart

( draft, not tested - steps , based on the current documentation )

• "check" : the docker; docker-compose versions ; wget exists ?
• docker-compose down
• docker-compose rm -f -v
• "clean the data directory " rm -f ./data/* ..
• wget https://s3.amazonaws.com/metro-extracts.mapzen.com/zurich_switzerland.osm.pbf -P ./data
• "check" : the downloaded osm.pbf ( it is valid ? )
• docker run -v $(pwd):/tileset openmaptiles/openmaptiles-tools make
• docker-compose up -d postgres
• docker-compose run import-water
• docker-compose run import-natural-earth
• docker-compose run import-lakelines
• make clean && make && docker-compose run import-sql
• docker-compose run generate-vectortiles
• docker-compose up mapbox-studio
• echo "Hello OpenMaptiles User! please start your browser on : localhost:3000 "

Why:

• First time impressions for the new users --> it is very easy
• Help for the project tech support team
  • the first question: the 'make quickstart' is working ?

implementation:

• Without the extra "check" steps - is easy ...
• "later" ( in a next month ,,, ) I will create code for the extra checks
  • "check" : the docker; docker-compose versions ; wget exists ?
  • "check" : the downloaded osm.pbf ( it is valid ? )

When drawing extruded buildings there may be artifacting due to multiple overlapping geometries.

Likely resolution:
Adding 'DISTINCT' to the building.sql file.

render_height: An approximated height from levels and height of building after the method of Paul Norman in OSM Clear. For future 3D rendering of buildings

I've been skeptical about the use of this particular formula. I don't have it designed for complex 3d rendering, but simple 3d rendering, and I ended up pulling it from one of my styles because mapnik's building symbolizer doesn't work very well.

Not so fond of the repetition in the name.

Why not rename openmaptiles/openmaptiles-tools to openmaptiles/tools
and openmaptiles/openmaptiles-gl-styles to openmaptiles/gl-styles?

Right now it is in the openmaptiles repo and you have to call make.
It could also be in the website or wiki.

Can we port something like bubblewrap to OpenMapTiles?
tangrams.github.io/bubble-wrap/

Be able to label waterways (could stay on same layer). #20

So we discussed this further.
We want to have the documentation in a website just like Mapzen https://mapzen.com/documentation/vector-tiles/layers/

There will be a distinction between "Cartographer" (layer schema) and "Developer" (diagrams made by @ImreSamu ) docs.

The README in the layer directory should not be there (since it is just duplicated code) , there should be a link to the website documentaiton however in the README.

Wikipedia and @pnorman are working on a nice implementation of disputed borders which should make a way to this tile schema at some point.

Details in https://phabricator.wikimedia.org/T113008

😄

Imho :

• This is an OPEN Open Source Project : http://openopensource.org/
• And we need a few basic ground-rules for contributors

What is the ideal github workflow for US ?
Testing requirements
Code conventions
....

Not critical, just a memento ..
Imho

• we need to create a simple documents about this , if we open this repo

I think the missing leisure=park mentioned in #20 are now there since the fix of 2d55f4f. But need to recheck again after a Switzerland render.

This issue is meant to check whether 2d55f4f really fixed it.

I'd open this in the CT repo, but this one is still private.

This is a checklist to make sure that ClearTables has the required content to produce the current layers. I've done the same thing with MB Streets, imposm default, and Geofabrik shapefiles to see if there's anything missing on the CT side.

• boundary
• building
• highway
• highway_name
• housenumber
• landcover
• • farmland
• • ice
• • wood
• • wetland
• landuse
  • park
  • school
  • landuse
• place
• poi
• railway
• water
• water_name
• waterway

The new vector tile schema v3.1 is based on making Positron and Dark Matter and it would be good to have someone read through the layer documentation and give feedback.

A lot of the layers have a query of the form

    query: (SELECT geometry, class, subclass, properties::text FROM layer_railway(!bbox!, z(!scale_denominator!))) AS t

where layer_railway is a SRF which is only used in this one place. Speaking from experience, this is overuse of functions which will be a barrier to maintenance and understanding without offering gains.

testing ./quickstart.sh -> generate-vectortiles

docker-compose -f docker-compose.yml -f docker-compose-test-override.yml  run --rm generate-vectortiles

I see lot of warnings .. Is it possible to remove ?

Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer boundary)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer boundary)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer highway)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer highway)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer highway_name)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer highway_name)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer building)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer building)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer housenumber)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer housenumber)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer place)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer place)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer poi)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer poi)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer railway)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer railway)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer water_name)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer water_name)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer water)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer water)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer waterway)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer waterway)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer landcover)
Mapnik LOG> 2016-11-16 01:32:45: 'maxzoom' is deprecated and will be removed in Mapnik 4.x, use 'maximum-scale-denominator' instead (encountered in layer landcover)
Mapnik LOG> 2016-11-16 01:32:45: 'minzoom' is deprecated and will be removed in Mapnik 4.x, use 'minimum-scale-denominator' instead (encountered in layer landuse)

POI_Classification.xlsx

The excel in the Attachment is an example of a poi layer structure, do you think this is a different way to display the real word. if you think this is helpful I will enrich the structure.

Start with ClearTables until we hit the wall. Then implement a table ourselves and then we have more technical experience with ClearTables.

We have a ./quickstart.sh - for start new users ..
But the expected next questions

• I want to generate my country, my city .......

idea /draft specifications :

./create_tiles.sh < extract_id >

V1 specification :

for downloading:
We can use https://github.com/julien-noblet/download-geofabrik

go get github.com/julien-noblet/download-geofabrik
go install  github.com/julien-noblet/download-geofabrik
download-geofabrik update
download-geofabrik download  albania
---> albania.osm.pbf 

We need to extract the BBOX info from this
And we need to create a docker-compose config

• like this : https://github.com/openmaptiles/openmaptiles/blob/master/docker-compose-test-override.yml

PRO:

• easy to implement

CON:

• lot of osm2vectortiles extracts is missing, not perfect

V2 specification :

based on:

• https://github.com/osm2vectortiles/osm2vectortiles/blob/master/src/create-extracts/country_extracts.tsv
• https://github.com/osm2vectortiles/osm2vectortiles/blob/master/src/create-extracts/city_extracts.tsv

We have to find an optimal osm (geofabrik?) extracts
We have to pre-generate for an every extract a docker-compose extending yml file

• or we need a tool ...

~ expected output ...

##  OpenMapTiles service definitions for zurich_switzerland ! 
version: "2"

services:
  download-osm:
    environment:
       download_osm_url=https://s3.amazonaws.com/metro-extracts.mapzen.com/zurich_switzerland.osm.pbf
 
services:
  generate-vectortiles:
    environment:
      BBOX: "8.25,46.97,9.58,47.52"
      MIN_ZOOM: "0"
      MAX_ZOOM: "7"

railway=tram_stop

I'm inclined to think this should go with highway=bus_stop into its own layer.

Because tilemaker is quite different than node-mapnik, it's a good example for including a layer definition that doesn't depend on node-mapnik.

A good example would be the demo tilemaker buildings and water layers.

From config.json the layer definitions are

water:

{ "minzoom": 11, "maxzoom": 14 }

buildings:

{ "minzoom": 14, "maxzoom": 14 }

The metadata properties are

{ "id": "water",     "description": "water",     "fields": {}},
{ "id": "buildings", "description": "buildings", "fields": {}}

The proocessing lua is in https://github.com/systemed/tilemaker/blob/master/process.lua, but this covers both layers, as well as other layers.

If we can merge building polygons for nicer buildings at low zoom levels it will look better at z13.

@klokan Any feedback on mobile performance with the latest release v3.1?

Minimal viable vector tiles that make Positron look good and have high quality.
Less is more.

There is an example of a water layer, but it might not be what everyone needs. It's not clear how to add two layers to the repo which cover the same content and have the same most-obvious name.

Before I lost my "first User Expericence" (UX) view :)

If possible ,

• we have to use "similar volume name" ( inside - an outside the container )

imho :

• Sometimes it is very hard to find a logic,
• And hard to debug ( for me ) - and
  • maybe more harder for the first time users , who have less experience than me ..

Now:

    volumes:
     - ./data:/import
     - ./build:/mapping

or

     - ./build:/sql

or

    - ./data:/export

My First idea ( for brainstorming ) but not so easy to find a perfect naming standard ..

    volumes:
     - ./host_data:/data
     - ./host_build:/build

Why?
For file management

• I am a Midnight Commander user
  and a similar naming standard - helps me, quickly detect, that I am inside or outside the container

but I am open for other suggestions ...

Landuse from OSM data is often tricky, and people can decide to go different ways with it. This being said, I just can't figure out what the cartographic decisions behind the selections in https://github.com/openmaptiles/openmaptiles/blob/master/layers/landuse/landuse.sql are.

I suggest either BSD-3 or MIT for the code and something like CC0 for the styles.

I'm looking at the place layer and have some feedback

Column changes suggested

• class (String): country, state, settlement, subregion, or other.
• capital (Numeric): For settlements the place is a capital, what admin_level it is capital of. 2 and 4 only.
• place_detail (String): For settlement, one of city, town, village, hamlet, isolated_dwelling. For subregion, suburb or neighbourhood. For other, one of locality or farm.

I'm not certain of rank. As described, it ties an implementation into using Natural Earth data.

Add continents to place layer.
osm2vectortiles/osm2vectortiles#477

In general length filters don't work because a waterway can be split into multiple parts without changing the meaning. When you use a length restriction, you get parts of the waterway not rendering.

As an example, in London much of the Thames is in sections under 1km long, and the z8 length restriction is 10km, which is 6% of a tile. Even closer to the water, the sections are still under 10km.

I am still prototyping but already working with it and quite happy with the modular approach. Really rocks.

Example of water:

layers/water
├── mapping.yaml
├── water.sql
└── water.yaml

The layers/water/water.yaml is the definition file of the layers.
It specifies docs, buffer size and the query and also which SQL files to get the schema from
and which imposm3 schema to use. In datasources we might add different type sources.

layer:
  id: "water"
  description: |
      Water polygons and linestrings representing oceans, lakes and waterways.
  buffer_size: 4
  datasource:
    query: (SELECT * FROM layer_water(!bbox!, z(!scale_denominator!))) AS t
schema:
  - ./water.sql
datasources:
  - type: imposm3
    mapping_file: ./mapping.yaml

This definition files are then pulled together in a tileset specification.

tileset:
  layers:
    - layers/boundary/boundary.yaml
    - layers/highway/highway.yaml
    - layers/ice/ice.yaml
    - layers/building/building.yaml
    - layers/state/state.yaml
    - layers/country/country.yaml
    - layers/place/place.yaml
    #- layers/rail/rail.yaml
    - layers/urban/urban.yaml
    - layers/water/water.yaml
    - layers/landcover/landcover.yaml
  name: OSM2VectorTiles v3.0
  description: "Free global vector tiles from OpenStreetMap. http://osm2vectortiles.org"
  attribution: "<a href=\"http://www.openstreetmap.org/about/\" target=\"_blank\">&copy; OpenStreetMap contributors</a>"
  center: [-12.2168, 28.6135, 4]
  maxzoom: 14
  minzoom: 0
  defaults:
    srs: +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over
    datasource:
      srid: 900913

With the mapping containing the imposm3 mapping and generalized tables.

generalized_tables:
  water_polygon_gen3:
    source: water_polygon
    sql_filter: area>9000000.0
    tolerance: 350.0
  water_polygon_gen2:
    source: water_polygon
    sql_filter: area>1000000
    tolerance: 200.0
  water_polygon_gen1:
    source: water_polygon
    sql_filter: area>500000
    tolerance: 100.0
tables:
  water_polygon:
    fields:
    - name: osm_id
      type: id
    - name: geometry
      type: validated_geometry
    - name: area
      type: pseudoarea
    - key: name
      name: name
      type: string
    - name: name_en
      key: name:en
      type: string
    filters:
      exclude_tags:
      - [ "covered", "yes" ]
    mapping:
      landuse:
      - reservoir
      natural:
      - water
      - bay
      waterway:
      - river
      - riverbank
      - stream
      - canal
      - drain
      - ditch
    type: polygon
  water_linestring:
    fields:
    - name: osm_id
      type: id
    - name: geometry
      type: geometry
    - name: waterway
      type: mapping_value
    - key: name
      name: name
      type: string
    - name: name_en
      key: name:en
      type: string
    mapping:
      waterway:
      - stream
      - river
      - canal
      - drain
      - ditch
    type: linestring

I switched to imposm3 now because it integrates much better with modular layers - actually switching the queries over from one schema to another is not much work.

I have a Python script now called omtgen.py (for Open Map Tile generation tool) which takes care of generating tm2source, imposm3 mapping and aggregating all the SQL of all the layers.

@klokan Could you create a repo omtgen where I develop the generation tool for OpenMapTiles? Better name for the CLI also welcome.

The generation tool must be able to do:

• Create tm2source (in future other outputs as well - including PostgreSQL native generation #3)
• Create imposm3 mapping
• Collect SQL
• Generate Markdown documentation

https://lists.osgeo.org/pipermail/postgis-devel/2016-October/025973.html
https://git.osgeo.org/gogs/postgis/postgis/pulls/5

This is very interesting - as it may save a lot of data transfers and reparsing between Postgres and Mapnik workers - which could mean a significant boost in the speed of the vector tile rendering (of the planet)

It plays well with the openmaptiles "layer-catalog" structure we have discussed - where tm2source is just one of the generated outputs. We could have a lightweight generator with core functionality done directly in SQL - especially for the microtiles sets.

This allows main lines to be distinguished from less important ones

See gravitystorm/openstreetmap-carto#1645 for an example, it makes a surprising difference.

I started planet rendering yesterday evening for v3.1 mainly to test performance and the modified distributed. And also so we can put it on the CDN to test in the mobile apps.

The performance has increased a lot... right now it rendered half the planet in 12hrs with 6 machines.
With speeds on the workers ranging from 30 tiles/s to 80 tiles/s for each process (4 processes on each server). I guess the savings come from the table functions.

If you have a feedback about POI layer - comment to this issue ...

Analyzing what is missing ( based on Taginfo count )

SQL Output: (secret gist ) : https://gist.github.com/ImreSamu/a666974defc8c67da494349239b30045
based on 2016.nov.04 taginfo data and imposm3 mapping
( ~ select key,value, count_all from taginfo where not in [['imposm3_mapping']] order by count_all desc Limit 100 )

My rule of thumb :

• has a MAKI icon ? ( manual checking , so not fool safe )
• taginfo count_all >= xxxx
• I know as a OSM mapper ? ( personal feelings )

My suggestion for the NEW POI-s based on the SQL output ..

should

   key   |         value          | count_all 
---------+------------------------+-----------
 amenity | drinking_water         |    127332   ! https://github.com/mapbox/maki/blob/master/icons/drinking-water-11.svg
 amenity | ice_cream              |      8726   ! https://github.com/mapbox/maki/blob/master/icons/ice-cream-11.svg 
 shop | car_parts             |     24385  !
 shop | variety_store         |     17220  ! 
 shop | tyres                 |     10265  !
 sport | football                    |      3418   !
 sport | fitness                     |      2347   !

maybe

   key   |         value          | count_all 
---------+------------------------+-----------
 amenity | atm                    |    109383   ?
 amenity | hunting_stand          |     85403   ? ( for outdoor )
 amenity | vending_machine        |     82017   ? ( city maps)
 amenity | social_facility        |     47957   ? 
 amenity | clinic                 |     36680   ?
 amenity | charging_station       |      9624   ?
 amenity | emergency_phone        |      4828  ?
 shop | funeral_directors     |      7484  ?
 shop | farm                  |      6554  ?
 shop | seafood               |      5851  ?
 shop | kitchen               |      5040  ?
 shop | paint                 |      4756  ?
 shop | trade                 |      4356  ?
 shop | bookmaker             |      3839  ?
 shop | estate_agent          |      3808  ?
 shop | pawnbroker            |      3756  ?
 shop | lottery               |      3680  ?
 shop | houseware             |      3511  ?
 shop | fashion               |      3208  ?
 sport | 10pin                       |      2104   ?
 sport | exercise                    |      1594   ?
 sport | trampoline                  |      1226   ?
 sport | softball                    |      1007   ?
 sport | 9pin                        |      1005   ?
 leisure | sauna               |      5276   ?
 leisure | fitness_centre      |      4812   ?
 leisure | fitness_station     |      4387   ?
 leisure | horse_riding        |      3876   ?

edited:

• the taginfo statistics :
  • http://taginfo.openstreetmap.org/keys/amenity#values
  • http://taginfo.openstreetmap.org/keys/shop#values
  • http://taginfo.openstreetmap.org/keys/sport#values
  • http://taginfo.openstreetmap.org/keys/leisure#values

The ID of the layer should be defined in the tileset not in the layer YAML.
This means you can pull in the layers as you want and still give them a custom name.

So not like this.

tileset:
  layers:
    - layers/boundary/boundary.yaml
    - layers/highway/highway.yaml
    - layers/highway_name/highway_name.yaml
    - layers/building/building.yaml
    - layers/housenumber/housenumber.yaml
    - layers/place/place.yaml

More like this.

tileset:
  layers:
    boundary: layers/boundary/boundary.yaml
    street: layers/highway/highway.yaml
    streetname: layers/highway_name/highway_name.yaml
    house: layers/building/building.yaml
    housenumber: layers/housenumber/housenumber.yaml
    place: layers/place/place.yaml

As I see most of the POI data is missing ( compare to taginfo )

Probably the osm_poi_polygon table is missing ? or we have to change the imposm3 mapping type ...

• https://github.com/openmaptiles/openmaptiles/blob/master/layers/poi/mapping.yaml

if we don't choose mapbox-gl.js which rendering component will be chosen?

vision

My ideal use case

• easy importing from
  • 3 different repos AND adding from a local defs ( similar like in Golang import )
    • and expecting NO table/view/function name collisions
  • define the layer name for the mbtiles ( layer=railway )
  • set the update schedule ( dataupdate=monthly or hourly or never? )
• minimal configuraions for job generating
  • area=albania ( from a metatable we can set the downloadURL and the BBOX defintions )
  • system=K8S ( Docker-compose , K8S , Docker Swarm, ... ) for the future ...
  • mbtiles_compress=... for the future ... like https://github.com/rastapasta/tileshrink ?
• RUN

like this:

openmaptiles clear_all
openmaptiles import_defs github.com/cleartables/omt_layers/cleartables/clearrailway_slim layer=railway   dataupdate=monthly
openmaptiles import_defs github.com/openmaptiles/layers/openmaptiles/building3d          layer=building  dataupdate=weekly 
openmaptiles import_defs github.com/openmaptiles/layers/openmaptiles/landuse             layer=landuse   dataupdate=NO 
openmaptiles import_defs github.com/klokantech/openvectordefs/klokantech/extendedpoi     layer=poi       dataupdate=daily 
openmaptiles import_defs /myprivate_dir/layers/extremehiking                             layer=hiking    dataupdate=daily
openmaptiles generate_jobs area=albania min_zoom=1 max_zoom=10 mbtiles_compress=4 parallelization=8 system=K8S update=yes
openmaptiles schedule_and_run_jobs 

And if possible we can use imposm3 and/or osm2pgsql tools in each layer

problem

IMHO : Our FUTURE Problems ( based on my experience in working on big DW-s )

• How we can avoid SQL table name collisions, when more organisations developing different layers in parallel ,
• in extreme case they choose same table/view/function names
  • and creating a very hard to debug problem types ..
• Company/organisations want to create a private layers and want to use with a community layers together
• without a problems
• ...

IMHO : we need to create and adapt some guideline, conventions for this ...

solutions ?

As I see we have 4 basic solutions for start of thinking ...

• V1(PostgreSQL_Schema as a layer_group_id)

  • create SQLSchema for every organisations, and they can write to only their shema
    • openmaptiles
    • cleartables
    • wikidata
    • klokantech
      * osmbuildings
      * ....

• V2(PostgreSQL_Schema as a layer_definition_id )

  • new schema for every layers

• V3(Table name prefix) based

  • 1 public schema and in inside every tables , the prefix separates them

• V4 mixed ( V1+V3 = grouping + table prefixing )

  • separating organisations and separating layers

So I am thinking about this ...

testing [new] ./quickstart.sh - I see lot of SQL warnings ..

cat quickstart.log | grep 'has type "unknown"'
psql:/sql/tileset.sql:952: WARNING:  column "class" has type "unknown"
psql:/sql/tileset.sql:958: WARNING:  column "class" has type "unknown"
psql:/sql/tileset.sql:964: WARNING:  column "class" has type "unknown"
psql:/sql/tileset.sql:1062: WARNING:  column "landuse" has type "unknown"
psql:/sql/tileset.sql:1062: WARNING:  column "natural" has type "unknown"
psql:/sql/tileset.sql:1180: WARNING:  column "landuse" has type "unknown"
psql:/sql/tileset.sql:1186: WARNING:  column "landuse" has type "unknown"
psql:/sql/tileset.sql:1192: WARNING:  column "landuse" has type "unknown"