Thanks for this tool, it's been really good to use.

I have a question about whether it is possible to include features depending on zoom level. Say I have a roads layer, I'd like this layer to only include ways of type "motorway" and "primary" at zoom 8, but then at zoom 9 I'd also like the same layer to include ways of type "secondary".

Is it possible to do this with a single "roads" layer, or do I have to split the road types into individual layers and use the config.json to specify the zoom levels for each layer?

Does this or will this support Windows OS?

Created tiled .mbtiles from osm .pbf but they not load ok when trying to display them.
Can you help me？

I've been testing tilemaker with larger extracts to see how RAM usage is with larger extracts

Doing a linear fit gives a RAM requirement for the current planet of 290GB. Sizing for 3 years growth I'd add 40% to the current planet size and a more complex conversion would require more RAM, which I'd put at a 50% growth, for a total estimated RAM need of 600 GB in 3 years. With AWS instances with 1 and 2TB of RAM this isn't impossible, but is expensive, and I'd be reluctant to recommend planning on doing Germany (2.9GB) or Canada (2.1GB) on a machine with 32GB of RAM.

The next steps depend on the intended direction of tilemaker, and what limitations and tradeoffs are acceptable.

The build of tilemaker I used wasn't multithreaded and I didn't get a good fit for conversion times, so I'm not going to focus on that.

Seeing errors as follows:-

/usr/include/boost/geometry/algorithms/disjoint.hpp: In instantiation of ‘bool boost::geometry::disjoint(const Geometry1&, const Geometry2&) [with Geometry1 = int; Geometry2 = boost::geometry::model::d2::point_xy<double>]’: /usr/include/boost/geometry/algorithms/intersects.hpp:99:53: required from ‘bool boost::geometry::intersects(const Geometry1&, const Geometry2&) [with Geometry1 = int; Geometry2 = boost::geometry::model::d2::point_xy<double>]’ src/osm_object.cpp:269:66: required from here /usr/include/boost/geometry/algorithms/disjoint.hpp:344:80: error: no type named ‘type’ in ‘struct boost::geometry::dimension<int>’ return dispatch::disjoint<Geometry1, Geometry2>::apply(geometry1, geometry2); ^ In file included from /usr/include/boost/iterator/iterator_categories.hpp:22:0, from /usr/include/boost/iterator/detail/facade_iterator_category.hpp:7, from /usr/include/boost/iterator/iterator_facade.hpp:14, from /usr/include/boost/filesystem/path.hpp:28, from /usr/include/boost/filesystem.hpp:16, from src/tilemaker.cpp:20:

Hi,

1. I downloaded this .mbtiles
   https://osm2vectortiles-downloads.os.zhdk.cloud.switch.ch/v2.0/extracts/liechtenstein.mbtiles

and broked down to .pbf tiles and was able to display it offline using Mapbox -gl like it outlined in
https://github.com/klokantech/mapbox-gl-js-offline-example

2. Now I built tilemaker on Windows (https://github.com/alex85k/tilemaker), then downloaded liechtenstein.pbf from http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf

and used tilemaker.exe (with default config.json and process.lua of tilemaker) to create tiled .pbf. The process created indeed zoom level 12, 13 and 14 (as outlined in config.json).

When I tried displaying them the same way as I displayed other .pbf in step 1, I get errors about null blobs and my map do not display any geometry at all. My .html start the zoom at 12 since my tiles started there.

I'm wondering if the produced .pbf are valid since I can't display them ?
Or may be I'm missing a settings when using TileMaker to produce my tiled .pbf ?

Thanks for any help

Noury

Is there any way we can host tilemaker output .pbf files,

I tried to host in apache and used leaflet vector plugin started getting unimplemented error.

If there is any other way or some help, kindly advise.

When using the default config with just turned off compression tilemaker produce all tile files in the 12 folder with zero size. Look at the screenshot: (Sorry for non-English interface)

First red box: 0 bytes, second: 720 files, 40 filders.

Why???

When running make on Ubuntu 15.04 I get the following:

root@helios:/home/simone/develop/tilemaker# make 
protoc --proto_path=include --cpp_out=include include/osmformat.proto include/vector_tile.proto
g++ -O3 -Wall -Wno-unknown-pragmas -Wno-sign-compare -std=c++11 -o tilemaker include/osmformat.pb.cc include/vector_tile.pb.cc src/tilemaker.cpp -I/usr/local/include -I./include -I./src -I/usr/local/include/lua5.1 -I/usr/include/lua5.1 -L/usr/local/lib -lz -llua5.1 -lboost_program_options -lluabind -lsqlite3 -lboost_filesystem -lboost_system -lprotobuf
In file included from src/tilemaker.cpp:22:0:
./include/sqlite_modern_cpp.h: In static member function ‘static void sqlite::binder<0>::run(sqlite::database_binder&, F)’:
./include/sqlite_modern_cpp.h:293:41: warning: typedef ‘traits’ locally defined but not used [-Wunused-local-typedefs]
    typedef function_traits<decltype(l)> traits;
                                         ^
In file included from src/tilemaker.cpp:22:0:
./include/sqlite_modern_cpp.h: In static member function ‘static void sqlite::binder<10>::run(sqlite::database_binder&, F)’:
./include/sqlite_modern_cpp.h:537:51: warning: typedef ‘type_10’ locally defined but not used [-Wunused-local-typedefs]
    typedef typename traits::template arg<9>::type type_10;
                                                   ^
src/tilemaker.cpp: In function ‘int main(int, char**)’:
src/tilemaker.cpp:335:45: error: converting to ‘std::unordered_set<unsigned int>’ from initializer list would use explicit constructor ‘std::unordered_set<_Value, _Hash, _Pred, _Alloc>::unordered_set(std::unordered_set<_Value, _Hash, _Pred, _Alloc>::size_type, const hasher&, const key_equal&, const allocator_type&) [with _Value = unsigned int; _Hash = std::hash<unsigned int>; _Pred = std::equal_to<unsigned int>; _Alloc = std::allocator<unsigned int>; std::unordered_set<_Value, _Hash, _Pred, _Alloc>::size_type = long unsigned int; std::unordered_set<_Value, _Hash, _Pred, _Alloc>::hasher = std::hash<unsigned int>; std::unordered_set<_Value, _Hash, _Pred, _Alloc>::key_equal = std::equal_to<unsigned int>; std::unordered_set<_Value, _Hash, _Pred, _Alloc>::allocator_type = std::allocator<unsigned int>]’
        unordered_set <uint32_t> tilelist = {};
                                             ^
Makefile:8: recipe for target 'all' failed
make: *** [all] Error 1
root@helios:/home/simone/develop/tilemaker# 

I get a crash trying to create .mbtiles from osm.pbf.
See below for the run and crash message:

./tilemaker ~/Downloads/montpellier.osm.pbf --output=montpellier.mbtiles
Layer water (z11-14)
Layer roads (z12-14)
Layer buildings (z14-14)
Layer pois (z13-14)
Reading /home/vagrant/Downloads/montpellier.osm.pbf
[libprotobuf FATAL /usr/include/google/protobuf/repeated_field.h:613] CHECK failed: (index) < (size()): 
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: (index) < (size()): 
Aborted (core dumped)

And this is the gdb backtrace:

gdb ./tilemaker core                                                          
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./tilemaker...done.
[New LWP 12232]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./tilemaker /home/vagrant/Downloads/montpellier.osm.pbf --output=montpellier.mb'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007ff569dbccc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
(gdb) bt
#0  0x00007ff569dbccc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ff569dc00d8 in __GI_abort () at abort.c:89
#2  0x00007ff56a6c7535 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ff56a6c56d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ff56a6c5703 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ff56a6c5922 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ff56a9aeb58 in google::protobuf::internal::LogMessage::Finish() () from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
#7  0x000000000042aabf in google::protobuf::RepeatedField<int>::Get (this=0x7fff01a21160, index=0)
    at /usr/include/google/protobuf/repeated_field.h:613
#8  0x000000000042992e in DenseNodes::keys_vals (this=0x7fff01a21100, index=0) at include/osmformat.pb.h:3735
#9  0x000000000043b367 in main (argc=3, argv=0x7fff01a31588) at src/tilemaker.cpp:258
(gdb)

Additional information:

• This is a on a fresh Ubuntu Trusy64 Vagrant install.
• Tilemaker is running from master branch (6a994ee).
• The PBF montpellier.osm.pbf was downloaded from http://metro.teczno.com/#montpellier.
• In CXXFLAGS I removed the -O3 and added the -g to get a better stacktrace.
• I installed all the dependencies as described in the README.mb (nice instructions by the way).

Installed tilemaker:

 $ brew install protobuf boost lua51 shapelib                                                                                             
 $ make
 $ make install

It exited with error, when I tried to run it:

$ tilemaker osm_extract.osm.pbf --output=/path/to/tiles
libc++abi.dylib: terminating with uncaught exception of type kaguya::LuaTypeMismatch: type mismatch!!
[1]    15510 abort      tilemaker osm_extract.osm.pbf --output=/path/to/tiles/

One thing I changed is install path in Makefile to my home/bin directory and I used git tag v1.4.0 when compiled tilemaker, using master didn't help.

I'm getting an error when loading a custom-made pbf file. I converted a big dataset of bustops into osm-data and then built an osm-pbf file. I'm able to read/write this file with both osmosis and josm but it fails when loading it into tilemaker. Just to make sure the pbf is using dense nodes I recreated the file using osmosis.

Reading stops1.osm.pbf
[libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "BlobHeader" because it is missing required fields: type, datasize
terminate called after throwing an instance of 'std::runtime_error'
  what():  Exception during zlib decompression: (-5) 
Aborted (core dumped)

A possible cause: there are no ways or relations in this file...

The actual file: https://www.dropbox.com/s/tttfctupw4okmhv/stops.osm.pbf?dl=0

Hi,
I'm getting the error trying to process fresh osm pbf file:

alex@ubuntu:~/tilemaker$ tilemaker-master/tilemaker japan-latest.osm.pbf --output=out/
terminate called after throwing an instance of 'kaguya::LuaTypeMismatch'
  what():  type mismatch!!
Aborted (core dumped)

Could you help me to solve this issue please?

I'd like to set up some CI for tilemaker with travis. Could you enable it at https://travis-ci.org/systemed/tilemaker?

If compiling boost from source, it saves a lot of time to only compile what will be used.

The docs do not say which parts of boost are required. Per conversation with @systemed, these are boost::geometry, boost::program_options, boost::filesystem (the latter only for mkdir -p support).

Of these, only program_options and filesystem require separate building and installation.

The tagline says "... without the stack". It seems to require quite a stack through, any plans to remove some or all these:

• Google Protocol Buffers
• Boost 1.56 or later (for boost::geometry, boost::program_options, boost::filesystem, boost::variant)
• Lua 5.1
• sqlite3
• shapelib

I have run tilemaker on the latest ireland-and-northern-ireland extract, and I get a pile of pbf files. However I don't think those tiles are valid protobuf files.

If I run:

tilemaker ireland-and-northern-ireland-latest.osm.pbf --output=tiles

Then protoc can't read those files:

$ protoc --decode_raw < tiles/13/3932/2588.pbf
Failed to parse input.
$ protoc --decode=vector_tile.Tile ./include/vector_tile.proto < tiles/13/3932/2588.pbf
Failed to parse input.

All the output file suffer from this problem:

$ find tiles -type f | while read FILENAME ; do protoc --decode_raw < $FILENAME ; done |& uniq -c
  53176 Failed to parse input.

Vector tiles downloaded from mapzen's vector tile format work and will give debugging/textual output.

(I asked this on StackOverflow)

Same structure as the JSON file. yaml-cpp appears to be the standard: https://github.com/jbeder/yaml-cpp

At present we use 32-bit unsigned integers (uint32_t) for node IDs. This won't last forever, or indeed, for too much longer.

In theory we should move to 64-bit uints. However, this is a massive increase in memory consumption, and I'm loth to increase the hardware requirements for people who just want to make vector tiles of one country using limited hardware.

So I'd suggest that we move to 64-bit as default, but have a compile-time switch to use 32-bit: make compact or similar. This would be implemented as ifdefs around a typedef NodeID. People could then renumber their extract into a 32-bit space (typically with osmium-renumber) and continue using tilemaker without so much RAM.

I'll tackle this when I get the proverbial spare minute.

I did consider bonkers schemes to pack the structs with 34-bit IDs or something like that, but that way lies madness and fragility.

Aide memoire: how to change the makefile.

Hi.

I tried to generate .mbtiles database for 12-18 zoom levels from OSM pbf file, but my tile server returned nothing. By manually requesting SQLite database I figured out that tile_column and tile_row has wrong values on all zoom levels. Here is my config file (default except 14->18):

{
	"layers": {
		"water": { "minzoom": 11, "maxzoom": 18 },
		"roads": { "minzoom": 12, "maxzoom": 18, "simplify_below": 13, "simplify_level": 0.0001, "simplify_ratio": 2.0 },
		"buildings": { "minzoom": 18, "maxzoom": 18 },
		"pois": { "minzoom": 13, "maxzoom": 18 }
	},
	"settings": {
		"minzoom": 12,
		"maxzoom": 18,
		"basezoom": 18,
		"include_ids": false,
		"name": "Tilemaker example",
		"version": "0.1",
		"description": "Sample vector tiles for Tilemaker",
		"compress": "gzip",
		"metadata": {
			"json": { "vector_layers": [
						{ "id": "roads",     "description": "roads",     "fields": {"name":"String","type":"String"}},
						{ "id": "water",     "description": "water",     "fields": {}},
						{ "id": "buildings", "description": "buildings", "fields": {}}
					] }
		}
	}
}

But with default parameters (12-14) everything works as expected.

"low_roads": { "minzoom": 9, "maxzoom": 11, "write_to": "roads", "simplify_below": 12, "simplify_level": 0.0001 }

In my understanding, the above configuration applies the same simplfy_level 0.0001 to each zoom level below 12.
Though it would be helpful to apply simplfy level 0.0002 to zoom level 11, 0.0004 to zoom level 10 and so on.

Firstly, thanks for making this great tool!

The ESRI Shapefile Technical Description states that a shapefile polygon may contain multiple outer rings, in effect representing a multi-polygon geometry.

Tilemaker currently assumes all shapefile polygons consist of only 1 outer ring. Therefore, any shapefile polygon containing multiple outer rings will fail Boost's geometry checks on ring orientation. An example of such a dataset where this occurs is Ordnance Survey Boundary-Line.

I couldn't find anything in the boost geometry docs for organising a set of rings into multiple polygons. It appears that OGR has code for handling this, but it doesn't use boost geometry and in any case is a bit too heavyweight a dependency to add just for this purpose.

Therefore, it would be nice to add a bit of code to tilemaker to support this, possibly making some assumptions about the ring ordering/overlaps in the input so that heavy spatial calculations can be avoided when reading shapefiles - something along the lines of how OGR does it.

I'd be happy to make a PR and give this a try.

Is it possible to generate one tile at x,y,zoom instead of generating all tiles?

As per dev@ post. The problem doesn't manifest itself with the default style.

Expected: http://osmapa.pl/w/area/?lat=51.10297&lon=17.03094&zoom=18&ol=NQ
Result:

Overpass query to get data:
[out:xml][timeout:2500];(way["area:highway"];relation["area:highway"]["type"="multipolygon"];);out meta;>;out meta qt;

config.json:

{
	"layers": {
		"area_highway": { "minzoom": 8, "maxzoom": 16}
	},
	"settings": {
		"minzoom": 8,
		"maxzoom": 16,
		"basezoom": 16,
		"include_ids": false,
		"name": "area:highway test",
		"version": "0.1",
		"description": "test QA render",
		"compress": "gzip",
		"metadata": {
			"json": { "vector_layers": [
						{ "id": "area_highway",     "description": "area highway",     "fields": {"type":"String","surface":"String"}}
					] }
		}
	}
}

process.lua:

-- Nodes will only be processed if one of these keys is present

node_keys = { "amenity", "shop" }

-- Initialize Lua logic

function init_function()
end

-- Finalize Lua logic()
function exit_function()
end

-- Assign nodes to a layer, and set attributes, based on OSM tags

function node_function(node)
--	local amenity = node:Find("amenity")
--	local shop = node:Find("shop")
--	if amenity~="" or shop~="" then
--		node:Layer("pois", false)
--		if amenity~="" then node:Attribute("type",amenity)
--		else node:Attribute("type",shop) end
--		node:Attribute("name", node:Find("name"))
--	end
end

-- Similarly for ways

function way_function(way)
	local area_highway = way:Find("area:highway")
	if area_highway~="" then
		way:Layer("area_highway", true)
		way:Attribute("surface", way:Find("surface"))
		way:Attribute("type",area_highway)
--		way:Attribute("id",way:Id())
--		way:AttributeNumeric("area",37)
	end
end

Isn't "tile_data" supposed to be jpg or png image?
Because when I dump it to a file it's not recognised as an image, but just "data".

This is the script I used for dumping:

#!/usr/bin/env python2
import sys
import sqlite3
import tempfile

database = sys.argv[1]
conn = sqlite3.connect(database)
c = conn.cursor()
c.execute("SELECT zoom_level, tile_column, tile_row, tile_data FROM tiles")
row = c.fetchone()
(zoom_level, tile_column, tile_row, tile_data) = row
# we're only interested in non-empty data
while len(tile_data) == 0:
    row = c.fetchone() # fetch next one
    (zoom_level, tile_column, tile_row, tile_data) = row
    # print "row(zoom_level, tile_column, tile_row, tile_data):", row
conn.close()
f = tempfile.NamedTemporaryFile(delete=False, suffix=".data")
f.write(tile_data)
f.close()
print "tile_data dumped to:", f.nam

Running the script:

$ ./dump_tile_data.py languedoc-rousillon.mbtiles 
tile_data dumped to: /tmp/tmpwJ1G_H.data
$ file /tmp/tmpwJ1G_H.data
/tmp/tmpwJ1G_H.data: data

However, when I try with another mbtiles which is not made from Tilemaker, the dumped data is actually a png:

$ ./dump_tile_data.py montpellier.mbtiles 
tile_data dumped to: /tmp/tmpyMet0n.data
$ file /tmp/tmpyMet0n.data
/tmp/tmpyMet0n.data: PNG image data, 256 x 256, 8-bit colormap, non-interlaced

Additional information:

• languedoc-rousillon.mbtiles was generated with Tilemaker (master branch 6a994ee) and "compress": false, the pbf was downloaded from Geofabrik
• montpellier.mbtiles was generated with makinacorpus/landez

My aim is to create MVT tiles from a set of shapefiles using tilemaker.

I have created a config file as follows:-

{
  "layers": {
    "roads": {
      "minzoom": 0,
      "maxzoom": 20
    },
    "om_roads": {
      "minzoom": 17,
      "maxzoom": 20,
      "write_to": "roads",
      "source": "Road_om_wgs84_2d.shp",
      "source_columns": [
        "classifica",
        "distname",
        "roadnumber"
      ]
    },
    "vd_roads": {
      "minzoom": 7,
      "maxzoom": 18,
      "write_to": "roads",
      "source": "Road_vd_wgs84_2d.shp",
      "source_columns": [
        "featcode",
        "roadnumber",
        "distname"
      ]
    }
  },
  "settings": {
    "minzoom": 0,
    "maxzoom": 20,
    "basezoom": 20,
    "include_ids": false,
    "name": "Tilemaker example",
    "version": "0.1",
    "description": "Sample vector tiles for Tilemaker",
    "compress": "gzip"
  }
}

I created a pbf file covering the UK land area uk_land_osm.pbf and used the sample lua file from this repo. I am only interested in generating tiles with the roads from the shapefiles

I then run tilemaker with:-

tilemaker --config config.json --process process.lua --output ./tiles --verbose uk_land_osm.pbf

This appears to work fine as the directory tree is created and I can see PBF files. However they are all empty only containing the GZIP headers. Am I missing a step or configuration? The SHP files are in the WGS84 SRS.

sudo add-apt-repository ppa:ostogvin/tjo-develop
sudo apt-get update

W: http://ppa.launchpad.net/gwibber-daily/ppa/ubuntu/dists/vivid/main/source/Sources 404 Not Found
W: http://ppa.launchpad.net/gwibber-daily/ppa/ubuntu/dists/vivid/main/binary-i386/Packages 404 Not Found

@systemed thank you for your help!

The example included has a water, roads, buildings, and pois layer. Is this enough to get a handle on performance and memory requirements? Are there features not demoed in the example (e.g. shapefiles)?

I've downloaded planet.osm.pbf and used osmosis to clip a bounding-polygon.
The result is the following osm.pbf file: clipped pbf file

When processing this file with tilemaker I get the following output:

Layer water (z0-15)
Layer waterway (z0-15)
Layer road (z0-15)
Layer road_bridge (z0-12) -> road
Layer bridge (z12-15)
Layer building (z15-20)
Layer poi (z0-15)
Layer admin (z0-15)
Layer aeroway (z0-15)
Layer barrier_line (z0-15)
Layer contour (z0-15)
Layer housenum_label (z0-15)
Layer landcover (z0-15)
Layer landuse (z0-15)
Layer country_label (z0-15)
Layer state_label (z0-15)
Layer place_label (z0-15)
Layer poi_label (z15-20)
Layer road_label (z0-15)
Layer tunnel (z0-15)
Layer water (z0-15)
Layer water_label (z0-15)
Layer waterway (z0-15)
Layer waterway_label (z0-15)
Layer marine_label (z0-15)
Reading /tmp/osm-latest-clipped.osm.pbf
terminate called after throwing an instance of 'std::out_of_range'
what(): _Map_base::at
Aborted

I'm currently working on a lua process to parse osm.pbf into .mbtiles. The goal is to have a source file which is compatible with the different mapbox stlyes. The Mapbox Streets v6 Documentation gives some details about how the source is handled:

The roads layers are some of the most complex ones in Mapbox Streets. Starting at zoom level 12,
tunnels and bridges are broken out of the #road layer into either #tunnel or #bridge.

It would be great to have some information about the zoom layer in way_function and node_function to decide to which layer the way/node should be written.
A workaround would be to have different config und lua, dependent on the zoom level:

...
tilemaker --config=./config_zoom_10.json --process=./process_zoom_10.lua test.osm.pbf --output=test.mbtiles
tilemaker --config=./config_zoom_11.json --process=./process_zoom_11.lua test.osm.pbf --output=test.mbtiles
...

This doesn't work well because tilemaker overwrites the database content on each call. Because of that it would be necessary to write into different .mbtiles files and then merge them together at the end.

A solution would be to introduce a "overwriteDatabase" switch in the config.json file.

I'm running debian so I couldn't follow the ubuntu instructions, but I installed the latest version of boost (1.62.0) and symlinked it into ./include. Everything compiles successfully, but I get this error:

$ ./tilemaker 
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::logic_error> >'
  what():  character conversion failed
Aborted

Although not all of a tilejson can be filled out by tilemaker as it doesn't know the URL, the vector tile layer information can.

It would be good to be able to generate this instead of manually writing it.

I am using Tilemaker to create vector tiles from a shapefile and there seem to be some issues with the outline of polygons. Here is an example for a single feature using the Tangram client (zoom in to see additional lines):

https://mapzen.com/tangram/view/?scene=https://mapzen.com/api/scenes/43045/224/resources/basic.yaml#8.2083/-2.312893/21.52367

The geometry is valid (Tilemaker did not flag it as invalid and it passes shapely.is_valid). Any ideas? The tool is fantastic, thanks!

Best wishes,
Andrew Cottam
Joint Research Centre
Ispra, Italy

I tried to render a seaside area (Genoa, Italy), but I got into a problem with coastline. In my extract the coastline isn't closed in the data, so when I try to render it, it closes the segment instead of filling the land.
Tried with both natural=coastline and the place=peninsula multipolygon (I wonder what this is, but... :-) )

Can you suggest a solution?
I thought I have to render water polygons from openstreetmapdata, or there's a way to close (correctly) the coastline when creating the extract or when running tilemaker?

Thanks

Thanks for sharing this tool, I'm keen to try it out but I'm running into an issue.

$ tilemaker liechtenstein-latest.osm.pbf --output=tiles/
Assertion failed: (fp_ != 0), function FileReadStream, file ./include/rapidjson/filereadstream.h, line 44.
zsh: abort      tilemaker liechtenstein-latest.osm.pbf --output=tiles/

I'm on OSX, and I installed the dependencies with homebrew:

$ brew install protobuf                    
Warning: protobuf-2.6.1 already installed
$ brew install boost                              
Warning: boost-1.58.0 already installed
$ brew install lua51   
Warning: lua51-5.1.5_2 already installed
$ brew install luabind
Warning: luabind-0.9.1_1 already installed
$ brew install sqlite3
Warning: sqlite-3.8.11 already installed

It seemed to install correctly:

$ make                             
protoc --proto_path=include --cpp_out=include include/osmformat.proto include/vector_tile.proto
g++ -O3 -Wall -Wno-unknown-pragmas -Wno-sign-compare -std=c++11 -o tilemaker include/osmformat.pb.cc include/vector_tile.pb.cc src/tilemaker.cpp -I/usr/local/include -I./include -I./src -I/usr/local/include/lua5.1 -I/usr/include/lua5.1 -L/usr/local/lib -lz -llua5.1 -lboost_program_options -lluabind -lsqlite3 -lboost_filesystem -lboost_system -lprotobuf
$ make install

I've tried with both liechtenstein-latest.osm.pbf and london_england.osm.pbf.

Any ideas what the issue could be?

Installing Luabind seems to be pretty much the pain point for everyone. Luwra is header-only, so should be less like dragging one's face across a bed of broken glass coated in Dave's Insanity Sauce while having the last month's posts on talk@ read out to you.

hi, when i do

./tilemaker --input=./osm/crimean-fed-district-latest.osm.pbf --output=./tile

where crimean-fed-district-latest.osm.pbf — file from geofabrik.de, all ok.
But when i crop pbf:

osmosis --read-pbf file=./osm/crimean-fed-district-latest.osm.pbf --bounding-box top=45.003934 left=34.017402 bottom=44.891783 right=34.196616 --write-pbf ./osm/simf.pbf

i get error:

./tilemaker --input=./osm/simf.pbf --output=./tile
Layer water (z11-14)
Layer roads (z12-14)
Layer buildings (z14-14)
Layer pois (z13-14)
Reading ../simf.pbf
libc++abi.dylib: terminating with uncaught exception of type std::out_of_range: unordered_map::at: key not found
Abort trap: 6

any idea?

Hello,

I tried to use the following in lua:

function way_function(way)
    way:Layer("layer1", true)
    way:Attribute("key", "value1")

    way:Layer("layer1", true)
    way:Attribute("key", "value2")
end

This code generates only one feature (instead of 2) in the final geojson.

Finding the problem

In the source code this behaviour is due to:
map< uint64_t, unordered_set<OutputObject> > tileIndex; in tilemaker.cpp and

bool operator==(const OutputObject& x, const OutputObject& y) {
    return (x.layer == y.layer) && (x.objectID == y.objectID) && (x.geomType == y.geomType);
}

in output_object.cpp because there is no verification of equality for attributes field.

Trying to resolve

For me there are 2 solutions:

• Compare attributes fields in bool operator==(const OutputObject& x, const OutputObject& y); in output_object.cpp
• Change the unordered_set type to a vector type

These 2 solutions give the same result. It works when there is only one zoom level but creates too much features when I tried:

{
    "layers": {
        "layer1": { "minzoom": 16, "maxzoom": 18 }
    },
    "settings": {
        "minzoom": 16,
        "maxzoom": 18,
        "basezoom": 18,
        "include_ids": false,
        "name": "OSM Vector Tiles",
        "version": "0.1",
        "description": "Vector Tiles from OSM data, created with tilemaker",
        "compress": "none"
    }
}

Have you an idea of How can we fix this problem ?

And thank you for your nice job :)

Thibaud

tilemaker.cpp:55:33:
error: ‘multi_polygon’ in namespace ‘boost::geometry::model’ does not name a template type typedef boost::geometry::model::multi_polygon MultiPolygon;

@systemed thank you for your help

This is not an issue, but maybe it helps someone to learn how to use the tilemaker > mapbox-gl-js stack:

At https://github.com/flamed0011/tilemaker you can find a process.lua to process a .osm.pbf (e.g. extracts from http://extract.bbbike.org) to vector tiles which are compatible with mapbox-gl-js.

An example to render these tiles via (a modified) mapbox-gl-js you can find at https://github.com/flamed0011/mapbox-gl-js-cordova-offline-example . The example extracts the tiles directly from the database without a server and renders the data in a cordova app. Important references are https://github.com/mapbox/mapbox-gl-styles and https://www.mapbox.com/developers/vector-tiles/mapbox-streets-v6/ .

I wil update the readme files of the two repos with more details. The process.lua is not really clean for now, but work in progress. Maybe it would be great to put a process_osm.lua to this repo?

Hi kind TileMaker developers,

Sorry, i am just new using tilemaker without good programming skills...

I have a shape file non OSM data that i want to convert to vector tiles

Folowing instructions CONFIGURATION.md, my config.json looks like :
{ "layers": {
"Iris": {
"minzoom": 11, "maxzoom": 14,
"source": "France_Iris_region.shp",
"simplify_below": 13, "simplify_level": 0.0003,
"source_columns": ["Code","Nom","Pop"]
}
}
}

But it is not clear what kind of tilemaker command do i have to execute : what input file (pbf?) do i have to refer to ?
tilemaker [what input file???] --output/Iris_Vtiles

Thanks for your lights, best

can we get the vector files in geojsonformat

I converted a *.osm.pbf file (http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf) to mbtiles with tilemaker, default config+lua (tilemaker liechtenstein-latest.osm.pbf --output=lichtenstein.mbtiles). When I try to upload to mapbox.com I get an error "unknown format" by mapbox. Does it need a special config to be compatible with mapbox?

On Debian testing I get this error message

pnorman@pippin:~/osm/tilemaker$ ./tilemaker ~/osm/planet/british-columbia-161023.osm.pbf --output=tiles/
terminate called after throwing an instance of 'std::runtime_error'
  what():  luabind::open() must be called with the main thread lua_State*
Aborted

luabind installed is 0.9.1+dfsg-11

Hi there,

I'm trying to produce map tiles with a greater level of road detail at lower zoom levels compared to those Mapbox provide. In a similar style to this: http://benfry.com/allstreets/map5.html

However i'm struggling to get the required level of detail at lower zooms, and it takes forever to render to try new config values. Do you have any tips for setting the right config values to achieve something similar to the link above?

I'm using British Isles OSM pbf data from Geofabrik. The default process.lua file and the following json config:

{
	"layers": {
		"water": { "minzoom": 6, "maxzoom": 14 },
		"roads": { "minzoom": 6, "maxzoom": 14, "simplify_below": 6, "simplify_level": 0.0001, "simplify_ratio": 1.0 },
		"buildings": { "minzoom": 14, "maxzoom": 14 },
		"pois": { "minzoom": 13, "maxzoom": 14 }
	},
	"settings": {
		"minzoom": 6,
		"maxzoom": 14,
		"basezoom":14,
		"include_ids": false,
		"name": "Tilemaker example",
		"version": "0.1",
		"description": "Sample vector tiles for Tilemaker",
		"compress": "gzip",
		"metadata": {
			"json": { 
                               "vector_layers": [
				       { "id": "roads",     "description": "roads",     "fields":{"name":"String","type":"String"}},
				       { "id": "water",     "description": "water",     "fields": {}},
				       { "id": "buildings", "description": "buildings", "fields": {}}
				]
                        }
		}
	}
}```

Many thanks.

Clipped .pbfs produced by Osmosis have a nasty habit of referring to non-existent nodes in ways (see #31, #68).

A few to-dos noted while putting the first version together:

• read Nodes from the .pbf, not just DenseNodes
• trap Lua errors when reading node_tags
• simplify geometries at small scales (specify with 'simplify_below' option)
• project-wide clipping box
• fill holes in areas
• route relations

Something along the lines of Emil Tin's old implementation for OSRM would be good, so that way_function can do this:

    while true do
        local role, route = routes:Next()
        if route==nil then break end
        local route_type=route.tags:Find("route")
        -- further processing...
    end

We could potentially specify the types of relations we're interested in (rather than hard-coding route only) in a similar manner to node_tags:

relation_types = { "route", "waterway" }

I have compiled and installed tilemaker according to the documentation. I'm using Ubuntu 14.04. When I am in the tilemaker source code directory tilemaker works fine. When I am not in that directory, I get the following error:

$ tilemaker ./ireland-and-northern-ireland-latest.osm.pbf --output=ietiles/
tilemaker: ./include/rapidjson/filereadstream.h:44: rapidjson::FileReadStream::FileReadStream(FILE*, char*, std::size_t): Assertion `fp_ != 0' failed.

I have seen this error on 3 different (linux) machines so far.