Giter Club home page Giter Club logo

tilestache's Introduction

TileStache

a stylish alternative for caching your map tiles

Build Status

TileStache is a Python-based server application that can serve up map tiles based on rendered geographic data. You might be familiar with TileCache, the venerable open source WMS server from MetaCarta. TileStache is similar, but we hope simpler and better-suited to the needs of designers and cartographers.

Synopsis

import TileStache
import ModestMaps

config = {
  "cache": {"name": "Test"},
  "layers": {
    "example": {
        "provider": {"name": "mapnik", "mapfile": "examples/style.xml"},
        "projection": "spherical mercator"
    }
  }
}

# like http://tile.openstreetmap.org/1/0/0.png
coord = ModestMaps.Core.Coordinate(0, 0, 1)
config = TileStache.Config.buildConfiguration(config)
type, bytes = TileStache.getTile(config.layers['example'], coord, 'png')

open('tile.png', 'w').write(bytes)

Dependencies

Required:

Optional:

Install the pure python modules with pip:

sudo pip install -U pillow modestmaps simplejson werkzeug uuid mapbox-vector-tile

Install pip (http://www.pip-installer.org/) like:

curl -O -L https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

Install Mapnik via instructions at:

http://mapnik.org/pages/downloads.html

Installation

TileStache can be run from the download directory as is. For example the scripts:

tilestache-render.py tilestache-seed.py tilestache-server.py

Can all be run locally like:

./scripts/tilestache-server.py

To install globally do:

python setup.py install
  • Note: you may need to prefix that command with 'sudo' to have permissions to fully install TileStache.

Quickstart

To make sure TileStache is working start the development server:

./scripts/tilestache-server.py

Then open a modern web browser and you should be able to preview tiles at:

http://localhost:8080/osm/preview.html

This is a previewer that uses ModestMaps and OpenStreetMap tiles from http://tile.osm.org as defined in the default config file 'tilestache.cfg'

Documentation

The next step is to learn how build custom layers and serve them.

See the docs for details.

Features

Rendering providers:

  • Mapnik
  • Proxy
  • Vector
  • Templated URLs

Caching backends:

  • Local disk
  • Test
  • Memcache
  • S3

Design Goals

The design of TileStache focuses on approachability at the expense of cleverness or completeness. Our hope is to make it easy for anyone to design a new map of their city, publish a fresh view of their world, or even build the next 8-Bit NYC (http://8bitnyc.com).

  • Small

The core of TileStache is intended to have a small code footprint. It should be quick and easy to to understand what the library is doing and why, based on common entry points like included CGI scripts. Where possible, dynamic programming "magic" is to be avoided, in favor of basic, procedural and copiously-documented Python.

  • Pluggable

We want to accept plug-ins and extensions from outside TileStache, and offer TileStache itself as an extension for other systems. It must be possible to write and use additional caches or renderers without having to modify the core package itself, extend classes from inside the package, or navigate chains of class dependencies. Duck typing and stable interfaces win.

  • Sensible Defaults

The default action of a configured TileStache instance should permit the most common form of interaction: a worldwide, spherical-mercator upper-left oriented tile layout compatible with those used by OpenStreetMap, Google, Bing Maps, Yahoo! and others. It should be possible to make TileStache do whatever is necessary to support any external system, but we eschew complex, impenetrable standards in favor of pragmatic, fast utility with basic web clients.

License

BSD, see LICENSE file.

tilestache's People

Contributors

aashish24 avatar adamfranco avatar alculquicondor avatar alno avatar andrewharvey avatar bwg avatar danzel avatar davidjb avatar gabefair avatar iandees avatar jessecrocker avatar juliomalegria avatar jwass avatar mattblissett avatar migurski avatar mojodna avatar ndimiduk avatar nvkelso avatar olegsson avatar paulsmith avatar philjung avatar rassie avatar rburhum avatar rjw57 avatar ryanbreen avatar sebastic avatar sk1p avatar thisisaaronland avatar tnightingale avatar zain avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tilestache's Issues

Apache mod_wsgi documentation

There's no example for mod_wsgi in the docs

Create a file: stash.wsgi on a path apache can read

import os, TileStache
application = TileStache.WSGITileServer('/full/path/to/tiles.cfg')

Add WSGI to apache

edit /etc/apache2/sites-available/defaults

WSGIScriptAlias /tilestache /var/www/wsgi/tilestache.wsgi

Install apache mod_wsgi

example on debian/ubuntu:

  sudo apt-get install libapache2-mod-wsgi
  sudo a2enmod wsgi
  sudo service apache2 restart

Documentation about TMS slightly misleading

"TileStache URLs are based on a simplified TMS scheme" is misleading:

  1. TMS has url/version#/layer/ in it's url
  2. TMS tiling order starts in the lower left, Tilestache is starting in the upper left which is google's tiling scheme
    http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/
  3. Openlayer TMS will fail on both of these, Tilestache has to be used as an OpenLayers XYZ

While not a big deal to people who know this stuff well, it could really impede someone just learning (as it did me for a good hour + my students). Seems reasonable to just update the docs with a better explanation and maybe examples using the tiles in common js clients.

Mapnik GridProvider does not `crop()` `keys` and `^data`

The Mapnik.GridProvider supports metatiles and can slice them up using crop() which is great!
However, it only crops the grid and not the keys and data. So all the individual tiles of a metatile share the same keys and data, even though they needn’t contain the keys at all.
Depending on the underlying data and metatile size, this can be A LOT of useless, duplicate data.

Proj4Projection.project(...) overrides ModestMaps.Geo.LinearProjection.project(...)

...with a different method signature. The issue can be resolved by renaming the project and unproject methods in Proj4Projection.

Traceback (most recent call last):
  File "/usr/bin/tilestache-seed.py", line 229, in <module>
    ul = layer.projection.locationCoordinate(northwest)
  File "/usr/lib/python2.6/site-packages/ModestMaps/Geo.py", line 132, in locationCoordinate
    point = self.project(point)
TypeError: project() takes exactly 3 arguments (2 given)

make failures

Just did a pass with the new user in mind. Thinking a new user may try to run the make. Even though it is mostly shortcuts for development would be great to make sure it works.

However I can't seem to get it past these errors:


pydoc -w scripts/tilestache-*.py
problem in scripts/tilestache-render.py - <type 'exceptions.ImportError'>: No module named simplejson
problem in scripts/tilestache-seed.py - <type 'exceptions.ImportError'>: No module named simplejson
wrote tilestache-server.html
mv TileStache.html doc/
mv: rename TileStache.html to doc/TileStache.html: No such file or directory
make: *** [doc] Error 1

I do have simplejson installed, so not sure what is up with pydoc:

dane$ python -c "import simplejson"

Hints on what I may be doing wrong?

Impossible to implement a provider which may or may not render a metatile

What I'd like to do is have a Provider which renders from an mbtiles file below a certain level, and renders an area with mapnik otherwise.

Under the current scheme, all requests to a given layer either do or do not return metatiles; my proposed provider would (optionally, according to config) provide a metatile when rendering from mapnik, but return a pre-rendered tile from an mbtile database otherwise.

Thus, I think that providers should tell Core whether or not they're returning a metatile, rather than Core handling that decision.

If you think this is a reasonable feature request, I'd be happy to implement it myself in the way that you think would be most reasonable, so please let me know. (We can discuss ways to implement this feature if you like it conceptually, I'm just asking here whether or not you think it's reasonable.)

My wholly untested code which sketches the type of provider I'm looking for:

from MBTiles import Provider as MBTilesProvider
from Providers import Mapnik as MapnikProvider

class MBCacheProvider:
  def __init__(self, layer, max_mb_zoom, tileset, mapfile, fonts=None):
    self.layer = layer
    self.max_mb_zoom = max_mb_zoom
    self.mbtileprovider = MBTilesProvider(self, layer, tileset)
    self.mapnikprovider = MapnikProvider(self, layer, mapfile, fonts)

  def renderTile(self, width, height, srs, coord):
    if coord.zoom < max_mb_zoom:
        return self.mbtileprovider.renderTile(width, height, srs, coord)
    else:
        xmin, ymin, xmax, ymax = self.layer.envelope(coord)

        if self.layer.doMetatile():
            xmin, ymin, xmax, ymax = self.layer.metaEnvelope(coord)
            width, height = self.layer.metaSize(coord)

            subtiles = self.layer.metaSubtiles(coord)

        return self.mapnikprovider.renderArea(width, height, srs, xmin, ymin, xmax, ymax, coord.zoom)

Error importing Vector

I'm using the master version of the library from github.

When i try to use the vector layer, i get the following nameError:

Error loading Tilestache config:
Traceback (most recent call last):
  File "./scripts/tilestache-server.py", line 55, in <module>
    app = TileStache.WSGITileServer(config=options.file, autoreload=True)
  File "/Library/Python/2.7/site-packages/TileStache/__init__.py", line 373, in __init__
    self.config = parseConfigfile(config)
  File "/Library/Python/2.7/site-packages/TileStache/__init__.py", line 166, in parseConfigfile
    return Config.buildConfiguration(config_dict, dirpath)
  File "/Library/Python/2.7/site-packages/TileStache/Config.py", line 217, in buildConfiguration
    config.layers[name] = _parseConfigfileLayer(layer_dict, config, dirpath)
  File "/Library/Python/2.7/site-packages/TileStache/Config.py", line 422, in _parseConfigfileLayer
    _class = Providers.getProviderByName(provider_dict['name'])
  File "/Library/Python/2.7/site-packages/TileStache/Providers.py", line 122, in getProviderByName
    return Vector.Provider
NameError: global name 'Vector' is not defined

PIL error when running tilestache-server.py

When trying to run tilestache-server.py I get the following error: AccessInit: hash collision: 3 for both 1 and 1.

From what I've read, this seems to be an issue where PIL ends up being imported more than once in a script and conflict.

I'm not sure what exactly is the cause of the issue, but adding this to tilestache-server.py got it running:

import sys
import PIL.Image
sys.modules['Image'] = PIL.Image

Vector provider should not output empty geometries

I am not certain if there is a dependency on my data source, but Vector outputs a lot of empty collections and geometries. I patched it locally using

    if geom["type"] == "Polygon" or geom["type"] == "LineString":
      features.append({'type': 'Feature', 'properties': prop, 'geometry': geom})

cutting tile size dramatically with the same results.

slow loading tiles

when viewing http://localhost:8080/osm/preview.html, tiles load really slow and console spits this out:

----------------------------------------
127.0.0.1 - - [18/Oct/2011 22:41:29] "GET /osm/13/1316/3161.png HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 61006)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/SocketServer.py", line 641, in __init__
    self.finish()
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/SocketServer.py", line 694, in finish
    self.wfile.flush()
  File "/usr/local/Cellar/python/2.7.2/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe

composite example broken

$ ./scripts/tilestache-server.py --config examples/composite/composite.cfg 
Traceback (most recent call last):
  File "./scripts/tilestache-server.py", line 55, in <module>
    app = TileStache.WSGITileServer(config=options.file, autoreload=True)
  File "/Users/dane/projects/TileStache/TileStache/__init__.py", line 297, in __init__
    raise Core.KnownUnknown("Error loading Tilestache config file:\n%s" % str(e))
TileStache.Core.KnownUnknown: Error loading Tilestache config file:
[Errno 2] No such file or directory: u'file:///Users/dane/projects/TileStache/examples/composite/composite-stack.xml'

The appears to happen because 'file:///foo' is passed to open() inside of expatbuilder.parse(file) which fails.

>>> file = 'file:///Users/dane/projects/TileStache/examples/composite/composite-stack.xml'
>>> open(file,'rb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'file:///Users/dane/projects/TileStache/examples/composite/composite-stack.xml'
>>> file = '/Users/dane/projects/TileStache/examples/composite/composite-stack.xml'
>>> open(file,'rb')
<open file '/Users/dane/projects/TileStache/examples/composite/composite-stack.xml', mode 'rb' at 0x108925420>

Since it appears that stackfile is deprecated I tried to look at converting the file to json but I have no idea how to do that.

Configurable out-of-bounds action

I'd like a per-layer configuration for what happens when an out-of-bounds (as specified with the "bounds" config option on the layer). I'd like to return a 404, a configurable image (via URL or local file?), or "transparency" (but I suppose that's just an all-transparent tile).

Support for DateTime OGR field

The vector provider throws an error when it is passed a DateTime field.

KnownUnknown at /tiles/layer/16/10652/25165.geojson
Found an OGR field type I don't know what to do with: ogr.OFTDateTime

memcached: constantly creates new connections

The memcached Cache backend seems to create new connections for each operation. After a while of heavy usage, memcached does not accept any new connections:

error 26 from memcached_get on 127.0.0.1:11211: Cannot assign requested address

Doing netstat -n | grep 11211 reveals many (>100, sometimes >1000) connections in TIME_WAIT state. This happened both using gunicorn and while testing using the tilestache-server.py script.

It seems like the origin of this problem lies somewhere in python-memcached, it constantly creates new sockets. Replacing it with pylibmc fixed the problem.

One possible "solution": make the memcached backend configurable, and push the Client import down to Cache.__init__

My pylibmc-backend-hack lives at

https://github.com/sk1p/TileStache/blob/fix_memcache_connections/TileStache/Goodies/Caches/PyLibmc.py

As you can see, I also made it reuse the Client instances, and removed the mem.disconnect_all() calls. Was there a particular reason for disconnecting and creating new Clients? Also note that just re-using Client instances and removing the disconnects is not enough: python-memcached still creates new sockets...

Allow max/min zoom without bbox

I'd like to be able to specify a maximum and minimum zoom for a layer without specifying the bbox (it should default to "the world" when not specified in the config).

bounds parameter requires too much

It would be nice if I could leave out the spatial limits and only specify a zoom limit. As it stands now, something like this in my layer parameter gives an error:

"bounds": {
   "low": 16, "high": 19
}

KnownUnknown: Error loading Tilestache config file:
Missing part of bounds for layer, need north, south, east, west, high, and low: {"high": 19, "low": 16}

Vector provider is unnecessarily clipping 10 degrees off WGS84 layer

I understand that when we use Web Mercator, most tile services clip at +/-85 latitude, but we are getting the same behavior with the vector provider and WGS84 layers. The problem with this is that features that go beyond this (for example, grids drawn as vectors) are becoming incomplete.

Perhaps we should not do this with WGS84 vectors.

Repro case:

In PostGIS:

DROP TABLE if exists silly_box;
CREATE TABLE silly_box(gid serial PRIMARY KEY, name VARCHAR);
SELECT AddGeometryColumn('public', 'silly_box', 'geom', 4326, 'MULTIPOLYGON', 2);
INSERT INTO silly_box (geom, name) VALUES (ST_GeomFromText('MULTIPOLYGON(((-180 -90, 180 -90, 180 90,-180 90, -180 -90)))',4326),'The World');

Configuration:

            {
                 "provider":{
                    "name": "vector",
                    "driver" : "PostgreSQL",
                    "parameters": {
                                    "dbname": "test_tilestache", 
                                    "user": "postgres",
                                    "table": "silly_box"
                    }                    
                 },
                 "projection" : "WGS84"
              }

Actual result for 0/0/0.geojson

{"type": "FeatureCollection", "features": [{"geometry": {"type": "Polygon", "coordinates": [[[179.99999999999983, 85.05112877996989], [179.99999999999983, -85.0511287799699], [89.99999999645502, -85.0511287799699], [0.0, -85.0511287799699], [-89.99999999645502, -85.0511287799699], [-179.99999999999983, -85.0511287799699], [-179.99999999999983, 85.05112877996989], [-89.99999999645502, 85.05112877996989], [0.0, 85.05112877996989], [89.99999999645502, 85.05112877996989], [179.99999999999983, 85.05112877996989]]]}, "type": "Feature", "properties": {"name": "The World"}}]}

Expected result:
Half of the box where the 85's should be 90 (or 89.99999999)

ExternalConfigServer

Hi all,

I apologize for asking a maybe stupid question but is there any tutorial or how-to to set-up the ExternalConfigServer -wsgi?

To explain why: I have a postgisdb, create tiles with tilestache in wsgi mode and call them via polymaps - everything is fine with that and I really love this tool. I want now to add a "search" functionality where the users enters a name of POI and tilestache serves a layer that contains that POI. Right know I fiddled together a php script that changes the *.cfg file with an according query and, voila, tilestache serves me the layer. So far so good. But with that script I have to change the "physical" cfg file everytime a user does a query and I think I'll get in trouble when multiple users join the game.

Then I found out about Tilestache.ExternalConfigServer what (I guess) is exactly what I need but I am not able to get this working. Right know I have a

"tilestacheexternalconfig.wsgi":

!/usr/bin/env python

import os
import TileStache

application = TileStache.Goodies.ExternalConfigServer:WSGIServer(url)

and a Apache WSGI Alias that calls that script (WSGIScriptAlias /tilestacheexternalconfig /pathtoscript/tilestacheexternalconfig.wsgi). And .. right now I don't now how to go on.

So my question is: can anyone here explain to me how this works and wether this would be a useful solution to handle a "multiple-user-want-to-search-a-POI"-Problem?

Thank you in advance!
davp87

tilestache-seed.py issue

Minor point, but still:
the coordinates list is truncated to the first 10 (line 252):
coordinates = list(coordinates)[:10]
So the script does not work... some debugging I guess...
Cheers,

Decio

Preview.html doesn't let me zoom out

I can zoom in with a double click but I can't zoom out. I seem to remember that I could scroll-wheel zoom, but that doesn't appear to work any more.

Pass current zoomlevel to query parameter of PostgreSQL vector provider

Hi, it would be great to pass current zoomlevel to variable and allow PostgreSQL query adjustment.

Imagine we have a table with multiple pregeneralized geometry columns and we want to switch between them depending on currrent zoom level to spare some data and speed up client SVG rendering.

This is currently IMHO realizable with tilestache-seed.py - seed only selected levels and modify .cfg file to adjust desired geometry column.

It would be great to have .cfg like (very simplified idea, in fact there's need to have own function to select right geometry column):

"provider": {
"name": "vector",
"driver": "PostgreSQL",
"parameters": {"user": "postgis", "password": "mypassword", "dbname": "mydatabase", "query": "SELECT CAST(code AS character) AS code, name, geom_$ZOOMLEVEL FROM mytable"},
"properties": ["code","name"],
"clipped": false,
"id_property": "code"
}

Alternatively we could have some kind of zoom-geometry column mapping like:

"provider": {
"name": "vector",
"driver": "PostgreSQL",
"parameters": {"user": "postgis", "password": "mypassword", "dbname": "mydatabase", "table": "mytable"},
"zoom_to_geometry_mapping": {"13": "geom_simplify50m", "14": "geom_simplify25m", "15": "geom_simplify10m", "16": "geom_simplify10m", "17": "geom", "18": "geom"}
"properties": ["code","name"],
"clipped": false,
"id_property": "code"
}

Vector Provider not populating tiles that wrap around the outer edge of the data source

This bug is reproducable using a layer with the vector provider and the world_merc sample shapefile as a data source.

When tile boundaries cross the edge of a shapefile, they fail to be populated properly. This is visible where the world_merc shapefile wraps around, in the Bering Strait between Russia and Alaska: The further you zoom in, the more land is correctly rendered. (You get closer to the shapefile's edge.) It might be related to the map wrapping in some way.

This is visible in the following series of images:

Bering Strait 1

Bering Strait 2

Bering Strait 3

Bering Strait 4

Separate Mapnik Rendering from Request Cycle

We should borrow some functionality from tirex/renderd for mapnik rendering and separate Mapnik rendering from the request cycle. When a request comes in that is not cached, the Mapnik provider should put the render request onto a PriorityQueue that a configurably fixed-size process pool pulls from. Each process should contain one Mapnik instance, render with it, then throw information about the resulting image back to the requestor. As an example, mod_tile will time out and give a 404 after some number of seconds (5 by default) if the rendering process doesn't finish in time.

The point of this is to have a fixed number of Mapnik rendering threads that can be different than the number of TileStache threads since there need to be more TileStache threads available to serve tiles that are in cache already.

The approach mentioned here might be tricky, though, because there will be multiple e.g. gunicorns running, each with their own TileStache parent process. They need to know to only create one pool of Mapnik-provider processes and share them with all gunicorn processes.

Broken Locking?

The OSM.us server seems to get bogged down quite a bit when people make tile requests. I finally caught it when someone was trying to load an area and see the following queries running on Postgres:

 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 36.59788913365854,-39.37499999999991 75.49715731951939)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 -27.05912578447814,-39.37499999999991 45.08903556472714)'::box3d, 4326)
 SELECT ST_AsBinary("the_geom") AS geom FROM (select the_geom
        from tiger_roads
       ) as tiger2012 WHERE "the_geom" && ST_SetSRID('BOX3D(-118.1250000000001 36.59788913365854,-39.37499999999991 75.49715731951939)'::box3d, 4326)

You can see that most of these queries are for the same exact area. I would expect the locking mechanism to prevent this (there should be one query and several gunicorns waiting for it to finish).

We're using the cascading cache with memcache and S3. I'm going to look closer at the code to see if I can spot anything going on.

Configuration to return 404 instead of/along with grey image when request is out of bounds

When a user specifies a bounds object in the config, it should be possible to return an HTTP 404 instead of or along with the grey image that comes back right now.

The use case here is that when JOSM encounters a 404 it will render the tile from the zoom level above it, scaled up, in the place of the image that 404'd. As it stands now, when a user zooms in too far or otherwise goes out of the bounds for the layer, a grey image is loaded, preventing the user from seeing whatever was there before.

Unknown child node: Map > Parameters

I'm actually not sure if this is a TileStache, Mapnik or TileMill issue, but TileStache (1.31.1, Mapnik 2.0.1+dev20120518) threw an error

.../Providers.py", line 262, in renderArea
mapnik.load_map(self.mapnik, str(self.mapfile))
RuntimeError: Unknown child node in 'Map': 'Parameters'

...when I added a recent Mapnik XML created with TileMill with a <Parameters /> element right after the <Map> opening tag. Removing the <Parameters /> solved it.

Proxy provider only works with Spherical Mercator

The proxy provider has a check to ensure that the projection is spherical mercator before rendering any tile (https://github.com/migurski/TileStache/blob/master/TileStache/Providers.py#L245)

I'm not sure why this check must be there? It seems legitimate to want to use the proxy even for other projections.

On the other hand, I can see that some of the proxy providers can only work with spherical mercator; for the general URL templated proxy, projections shouldn't matter, though.

Am I missing something, or should I attempt to write a patch to refine the check to only apply to the providers that explicitly require spherical mercator?

MBTiles Provider doesn't serve UTFGrid tiles

MBTiles created by Tilemill (and others) can have UTFGrids. Tilestream, for example, serves the grids alongside the associated tile image simply by swapping out ".png" for ".grid.json" as the file extension.

I'm not sure if serving utfgrids should be handled through the current MBTiles provider or if another Provider would be necessary. Thoughts?

TileStache.WSGITileServer & mod_wsgi TypeError

Hi Guys,

Just started using TileStache and I'm really excited about the new WSGI class. Tried it out using Apache2 + mod_wsgi, but was receiving the following error:

"TypeError: expected byte string object for header value, value of type int found"

I think it has something to do with the Content-Length response header. Wrapping it in a str() eliminates the error, and I can request JSON tiles just fine.

def _response(self, start_response, code, content='', mimetype='text/plain'):
    """
    """
    start_response(code, [
        ('Content-Type', mimetype),
        ('Content-Length', str(len(content))), # Wrapped due to TypeError
    ])

Anyways, I'm not experienced using mod_wsgi, so it could definitely be something on my end that was causing the problem, too. But this fix worked in my particular case.

Thanks for the hard work and great product.

Cheers,
Matt

Empty png/html outputs.

Hey guys,

So I just installed tilestache and all dependencies. I fired up the server and the shell showed me its running at the port I explicitly specified (9000 range). When I open the preview html page, I see absolutely nothing. The entire page is blank, though the url changes according to zoom and center. The shell also shows me that the respective pngs are being pulled. When I checked the cache folder, I can see all the png files, but all are 0 bytes in size and have no image.

The OSM preview worked for me, so I think it's an issue with caching locally when using MBtiles or Mapnik xml. Can someone please look into this?

My config file:

{
"cache":
{
"name": "Disk",
"path": "/home/kaushik/Downloads/stache"
},
"layers":
{
"roads":
{
"provider": {"name": "mbtiles", "tileset": "/home/kaushik/ward176.mbtiles"},
"preview": {"zoom":18, "ext":"jpg"}
}
}
}

My shell output:

  • Running on http://127.0.0.1:9011/
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/preview.html HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42044/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42045/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42044/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42045/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42044/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42043/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42045/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42046/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42044/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42043/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42046/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42045/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42043/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /favicon.ico HTTP/1.1" 400 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42044/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42042/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42046/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42047/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42045/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42043/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42042/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42046/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42047/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42043/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42042/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42047/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42048/101301.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42046/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42042/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42047/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42048/101300.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42042/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42047/101303.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42048/101302.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42048/101299.jpg HTTP/1.1" 200 -
    127.0.0.1 - - [30/May/2012 19:27:33] "GET /roads/18/42048/101303.jpg HTTP/1.1" 200 -

Exception when using new Cache Control header on 1.24.0

I'm experiencing an exception using the latest version (1.24.0).

Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/Werkzeug-0.8.2-py2.6.egg/werkzeug/serving.py", line 159, in run_wsgi
    execute(app)
  File "/usr/local/lib/python2.6/dist-packages/Werkzeug-0.8.2-py2.6.egg/werkzeug/serving.py", line 146, in execute
    application_iter = app(environ, start_response)
  File "/ebs/data/tilestache/dev/TileStache/__init__.py", line 353, in __call__
    return self._response(start_response, '200 OK', str(content), mimetype, allowed_origin, max_cache_age, request_layer)
  File "/ebs/data/tilestache/dev/TileStache/__init__.py", line 364, in _response
    expires = datetime.utcnow() + timedelta(seconds=layer.max_cache_age)
NameError: global name 'layer' is not defined

A simple fix is to just pass in the request_layer from the caller.

        return self._response(start_response, '200 OK', str(content), mimetype, allowed_origin, max_cache_age, request_layer)

    def _response(self, start_response, code, content='', mimetype='text/plain', allowed_origin='', max_cache_age=None, layer = None):

Vector provider returns "Couldn't open datasource" when table has no records

If the table that is configured has no records, TileStache is returning "KnownUnknown: Couldn't open datasource".

Repro case in postgis:

DROP TABLE if exists silly_box;
CREATE TABLE silly_box(gid serial PRIMARY KEY, name VARCHAR)
SELECT AddGeometryColumn('public', 'silly_box', 'geom', 4326, 'MULTIPOLYGON', 2)

Configuration:

        {
           "layers":{
              "vector_test":{
                 "provider":{
                    "name": "vector",
                    "driver" : "PostgreSQL",
                    "parameters": {
                                    "dbname": "test_tilestache", 
                                    "user": "postgres",
                                    "table": "silly_box"
                    }                    
                 }
              }
            },
            "cache": {
                "name": "Test"
            }
        }

Expected behavior:
Return an empty response

Spacing parameter : buffer wrong value

Hi and thanks again for Tilestache !

I'm trying to use the spacing parameter with vector layers and i see stange values for the buffer value when i log it form Vector/init.py.

Here are the files i use : tilestache.cfg + my geojson source file + init.py with logs + log results

https://gist.github.com/2700390

My GeoJSON file has no CRS defined so it use "geographic coordinate reference system, using the WGS84 datum, and with longitude and latitude units of decimal degrees." (source http://www.geojson.org/geojson-spec.html).

My Layer has no projection defined so it use TileStache default projection : spherical mercator

The strange thing is that with a spacing value set to 16 i got a buffer value of buffer 19567.879241. This value cause the spacing parameter to generate au huge buffer and i got only one feature in response outpout for 355 in input. I also see that the polygon of the mask contains a strange mix of WGS84 and spherical mercator coords.

If i set the projection of the layer to WGS84 in tilestache.cfg i got a "reasonable" value of 0.087890625 for the buffer, but this is not a good solution cause i don't need my layer to be projected in WGS84.

I'm a total newbie with python and ogr things but it seems that the buffer value should be "transformed" to the layer_sref like it's done with the bbox value here : https://github.com/migurski/TileStache/blob/master/TileStache/Vector/__init__.py#L495

Am i missing something in my conf or is it a bug ?

Vector provider does not handle unicode in queries

Traceback (most recent call last):
File "/usr/bin/tilestache-seed.py", line 273, in
mimetype, content = getTile(layer, coord, extension, options.ignore_cached)
File "/usr/lib/python2.6/site-packages/TileStache/init.py", line 103, in getTile
tile = layer.render(coord, format)
File "/usr/lib/python2.6/site-packages/TileStache/Core.py", line 307, in render
tile = provider.renderTile(width, height, srs, coord)
File "/usr/lib/python2.6/site-packages/TileStache/Vector/init.py", line 520, in renderTile
layer, ds = _open_layer(self.driver, self.parameters, self.layer.config.dirpath)
File "/usr/lib/python2.6/site-packages/TileStache/Vector/init.py", line 400, in _open_layer
layer = datasource.ExecuteSQL(str(parameters['query']))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf8' in position 84: ordinal not in range(128)

Readme not mentioning GDAL python bindings

If you want to use the vector tile provider, you need to have GDAL and the python GDAL bindings installed. This isn't mentioned in the readme. Installing on a Linux is dead simple (intall the python-gdal package), but might be more complex on Windows?

Error in TileStache.Goodies.Providers.UtfGridComposite

I've been attempting to use the TileStache.Goodies.Providers.UtfGridComposite provider but am getting the following error:

File "/usr/local/lib/python2.7/site-packages/TileStache-1.31.1-py2.7.egg/TileStache/Goodies/Providers/UtfGridComposite.py", line 90, in addLayer
    for ext in xrange('a', 'z'+1):
TypeError: cannot concatenate 'str' and 'int' objects

Here's my provider config & the full stack trace

I'm not familiar with python or the internal workings of tilestache & providers but my guess is what is desired here is a range of ascii codes for characters a through z?

tilestache-compose.py only does center, zoom, and dimensions despite docs

The docs say that tilestache-compose.py can specify a map three ways but only the first actually works.

Steps

  1. python /usr/local/bin/tilestache-compose.py -c tilestache.cfg --l terrain -e 50.1092810 -122.9948797 50.1553899 -122.9300267 -z 14 output.jpg

Expected results

  • A jpg to be generated

Actual results

Error: bad map coverage, center and extent can't both be set.

Problem

The defaults are set leading to the condition at tilestache-compose.py line 166 and similar to succeed and the error is thrown

Workaround

Empty the defaults at line 101.

No features detected using PostGIS-backed vector provider

Pulling from a table with a few MultiPolygon geometries, no features seem to be detected no matter the zoom level or x y positions.

print "pre-filter features: %i projection: %s" % (layer.GetFeatureCount(), projection,) 
# 
# Spatially filter the layer 
# 
bbox = _tile_perimeter_geom(coord, projection, clipped == 'padded') 
bbox.TransformTo(layer_sref) 
layer.SetSpatialFilter(bbox) 
print "post-filter features: %i" % (layer.GetFeatureCount(),) 

which shows 2245 features before filtering, and 0 after.

Configuration: https://gist.github.com/2650080
PostGIS table is using the 900913 SRID with the definition provided here: http://tilestache.org/doc/TileStache.Vector.html

Turns out that
layer.SetSpatialFilter(bbox)
is filtering out everything, because the problem disappears when I comment it out.

Getting "ReadError: not a gzip file" when trying to pip install TileStache on some machines

I hope you help me shed some light on this problem!

I found that when I run pip install TileStache on an ubuntu machine, it is failing with the error:

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 1676, in open
    return func(name, filemode, fileobj, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 1727, in gzopen
    raise ReadError("not a gzip file")
ReadError: not a gzip file

I noticed that when I wget http://tilestache.org/download/TileStache-1.46.7.tar.gz, I end up with the file as tar file (1.0MB), rather than a gzipped tar file (200KB). However when I download in the browser it is the gzipped tar file.

I think it might be something to do with the server response headers? As a comparison, I uploaded the gzipped tar file to CloudApp and ran pip install http://cl.ly/1O3I322n0S3m/TileStache-1.46.7.tar.gz, which works!

Any help much appreciated. Thanks!

PostgreSQL primary key not included in GeoJSON feature properties

I'm using a vector provider with the PostgreSQL driver. I specify in that provider definition,
"properties": ["code", "name"]
In this case "code" is the primary key of the table I'm pulling from, yet in the geojson that tilestache creates only the "name" property is included. I want the "code" as well.

I've tried omitting the properties entry in the tilestache cfg but the same happens.

I've tried updating my postgres table with another column with the same values and type as the primary key and it is included in the geojson. How can I get the "code" primary key included?

Thanks.

error in composite example

Having trouble getting TileStache to work (I'm not particularly programming literate) but have found an error in the composite.cfg file.

It has:
"composite":
{
"provider": {"class": "TileStache.Goodies.Providers.Composite.Composite",
"kwargs": {"stackfile": "composite-stack.xml"}}
}

This gives an error about being unable to find TileStache.Goodies.Providers.Composite . Looking at the docs I think the provider line should read:

"provider": {"class": "TileStache.Goodies.Providers.Composite:Provider",

That moved the scripts on but I've run into what appears to be a problem with python in ubuntu natty lacking zlib support....

sigh

Thanks
Gary

PS really like the idea behind the project

Allow dynamic jsonp callbacks via GET params

Instead of configuring MapnikGrid layers with hardcoded callbacks, some clients may want to pass a ?callback=... on the GET request in order to determine the callback function dynamically. This can come in quite useful in using the grids with a variety of clients without having to cache a set of json files for every possible callback function.

This simple patch seems to work pretty well and provide this functionality with minimal trouble.

produce TileJSON

An idea is to support providing TileJSON through the TileStache REST API for layers which have been configured in your TileStache config file.

For example if you specify bounds of a layer in your TileStache config file, then the TileJSON for that layer would include those bounds.

The TileJSON would be served up at somewhere like tilestache.cgi/layer-name.jsonp

If TileStache were to support this it could add extra fields to the config file like description, attribution, layer name of UTF grid, etc.

It would also need an extra config option for the public facing URL prefix. ie. http://example.com/ because TileStache won't know this.

There are both pros and cons of supporting this in TileStache, but it is an idea.

MapnikGrid provider doesn't find mapnik file in configuration directory

When using the mapnik provider, if I just give a relative filename for the mapfile parameter, it will look for the mapfile in the same directory as the tilestache configuration file, but for the MapnikGrid provider, this gives the following error:

Traceback (most recent call last):
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line 1537, in HandlerDispatch\n default=default_handler, arg=req, silent=hlist.silent)
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line 1229, in _process_target\n result = _execute_target(config, req, object, arg)
File "/usr/lib64/python2.6/site-packages/mod_python/importer.py", line 1128, in _execute_target\n result = object(arg)
File "/var/www/html/tiles/geoiq.py", line 233, in handler\n return handle_get(request)
File "/var/www/html/tiles/geoiq.py", line 130, in handle_get\n mimetype, content = TileStache.requestHandler(config, path_info, query_string)
File "/var/www/html/tiles/TileStache/init.py", line 287, in requestHandler\n mimetype, content = getTile(layer, coord, extension)
File "/var/www/html/tiles/TileStache/init.py", line 96, in getTile\n tile = layer.render(coord, format)
File "/var/www/html/tiles/TileStache/Core.py", line 373, in render\n tile = provider.renderTile(width, height, srs, coord)
File "/var/www/html/tiles/TileStache/Goodies/Providers/MapnikGrid.py", line 67, in renderTile\n mapnik.load_map(self.mapnik, str(self.mapfile))
[RuntimeError: Could not load map file 'style.xml': File does not exist

it seems like MapnikGrid requires a full path to the map file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.