Giter Club home page Giter Club logo

cartocsshelper's Introduction

CartoCSS Helper

It is a tool making development of CartoCSS styles more efficient. Allows to write simple Ruby scripts that generate comparison images of how rendering changed between two versions of map style. It is also possible to generate validation reports detecting for example too low values of text-dy. Loading data using osm2pgsql, rendering with TileMill, obtaining test data from overpass turbo.

Proper testing of map rendering requires testing covering mutiple situations. Doing it manually means preparing test data and locating places with applicable real data, loading map data into database, then moving viewport of TileMill to check every data location across multiple zoom levels. It is painfully slow.

Fortunately it is possible to automate it almost entirely. With this tool testing new rendering change requires only specification of tag combination that should be tested and type of element.

It is work in progress, major problems that should be solved include:

  • Improving documentation
  • Executing git checkout commands on map style git repository during normal script operation (#9)
  • Current rendering method is obnoxiously slow. (#1)

Installation

Assumes that osm2pgsql and TileMill are installed.

This tool is available as cartocss_helper Ruby gem.

Install ruby - see https://www.ruby-lang.org/en/documentation/installation/ for details.

Unfortunately, standard gem install cartocss_helper may not be enough as CartoCSS Helper depends on RMagick gem that requires special installation.

  1. install software necessary to install RMagick gem. On Lubuntu 14.04 it was enough to run sudo apt-get install ruby-dev imagemagick libmagickcore-dev libmagickwand-dev.
  2. gem install --user-install rmagick
  3. gem install --user-install cartocss_helper.

Examples

It is assumed that CartoCSS project with Default OSM Style (https://github.com/gravitystorm/openstreetmap-carto/) is located at ~/Documents/MapBox/project/osm-carto.

require 'cartocss_helper'

CartoCSSHelper::Configuration.set_style_specific_data(CartoCSSHelper::StyleDataForDefaultOSM.get_style_data)
project_location = File.join(ENV['HOME'], 'Documents', 'MapBox', 'project', 'osm-carto', '')
CartoCSSHelper::Configuration.set_path_to_cartocss_project_folder(project_location)
output_folder = File.join(ENV['HOME'], 'Documents', 'CartoCSSHelper-output', '')
CartoCSSHelper::Configuration.set_path_to_folder_for_output(output_folder)
cache_folder = File.join(ENV['HOME'], 'Documents', 'CartoCSSHelper-tmp', '')
CartoCSSHelper::Configuration.set_path_to_folder_for_cache(cache_folder)

tags = {'landuse' => 'village_green', 'tourism' => 'attraction'}
CartoCSSHelper.test tags, 'master', 'v2.28.1'

or, to just test on real examples of landuse=quarry tagged on ways (most will form areas)

require 'cartocss_helper'

CartoCSSHelper::Configuration.set_style_specific_data(CartoCSSHelper::StyleDataForDefaultOSM.get_style_data)
project_location = File.join(ENV['HOME'], 'Documents', 'MapBox', 'project', 'osm-carto', '')
CartoCSSHelper::Configuration.set_path_to_cartocss_project_folder(project_location)
output_folder = File.join(ENV['HOME'], 'Documents', 'CartoCSSHelper-output', '')
CartoCSSHelper::Configuration.set_path_to_folder_for_output(output_folder)
cache_folder = File.join(ENV['HOME'], 'Documents', 'CartoCSSHelper-tmp', '')
CartoCSSHelper::Configuration.set_path_to_folder_for_cache(cache_folder)

tags = {'landuse' => 'quarry'}
branch_name = 'quarry-icon'
base_branch_name = 'master'
zlevels = 10..22
type = 'way' #or 'node'
test_locations_count = 10
CartoCSSHelper.test_tag_on_real_data_for_this_type(tags, branch_name, base_branch_name, zlevels, type, test_locations_count)

First part of this Ruby script is responsible for loading gem and configuration, including location of cache folder and folder where produced comparison images will be saved.

Second part runs quick test for specified tag combination rendering only this element as node, way and a closed way. It is followed by locating multiple places across globe where such tag combination is used.

For each test case images are produced both for current master branch and release v2.28.1 across multiple zoom levels. Finally tool generates before/after comparisons for each case. Some of generated images were used in gravitystorm/openstreetmap-carto#1371.

It is also possible to look for certain keys, with any value:

tags = {'landuse' => 'village_green', 'tourism' => 'attraction', 'name' => :any_value}
CartoCSSHelper.test tags, 'master', 'v2.28.1'

Tests

Tests are written using rspec. Use rspec command to run them.

Automated stuff

Code Climate Build Status

RubyGems link

https://rubygems.org/gems/cartocss_helper

cartocsshelper's People

Contributors

matkoniecz avatar vassilevsky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

cartocsshelper's Issues

handle failed overpass responses not marked by HTTP error code

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="2015-07-03T17:03:02Z"/>

<remark> runtime error: Query run out of memory in "recurse" at line 1 using about 541 MB of RAM. </remark>

</osm>

see tyrasd/overpass-turbo#176

remove png gamma information from generated images

Reported by @imagico in gravitystorm/openstreetmap-carto#1728 (comment)

If you use convert +gamma - (preventing imagemagick to generate png gamma information) for generating your preview pngs you can probably make them show up with correct colors in firefox (why firefox and chrome show this differently is a mystery to me).\

see https://hsivonen.fi/png-gamma/ http://superuser.com/questions/579216/why-does-this-png-image-display-differently-in-chrome-firefox-than-in-safari-a

support rest-client chunking to allow large downloads

rest-client/rest-client#359 (comment)

require 'rest-client'
File.open('test', 'w') {|f|
    block = proc { |response|
      response.read_body do |chunk|
        puts "Working on response" 
        f.write chunk
      end
    }
    RestClient::Request.new(method: :get, url: 'http://overpass-api.de/api/interpreter?data=[timeout:3600];(node(51.8455934,4.8238493,52.3455934,5.3238493);%3C;);out%20meta;', block_response: block).execute
}

Check for missing labels

input: tag where lack of label is OK
output: tags with missing labels

separate check for ways and nodes (see waterway=dam bug caused by a missing _)

Use post instead of get to allow very large querries

see https://josm.openstreetmap.de/ticket/15141 for an equivalent issue in JOSM

very large queries hit URL limit that may be solved by using POST rather than get

see https://gist.github.com/matkoniecz/b27f17f34832fdf2115c915cee46c158 for an example resulting in

/home/mateusz/.gem/ruby/2.3.0/gems/cartocss_helper-5.0.1/lib/cartocss_helper/util/generic_downloader.rb(39) : ExceptionWithResponse
/home/mateusz/.gem/ruby/2.3.0/gems/cartocss_helper-5.0.1/lib/cartocss_helper/util/generic_downloader.rb(40) : 414 URI Too Long
/home/mateusz/.gem/ruby/2.3.0/gems/cartocss_helper-5.0.1/lib/cartocss_helper/util/generic_downloader.rb(60) : <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

It seems that one way to do that is

require 'net/http'
require 'json'

query = '[out:json][timeout:25];(way["surface"="beton"];);out body;>;out skel qt;'
url = "http://overpass-api.de/api/interpreter"
uri = URI(url)
response = Net::HTTP.post_form(uri, { 'data' => query })
puts JSON.parse(response.body)

cache git repository

Currently I am caching git repository in an external wrapper. But I am unsure whatever it would be worse to force everybody to do this or risk corrupt data.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.