Giter Club home page Giter Club logo

skinnywms's Introduction

The skinny WMS is a small WMS server that will help you to visualise your NetCDF and Grib Data. The principle is simple: skinny will browse the directory, or the single file passed as argument, and try to interpret each NetCDF or GRIB files. From the metadata, it will be built the getCapabilities document, and find a relevant style to plot the data.

Docs Upload Python Package PyPI version Upload Python Package Docker Pulls Anaconda Version Anaconda Downloads License

Features:

SkinnyWMS implements 3 of the WMS endpoints:

  • getCapabilities: Discover the data, build an XML Document presenting each identified parameter in the file(s) as a layer with the list of their predefined styles. (There is always a default style)
  • getMap : Return the selected layer suing the selected style.
  • getLegendGraphic: Return the legend.

Usage:

There are 2 ways to start using it, they both will start a small Flask server. Once running, a small leaflet client is accessible [http://127.0.0.1:5000/]

  • The demo:
python demo.py --path /path/to/mydata
  • The command line:
skinny-wms --path /path/to/mydata
  • Or with uwsgi:
uwsgi --http localhost:5000 --master --process 20 --mount /=skinnywms.wmssvr:application --env SKINNYWMS_DATA_PATH=/path/to/mydata

Run using Docker

By default the docker image will start the application using uwsgi and will load and display some demo data.

  • Run the demo:
docker run --rm -p 5000:5000 -it ecmwf/skinnywms 

Now you can try the leaflet demo at http://localhost:5000/

  • Run using data on your machine:
docker run --rm -p 5000:5000 -it \
    --volume=/path/to/my/data:/path/inside/the/container \
    --env SKINNYWMS_DATA_PATH=/path/inside/the/container \
      ecmwf/skinnywms

Now you can access the leaflet demo with your data at http://localhost:5000/

  • Configure different options by setting environment variables accordingly:
docker run --rm -p 5000:5000 -it \
    --volume=/path/to/my/data:/path/inside/the/container \
    --env SKINNYWMS_DATA_PATH=/path/inside/the/container \
    --env SKINNYWMS_HOST=0.0.0.0 \
    --env SKINNYWMS_PORT=5000 \
    --env SKINNYWMS_MOUNT=/mymodel/ \
    --env SKINNYWMS_UWSGI_WORKERS=4 \
    --env SKINNYWMS_ENABLE_DIMENSION_GROUPING=1 \
      ecmwf/skinnywms

Now you can access the ```GetCapabilities`` document for your data at http://localhost:5000/mymodel/wms?request=GetCapabilities

Installation

SkinnyWMS depends on the ECMWF Magics library.

If you do not have Magics installed on your platform, skinnywms is available on conda forge https://conda-forge.org/

conda config --add channels conda-forge
conda install skinnywms

If you have Magics already installed you can use pip:

pip install skinnywms

Limitations:

  • SkinnyWMS will perform better on well formatted and documented NetCDF and GRIB.

  • grib fields containing corresponding wind components u,v need to be placed together in a single grib file in order to be displayed as vectors/wind barbs in SkinnyWMS. You can combine multiple grib files into a single file using ecCodes grib_copy (included in the docker image), e.g.:

grib_copy input_wind_u_component.grb2 input_wind_v_component.grib2 output_wind_u_v_combined.grb2

Add your own styles

Multi-process

Cache

How to install Magics

that must be installed on the system and accessible as a shared library. Some Linux distributions ship a binary version that may be installed with the standard package manager.

As an alternative you may install the official source distribution by following the instructions at https://software.ecmwf.int/magics/Installation+Guide Magics is available on github https://github.com/ecmwf/magics

Note that Magics support for the Windows operating system is experimental.

Alternatively you can use the ecmwflibs package (currently in Alpha development stage) to install magics and eccodes libraries:

pip install ecmwflibs

Start up a local development environment (Docker)

Make sure you have Docker and docker-compose installed. Then run:

docker-compose up

This will build a dev image and start up a local flask development server (with automatic reload on code changes) at http://localhost:5000 based on the configuration stored in docker-compose.yml and .env and by default try to load all GRIB and NetCDF data stored in skinnywms/testdata.

Contributing

The main repository, as well as related projects are hosted on GitHub. Testing, bug reports and contributions to all our projects are highly welcomed and appreciated:

Lead developers:

  • Sylvie Lamy-Thepaut <https://github.com/sylvielamythepaut>_ - ECMWF
  • Baudouin Raoult <https://github.com/b8raoult> - ECMWF
  • Eduard Rosert <https://github.com/EduardRosert> - ECMWF

Main contributors:

  • Stephan Siemen <https://github.com/stephansiemen>_ - ECMWF
  • Milana Vuckovic <https://github.com/milanavuckovic> - ECMWF

License

Copyright 2017-2019 European Centre for Medium-Range Weather Forecasts (ECMWF).

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

skinnywms's People

Contributors

b8raoult avatar bjoern-reetz avatar brancomat avatar edigiacomo avatar eduardrosert avatar francesconazzaro avatar maxdow avatar sandorkertesz avatar sylvielamythepaut avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

skinnywms's Issues

Legend is incorrectly handled when layer not found

See this code in WMSServer.getLegend():

try:
     legend = self.availability.layer(layer, time)
except errors.LayerNotDefined:
     legend = self.plotter.layer

The problem is that in the except branch plotter.layer is a function and should properly called with arguments.

Allow specifying the vertical longitude for polar stereographic projection

The EPSG codes do not allow us to specify the vertical longitude for polar stereographic projections. For both EPSG:32661 (polar north) and EPSG:32761 (polar south) the vertical longitude is 0. So we go for a non-standard solution and allow encoding the vertical longitude into the EPSG code as follows:

polar north: EPSG:32661:vertlon
polar south: EPSG:32761:vertlon

where vertlon is a number in the range of [-180, 180]

Parse a specific NetCDF attribute to set the WMS layer name

Is your feature request related to a problem? Please describe.

At the moment if I set skinnywms to parse a folder containing several NetCDFs of the same variable I can only distinguish the resulting WMS layers by their timestamp or predefined levels with specific standard names ("air_pressure", "altitude", "plev", "isobaricInhPa", "model_level_number").

For any other coordinates there is no way to distinguish the WMS layer and only the fist or last NetCDF parsed by skinnywms is visible in the GetCapabilities.

The workaround I found is to rename the NetCDF variable to be unique and specific to each 2D file to that I can find the unique WMS layer name in the GetCapabilities.

Describe the solution you'd like

Changing the NetCDF variable name to obtain a unique WMS layer name is rot really a nice thing to do but a handy alternative would be to parse for a specific NetCDF attribute to set the WMS layer name such as wms_layer_name or skinnywms_name.

This would allow to create unique WMS layers from a folder containing 2D NetCDFs of the same variable based on that attribute.

A typical example I am currently working on are storm footprints which are a 2D fields of wind gust. The "levels" which I am not able to distinguish with skinnywms are for example:

  • which algorithm was used to generate the footprint,
  • whether the footprint is downscaled or not,
  • whether the footprint is decontaminated or not.

All these "levels" would be easy to include in a WMS layer name.

Describe alternatives you've considered

The alternative would be to support more coordinates with a wider variety of standard names but this seems more complex and less generic.

Additional context

A specific attribute (magics_style) is already parsed to set the WMS style and provides a handy alternative to the automatic style setting and a great way to pass a custom style.
The solution I propose to set the WMS layer name seems very similar to me and I hope it will be accepted and easy enough to implement.

Organisation

B-Open

Adding support for EPSG:3035

Best projection for european area

Bounding box :
1896628.62 1507846.05
4662111.45 6829874.45
proj string
+proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs

HRRR GRIB2 File

I get the following error message when launching the app from the command line:
$ skinny-wms --path /home/rik/vis.t17z.f00.grib2

The file is a HRRR GRIB2 file using Lambert Conformal projection.

Error message:
Exception: Unsupported grid type 'lambert' in grib /home/rik/vis.t17z.f00.grib2
127.0.0.1 - - [27/Jul/2020 15:43:19] "GET /favicon.ico HTTP/1.1" 404 -

Temporary files management

If I'm not mistaken the temporary files (wms-server-????????.png) are not deleted nor reused once served, this could trigger some disk space issues in the long run.

It would be also great to have the possibility of optionally configure the tmp dir (e.g. for assigning it to a /dev/shm device for performance enhancement)

incorrect unit in temporal extent within getCapabilities document

The time step of the temporal extent within the WMS capabilities document is rendered incorrectly when the time steps are not an integer number of hours.

Expected behaviour (step of PT15M):

<Dimension name="time" default="2022-02-14T03:00:00Z" units="ISO8601" multipleValues="0" nearestValue="0">
2022-02-14T03:00:00Z,2022-02-14T03:15:00Z/2022-02-15T03:45:00Z/PT15M
</Dimension>

Actual behaviour (step of PT0H):

<Dimension name="time" default="2022-02-14T03:00:00Z" units="ISO8601" multipleValues="0" nearestValue="0">
2022-02-14T03:00:00Z,2022-02-14T03:15:00Z/2022-02-15T03:45:00Z/PT0H
</Dimension>

Steps to reproduce:

  • download and extract some total precipitation data of DWD's ICON-D2 model in regular_lat_lon grid format (which has a temporal resolution of 0.25 hours)
  • launch SkinnyWMS with the data in it's SKINNYWMS_DATA_PATH, e.g. docker run --rm -p 5000:5000 --mount type=bind,source=$(pwd),target=/grib_data -e SKINNYWMS_DATA_PATH=/grib_data docker.io/ecmwf/skinnywms:latest
  • look into the layer info within the WMS capabilities document, e.g. http://localhost:5000/wms?request=getCapabilities

The PT0H is most likely an artefact due to rendering 0.25 hours with "PT%d%s" % (step, unit) in datatypes.py.

Problem with axis order and direction in BBOX parameter

First of all, thank you for skinnywms!

Unfortunately, it seems that the server has some problems with the axis order and direction in the BBOX parameter.

  • In WMS < 1.3.0, the axis order and direction in BBOX are always left,bottom,right,top:

    • In EPSG:4326 the order is min_lon,min_lat,max_lon,max_lat
    • In EPSG:3857 the order is min_x,min_y,max_x,max_y
  • In WMS 1.3.0, the axis order and direction in BBOX depend on the spatial reference:

    • In EPSG:4326 the order is min_lat,min_lon,max_lat,max_lon
    • In EPSG:3857 the order is min_x,min_y,max_x,max_y

If I understand correctly, skinnywms considers every BBOX as
min_lat,min_lon,max_lat,max_lon. With EPSG:4326 and WMS 1.3.0 everything
works fine, but e.g. with EPSG:4326 and WMS 1.1.1 the tiles are wrongly placed.

References:

EPSG:3857 doesn't render the map correctly.

With EPSG:4326 the map is not rendered correctly. I think that the bounding box should be converted to lat/lon before passing it to macro.mmap. I'll file a PR soon to show a possible fix using pyproj.

Adding user defined, timeless geo-referenced data i.e cities

Is your feature request related to a problem? Please describe.

Hello, looking for a way to add above kind of data. I've tried to add them via geojson file but it seems class GeoJSONReader requires a property 'time'.

Describe the solution you'd like

Perhaps it would be nice to have the possibility to add geojson files with timeless data into SKINNYWMS_DATA_PATH

Describe alternatives you've considered

It seems a way could be via a StaticLayer, defined in magics.py, similar to what is described here: https://confluence.ecmwf.int/display/MAGP/Symbol+examples+symbol5, but feed input_x/y_values & symbol_text_list filled with the data from a (i.e geojson) file.
I've also seen the --baselayer parameter but I got a "shp file not found" error and didn't found an example

Additional context

No response

Organisation

No response

Unsupported grid type 'mercator'

Hi,
I was trying skinnywms with some wrf model which have a mercator projection and i got the exception in subject?
Any suggestion or turnaround?

Thanks,
Davide

documentation for creating custom styles

If I understand correctly, skinnywms uses magics' default styles in /usr/share/magics/styles/, but it's possible to use a different path with the --style parameter.

Is there any documentation or best practices for the json files in that directory? Particularly on matching products/levels with the desired style.
It seems that there are generic style (contour) definitions and other files with a match between a combination of levels/parameters and those styles.

Thank you.

different name layer for similar grib files

Good morning i have a question using skinnywms with grib file with same name of the field
example temp 2m for two hours (00, 12Z) have same name of field t2m.
how i can change the LAYER name in the wms request ?
tks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.