Giter Club home page Giter Club logo

geometric_features's Introduction

Geometric Features

Build Status

This repository houses definitions of geometric features. These features can include regions, transects, and points, described in geojson format. For example, here are some regions for Antarctica. alt text

Documentation

http://mpas-dev.github.io/geometric_features/stable/

Contributors

Data for regions has been derived from sources listed in the contributors file as specified via the author field in each geojson file.

Quick Start

The python geometric_features package can be used to help maintain and use this repository. Several example scripts that make use of the package can be found in the examples directory. Each of the classes and functions that make up the package have extensive documentation. More user-level documentation will be added shortly.

To use geometric features, you can install it in a conda environment:

conda create -n geometric_features -c conda-forge python=3.9 geometric_features
conda activate geometric_features

To develop geometric_features (e.g. to add new features), create and activate an environment with all of the required dependencies:

conda create -y -n mpas_dev --file dev-spec.txt
conda activate mpas_dev
python -m pip install -e .

A typical workflow will look like:

  • Create a GeometricFeatures object and point it to a location where you have stored (or would like to store) geometry data.
    • gf = GeometricFeatures(localCache='./geometric_data')
  • Read in one or more FeatureCollections from the geojson files in the geometric_data directory.
    • fcArctic = gf.read('ocean', 'region', featureNames=['Arctic Ocean'])
    • fcAtlantic = gf.read('ocean', 'region', tags=['Atlantic_Basin'])
  • Edit features:
    • Merge, combine, tag, mask out or simplify the features, see below.
    • Use the shapely package to edit the geometry in more sophisticated ways
  • Visualize features:
    • fc.plot(projection='cyl')
  • Split feature collection back into individual features for inclusion in the repo:
    • gf.split(fc)

Available functionality includes:

  • fc.merge(other) - Merge two feature collection together.
  • fc.combine() - Combine features into a single feature.
  • fc.difference() - Mask features using shapes in a second feature file.
  • fc.fix_antimeridian() - Split a feature at the antimeridian (+/- 180 longitude). The resulting feature has all points between -180 and 180 lon.
  • fc.set_group_name() - Set the "groupName" property of the FeatureCollection
  • fc.tag() - Add one or more tags to the "tag" property of each feature in a collection. This can be useful for reading back a collection of features with that tag.

IMPORTANT: Always use the gf.split(fc) script when placing features into the geometric_data directory. This will help everyone maintain the repository, and allow tools to parse them cleanly.

Many of this functionality can also be accessed with a command-line interface:

merge_features
combine_features
difference_features
set_group_name
split_features
simplify_features
tag_features
plot_features

Use the -h flag to find out more.

An example workflow to read in, plot and write out a set of features is

#!/usr/bin/env python

from geometric_features import GeometricFeatures
import matplotlib.pyplot as plt

# create a GeometricFeatures object that points to a local cache of geometric
# data and knows which branch of geometric_feature to use to download
# missing data
gf = GeometricFeatures(cacheLocation='./geometric_data')

# read in a FeatureCollection containing all ocean regions in the Atlantic
# basin
fcAtlantic = gf.read(componentName='ocean', objectType='region',
                     tags=['Atlantic_Basin'])

fcAtlantic.plot('cyl')
plt.title('Atlantic merged')

# combine them all into a single feature
fcAtlantic = fcAtlantic.combine(featureName='Atlantic_Basin')
fcAtlantic.plot('cyl')
plt.title('Atlantic combined')

# make another feature containing the regions in Filchner-Ronne Ice Shelf
fcFilchnerRonne = gf.read(componentName='iceshelves', objectType='region',
                          featureNames=['Filchner_1', 'Filchner_2',
                                        'Filchner_3', 'Ronne_1', 'Ronne_2'])
fcFilchnerRonne.plot('southpole')
plt.title('Filchner-Ronne')


# make one more collection of all the IMBIE basins in West Antarctica
fcWestAntarctica = gf.read(componentName='landice', objectType='region',
                           tags=['WestAntarcticaIMBIE'])

fcWestAntarctica.plot('southpole')
plt.title('West Antarctica')

fcWestAntarctica.to_geojson('west_antarctica.geojson')
plt.show()

geometric_features's People

Contributors

cbegeman avatar douglasjacobsen avatar mark-petersen avatar matthewhoffman avatar milenaveneziani avatar pwolfram avatar stephenprice avatar xylar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

geometric_features's Issues

Version of coastline data?

I am looking for the name of the dataset that we use for the coastlines for culling land cells from global meshes. It is this file:

https://github.com/MPAS-Dev/geometric_features/blob/master/geometric_data/natural_earth/region/Land_Coverage/region.geojson

which only has a label of

            "properties": {
                "name": "Land Coverage",
                "tags": "",
                "object": "region",
                "component": "natural_earth",
                "author": "http://www.naturalearthdata.com/"

I suspect it is 10m coastline data set found here:
https://www.naturalearthdata.com/downloads/10m-physical-vectors/10m-coastline/

Does anybody know? We should add the name and also a version number, if possible, to the meta data.

setup_ocean_region_groups does not appear to work properly

This came up because I am trying to create consistent MOC regions (ocean basin regions for which we wish to compute the MOC streamfunction and zonal MHT). The plan is to:

  1. create the MOC regions feature collection, like we intended to in the beginning;
  2. create the southern transect manually, looking at the southern transect in the features in 1) and adding many more points in between (to ensure a better transformation into a maskfile by MpasMaskCreator);
  3. create the new maskfile from 1) and 2), to be used for the online/offline MOC (and eventually the online/offline MHT).

I started using setup_ocean_region_groups from the examples/ folder to get to step 1). But I am getting a feature collection that is incorrect, in the sense that, instead of a merge, it seems more like a combine (only one huge feature is there).
I believe there is a similar problem for the oceanBasins geojson file.

Testing to ensure requirements are met

Following the accidental removal of groupName, we should consider implementing an integrated testing environment that ensures requirements are met, e.g., minimally that groupName and other properties are outputed at appropriate points within the workflow. Note, there are probably other things that should be tested.

cc @xylar, @milenaveneziani

Provenance producing git errors when not in a valid git repository

Currently, when any scripts in this repo that create new geojson files are run in a directory that is not a git repo (e.g. when setting up MPAS test cases), the following error is produced:

fatal: Not a git repository (or any parent up to mount point /home/xylar)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

This is confusing, as it sounds serious but simply means that the git repo for geometric features could not be located.

This could be addressed in one of 2 ways:

  1. cd to the location of the scripts themselves before calling any git commands or
  2. simply make the script fail silently

My inclination would be toward 1, though 2 would be slightly easier.

setup_MOC_basins re-defines three Basin regions that are already defined in setup_ocean_basins

Atlantic_Basin.geojson, Indian_Basin.geojson and Pacific_Basin.geojson are defined by setup_ocean_basins.py (together with 3 more Ocean Basin regions). They are then re-defined in setup_MOC_basins.py and they are different from the equally-named features defined in setup_ocean_basins.

To avoid confusion and also unify the way that all this basin-scale regions (and their groupnames) are defined it seems like we should have one single script that:

  • creates Ocean Basins (groupName=OceanBasinRegionsGroup);
  • creates MOC Basins (groupName=MOCBasinRegionsGroup), without overriding the previously defined Basin features;
  • creates MOC southern transects, as in issue #69;
  • defines the group for the Global* features (already in the repo, in ocean/region) as the same groupName as the other Ocean Basins.

Option to select remote fork other than MPAS-Dev and remote branch/fork with command-line tools

It is not currently possible to select a remote branch on a fork other than MPAS-Dev for downloading geometric_data. Support for other forks would likely be useful for debugging.

It is also not currently possible to make the command-line scripts to point to any remote branch other than the tag associated with the version in __init__.py. This makes debugging similarly challenging.

manipulate_features.py has identical functionality to merge_features.py

I think that @douglasjacobsen had intended to make manipulate_features.py be a general script that could perform several operations on features. However, the current version in the repository is identical to merge_features.py other than having a command line argument for an operation and a docstring that describes functionality it doesn't actually implement.

Unless a functioning version of this script will be added to the repo, I suggest we delete this script.

Update critical passages with depths based on bathymetry

Currently, critical passages have depth properties that were chosen ad hoc to ensure the ocean was deep enough to allow flow along the passage. A better method would be to break each critical passage into a number of smaller features (say 0.1 degrees long), each with its own depth taken from the ETOPO1 bathymetry. This way, critical passages could be used without harm in high resolution and lower resolution setups alike.

Support for python 3

Python 2 is going away and we need to start supporting python 3 before we run into trouble.

Plotting error with moll projection for combined Atlantic and Southern Ocean

Using the file at https://gist.github.com/43b8c23a27a84da35eadd9304a1b3753, there is a plotting error for the moll projection (which does not show up for moll2):

screenshot 2016-09-26 16 18 41

Script to reproduce error from repository root:

mkdir test_feature_collection
cd test_feature_collection/
ln -s ../ocean/region/North_Atlantic_Ocean/region.geojson NA_region.geojson
ln -s ../ocean/region/South_Atlantic_Ocean/region.geojson SA_region.geojson
ln -s ../ocean/region/Southern_Ocean/region.geojson SO_region.geojson
cd ..
./merge_features.py -d test_feature_collection/ -o AO_SO.geojson
./plot_features.py -f AO_SO.geojson 

Note, found in the process of testing #52. I believe this is a plotting error but it could be an error with preliminary changes in #52.

script to join a number of regions together to make one single region

@xylar, @pwolfram, @douglasjacobsen: tugging you on this, as you are probably most interested to provide input here.

Yesterday, I was trying to create a mask file for ocean basins, and specifically for 2 groups: oceanBasins and oceanBasinsMOC. These groups contain the ocean basins that can now be reproduced by using the drivers that @xylar wrote a few months back.
Let us take Atlantic_MOC as an example.
The problem is that the corresponding geojson file for this region is a collection of the subregions that make up Atlantic_MOC, whereas what we really need is a single region, called Atlantic_MOC, that is the result of the joint subregional polygons.
This way, say for the oceanBasinsMOC group, we will have 2 single regions: Atlantic_MOC and IndoPacific_MOC.
Does this make sense?

Would it be hard to write such a script?

Redefining Arctic and North Atlantic subpolar regions

I see the following two main issues with the current definition of 'Arctic' and sub Arctic regions:

  1. the current Arctic geojson feature is a weird region that covers most of the Arctic, but not quite all of it (excluding coastal seas such the Beaufort Sea, Chukchi Sea, etc);
  2. the subpolar North Atlantic regions are defined weirdly sometimes, perhaps more in line with geographical definitions rather than oceanographic ones: one example is the Lab Sea, which is a small triangle, south of a DavisStrait feature..

Below, I am suggesting a possible redefinition. Please let me know if you have an opinion on this, and your suggestions, so that I can go ahead and make a PR to address it.

Consolidate output from driver scripts into single geojson files

The setup_MOC_basins.py and setup_ocean_basins.py produce several individual geojson files, each with a single feature but all with the same groupName. Depending on how these features are being used by the MPAS mask creator, it would seem more logical to generate a single file (e.g. MOCBasins.geojson) that contains all the features with the same groupName.

Update of Maritime Boundaries Geodatabase

We may want to consider updating our regions based on the new standards published at http://www.marineregions.org, e.g., see the email below I received this morning:

Dear Marine Regions User,

MarineRegions.org is proud to launch the new version of the Maritime Boundaries Geodatabase. In this new release, Marine Regions updates the global Exclusive Economic Zones (EEZ) (version 9), and launches global datasets for Territorial Seas (TS), Contiguous Zones (CZ), Internal Waters (IW) and Archipelagic Waters (AW). The global baseline used in this new version is a combination of the publically available straight and archipelagic baselines and the coastline as a proxy for the low-water line. The source for the straight baselines is the United Nations repository on maritime claims, together with national publications on maritime delimitations and agreements. The ESRI Countries 2014 database has been used as the primary source for the Maritime Boundaries coastline. The coastline was further updated with reefs data extracted from the Coral Reef Distribution database from UNEP for the countries where reefs were fundamental for the correct calculation of the maritime areas.

With the new release, we hope to further serve the global marine data user community working on geospatial analysis. As Marine Regions depends on data and knowledge sharing from global, European, regional and national data providers and relevant experts, we aim to set up Collaboration Agreements with our users and data providers. An example template of a Collaboration Agreement can be found online, please contact [email protected] if your organization is interested to explore this collaboration.

All datasets from marineregions.org’s Maritime Boundaries Geodatabase continue to be the only freely-available global datasets on the World’s Maritime Boundaries.

With Kind Regards

The Marine Regions Team

I'd recommend we try to stay as faithful to this "standard" as possible, but at the same time I don't think we really can afford to change out region definitions too frequently either. This is obviously an issue up for broad discussion.

Adding definition of MOC regions southern transects to setup_MOC_basins

The MOC and MHT AM's need both the MOC regions and their southern border for doing their computation. Since these two features need to be consistent with each other, I thought that the driver setup_MOC_basins could do both things:

  1. define the MOC ocean basins;
  2. pick the southern border of each so defined basin and make a new transect feature.
    Does this sound like a reasonable thing to do?

Potentially remove fix_features_at_prime_meridian.py

Following discussions with @xylar, fix_features_at_prime_meridian.py is not used in current scripts and may not be needed anymore. If this is the case, we should consider removing this script (or merging it with fix_features_at_antimeridian.py).

region geojson files do not parse properly in standard tools

I found that geojson files in this repo would not display properly in any off-the-shelf tools I tried:

  1. QGIS
  2. http://geojson.io/
  3. http://geojsonlint.com/
  4. http://jansanchez.com/geojson-viewer/
  5. https://developers.google.com/maps/documentation/javascript/examples/layer-data-dragndrop

I spent a little while trying to figure out if there was mismatched brackets or something and could not find anything wrong. I also tried stripping out newlines, tabs, and even spaces but that did not help. I tried a few of the landice regions, and I tried a few of the ocean regions, even reducing them to a dozen or so points (in case any of those tools couldn't handle large data).

Anyway, as long as our MPAS tools work with format, this isn't a big deal, but it would be nice to quickly visualize or validate files with other off-the-shelf tools.

Antarctica_IMBIE26 and Antarctica_IMBIE27 have the same geometry

The Antarctica_IMBIE26 region (the basin for Larsen C) was not added correctly and instead contains the same geometry as Antarctica_IMBIE27.

$ diff Antarctica_IMBIE26/region.geojson Antarctica_IMBIE27/region.geojson
6c6
<           "name": "Antarctica_IMBIE26",

---
>           "name": "Antarctica_IMBIE27",
8c8
<           "tags": "AntarcticPenninsulaIMBIE;LarsenCIceShelfDrainage",

---
>           "tags": "AntarcticPenninsulaIMBIE",

Region file corresponding to hard-wired regions

Hi!

As part of my work on adding and testing region file support to some analysis members, I've created geojson files for the hard-wired regions that are used for some analysis members (e.g. Water Mass Census, and Surface Area Weighted Averages).

I propose to add them to this repository, I'm just not sure where exactly and under which name.

Currently, this is a region group containing:

  • Arctic: lat >= 60
  • Equatorial: -15 <= lat <= 15
  • Southern Ocean: lat <= -50
  • Nino 3: -5 <= lat <= 5; 210 <= lon <= 270
  • Nino 4: -5 <= lat <= 5; 160 <= lon <= 210
  • Nino 3.4: -5 <= lat <= 5; 190 <= lon <= 240
  • Global

This is what they look like

I can:

  • Add the entire region group (I propose to put that in ocean/region as Hard-Wired_Regions.geojson); or
  • Add each region individually, as it's already the case for other regions.

.. or something entirely different; I'll do whatever would be most useful.

Let me know what you prefer!

Ross Ice Shelf melt time series labels

Regional labels for the Ross Ice Shelf sectors should be changed from the Rignot et al. labels "Ross West" and "Ross East" to "eastern Ross" and "western Ross," respectively, or similar.

Plotting doesn't seem to work, at least not for Arctic basin

I just tried to plot all ocean basins and I found that the Arctic basin is plotting very differently in the 3 different views. It looks correct on GitHub so I think the problem is somehow with the plotting and not with data itself.

ocean_basins

In the figure, I think the North Pole plot is correct but the two mill plots are both very glitchy. I'm not sure the Arctic basin is the only one that isn't plotting correctly but it seems to be the most extreme case.

plot_features.py does not appear to support points

On IC, I am getting an error when trying to plot a features file containing points:

$ rm features.geojson
$ ./merge_features.py -d ocean/point/ -o features.geojson
$ ./plot_features.py -f features.geojson -m cyl
plot type: cyl
  feature: Equatorial_Pacific_E145.0_N00.0
  feature: Equatorial_Pacific_E155.0_N00.0
  feature: Equatorial_Pacific_E165.0_N00.0
  feature: Equatorial_Pacific_E175.0_N00.0
  feature: Equatorial_Pacific_W090.0_N00.0
  feature: Equatorial_Pacific_W095.0_N00.0
  feature: Equatorial_Pacific_W105.0_N00.0
  feature: Equatorial_Pacific_W115.0_N00.0
  feature: Equatorial_Pacific_W125.0_N00.0
  feature: Equatorial_Pacific_W135.0_N00.0
  feature: Equatorial_Pacific_W145.0_N00.0
  feature: Equatorial_Pacific_W155.0_N00.0
  feature: Equatorial_Pacific_W155.0_N02.0
  feature: Equatorial_Pacific_W155.0_N04.0
  feature: Equatorial_Pacific_W155.0_N06.0
  feature: Equatorial_Pacific_W155.0_N08.0
  feature: Equatorial_Pacific_W155.0_N10.0
  feature: Equatorial_Pacific_W155.0_S02.0
  feature: Equatorial_Pacific_W155.0_S04.0
  feature: Equatorial_Pacific_W155.0_S06.0
  feature: Equatorial_Pacific_W155.0_S08.0
  feature: Equatorial_Pacific_W155.0_S10.0
  feature: Equatorial_Pacific_W165.0_N00.0
  feature: Equatorial_Pacific_W175.0_N00.0
  feature: Southern_Ocean_Drake_Passage_W070.0_S55.0
  feature: Southern_Ocean_Drake_Passage_W070.0_S60.0
  feature: Southern_Ocean_Drake_Passage_W070.0_S65.0
saving features.png
Traceback (most recent call last):
  File "./plot_features.py", line 265, in <module>
    plot_features_file(args.features_file, mapInfo, max_length=args.max_length)
  File "./plot_features.py", line 215, in plot_features_file
    fig.savefig(plotFileName)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/figure.py", line 1563, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2232, in print_figure
    **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 527, in print_png
    FigureCanvasAgg.draw(self)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 474, in draw
    self.figure.draw(self.renderer)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/artist.py", line 62, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/figure.py", line 1159, in draw
    func(*args)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/artist.py", line 62, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/cartopy/mpl/geoaxes.py", line 372, in draw
    inframe=inframe)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/artist.py", line 62, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 2319, in draw
    a.draw(renderer)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/artist.py", line 62, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/cartopy/mpl/feature_artist.py", line 175, in draw
    **final_kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/collections.py", line 831, in __init__
    Collection.__init__(self, **kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/collections.py", line 139, in __init__
    self.update(kwargs)
  File "/usr/projects/climate/SHARED_CLIMATE/anaconda_envs/default-2.7/lib/python2.7/site-packages/matplotlib/artist.py", line 859, in update
    raise AttributeError('Unknown property %s' % k)
AttributeError: Unknown property marker

Create develop branch or an ACME v1 tag

We need to create a develop branch and/or a tag for ACME v1. Presumably we need to stop making merges to master except where compatible with ACME v1. This is because we need a way to recreate ACME initial conditions using features from the repo.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.