Giter Club home page Giter Club logo

qgreenland's Introduction

NSIDC logo NSF logo

QGreenland

NSF-1928393 NSF-1928393 DOI DOI

This repository is responsible for the code and configuration for creating the QGreenland Core zip package. To download the package and learn more about QGreenland, visit our our website.

For more detailed information about using the QGreenland Core zip package and on how to contribute to QGreenland, see our Documentation

🎉 QGreenland v3 has been released! Please visit our website to download it now! Note that an official annoucement and more exciting news is planned for early September. Subscribe to the QGreenland newsletter to learn more!

A Free GIS package for Greenland

QGreenland example images

QGreenland is a free mapping tool to support interdisciplinary Greenland-focused research, teaching, decision making, and collaboration. It combines key datasets into a unified, all-in-one GIS analysis and visualization environment for offline and online use.

An international Editorial Board and Project Collaborators connects the QGreenland Team to data and user communities.

Learn more about What is QGreenland?

Usage

For contributors

Those wishing to utilize the qgreenland code to create their own QGreenland data package should see the contributor How to build QGreenland Core guide.

For users of the QGreenland Core data package

See our Get started with QGreenland Core tutorial!

What is inside the QGreenland Core data package zip file?

At the root of the zip file, you will find useful files such as a UserGuide.pdf, the qgreenland.qgs QGIS project file and scientific discipline-specific directories containing data (GeoTIFFs and GeoPackages).

For more detailed information, see our documentation on the QGreenland Core Download Package.

Educational resources

We keep the QGreenland official website up-to-date with links to helpful educational resources, including our own QGreenland User Guide.

Troubleshooting

See our user troubleshooting guide here.

Contributing

See our discusson page on contributing to get started!

Contributor documentation contains technical instructions about running the QGreenland pipeline code, but we also strive to describe everything clearly. Our goal is to make it as easy as possible for any user of QGreenland to contribute to the project, so please do not be deterred from sharing your ideas.

If you have an idea for a new feature or have a bug to report, please submit an Issue.

If all else fails, please email us!

Acknowledgements

Please see our acknowledgements for our best effort to acknowledge all of the giants upon whose shoulders we stand.

qgreenland's People

Contributors

alysethurber avatar ansa6061 avatar dylenswan avatar hsimonoko avatar mfisher87 avatar trey-stafford avatar twilamoon-science avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

qgreenland's Issues

Update research stations layers (PROMICE + GC-NET)

Our current station locations for PROMICE and GC-NET layers is from the map_GC-Net_PROMICE_kml GitHub repo, which used "2017-2018 average of station GPS data" for station locations.

New stations have been added. The PROMICE and GC-Net automated weather station data in Greenland dataset from GEUS includes a AWS_metadata.csv with location data for stations (published May 2023 w/ citation indicating 2022; the last station added in the AWS_metadata.csv is QAS_Lv3 on 2022-08-29)

Validate `.qml` style files don't contain any unexpected fonts

PyQGIS seems to be translating some fontFamily= values from the style files to Helvetica, which is a non-free font. This would be fine if it would translate the fonts to an open font that can be downloaded by QGIS >= 3.28.

We could either throw an error for style files which contain any value other than a few allow-listed values, and/or we could provide a script that updates all ancillary/styles/*.qml files to use Open Sans with e.g. s/fontFamily="[A-Za-z ]+"/fontFamily="Open Sans"/g

QGreenland data access API?

Thanks for putting this package together!

I'm wondering if there is a way to retrieve individual data files without first downloading the entire QGreenland package. This would be very helpful for butting together a mybinder notebook as part of the NSF EarthCube notebooks call.

GEUS 1:500k geological map

  • Joe M. has a link to some data. It’s apparently really large but we could point users to it.

  • Someone at GEUS noted making a QGIS compatible version of the dataset from Joe M.

Consider renaming layers with ambigous titles

This ticket: #579

Reminded me that we have a problem with layer titles being ambigious without the context of the layer groups. Many tools in QGIS only display layer titles in dialog boxes, so a user is stuck with trying to figure out what e.g., a layer titled "January" is (which is a layer under the "Median extent" layer group, which is itself under "Sea ice")

Expose QGreenland data to the internet

Is there any way to access QGreenland data outside of the ZIP file here https://nsidc.org/qgreenland/stable/ ?

Can you unzip that on the server into a 'data' or 'src' sub-folder so that we can access the layers individually? If so, you would provide alternate access methods that support older hardware, smaller disks, faster partial downloads, etc. I as a 3rd party can take care of all that via my own interface (probably using datalad).

[Bug]: UTM Zones 404 on download

Issue description

The data is not hosted at the currently-configured URL anymore https://sandbox.oarc.ucla.edu/mapshare/data/world/data/utmzone.zip

System information

N/A

[Bug]: Docker image should have `micromamba`, but not `mamba`

Issue description

I'm not sure what is causing the issue that was resolved by #547 . We use micromamba to install environments, so we shouldn't ever need mamba in the Docker image, especially at runtime. I'm so confused about that... I think we need to look closer at whatever is causing that error!

TODO: Reproduction instructions for error in description of #547

System information

N/A

Frozen ground layer updates/additions

There are 3 "Frozen ground/" layers currently:

  • "Ground temperature 2000-2016 (10km)"
  • "Ground temperature standard deviation (10km)"
  • "Permafrost probability (10km)"

All three of these layers are from Obu et al., 2018:

Obu, Jaroslav; Westermann, Sebastian; Kääb, Andreas; Bartsch,
Annett (2018): Ground Temperature Map, 2000-2016, Northern
Hemisphere Permafrost. Alfred Wegener Institute, Helmholtz
Centre for Polar and Marine Research, Bremerhaven, PANGAEA

https://doi.org/10.1594/PANGAEA.888600

This dataset does not have an update with more recent data.

Are there other datasets we could add? Newer dataset we could use to replace the above?

Twila suggests reaching out to colleagues for suggestions:

Migrate from CircleCI -> GitHub Actions?

We can get CI for free for this project, plus we're moving that way in general as a dev group. Our CI isn't very complex, it'd be pretty easy to migrate. And probably easier to read post-migration as well.

While migrating, we might want to consider adding environment caching (#527); the micromamba action can do that pretty well.

[Bug]: `scripts/cli.sh` doesn't handle quoted strings well

Issue description

./scripts/experimental/local_cli.sh layers '*' lists all layers, but ./scripts/cli.sh layers '*' returns:

Usage: __init__.py layers [OPTIONS] PATTERN
Try '__init__.py layers -h' for help.

Error: Got unexpected extra argument (tasks)

Workaround

Escape the string aggressively:

./scripts/cli.sh layers \'\*\'

System information

  • Operating system: PopOS
  • Operating system version: 22
  • QGIS version: N/A
  • QGreenland version: 2.0.0

Troublesome characters in folder names

Hi QGreenland- thanks for setting this up- great resource.

I tried to look and see if I could resolve this myself in git, but it looks like not all the actual layers are on git. I'm seeing through a fresh install (v0.54.0 downloaded today) a few troublesome filenames, with "bad" characters. These are:

  • the forward slash ('/') in the topo map scale: QGreenland_v0.54.0/Terrain models/Topographic map (1/500,000)/
  • greater-than symbol ('>') in the earthquake magnitudes: QGreenland_v0.54.0/Geology/Earthquakes M>2.5 1900-2020/earthquakes.gpkg

Thanks!

Release next alpha version

Release the next alpha version with changes so far.

Tasks

[Bug]: Warning `Font "Helvetica" not available on system` in QGIS >=3.22

Issue description

From: QGR-134

TODO: Is the version specification correct? Maybe started in 3.18?

In older versions of QGIS, the error may manifest in a less visible way like this:

image

In newer versions, the error is surfaced to the viewport:

image

Workaround

Install helvetica.ttf (TODO: Find a safe source)

Proposed solution

Require minimum version of QGIS 3.28, which is capable of downloading missing fonts. Pick an open source font that will be used on all operating systems.

Commentary

Matt Fisher

Found this issue on GitHub: qgis/QGIS#46206

Looks like this warning was promoted from being hidden in the symbology menu to the main viewer, but it's still just a warning. I think we need:

Add a note about this to troubleshooting docs
If possible, select a default font for the whole project that is available in all major OSes to minimize the occurrence of the warning.

Trey Stafford

The "Helvetica" font seems to be set in the project file (qgreenland.qgs). I manually did a global replacement of all occurrences of "Helvetica" with "Sans Serif", which resolved the issue when opening the project in QGIS.

Unclear how this attribute gets set. It might be the QT library that's determining a default system font? Perhaps Helvetica is available in the docker image that we use to generate the project file?

I tried getting the default font using the PyQT library but was unsuccessful:

(base) root@1bfc695b3a88:/luigi# python
Python 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:23:14) [GCC 10.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.QtGui import QFont
>>> QFont().defaultFamily()
Segmentation fault (core dumped)
(base) root@1bfc695b3a88:/luigi#

System information

  • Operating system: Any except Windows
  • Operating system version: Any
  • QGIS version: >=3.22
  • QGreenland version: Any

[Dataset change] ocean temp layers to 'background' boundary

Does the package get unreasonably large?

Can we use downsampling? Probably don't need full res for background layer.

The reason we gave up on this in the past was because cutting this layer to the 40° extent resulted in a weird ring-shaped cut where the only data extant afterwards is the data between 40°N and 41°N

I think it's worth taking one more stab at this now that we're using CLI OGR tools. Maybe it will be easier than wrangling the Python bindings?

Then again, maybe we already tried making this work at the CLI. See if @trey-stafford remembers

Replace HOTOSM layers?

Can we replace the HOTOSM layers with something else, e.g., from Asiaq?

Aside from populated places, HOTOSM layers were:

  • "health_facilities"
  • "airports"
  • "seaports"
  • "waterways"
  • "financial_services"
  • "education_facilities"
  • "points_of_interest"
  • "roads"
  • "buildings"

Frederik H. from Asiaq says:

You can find the data here: https://kort.nunagis.gl/refserver/rest/services/Kortportal/Kortportal_TekniskGrundkort/MapServer

The building layer is called ‘BYGNING’, and the attribute containing building use is called ‘Info’. It is only in Danish though.

[Dataset]: MAR atmospheric model

Dataset URL

https://arcticdata.io/catalog/view/doi%3A10.18739%2FA2MV9S

Rationale

From: QGR-17

The data is available from ADC in 10+ GB decadal files. Is this the only format to access the data? We are waiting for information from Marco Tedesco. Unclear whether he can provide a publicly archived datasource or if they will email us file(s). The data are currently NetCDF files. Waiting for more emails! See [email protected] mailbox for discussion.

Next step: E-mail Marco again!

Update place names layer (Asiaq)

We want to update the Asiaq place names layer ("Places/Towns and settlements") because there are a number of noted data issues.

[Dataset upgrade]: Bedmachine to v5

Dataset URL

https://nsidc.org/data/idbmg4/versions/5

More info

From: QGR-173

Probably little/no change in our processing. Here are the updates to the dataset:

Added bathymetry measurements from the Alfred Wegener Institute,
Helmholtz Centre for Polar and Marine Research, in front of the Northeast Greenland Ice Stream

Added ice thickness for ice caps from Millan et al. (2022) (https://www.nature.com/articles/s41561-021-00885-z)

Included bathymetry data from NASA's 2021 Oceans Melting Greenland (OMG) campaign

Provided a geotiff file of bed elevation

[Dataset change] Consider removing alpha band from background layer

The background layer has an alpha band that is not currently used and appears to be all-nan (double-check this). If this is the case, consider removing that alpha band!

QGIS does not seem to like the 4th band, the following error message gets logged repeatedly while this layer is toggled on:

ERROR 5: /share/appdata/qgreenland/working-storage/wip-package/QGreenland/Basemaps/Background (500m)/background.tif: GDALDataset::GetRasterBand(4) - Illegal band #

Linux: "Too many open files"

QGreenland_v1.0.1 has quite a few issues.
I downloaded the zip from the webpage, unzipped it, and then opened the qgreenland.qgs

  1. there are 45 layers that are not available (dialog box)
  2. startup is extremely slow
  3. Critical: Could not create AF_NETLINK socket (Too many open files)
  4. Warning 4: Failed to open /home/data/QGreenland_v1.0.1/Places/Towns and settlements/populated_places.gpkg: Too many open files.

lots of the above two error messages occur (3./4.).

QGIS 3.18.1-Zürich
Arch Linux as of today
very performant laptop with lots of RAM (32G) of which almost nothing is used.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.