Giter Club home page Giter Club logo

qgis-stac-browser's Introduction

qgis-stac-browser's People

Contributors

kbgg avatar suricactus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

qgis-stac-browser's Issues

Downloaded image is in the wrong location wrt the query vector

Steps:

  1. Query the staccato api with the attached vector.
    study_area.zip
  2. Select a random scene and download a band with "Add to Layers" and "Stream COGs" option.

Expected:
query vector and downloaded image are coincident.

Actual:
query vector and band are very far away from one another. The distance is probably thousands of world sizes, so it seems to be an issue with CRS.

Screen Shot 2019-06-25 at 11 53 33 AM

Remove external package dependencies

QGIS doesn't have an easy way to allow plugins to have external dependencies. Right now the only external dependency is requests which would require users to install that package manually. The plugin should move to use urllib instead to remove this requirement.

improve http error messages, trying out pygeoapi with stac browser

tried to add api endpoint https://dev.api.weather.gc.ca/msc-wis-dcpc/stac
fails with a generic http error,

image

would be good to see some more details in log why connection failed
seems i had to add it without /stac:
https://dev.api.weather.gc.ca/msc-wis-dcpc
then is added fine

image

in next step to query the catalog fails also with very generic error

image

Would be good to have in error message some indication of an expected element in server answer

in this case the browser may not have been able to detect the available stac collections in the service, the payload was:

{
	"collections": [],
	"bbox": [223181.425, 480387.771, 245110.0, 503192.755],
	"time": "2000-01-01T00:00:00Z/2020-09-10T09:34:00Z",
	"limit": 50,
	"page": 1
}

so browser should have indicated it was not able to detect any stac collections in service

Error querying Planet Staccato

Version: dev/f5a35fba113c72129efc397a8a036ca5b8080af8

Repro:

  1. Open STAC dialog
  2. check Planet[Staccato]
  3. Search button
AttributeError: 'NoneType' object has no attribute 'extent' 
Traceback (most recent call last):
  File "/Users/pvarner/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/stac_browser/ui/query_dialog.py", line 59, in on_search_clicked
    self.get_time_period())
  File "/Users/pvarner/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/stac_browser/stac_browser.py", line 51, in on_search
    extent_rect = extent_layer.extent()
AttributeError: 'NoneType' object has no attribute 'extent'

The extent attribute makes me think that the Extent dialog shouldn't be empty?

Screen Shot 2019-06-14 at 2 10 29 PM

Select extent by current canvas extent or drawing a rectangle

The current layer extent only approach is very inconvenient in the general case.
Why limit only to layer extent, when you can use the current zoom level of the map canvas to get the extent? Further, the user should be also able to draw a rectangle extent on the map to be used.

Similar behavior control already exists in QGIS, however, it's part of the processing framework and it's not a reusable component (check the second param in "Clip raster by extent" algorithm). I would like to write it first for this plugin first and then maybe reimplement it directly in C++ to make it available to all plugin developers.

Handle errors

Handle network errors, authentication errors, etc.

Hang if API endpoint cannot be connected to

The loading bar window hangs if the server is down. For example, I added to Config.STAC_APIS 'http://localhost:22902/api/v2', and started the plugin, even though I'd neglected to start the server locally.

This is likely a general issue with how the API requests are being done, with no timeouts. Recommendation is to at least add connections and socket timeouts, and possibly add code like [https://pypi.org/project/timeout-decorator/] that will timeout a request even if its not causing the socket timeout, but the data return rate is intolerably slow.

Project style guide discussion

I'm creating this issue for discussion about the style guide in the project, that has to be developed.

I would suggest thinking carefully before you decide to switch to one style or another. I've checked the QGIS repo and even there is no consistency, for example here. I've checked the top most downloaded plugins about camelOrPascalCase vs pep8 recommendation: Use the function naming rules: lowercase with words separated by underscores as necessary to improve readability.. The conclusions from my investigations are: I see a lot of diversity :) . Either lack of consistency or one of the two styles is used. However, it seems there is a little preference towards QT style.

IMHO when dealing with QGIS/QT, it's better to follow their style guide, when everything is depending so much on them. Otherwise mixing is inevitable in the same file. For lib files that are completely separate and independent (therefore might be reused outside the current project), I would use the PEP8 recommendation. PEP8 is not a religious book, even python core libraries break the style guide: logging.getLogger(name).

As for PEP484, I see that in the beginning, the syntax might seem a bit funky to people who are not fully in the CS field. However, in the long term, I think it pays off as smart and easy documentation. Even for beginners, who often use IDEs, can get type checking for free, therefore making it easier to contribute with fewer bugs. PEP484 is a relatively new feature in the Python world (v3.5), QGIS supports Python 3 since Feb 2018 and it's understandable why it's not widely adopted in the community yet. So at least for separate, completely independent files, I would suggest to use it.

As a short summary, I would suggest the following convention.

  • max chars per line: 119, preferably 80
  • use of PEP484 as much as possible to secure maintainability and the need to write and check the documentation.
  • classes that interact with Qgis/Qt should use camelCase for their property, method and variable names.
  • QT widgets ids have to be in camelCase to match the QT convention; They should end with the widget type name (*Label, *LineEdit, *Button).
  • event callbacks should be marked as private, as we do not want to pollute the class interface.

Of course, all of this is my subjective opinion and it is based on writing and investigating a few public and private QGIS plugins. Feel free to disagree but I think it's a good starting point for a discussion.

# ./utils.py
def reverse_msg(msg: str):
    return msg[::-1]

# ./controllers/deals_with_ui.py

from QtWidgets import QDialog

from ..utils import reverse_msg

class RandomDialog(QDialog):
    def showReversedMsgDialog(self, msg: text = '!ymmuD') -> None:
        importantLabel = self.widgetIdLabel
        importantLabel.setText(reverse_msg(msg))
        importantLabel.clicked.connect(self._onImportantLabelClicked)

    def _onImportantLabelClicked(self) -> None:
        importantLabel.setVisible(False)

Page through search for items

Blocked by the API spec in STAC. Currently there is no standard way for paging through items so until that gets added (most likely v0.8.0) then this plugin will use unofficial paging APIs.

Temporal filter incompatible with STAC API spec

Queries made by the STAC browser are likely not compatible with the STAC API specification, in which time is parsed with the datetime argument.

For example, an HTTP request sent was:

POST https://.../api//search HTTP/1.1
Accept-Encoding: identity
Host: ...
User-Agent: Python-urllib/3.7
Content-Type: application/json; charset=utf-8
Content-Length: 183
Connection: close

{"collections": ["markham"], "bbox": [-123.6308868661519, 36.02789736179574, -28.151779495509228, 52.37970698302373], "time": "1959-01-01T00:00:00Z/1961-01-01T00:00:00Z", "limit": 50}

Which uses the time keyword for temporal filtering. This leads to ignoring the filter from the API's standpoint and loading all the items.

Assuming title is always available in collections

@kbgg congrats on this awesome project!

I'm tinkering with it to submit a PR to include the CBERS stac api server, and noticed that the browser assumes that title is always available for collections, more specifically here:

title = collection.title.replace("\n", " ")

This breaks if no title is available. Since this field is not currently required it would be better to use id if title is not available. This is what I did in my fork,

Syntax Error in stac_browser.py

Line 189 in stac_browser.py:
logging.error(f'Window {self.current_window} does not exist')
fails on Ubuntu 16.04

Tracking it down f formatting of strings was added in Python 3.6, this version of Ubuntu has 3.5.
Making a note here in case others hit this issue. Not sure it warrants a change in the code of this plugin, as most people have moved on to Python 3.6+ (osgeo4w, mac installer, and Ubuntu 18.04)

Check API Version Compatibility

APIs should only be included in the query window if the STAC version the API is providing is compatible with the plugin. Currently the plugin is targeting >= 0.7.0

Hangs when trying to add API or test existing API

I've just installed this extension to try and test it with my new STAC API, and I've found it seems to keep hanging whenever it tries to make a network request.

If I go to Configure APIs, and try and add my new API, it hangs when testing the API. Similarly, it hangs whenever testing any of the APIs that are originally listed there.

Do you know if there is some major network access problem with this plugin?

Can't upgrade 1.0B1 to 1.b1

QGIS tells me there's a new version, but the upgrade doesn't work. I'm not sure if the plugin manager handles letters in version numbering or not. This is the only plugin I've ever seen with a letter. Might be an upstream bug.

Also I can't find which branch in this repo has either version number, all the code still shows 0.1

Can't download Sentinel image with sat-api, HTTP Error

I'm using the latest stable plugin and qgis version 3.8. I can view Sentinel and Landsat images intersecting with an aoi drawn on the map but when I try to download the true color image I get an HTTP ERROR. Downloading Landsat works (the Extent layer invalid error was because I accidently used an empty extent). Here is the traceback.

2019-09-21T18:18:48     INFO    [1/1] Downloading https://sentinel-s2-l1c.s3.amazonaws.com/tiles/35/J/MF/2019/5/12/0/TKI.jp2
2019-09-21T18:18:51     CRITICAL    Error : Failed to load S2A_35JMF_20190512_0; HTTPError
2019-09-21T18:23:11     INFO    [1/1] Downloading https://sentinel-s2-l1c.s3.amazonaws.com/tiles/35/J/MF/2019/1/12/0/TKI.jp2
2019-09-21T18:23:11     CRITICAL    Error : Failed to load S2A_35JMF_20190112_0; HTTPError
2019-09-21T18:26:03     CRITICAL    Error : Extent layer is not valid
2019-09-21T18:26:23     INFO    [1/1] Downloading https://landsat-pds.s3.amazonaws.com/c1/L8/170/082/LC08_L1TP_170082_20190319_20190325_01_T1/LC08_L1TP_170082_20190319_20190325_01_T1_thumb_large.jpg

bad request searching for Planet data with Staccato

Thanks for developing this plugin, looks really useful!

I can't get request Landsat or Planet using Staccato, here is my Messages console

2019-09-21T18:08:35     INFO    Plugin installed successfully
2019-09-21T18:09:17     CRITICAL    Error : Extent layer is not valid
2019-09-21T18:10:14     INFO    Plugin installed successfully
2019-09-21T18:13:32     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:13:45     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:13:54     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:14:26     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:16:15     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:16:26     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:16:44     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:16:57     CRITICAL    Error : Network Error: [400] Bad Request
2019-09-21T18:17:04     CRITICAL    Error : Network Error: [400] Bad Request

I'm on Ubuntu 18.04 and QGIS 3.8 Zanzibar. I drew multiple different sized bounding box extents using the tool to test it out, I think only the first box was invalid because I didn;t get the Extent layer is not valid error multiple times. I'm using the most recent version of the plugin.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.