Add a simple site for documenting the plugin and then use github actions and github pages to re-deploy it on every push to the repo.

As the underlying site technology, mkdocs seems like a good choice

Expand the current github actions workflow to also run automated tests on every push.

We are already using the official QGIS docker image as a base for the workflow's job steps. It will likely be necessary to tweak the way tests are run in order to ensure they run with the QGIS docker image.

Add a section in the documentation explaining how to release a new version of the plugin

When the search results message bar appears on the GUI it forces other widgets to move.
Lets move the message bar below the search results scroll area to see if it is less disruptive there

QGIS datasource manager dialogue cannot currently be accessed via Python. For our concrete use-case it would be beneficial to be able to embed our GeoNode dialog into QGIS datasource manager.

Evaluate the effort needed to add this new functionality into QGIS core.

Add a basic plugin skeleton suitable for initial development.

This may be based upon the plugin builder plugin

When the user presses either the Search or one of the pagination buttons we should clear the current search results, in anticipation of the new results which shall be arriving soon

Lets add an initial implementation of a github actions pipeline to perform continuous integration of the code. Eventually we'll turn this into a more continuous deployment pipeline, with automated releases.

For now, lets add a pipeline that does checking of code formatting with black.

As #15 is completed we will then extend this to run the tests too

Add our layer GeoNode management GUI to the QGIS layer properties dialog. This is to provide an early implementation of the design shown in #2.

The GeoNode API supports specifying custom ordering when searching for resources. Lets add support for that in the plugin.

• Add support for ordering in all GeonodeClient methods that fetch lists of resources
• Add support for ordering in the main search GUI part of the plugin

Suggested design for the new ordering GUI controls:

Use github actions and github pages to configure a plugin repository that publishes new versions of the plugin when a new tag is created.

In order to provide maximum visibility on this work, lets configure a custom plugin repository so that we can push new releases without being reliant on the official QGIS repository. Obviously we will want to publish the plugin on the official QGIS plugins repo too, but lets do that at a later stage.

The main idea is to have a way to automate publishing new versions of the plugin from git tags. This shall become the way to publish all versions of the plugin.

Envisioned workflow:

• repo admin creates a new tag, wich kicks off a github action that performs several tasks in an automated way
• lint and test the code
• create a github release
• zip the plugin and add it as a release artifact
• update github-pages generated plugin repo with information about the release, so that it is published

this means that shortly after the repo admin has created a tag a new version of the plugin shall become visible inside QGIS.

The following github actions will likely be useful:

• https://github.com/marketplace/actions/create-a-release
• https://github.com/marketplace/actions/upload-a-release-asset
• https://github.com/marketplace/actions/github-pages

Current error handling is very crude, we should provide at least a bit more detail when something goes wrong. For example, I have the following error:

The message speaks about some error code 203, which is even more confusing because HTTP status codes of 2XX usually mean success. Lets at least show the HTTP error code instead of the Qt error code. Additionally, if possible, lets also show the description of the error.

After #29 is done, move to adding support for the various search-filter widgets that we currently have on our GUI

The current GeoNode connection editor dialog looks like this:

Instead of the highlighted User name and Password widgets we need to be using the QgsAuthConfigSelect widget. This widget should be available on Qt Designer, under the QGIS custom widgets section (as shown on the below screenshot)

According to qgis/QGIS#28060 there is currently no support in QGIS for showing a custom section on the layer properties dialogue of raster layers. This feature exists for vector layers though and we are planning on making it an important part of our UI.

Please add support for this in QGIS core.

Read the mentioned issue in order to get more details.

In order make restoring previous searches easier, lets allow the user to save and load search parameters. We already have some of the GUI elements in place, now we need to make them work.

Allow the saving the search parameters in QgsSettings. These shall be saved under the relevant connection key, so that each connection has its own set of stored searches. Additionally, each search shall have a unique id that is never shown to the user and is only used for organizing the settings. A search shall also have a _search_title, which is used as the name of the search for letting the user select it.

Allow restoring saved search parameters too.

# this is a title chosen by the user to refer to the stored search, 
# it is used for showing the stored search in the GUI
qgis_geonode/connections/{connection-uuid}/stored_searches/{search-uuid}/_search_title  

# in this snippet, {search-parameter} is to be replaced with 
# `title`, `abstract`, `category`, etc. This means that there 
# should be a key for each parameter of the search
qgis_geonode/connections/{connection-uuid}/stored_searches/{search-uuid}/{search-parameter}

Expand on the initial testing infrastructure added in #11 and add automated tests to verify correct behavior of the api_client

After #30 is done we shall be able to perform basic GeoNode layer searches.

Next up is being able to pick one GeoNode item from the ones returned in the search and load it into QGIS as a new layer.

The intended workflow is for a user to search for available GeoNode resources, then find the one(s) that are relevant and load them as QGIS layers by selecting one of the presented formats (i.e. WMS or WFS for vectors and WMS or WCS for rasters - maybe at a later stage we will implement direct data download, but not yet).

I understand that the current GeoNode REST API does not return URLs for accessing a layer's respective OGC services, so for now we will need to infer them from the GeoNode base URL and the layer's properties.

Do not worry about populating the QGIS layer's metadata tab with GeoNode-related metadata info yet. That shall be done at a later stage. Let's also leave retrieval and setting of the QGIS layer style to match GeoNode's for a later task too.

Search results should show the GeoNode resource's thumbnail, in addition to the alphanumeric fields.

Load the thumbnail asynchronously in order to avoid locking up the main QGIS UI.

Each search result shall be responsible for loading up its own thumbnail. Suggested implementation:

• Pass the retrieved resource details to the instance of SearchResultWidget upon initialization - you probably will also need to pass it the current geonode connection settings;
• Let each search result fetch the corresponding thumbnail by using QgsNetworkContentFetcher, in an analogous way as we are doing in api_client.GeonodeClient

Search results may return different types of resources, which in turn shall be loaded as different layer types:

• Vector layer - Loadable as WMS, WFS` (and possibly direct file download, in the future)
• Raster layer - Loadable as WMS, WCS (and possibly direct file download, in the future)
• Map - Loadable as WMS

The SearchResultWidget class needs to take this into account and present suitable buttons to the user

The bulk of this work is about supporting GeoNode's new v2 API. However, since the plan is to eventually go for removing GeoNode support from QGIS core, we will need to provide some level of support for the older GeoNode API. Otherwise we would drop support for existing GeoNode deployments, and that is not really acceptable

When attempting to install the plugin from our custom plugin repo QGIS throws a warning that it cannot find the plugin due to it having disappeared. It seems likely that there is some name mismatch in the plugin metadata or the zip name

Current search results does not look very pleasing, as seen in the following picture:

Lets improve on that:

• Provide better separation of consecutive search results - perhaps by placing the contents inside a QFrame?
• Experiment with using a QTextEdit for displaying each search result's description instead of a QLabel
• Use smaller buttons, with icons, for loading resources onto QGIS

Use GeoNode API's existing pagination facilities to add pagination to the API client method that retrieves a list of layers.

When the user presses one of the layer-loading buttons on the search results screen there is no visual feedback to let the user know that something is happening. The user is waiting for the layer to be added to QGIS canvas without knowing if QGIS is actually doing anything.

Add some progress indicator

Our custom plugin repo should only show the latest version of both the experimental and stable channels. Otherwise it seems QGIS does not pick up the most recent version

The GeoNode v2 API returns a layer's style in SLD format. Lets use that to style a raster layer that is loaded into QGIS.

Unfortunately, QGIS does not currently support import of SLD for raster layers (export is supported though) We'll likely need to read in the SLD and convert it to QGIS' native style format (QML). There are a couple of plugins that implement such functionality:

• SLD4raster - its sld4raster.py has two interesting methods which we can likely adapt: qml2Sld() and sld2qml()`
• GeoCatBridge - is also able to convert qml to sld but the code is more complex

Let's study these and come up with a suitable implementation that would allows us load a raster SLD style into QGIS.

NOTES

• A GeoNode layer can have multiple styles, not just one. Regardless, there is always a default style. When a layer is first loaded we shall assign it the default style it has on GeoNode. GeoNode is moving towards a single style per layer
• It shall be possible to select one of the other styles the layer may have for loading. This shall be implemented by means of an additional GUI control, so it will be better described in a later issue.
• The layer's SLD style shall be retrieved from the API's detail response (i.e. /layers/{layer-id}), not from the list response (/layers)
• The current version of GeoNode API returns the SLD document directly in the layer details. However, this may change in the future. As such, lets not assume that we will always have the style readily accessible for loading.

It seems the workflow is not defined correctly, as it should be listening to both push and pull request events

Lets add a rough version of the UI in order to start making things a bit more concrete.

Based on current discussion, plugin UI will make use of the following new elements:

plugin dock and connection configuration dialog

This shall eventually be replaced by integrating with the QGIS datasource manager dialog. Nevertheless, the container widget shall be reusable between this initial dock-based implementation and the future datasource manager one

Dedicated layer properties section

Individual layer operations (publish to GeoNode, etc.) shall be available in a new GeoNode section of the layer properties dialog.

there is currently a very long timeout for GeoNode API requests - I haven't measured exactly, but it feels like more than 30 seconds.

Use a lower value - 10 seconds should be enough

Fields to update:

• homepage
• about
• description

Lets tidy up the plugin code a bit by removing unused code and moving some files around in order to simplify imports:

• Remove qgis_geonode/apiclient/layer.py - it is not being used
• Remove qgis_geonode/apiclient/maps.py - it is not being used
• Remove qgis_geonode/apiclient/api_client.ApiClient class - it is not being used
• Remove the qgis_geonode.apiclient package altogether, and move the qgis_geonode/apiclient/api_client.py module to the src/qgis_geonode directory
• Remove the qgis_geonode.qgisgeonode package and move the following files to the src/qgis_geonode directory:
  • qgis_geonode/qgisgeonode/conf.py
  • qgis_geonode/qgisgeonode/main.py
  • qgis_geonode/qgisgeonode/utils.py

We need a way to run automated tests.

Lets add a way to run them using pytest and pytest-qt.

Lets implement saving a GeoNode connection information.

• We need to hook up the GeoNode connection manager ui to the main plugin ui, as depicted in the picture below

• Then we need to capture user input and save all connection information in the main QSettings object under our plugin's group.

• We also need a way to show all available connections in the connection-choosing combobox

In addition to a link in the description of the project, also use mkdocs' own configuration to add it to the main header

Remove the following unused files that were created during the plugin generation with the plugin builder:

• icon.png
• Makefile
• metadata.txt
• pb_tool.cfg
• plugin_upload.py
• help/
• scripts/run-env-linux.sh
• test/__init__.py
• test/qgis_interface.py
• test/tenbytenraster.*
• test/test_init.py
• test/test_qgis_environment.py
• test/test_Qgis_GeoNode_dockwidget.py
• test/test_resources.py
• test/test_translations.py

Hook up GeonodeClient to the main search GUI so that:

• When a user presses the Search GeoNode button the relevant API client code is executed and results are shown back on the GUI.
• When a user presses the Next and Previous buttons, the relevant paginated search is done and updated search results are shown on the GUI

Search results should show at least:

• Layer title
• Layer abstract

We can worry about improving the search results with more information later

Do not bother implementing search filters yet, those will be dealt with in #29 for the api client and later on in the GUI.

The GeoNode v2 API returns a layer's style in SLD format. Lets use that to style a layer that is loaded into QGIS.

QGIS currently supports import/export of SLD for vector layers. I did some research of the QGIS API and it seems like we should be able to call QgsVectorLayer.readSld() in order to have QGIS load an SLD style for a vector layer.

Lets then implement loading a QGIS vector layer style from the corresponding GeoNode SLD.

NOTES

• A GeoNode layer can have multiple styles, not just one. Regardless, there is always a default style. When a layer is first loaded we shall assign it the default style it has on GeoNode.
• It shall be possible to select one of the other styles the layer may have for loading. This shall be implemented by means of an additional GUI control, so it will be better described in a later issue.
• The layer's SLD style shall be retrieved from the API's detail response (i.e. /layers/{layer-id}), not from the list response (/layers)
• The current version of GeoNode API returns the SLD document directly in the layer details. However, this may change in the future. As such, lets not assume that we will always have the style readily accessible for loading.

Add an additional setting to the connection settings named page_size that defaults to 10. This shall be used to control the size of each search page.

• Add setting
• Use it when calling the GeoNode API

Use GeoNode API's existing pagination facilities to add pagination to the API client method that retrieves a list of maps.

Currently our GeoNode search UI looks like this:

since we are not making use of any search filtering functionality on our GeoNode API client class (I'm not even sure the GeoNode API supports these search filters yet), lets disable the widgets that allow filtering GeoNode layers (the ones in the section highlighted in red). It is very likely that filtering will only be implemented after our initial release.

Add our initial search GUI, as implemented in #2 to the QGIS data source manager dialog, as researched in #4.

The main objective of this issue is to have our custom GUI show up when a user has the plugin loaded and opens up the QGIS data source manager. No need to actually connect our GUI's buttons to any actions at this point

I've done some research on the current state of GeoNode's v2 API and could figure out how to filter requests (documentation on this is still lacking unfortunately). This means we can start adding support for filtering searches.

Lets first focus on adding support and tests for these in the apiclient.api_client.GeonodeClient class. In a subsequent issue we will focus on hooking these to the GUI.

Implement support for filtering layers by the following attributes:

• Free text - Ideally we'd filter across multiple attributes for this like a resource's title and abstract. That does not seem to be supported by dynamic-rest, which is the underlying API generation tech being used by geoNode, as shown in this open issue

  We need to decide how to handle this (filter only by layer title? do multiple requests to filter in the layer title and the layer abstract and then merge the results? other solution?).

  Here is an example on how to filter on a layer's title:

  # find all layers that have `park` in their title
  https://master.demo.geonode.org/api/v2/layers?format=json&filter{title.icontains}=park
  

• Keywords - There does not seem to be an endpoint for knowing which keywords exist in GeoNode, at least not yet. As such, the best we can do for now is to use a line edit widget and let the user write down whatever words to be searched for. Example keywords search:

  # find all layers that have a keyword with a name that includes the string `drain`
  https://master.demo.geonode.org/api/v2/layers?format=json&filter{keywords.name.icontains}=drain
  

• Topic category - Topic category fields are taken from the ISO 19115 topic categories list, so we just need to define an internal enum with their names (or maybe QGIS already has something like this, since it also uses these categories in its own metadata stuff) and use those to do a search like the following example:

  # filter by ISO Topic Category named `boundaries`
  https://master.demo.geonode.org/api/v2/layers?format=json&filter{category.identifier}=boundaries
  

• Resource type - Lets allow choosing between vector layer, raster layer and map. These will require performing alternative requests, as depicted in this example:

  # filter for raster layers
  https://master.demo.geonode.org/api/v2/layers?format=json&filter{storeType}=coverageStore
  
  # filter for vector layers
  https://master.demo.geonode.org/api/v2/layers?format=json&filter{storeType}=dataStore
  
  # filter for maps (uses a different endpoint)
  https://master.demo.geonode.org/api/v2/maps?format=json
  

Similarly to what is being done when searching for GeoNode resources, when testing a layer connection we should also show a testing connection... message in the message bar

Lets add support for GeoNode's v2 REST API.

This new API is still evolving but it should already provide enough for us to have basic read support of existing GeoNode layers.

The aim is to develop a python package that can be used as a client for this API inside our plugin in such a way to make it easily testable. It should:

• Use QGIS network request infrastructure. This means using QgsNetworkAccessManager and QgsNetworkContentFetcherTask instead of other python alternatives (like urllib, requests, httpx, etc)
• Integrate with QGIS' authentication infrastructure - this comes for free when using QGIS network request classes
• Be runnable from QGIS' python console
• Not contain any GUI code
• Include some tests to verify its behavior

The actual design of this package is left for the implementer to decide, but it probably will need to be a class-based approach, using QObject as the base class (the reason being that it will probably need to emit its own signals)

Use cases that shall be supported for the initial implementation:

• get a paginated list of existing layers
• get a paginated list of existing maps
• get individual layer details
• get layer's metadata fields, as suitable for using in QGIS layer metadata
• get a list of layer's styles with their respective SLD definition
• identify the layer's default style

Currently the api_client.GeonodeClient class' methods that retrieve data from a remote GeoNode instance just return the JSON payload of the response. This is not very convenient to work with, as the caller needs to have implicit knowledge of the payload's schema.

Create a suitable GeonodeBriefResource class which we will use to hold the response data for each returned resource. Implementation can probably use python's dataclasses. For example:

import dataclasses

@dataclasses.dataclass
class GeoNodeBriefResource:
    title: str
    abstract: str
    thumbnail_url: str

    @classmethod
    def from_payload(cls, payload: typing.Dict):
        return cls(
            title=payload["title"],
            abstract=payload["abstract"],
            thumbnail_url=payload["thumbnail_url"]
        )

When a GeoNode resource is loaded as a QGIS layer, its metadata shall be used to populate the QGIS layer metadata section of its properties dialog.

The GeoNode section of the Layer Properties dialog already features a Open in Browser button, but it currently does not do anything. Let's make it open the main layer details page on the remote GeoNode instance.

We need to be able to establish a link between a layer that is loaded in QGIS and its respective GeoNode resource. One way this can be achieved is to rely on the layer's metadata, specifically the identifier field.

The set of layer identifier metadata and the geonode instance referred to in the GUI shall be enough to construct the layer details URL on the actual GeoNode instance.