thuenen-geonode-development / thuenen_atlas Goto Github PK

If I follow https://docs.geonode.org/en/master/devel/workshops/index.html#customize-the-look-and-feel and add get_menu_json.py under geonode/apps/thuenen_app/templatetags/get_menu_json.py to override GeoNodes default menu, it should change the TOPBAR menu entries.

I have added some additional lines to import needed modules in get_menu_json.py

from django import template

register = template.Library()

@register.simple_tag(takes_context=True)
def get_base_right_topbar_menu(context):

    is_mobile = _is_mobile_device(context)

    if is_mobile:
        return []

    return [
        {
            "type": "link",
            "href": "/",
            "label": "Custom 3"
        },
        {
            "type": "link",
            "href": "/",
            "label": "Custom 4"
        },
    ]
@register.simple_tag(takes_context=True)
def get_base_left_topbar_menu(context):

    is_mobile = _is_mobile_device(context)

    return [
        {
            "type": "link",
            "href": "/",
            "label": "Custom 1"
        },
        {
            "type": "link",
            "href": "/",
            "label": "Custom 2"
        },
    ]

But I will get an error:

Request Method: 	GET
Request URL: 	http://172.17.0.1:8001/
Django Version: 	3.2.20
Exception Type: 	TemplateSyntaxError
Exception Value: 	

Invalid block tag on line 14: 'get_menu_json'. 
In template /usr/src/django_geonode_mapstore_client/geonode_mapstore_client/templates/geonode-mapstore-client/_geonode_config.html, error at line 14

Did you forget to register or load this tag?

Exception Location: 	/usr/local/lib/python3.10/dist-packages/django/template/base.py, line 531, in invalid_block_tag
Python Executable: 	/bin/python3
Python Version: 	3.10.12
Python Path: 	

['/usr/src/geonode',
 '/usr/src/geonode',
 '/usr/src/geonode',
 '/root/.vscode-server/extensions/ms-python.python-2023.22.0/pythonFiles/lib/python/debugpy/_vendored/pydevd',
 '/usr/lib/python310.zip',
 '/usr/lib/python3.10',
 '/usr/lib/python3.10/lib-dynload',
 '/usr/local/lib/python3.10/dist-packages',
 '/usr/src/geonode-importer',
 '/usr/src/django_geonode_mapstore_client',
 '/usr/src/importer-datapackage',

> Invalid block tag on line 14: 'get_menu_json'. Did you forget to register or load this tag?

The file I would like to 'overwrite' is:
https://github.com/Thuenen-GeoNode-Development/geonode-mapstore-client/blob/thuenen_4.1.x/geonode_mapstore_client/templatetags/get_menu_json.py

What did I do wrong?

Part of #27

Problem: As a creator, it is not possible for me to specify for the map widgets whether they should be filtered by time slider or not. As a result, some widgets, such as a pie chart, are misleading in their data display. An example of a map with corresponding widgets can be seen here. The feature is part of the following GitHub issues: GeoNode/geonode-mapstore-client#1473

Objective: As a creator, I would like to be able to configure for a widget whether a widget should accept the time-slider filter or not. This configuration is saved with the map. By default, the time slider is ignored.

The new facet config is not ported to _geonode_config.html and still contains the old filter configuration:

Replace them with the new filters.

This issue is part of #24

Problem: The current GeoNode Version 4.x is a single page application. It will not show the project partners in the footer element like version 3.x.

Objective: As administrator I want to show project partners logos. An example of the functionality is shown at the current Thünen ATLAS (https://atlas.thuenen.de/) which is based on version 3.x of GeoNode:

In the screenshot below the homepage of the Thünen ATLAS is shown. In the footer area the partners are listed with their logos.

The screenshot below is showing the version 4.x homepage. The partners and logos are no longer displayed. A possible placement would be in the footer element of the start page.

Reported by @ahmdthr while uploading Wuchsbezirke 2011.

Failed with message tuple.index(x): x not in tuple. Request: 75122b9b-2610-410e-bdaa-e638ba42b0e6. Complete error response:

{
  "exec_id": "75122b9b-2610-410e-bdaa-e638ba42b0e6",
  "status": "failed",
  "func_name": "rollback",
  "created": "2023-09-18T15:05:09.098812Z",
  "finished": "2023-09-18T15:05:20.039114Z",
  "last_updated": "2023-09-18T15:05:20.039121Z",
  "input_params": {
    "files": {
      "cpg_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.cpg",
      "dbf_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.dbf",
      "prj_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.prj",
      "shp_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.shp",
      "shx_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.shx",
      "base_file": "/usr/src/geonode/geonode/uploaded/tmpjtvfx4cg/wgwb_wuchsbezirke_2011.shp"
    },
    "total_layers": 1,
    "store_spatial_file": true,
    "handler_module_path": "importer.handlers.shapefile.handler.ShapeFileHandler",
    "skip_existing_layers": false,
    "overwrite_existing_layer": false
  },
  "output_params": {},
  "step": "importer.rollback",
  "log": "tuple.index(x): x not in tuple. Request: 75122b9b-2610-410e-bdaa-e638ba42b0e6",
  "name": "wuchsbezirke_2011.zip",
  "action": "import",
  "source": "upload",
  "user": 1000,
  "geonode_resource": null,
  "link": "http://172.17.0.1/api/v2/executionrequest/75122b9b-2610-410e-bdaa-e638ba42b0e6"
}

Geoserver error log:

18 Sep 15:05:19 WARN   [org.geoserver] - Error decode epsg code: EPSG:6258
org.opengis.referencing.NoSuchIdentifierException: No transform for classification "Colombia Urban".
        at org.geotools.referencing.operation.DefaultMathTransformFactory.lambda$getProvider$1(DefaultMathTransformFactory.java:267)
        at java.base/java.util.Optional.orElseThrow(Optional.java:408)
        at org.geotools.referencing.operation.DefaultMathTransformFactory.getProvider(DefaultMathTransformFactory.java:264)
        at org.geotools.referencing.operation.DefaultMathTransformFactory.getDefaultParameters(DefaultMathTransformFactory.java:298)
        at org.geotools.referencing.factory.epsg.DirectEpsgFactory.createCoordinateOperation(DirectEpsgFactory.java:2845)
   
[cut for brevity]

EDIT: EPSG:6258 seems a bit weird for Wuchsbezirke dataset which is located in Germany. Note the difference between EPSG:6258 and EPSG:6258 (Datum)!

However, I am sure I downloaded the dataset directly from Thuenen Atlas instance last year. Will have to further investigate tomorrow.

As mentioned by @gannebamm we need to upgrade the whole setup to most current 4.2 sooner or later (preferred sooner).

This includes:

• All dependent upstream projects
  • MapStore2
  • geonode-mapstore-client
  • geonode (52north repo, as the thünen specific one should not be needed anymore)
• Once updated, the actual projects can be updated
  • thuenen_atlas
  • geonode-blueprint

Upgrading means to

• Merge with released upstream version
• Update submodules
• Update CI build pipeline
• Check for configuration changes (during setup, i.e. check entrypoints, settings, envs, task.py, ...)

(based on 2.1.1 specification document)

Problem: The HoliSoils project datasets are difficult to distinguish from Thünen's own data products. The HoliSoils color scheme and logo cannot be displayed.

Objective: The data records of the HoliSoils project should be findable in ATLAS. At the same time, however, the data records should also be able to be found pre-filtered on a separate portal page. The geonode-subsites module published on GitHub (https://github.com/geosolutions-it/geonode-subsites) is to be used as the technical basis for this. The aim is to transfer the datasets to the portal only once and display them in different views.

• #25
• #26

Multiple roles not available in downloaded meta-data. Possibly has to do with XML-metadata-template. Related to #23

Problem: As an administrator, I have to integrate the latest security updates and minor patches of the main project into the Thünen-ATLAS fork (https://github.com/Thuenen-GeoNode-Development/thuenen_atlas). This is time-consuming and requires some adjustments to the existing code.

Objective: As an administrator, I would like to be able to update the Thünen ATLAS via a pull of the customized fork. If special migration steps need to be taken into account, I would like to receive a description of these. The maintenance should include the next minor release (version 4.3.x), which is currently being developed: 4.3.1

Info: Some of the layout changes like mutually exclusive layers, TOC line breaks for titles and others are not needed anymore, since they are now part of core. The fork needs to get cleaned to get rid of those old deprecated code.

• #32
• Integrate "local" update on tisdar.dev.thuenen.de (#23) into tuenen_atlas 4.x.

Due to an urgent bug in the contact roles metadata editor (GeoNode/geonode#12170) we need to upgrade our branch to incorporate soon-to-be-merged PR GeoNode/geonode#12191 / edit closed in favor for new proper named GeoNode/geonode#12207

This will be included in release 4.2.4

The action items Delete and Copy are missing from the actions drop down list:

Perhaps, this is a regression from updating to 4.2.5.

There are lots of changes in the localConfig.json. We have to review and sync it with the latest changes from upstream.

Part of #30

settings.py sets X_FRAME_OPIONS=DENY by default. Upstream "fixes" this by overriding the setting in its sample.env. However, if not set explicitly, the DENY option will be set and embedded documents are blocked by the browser to be displayed:

Solution:

Add X_FRAME_OPIONS=SAMEORIGIN to the settings_override.py.

/cc @gannebamm

In the 'Create Dashboard' module, the WFS layers in the geonode catalog are not being able to be accessed while creating a CHART, TABLE And COUNTER widget. The screenshots are attached.

The widgets are working fine for the same layers when these layers are inserted in a map.
https://geonode-demo.thuenen.de/catalogue/#/map/22
This map shows all the types of widgets created using the same layers

Part of #24

Problem: The current portal shows the Thünen Institutes corporate design. It is therefore not clear to visitors that they are viewing HoliSoil's project results.

Objective: As an administrator, I would like to be able to customize the layout and colors of the sub-portal. The HoliSoils logo and the HoliSoils project colors should be displayed (see https://holisoils.eu/). These customizations should be made possible via templates of the subsites. The HoliSoils CD should be implemented as an example for this and the necessary work steps for future subsites should be documented.

For some cases, we need to modify uwsgi.ini settings like

harakiri = 600                       ; forcefully kill workers after 600 seconds

See: https://github.com/GeoNode/geonode/blob/master/uwsgi.ini

In theory, we could create our own uwsgi.ini file and mount it over the currently inherited one from https://github.com/GeoNode/geonode/blob/master/uwsgi.ini

Are there better alternatives?

Tracks status of feature branches developed in GeoNode SEP 2023.

• Map Layer Infos, LB 2.1.2.3
  • Ticket: Thuenen-GeoNode-Development/Sprints#10
  • PR geonode-mapstore-client: Thuenen-GeoNode-Development/geonode-mapstore-client#22
  • thuenen_atlas: 6efe1ef
  • Upstream Ticket: GeoNode/geonode-mapstore-client#1616
  • Upstream PR: GeoNode/geonode-mapstore-client#1661
• Compliant GDI-DE, LB 2.1.1.4
  • Ticket: Thuenen-GeoNode-Development/Sprints#6
  • PR: #7
  • thuenen_atlas: 06a5a1f
  • Upstream PR
  • PR for non-local decimals in template
• Copied Metadata, LB 2.1.1.5
  • Ticket: Thuenen-GeoNode-Development/Sprints#7
  • PR: Thuenen-GeoNode-Development/geonode#1
  • thuenen_4.x image build (docker-compose pull 52north/geonode_thuenen:4.x needed)
  • Upstream PR: GeoNode/geonode#11903
• Patch Layer Infos, LB 2.1.1.5 (see also Copied Metadata)
  • Ticket: Thuenen-GeoNode-Development/Sprints#14
  • PR: Thuenen-GeoNode-Development/geonode#4
  • thuenen_4.x image build (docker-compose pull 52north/geonode_thuenen:4.x needed)
  • Upstream Ticket: GeoNode/geonode#10873
  • Upstream PR: @kilichenko-pixida create an upstream PR
• DataCite Download, LB 2.1.1.3
  • Ticket: Thuenen-GeoNode-Development/Sprints#5
  • PR Thuenen-GeoNode-Development/geonode#3
  • PR Thuenen-GeoNode-Development/geonode-mapstore-client#27
  • thuenen_4.x image build (docker-compose pull 52north/geonode_thuenen:4.x needed):
    • @ahmdthr wait for pycsw PR to be merged and new alpha version released (contact community for this)
    • @ridoo update submodule once finished
  • Upstream PR: ❓
• Groups Facet, LB 2.1.2.4
  • Postponed as of GeoNode/geonode#11853 (will be part of upstream 4.3.0)
  • Ticket: Thuenen-GeoNode-Development/Sprints/issues/11
  • thuenen_atlas: ❓
  • Upstream PR: ❓
• Push GeoNode Instance
  • Thuenen-GeoNode-Development/Sprints#17
  • thuenen_atlas: ❓
  • Upstream PR

I expect to see an external application listed under 'Kartenanwendungen' like here: https://atlas.thuenen.de/atlanten/fernerkundungsatlas#apps

But they will neither be used to determine the quicklinks: https://github.com/Thuenen-GeoNode-Development/thuenen_atlas/blob/thuenen_4.x/geonode/apps/atlas/templates/geonode-mapstore-client/atlas_detail.html#L125-L127

Nor will they be shown in the list of mapping applications: https://github.com/Thuenen-GeoNode-Development/thuenen_atlas/blob/thuenen_4.x/geonode/apps/atlas/templates/geonode-mapstore-client/atlas_detail.html#L145-L162

I need to add two urls to urls.py for rendering the legal notice and accessibility htmls. See:

https://atlas.thuenen.de/accessibility/
https://atlas.thuenen.de/legal_notice/

in our old geonode fork based instance I have added those lines to urls.py ():

    url(r'^legal_notice/$',
        TemplateView.as_view(template_name='legal-notice.html'),
        name='legal-notice'),
    url(r'^accessibility/$',
        TemplateView.as_view(template_name='accessibility.html'),
        name='accessibility'),

How would I accomplish such task with geonode-project? I have found this but it is still not clear how to change / add in our project based setup.

https://stackoverflow.com/questions/23614960/add-urls-from-a-django-app

I likely have to create a urls.py in geonode/geonode (https://github.com/Thuenen-GeoNode-Development/thuenen_atlas/tree/thuenen_4.x/geonode/geonode) next to settings_override.py?

Könntet ihr bitte prüfen, ob hier die falsche Branch Grundlage verwendet wird? Prinzipiell sollte das Projekt auf den branch 4.0.x verweisen und nicht 4.x. Der 4.x Branch ist deprecated (wie in der geonode-dev liste angekündigt).

Aufgefallen ist mir dies durch:

django4thuenen_atlas |   File "/usr/local/lib/python3.10/site-packages/OpenSSL/crypto.py", line 3268, in <module>
django4thuenen_atlas |     _lib.OpenSSL_add_all_algorithms()
django4thuenen_atlas | AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

Beim Hochfahren und GeoNode/geonode#10486

Vllt habe ich aber auch einen Fehler begangen?

Is part of #27

Problem: As a creator I am not able to create widgets for raster data sources.

Objective: As a creator I want to create charts based on zonal statistics for an area. The areas polygon shall be drawn on the map as rectangle or polygon.

When container gets rebuilt, all uploaded files get deleted as they are stored by default under the PROJECT_ROOT which points to an inner container path. The default upstream settings override this:

MEDIA_ROOT=/mnt/volumes/statics/uploaded/

/mnt/volumes/statics/ is a docker volume and should keep the files unless the volumes do not get deleted explicitly (by docker volume rm or docker compose down -v)

Let's take over MEDIA_ROOT and STATIC_ROOTand place them in thesettings_override.py`.

GeoNode/QGISGeoNodePlugin#248

We better should rename settings.py to settings_override.py to make clear that this will not override the actual
file geonode/settings.py but provides overridden settings in a file laying next to geonode/settings.py.

Somehow the config for datapackage import has vanished. We would have to re-add it into the current setup.

It can be found here:

First described here: GeoNode/geonode#11511 (comment)
In context of: zalf-rdm/geonode#97 / GeoNode/geonode#10342

Problem: One of the major issues we discovered is that metadata entries for publications (in terms of research) are static and must be static by design. The dynamic approach of GeoNode which saves user-objects to contacts is therefore against that principle. Here is an example to make this more clear:

Scientist sci-A works currently for organisation unit org-A. If sci-A uploads dataset data-A he/she will be listed as owner with his/her other contact infos like affiliation to org-A. Later sci-A switches to a new organisation unit like org-B. Due to the dynamic nature of GeoNodes metadata it will result in changed metadata if retrieved in the future for data-A.

Even though this is the prefered way to e.g. contact sci-A, this shall not happen from our point of view. It would for example change the list of datasets from org-A even though data-A was created while sci-A was part of it.

Idea: One briefly drafted idea was to use the new GeoNode/geonode#12124 to save static metadata files per dataset. Additionally there is the "Metadata upload preserver" option to keep the original uploaded metadata file, and to avoid it being regenerated by GeoNode. as stated by Giovanni (GeoNode/geonode#11511 (comment)).

Objective: We should discuss options to solve this issue

@gannebamm this may be an opportunity to update the datapackage module. @mwallschlaeger suggested to test the datapackage module after bugfixing and adding new features in https://github.com/GeoNodeUserGroup-DE/geonode-dev-datapackage.

However, this would mean to merge necessary changes into geonode-mapstore-client and geonode fork.

As part of #30

The docker container registry re-organisation and the concept of blueprinting is now upstreamed. The thuenen_atlas repo should be aligned with the latest docker-blueprint and the docker containers switched to the official ones like done in this commit: GeoNodeUserGroup-DE/geonode-blueprint-docker@c6a3979

Is related to 2.1.2 in the specification document

Problem: The mapping client currently in use does not provide a good user experience for some datasets and maps. This impairs information retrieval and leads to a low number of visitors. Therefore, the user interface needs to be further improved and developed. The provision of this functionality is complex and is therefore described as EPIC, which consists of the following sub-functions

• #28
• #29