We may want to load and test the thesauri we'll be using in RNDT.

These thesauri will be used in the keyword GUI:

• INSPIRE themes: https://www.eionet.europa.eu/gemet/it/inspire-themes/
• Priority datasets: https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/httpinspireeceuropaeumetadatacodelistPriorityDataset-PriorityDataset.rdf
• Registro dei dati di interesse generale: https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/rndt-all1.rdf
• Spatial scope: https://inspire.ec.europa.eu/metadata-codelist/SpatialScope

Furthermore, we need these thesauri as well: they will not be used in the GUI for keywords, but will be useful in other GUI tabs:

• http://inspire.ec.europa.eu/metadata-codelist/LimitationsOnPublicAccess,
  • RDF at https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/httpinspireeceuropaeumetadatacodelistLimitationsOnPublicAccess-LimitationsOnPublicAccess.rdf
• http://inspire.ec.europa.eu/metadata-codelist/ConditionsApplyingToAccessAndUse,
  • RDF at https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/httpinspireeceuropaeumetadatacodelistConditionsApplyingToAccessAndUse-ConditionsApplyingToAccessAndUse.rdf

We need to add a facet for PubblicaAmministrazione.
The labels in the facet should be the values from PubblicaAmministrazione.name.
Once a PubblicaAmministrazione is selected, the related iPA code should be used for filtering the Layers; filtering may be performed either according to the prefix in their UUID or the associated group profile.

The facet should be added to the existing ones:

We need to be able to edit instances of:

• PubblicaAmministrazione
• GroupProfileRNDT

Set up the minimum set of files needed for a django app called rndt.
Add a README file for doc about basic installation.

Investigate about which elements are read from an uploaded XML metadata document and set into the geonode model.
Check whether more fields can be imported.
Check for possible improvements within geonode, and if parsing can be refined by other django apps

We need a way to create a brand new tab in the editor page, for handling custom elements.
Such elements shall be defined in django apps that need editing for customized fields.

The elements in the tab will:

• display the current values of the customized fields
• allow the editing of the values
• return back the values of the fields in order to be persisted in the db
• tell if the field content is valid or not, in order to let the editor know if the metadata editing is complete

e.g.
we have an istance of LayerRNDT class, which is 1:1 with Layer. In this tab we'll be editing the properties of the LayerRNDT instance.

Please note that the fields editing could be quite complex (also allowing interdependencies between fields), so the whole handling should be implemented in the custom app.

The getCapabilities operation in a sub-entrypoint (csw/<IPA>) should report the proper URL for all documented operations.

• Related to #7
• Need implementation in GeoNode: GeoNode/geonode#7118

Il servizio CSW dovrebbe rendere disponibili i metadati di servizio per i servizi WMS, WFS, WCS esposti da geoserver.

RNDT own information (constraints, resolution, accuracy) are not shown in metadata_details/ page.

• investigate
• create issues
• estimate

Check, fix, amend, complete the metadata template in order to be fully RNDT compliant.

RNDT implementation requires a 1:1 relation from LayerRNDT->Layer.

At the moment LayerRNDT instances are created at Layer creation, or on demand when a Layer metadata is edited.

The RNDT model partially overrides the exisiting one, so that at least the old value of other_constraints may be lost when

1. an existing layer is created before RNDT app is installed
2. RNDT app is installed
3. the old layer is edited

A migration scripts in the rndt app should:

• for each existing layer
  • create the related LayerRndt
  • copy the layer.constraints_other value into LayerRNDT.constraints_other

When creating a new Layer with the geonode-rndt app active, their resource link will be pointing to catalogue/, e.g.:

http://localhost:8000/catalogue/geonode_dev-data:geonode:PAT_PO

instead of layers/, e.g.:

http://localhost:8000/layers/geonode_dev-data:geonode:ITA_adm1

Review the existing command that imports Thesauri.

• Check that it still works.
• Populate the new fields and tables

As explained in #15 , we'll add a panel in the tab created in #17 that will contain:

• a dropdown for populating LayerRNDT.access_constraints with choices from LimitationsOnPublicAccess thesaurus

• a radio button selecting either

  • a dropdown populated with choices from the ConditionsApplyingToAccessAndUse thesaurus
  • a textarea for free text

  The value will be stored into Layer.constraints_other

An XML document is used as a template for creating ISO19139 metadata.

At the moment it's fixed (https://github.com/GeoNode/geonode/blob/3.1/geonode/catalogue/models.py#L103)

md_doc = catalogue.catalogue.csw_gen_xml(instance, 'catalogue/full_metadata.xml')

Make the template file configurable using a config entry CATALOG_METADATA_TEMPLATE

Since the thesauri faceting is always visibile, thesauri keywords should be editable also for maps and documents.

In the DB a tuple is stored, instead of plain text:

We want to update a Layer UUID according to the associated GroupProfile.

When a Layer has an associated GroupProfile, we need to read the related IPA code from
Layer --> GroupProfile --> GroupProfileRNDT --> PubblicaAmministrazione.ipa

The UUID will be trasformed as:
IPA + ":" + UUID

The UUID should always reflect the associated IPA.

The main actions to check for are (not limited to)

• When a layer is created
• When a layer is updated (its GroupProfile may have been changed)
• When a GroupProfileRNDT instance changes its reference to a PubblicaAmminstrazione
• ...

We do not want to customize GeoNode logic; we need instead to create hooks wherever needed in order for GeoNode to be extensible. For instance, we may want to create an UUIDHandler class that should be used at UUID initialization and whenever the Layer is updated.
For instance, in geonode/layers/models.py, function pre_save_layer() we have:

if instance.uuid == '':
    instance.uuid = str(uuid.uuid1())

We may want to check the configuration if a constant LAYER_UUID_HANDLER exists. If it's there, we may call its methods, for instance handler.create_uuid(instance).

We need to handle the value for Positional Accuracy.
The element is a gco:Real and has cardinality [1].
The UoM is fixed to meter.

We need to implement changes in:

• model
• GUI
• xml template

Model

As for the resolutions, the new field will be stored in LayerRNDT.
We're storing the value in meters, and the precision should be in the order of the cm, so 2 decimal digits shold be enough.

GUI

The panel for the Positional Accuracy (Accuratezza posizionale) could be placed just below the Resolution panel.

xml template

In the template there is already the element related to the gmd:DQ_AbsoluteExternalPositionalAccuracy with a fixed placeholder value 1.0; the placeholder should be replaced with the proper value:

         <gmd:DQ_AbsoluteExternalPositionalAccuracy>
            <gmd:result>
               <gmd:DQ_QuantitativeResult>
                  <gmd:valueUnit>
                     <gml:BaseUnit gml:id="UD1">
                        <gml:identifier 
                           codeSpace="http://www.bipm.org/en/si/base_units">m</gml:identifier>
                        <gml:unitsSystem xlink:href="http://www.bipm.org/en/si"/>
                     </gml:BaseUnit>
                  </gmd:valueUnit>
                  <gmd:value>
                     <gco:Record> 
                        {# TODO #}
                        <gco:Real>1.0</gco:Real>
                     </gco:Record>
                  </gmd:value>
               </gmd:DQ_QuantitativeResult>
            </gmd:result>
         </gmd:DQ_AbsoluteExternalPositionalAccuracy>

Create tables:

• PubblicaAmministrazione:
  • ipa (varchar 32),
  • name (varchar 128)
• GroupProfileRNDT
  • FK GroupProfile OneToOne
  • FK PubblicaAmministrazione

Add the resolutions edit field in the GUI in the custom tab created in #17

Implement hooks and configurability in GeoNode in order to be able to implement #5

Some fields that are optional in the base ISO19115 model, may become mandatory in sub profiles.

We need a way to flag which fields are mandatory.
User should be able to understand which fields are mandatory.
The mandatory/optional indication above the tab name should report if all fields in that tab are mandatory
A validation check should be enforced when saving the metadata.

Align to GeoNode/geonode#7421

In §3.6.3 "Risorsa on-line", il Requisito R.10 recita:

L’elemento gmd:CI_OnlineResource deve includere anche:

• gmd:protocol/gmx:Anchor che deve indicare il protocollo utilizzato per il Servizio. Nel caso di
  servizi di rete (discovery, view, download, transformation) esso deve puntare a uno dei valori
  dell’elenco di codici Protocol disponibile nel Sistema di Registri INSPIRE (40). Nel caso di altri servizi
  (other), deve puntare a uno dei valori del registro Protocol disponibile nel Sistema di Registri
  INSPIRE Italia;
• gmd:applicationProfile/gmx:Anchor che deve puntare a uno dei valori dell’elenco di codici “Tipo
  di servizio di dati territoriali” disponibile nel Sistema di Registri INSPIRE (41);
• gmd:description/gmx:Anchor che deve puntare al valore “accessPoint” dell’elenco di codici “Online
  description code” disponibile nel Sistema di Registri INSPIRE(42).

(40) http://inspire.ec.europa.eu/metadata-codelist/ProtocolValue
(41) http://inspire.ec.europa.eu/metadata-codelist/SpatialDataServiceType
(42) http://inspire.ec.europa.eu/metadata-codelist/OnLineDescriptionCode

(40) definisce soltanto i seguenti elementi:

• ATOM Syndication Format
• OGC Catalogue Service for the Web
• OGC Sensor Observation Service
• OGC Web Coverage Service
• OGC Web Feature Service
• OGC Web Map Service
• OGC Web Map Tile Service

(41) definisce:

• Altri servizi
• Servizio di consultazione
• Servizio di conversione
• Servizio di ricerca
• Servizio di richiesta dei servizi di dati territoriali
• Servizio di scaricamento

Nel caso in cui si voglia riportare il link alla pagina di consultazione del layer geonode, opzione contemplata nel punto "una pagina web dove reperire ulteriori informazioni per accedere al dataset;" della "Raccomandazione 1.9", non abbiamo un valore Protocol possibile tra quelli in (40). Putroppo il registro alternativo citato nel Requisito 1.8, ossia il "registro Protocol disponibile nel Sistema di Registri INSPIRE Italia;" non esiste.

Identico problema nel caso si voglia fornire un link di download diretto al file dati originale, anche questo contemplato nell'opzione "accesso diretto per scaricare il dataset;" della "Raccomandazione 1.9"

In GeoNode/geonode#7263 a new hook has been added for creating custom data.

Since we may need values extracted from the optional metadata when creating a new Layer, the LayerRNDT instance creation should be implemented using such a hook.

We'll maintan the old logic as well, in order to be able to add RNDT info to existent layer.
The new logic will be used for brand new layers.

Since we need a different GUI control for the constraints, we need to make the constraints controls disableable:

If possible, re-layout the page in order to fill up the available space with the input controls left.

In the thesaurus faceting, if a keyword does not have a translation in the currently selecte language, it will not be displayed at all.

This is a fix needed in the GeoNode repository.

Quando si salvano i metadati, il messaggio di salvataggio non sparisce.

Controllare se è dovuto alla cardinalità del thesaurus INSPIRE themes.

Add the spatial resolution info in the template

The metadata template should be updated in order to add all the thesauri and keywords in the metadata document.

GeoNode/geonode#7095

Expected Behavior

• The editor should clearly tell which fields are mandatory
• The "mandatory" label above the tab name should turn green/red accordingly to the mandatory fields status

Actual Behavior

The "mandatory" label stays red even if all
The fields

• abstract field
• thumbnail
• ..

Thesaurus table (update)

We need to support an href referencing the thesaurus; it's stored into the rdf:about field.

• Add field about = models.CharField(max_length=255, null=True, blank=True)

We need to move into the DB the config currently kept in SETTINGS (see https://docs.geonode.org/en/master/admin/thesaurus/#configure-a-thesaurus-in-geonode).
This is an excerpt from the current doc:

name: (mandatory string) the identifier you used in the load_thesaurus commands.
required: (optional boolean) if True, a keyword of this thesaurus is mandatory to complete the metadata. Currently not implemented.
filter: (optional boolean) if True, a faceted list of keywords of this thesaurus will be presented on the search page.

• name will not be ported, since it's a FK to the Thesaurus identifier;
• required will be enhanced, in order to better support cardinality info; we'll have card_min and card_max
• filter is a too general concept, so we can transform it into a field named facet

We'll add:

• card_min integer
• card_max integer
• facet boolean

Warning: the change about moving the settings from the SETTINGS file into the DB is breaking backcompatibility, so we'll have to change the logic accordingly:

• when one of the moved settings is needed,
  • we'll first check in the SETTINGS; if the setting is there, we'll use it and notify a warning about a deprecated setting.
  • only as a fallback step we'll check for the value in the DB

ThesaurusLabel (create)

Contains localized version of the thesaurus title

• lang = models.CharField(max_length=3)
• label = models.CharField(max_length=255)
• FK to Thesaurus

• All PA items are listed --> only the ones with associated layers should be visible
• can not tell when a PA is selected --> other facet when selected change color.

A published ResourceBase of type cswService is returned by a getRecordById request, but not by a getRecords.

We're using a custom template for RNDT metadata.
XSL file should be customized as well in order to align to the expected format.

• Related GeoNode issue: GeoNode/geonode#7097
• GeoNode PR (master): GeoNode/geonode#7099
• GeoNode PR (3.2.x): GeoNode/geonode#7100
• Doc PR: GeoNode/documentation#87

Modify the editing GUI so that thesauri selection is rendered in the first tab.

We want to display all of the thesauri set as required in the GUI.
The requirement is set in the DB (card_min, card_max), but we also have a legacy setting (https://docs.geonode.org/en/2.10/admin/thesaurus/index.html#configure-a-thesaurus-in-geonode)

In the GUI,

• card_max=0 --> Disabled, won't be shown in the GUI (card_min is not even checked in this case)
• card_max=1 --> Single choice
  • card_min = 0 --> Single choice, optional --> rendered as a dropdown, with an empty/unselected element
  • card_min = 1 --> Single choice, required --> rendered as a dropdown
• card_max=-1 --> Multiple choice
  • card_min = 0 --> [0..N] Multiple choices, optional --> rendered as a typeahead badges(?)
  • card_min = 1 --> [1..N] Multiple choices, required --> rendered as a typeahead badges(?) + apply check for the mandatory warning

If the legacy required boolean setting is in the configuration, we'll use it as if we'd found:

• required = true
  • card_min = 1
  • card_max = -1
• required = false
  • card_min = 0
  • card_max = -1

As stated in #11, when a legacy settings is found, we'll use it and notify a warning about the deprecated setting.

GUI should be restructed as follows:

• free keywords moved to the bottom
• list of Thesauri keywords selectors

Also, the validation checks should act according to the mandatory requirements of the displayed thesauri.

We need CSW entrypoints for pre-filtered CSW queries.

At the moment the CSW service is mapped through this URL definition:

url(r'^csw$', views.csw_global_dispatch, name='csw_global_dispatch'),

We need to enable calls for URLs ending with csw/{IPA}.
If the IPA code exists in PubblicaAmminstrazione.ipa, we want to prefilter CSW results by fileIdentifier field beginning with ipa:.
pyCSW pre-filtering is documented at https://docs.pycsw.org/en/2.6.0/repofilters.html.

• replace info of constraints
• add resolution info
• add positional accuracy info

Need changes in GeoNode to be implemented

Allow creation of Thesaurus, ThesaurusLabel, ThesaurusKeyword, ThesaurusKeywordLabel by importing an rdf file from web admin

We need to handle the element spatial resolutions.

It may be expressed either via the "equivalent scale", or using a resolutions expressed in meter.
The element is mandatory.
We will force here the use of the meter resolution, because it can be also used for vector layer -- in that case the resolution value represents the sample distance between points.

RNDT requires constraints for both access and use, while GeoNode only defines on set of costraints.

According to ISO19115, constraints are defined through a value in a codelist. If such value in the codelist is set to "otherRestrictions", the text element "otherConstraints" should be populated.

GeoNode model defines

• restriction_code_type = models.ForeignKey(RestrictionCodeType, ....
• constraints_other = models.TextField(...

and in the template these values are set as useConstraints.

RNDT requires that the RestrictionCode for both access and use are set to "otherRestrictions".

Limitations on public access

The otherConstraints element for access should be expressed as a URI + text from a controlled set.

The related vocabulary is defined at https://inspire.ec.europa.eu/metadata-codelist/LimitationsOnPublicAccess, while a valid RDF thesaurus can be downloaded from here: https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/httpinspireeceuropaeumetadatacodelistLimitationsOnPublicAccess-LimitationsOnPublicAccess.rdf

We may simply store the item URI in the DB, and then retrieve the related label from the thesaurus when needed.

Use constraints

The otherConstraints element for use should be expressed either as

• a URI + text from a controlled set.
• or a free text

The vocabulary is defined at https://inspire.ec.europa.eu/metadata-codelist/ConditionsApplyingToAccessAndUse while a RDF thesaurus can be found at https://github.com/geosolutions-it/iso19139.rndt/releases/download/v1.0-RC1/httpinspireeceuropaeumetadatacodelistConditionsApplyingToAccessAndUse-ConditionsApplyingToAccessAndUse.rdf

We can use the existing field constraints_other for storing such a value, and we'll tell which case is used (URI or free text) simply by checking if the contents is a URI.

Changes needed

We only need to add a single field to the Layer.
Since we don't want to add fields to existing table, we'll create a new table LayerRNDT, which is 1:1 with table Layer, and that will store a single field accessConstraints

RNDT requires separate elements for access and use costraints, while GeoNode only defines use constraints.

We need changes in the model and the GUI.

Add the resolution numeric field in LayerRNDT

The metadata template shall be revised in order to deal with the new constraints data.

We need to review and improve the current Thesauri implementation in GeoNode.

All of the related issues should be implemented in GeoNode, not in this django app.

GeoNode/geonode#7101

Actual Behavior vs Expected Behavior

If a required field is not filled, and the preview tab is clicked, an Error dialog pops up:

Steps to Reproduce the Problem

The error is caused by an automatic update when the preview is requested.