Evaluate the implementation of a raster handler about geonode-importer HOT 3 CLOSED

Raster import could be implemented in way similar to vectors, using a shared volume from which the raster file could be published.

The handler now contains all the methods needed to import and publish the resource

Meanwhile the current importer-based flow could be wrapped inside a geonode-importer handler. This way we could get rid of the old implementation, both upload, and import.

Yes, thanks to the creation of the registry, we can set which kind of file the new importer can handle, so once the handler is ready is enough to register the new app here https://github.com/geosolutions-it/geonode-importer/blob/d92d796d485366ae6f693d08e728523ef5e09d4e/importer/handlers/apps.py#L14-L16

This approach needs careful analysis of the implications, although I don't see blockers.

Indeed, the main sensitive point is related to where to store these images. I suggest adding a settings/env_var to configure the path outside the normal upload folder. The storage manager may need an improvement to clone the files into a different directory from the standard one by adding a param here

from geonode-importer.

TODO:

1. To be decided the usage of rasterio to handle the raster files. Maybe is better to continue with ogr2ogr with its drivers to avoid the usage of too many libraries
2. Change the storage manager to le the destination folder of the clone being passed as param. This is needed for example if we want to store the images in the non-default upload path:
   The changes must be done in the storage manager and the data-retriever
3. To be analyzed if smart_open can save the file in another destination (even in another container if needed)
4. create a "raster" handler. Probably it will contain less customization than the vector, it will be unique unless validation for each format is required
5. basic validation is needed
6. publish the resource in geoserver, probably this method is needed
7. the creation of the geonode resource and the dynamic model
8. handle the "copy" of the resource
9. Test coverage with unittest and end2end testing

Integration:

Option 1

1. Test with docker instances and relative volume share (this test may require different changes in the code to let it possible)
2. Fix project docker-compose to let the volume be visible between the two instances (Geonode/Geoserver) (it may be needed the support of a DevOps)

Option 2:
Let the geoserver be able to access the default (or the one defined in the settings) upload folder. So no additional volumes are needed.

Estimation without docker: 13 (~ 1 week)
Estimation with docker: 18 (~ 1 and 1/2 week)

@giohappy can I proceed?

from geonode-importer.

Some points about your notes:

• at the moment I don't foresee the need for a conversion utility (which by the way would be gdal_translate for rasters, not ogr2ogr). Currently, we only support GeoTiff and I would keep this constraint for the time being
• the handler should support the execution of pre-processing steps (e.g. for the optimization of the GeoTiff, tiling, adding overviews, etc.). It's not a strict requirement now, but it should be taken into account.
• do we really need dynamic models here? I don't see why...
• for sure we must support the Docker-based deployment. The Geoserver data dir volume is probably the best candidate for the shared folder.

from geonode-importer.