Containering of processes about pywps HOT 5 OPEN

Just skim read @cehbrecht's linked article - it looks like a good concept to a not-really-developer.

If PyWPS implements containerised services, could we please have options:

a. to not run containers and let processes use the host system resources as they do now
b. to fire up a scalable container set. My near term use case for this is scaling processes beyond the machine where PyWPS sits - either to my organisation's own cloud resources or (insert commercial provider here)'s.

from pywps.

just got a link to "function as a service":

https://dev.to/developius/functions-as-a-service---deploying-functions-to-docker-swarm-via-a-cli

Haven't really looked into it ... but might be useful as an example for using docker/swarm.

from pywps.

Email from Adam that made a master thesis on the topic, he implemented docker

Hi devs,

during work on my thesis[1] I've added Docker extension to pywps. At my git there is a demo server (pywps-demo with extended config file) [2]. To run it with Docker extension it's necessary to use pywps[3] and OWSLib[4] from my git. Also you need Docker engine to build image and create container (the extension was tested only on Linux).

I will be glad for any comment. Recently, I faced a problem with zombie processes when using multiprocessing python package for deamon that manages created docker containers. Eventually, I used threading[5] instead but not really sure if it's the right solution. So any opinion will be appreciated.

Cheers,
Adam

[1] https://github.com/ctu-geoforall-lab-projects/dp-laza-2018/blob/master/text/adam-laza-dp-2018.pdf
[2] https://github.com/lazaa32/pywps-flask
[3] https://github.com/lazaa32/pywps
[4] https://github.com/lazaa32/OWSLib/tree/sync_request
[5] https://github.com/lazaa32/pywps/blob/develop/pywps/processing/container.py#L73

from pywps.

An HPC analyst working with us on deploying PyWPS on a cluster suggested that Singularity would be a better fit than docker for integration into HPC architectures (at least in Canada). I can't answer questions about any of this, but I can redirect them.

from pywps.

from pywps.