gem / oq-platform2 Goto Github PK

All applications used for the platform with another branch instead of master:

oq-platform-building-class branch: oq-platform2
oq-platform-taxtweb: oq-platform2
git+git://github.com/gem/django-chained-selectbox.git@pla26#egg=django-chained-selectbox-0.2.3
git+git://github.com/gem/django-nested-inlines.git@pla26#egg=django-nested-inlines-0.1.4
git+git://github.com/gem/django-chained-multi-checkboxes.git@pla26#egg=django-chained-multi-checkboxes-0.4.1
git+git://github.com/gem/wadofstuff-django-serializers.git@pla26#egg=wadofstuff-django-serializers-1.1.2

Sometime the django serve does not start during tests (see https://ci.openquake.org/job/master_oq-platform2/693/ and https://ci.openquake.org/job/master_oq-platform2/691/) but we loose the information about what was going wrong because we do not have access to the server logs.

We should save logs (djnago and maybe geoserver too) on a text file that then is exported by jenkins as an artifact, like we do on oq-platform.

Something like GitLab does https://docs.gitlab.com/ee/user/admin_area/monitoring/health_check.html

It should report status of GeoNode, Engine server and GeoServer

Instead of use title to get proper GHEC and ISC maps we move to use UUID.
Some cleanup of map loaders scripts is required (dev prints & Co.)

How to reproduce:
Open the "Vulnerability" tab in the Physical Vulnerability web app.
Note that there are many functions in the list including "CR+CIP/LPB", "CR/LRINF+DNO" etc.
Select Material "Concrete, reinforced" and press the "filter" button - an empty list is shown.

Cause
The newly added "Silva and Martins" functions have a name field corresponding to a new taxonomy string, but the "taxonomy gem" field is empty. At least some of the new models make use features from version 3 of the GEM taxonomy such as "DUM" and so are not compatible with the vulnerability DB software.

Related
This issue is related to gem/oq-platform-taxtweb#28; in both cases our software needs to be updated to handle the new version of the taxonomy.

We discovered that the platform has an hard dependency on tilestream which is a very fragile service (unmaintained software on an unsupported os, known to crash after X days).

We should, if possible, remove the tight dependency on Tilestream: if it is not available we should just be unable to load its layers, it should not prevent the platform and other maps to work if those layers aren't required/loaded.

The applications affected : Map's visualization, GeoExplorer, Grv

There are some problems to visualize correctly the maps and relative thumbnails

The default TZ is America/Chicago. It must be overwritten in the local_settings.py to UTC

The supplemental information filed on objects is truncated after few words. This is hiding from the user some important information like the copyright.

It's a regression since in the old platform we were showing all of its content.

The visualization of more details for each function of vulnerability application doesn't works with simplejson version superior of 2.0.9

Allowing anonymous users to perform actions on the platform could open the doors to malicious attackers, please check at least principal functionalities that must be denied to not logged in users.

GENERAL

• add geoserver management to verifier-guest.sh
• geofence mgmt (add, list, drop rules in oq-gs-builder.sh) to allow custom oqplatform workspace
• add 'geonode' workspace to our imported geoserver structure

SLD

• set_maximum_form_content for jetty (to use big sld files)

ISC

• isc sql import (using migrations)
• isc dev data import

GHEC

• ghec sql import (using migrations)
• ghec dev data import

In the old platform we were using django-request to get some access statistics. It looks not available in platform2.

The released version of django-request is a bit old and does not play very well with proxies: we have now a fork with some fixes made for GeoViewer which runs on maps.openquake and SERA (https://github.com/gem/django-request).

It can be installed via:

pip install https://github.com/gem/django-request/releases/download/1.5.5%2Bgem/django_request-1.5.5+gem-py2-none-any.whl

an example of configuration is

settings.INSTALLED_APPS.append('request')
settings.MIDDLEWARE.append('request.middleware.RequestMiddleware')

REQUEST_IGNORE_IP = ['127.0.0.1']
REQUEST_IGNORE_PATHS = (
    r'^admin/',
)

The functions with that as name start with Taller OQ (SGC 2018) -should be removed from the platform. They were used for a workshop in 2018. They are now obsolete.

Implement a test to check if works correctly the search on the platform

There's no any doc about installation.

Add map creation to be also able to run tests

The tomcat log in production is full (more than 1/s) of errors like

29 Aug 10:07:04 WARN [geofence.internal] - GeoFence was not able to find any matching user on Security Context amd Services.
Aug 29, 2019 10:07:04 AM org.geoserver.security.GeoServerRestRoleService connectToRESTEndpoint
WARNING: null
org.apache.http.conn.HttpHostConnectException: Connect to localhost:8000 [localhost/127.0.0.1] failed: Connection refused (Connection refused)

The log, which has been rotated less than 3 hrs ago is already almost 90MB big.

It looks like that tomcat is trying to contact a non existing service (localhost:8000)

ISC

• isc production data import
• dev_data for any version of isc

GHEC

• ghec production data import
• dev_data for any version of ghec

When I tried to install oq-platform2 by this command

## first download release 2.2.1 and extract the tar.gz file
sudo -H oq-platform2-2.2.1/openquakeplatform/bin/deploy.sh

It reported an error :

cp: cannot stat '/root/oq_install.sh': No such file or directory

It was caused by line 221:

    cp $HOME/oq_install.sh .

To add test for download csv from exposure application

For the import project from IRMT (plugin qgis) to platform there is a problem with import the style.
To reproduce this problem:

Install Platform from branch pla26-irv and start
Open qgis
Connect to platform (IRMT settings (settings icon) -> create connection to platform)
Menu IRMT -> oq platform -> Load socioeconomic data -> get indicator -> select indicators and countries
Save project
Click weight data (balance icon) and save
Menu IRMT -> oq platform -> upload project to platform