gns-science / kororaa-graphql-api Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 0.0 3.86 MB

The graphql application API for NZHSM Kororaa.

License: GNU Affero General Public License v3.0

Python 99.52% Makefile 0.48%

kororaa-graphql-api's People

Contributors

Watchers

kororaa-graphql-api's Issues

K-API Feature: Support for Markdown Content

AS Kororaa support team

we want to add and modify markdown text that will be be rendered in 'Info' links in the Kororaa UI (or any other purpose that MD is appropriate for)

so that it's possible to enrich this content without redeploying the UI.

Done when:

add API endpoint for markdown content
add API configuration to that markdown content is mapped to a specified S3 root folder
when API maps to valid markdown file (in s3) return the markdown

Feature: modify hazard_curve attribute for app use-case

We want the query form like below, and the mappings of hazard_mode to various toshi-ids to be internal to API.

query {
  about
  hazard_curves (
    hazard_model: "TEST1"
    vs30s: [250, 350 , 450]
    imts: ["PGA", "SA(0.5)"]
    locs: ["WLG", "AKL"] # Include lat,lon for these
    aggs: ["mean", "0.1", "0.9"]
  ) {
    ok
    curves {
      loc   
      imt
      agg
      vs30
      curve {
        levels
        values
      }
    }
  }
}

Feature: Gridded Hazard API query should return a NamedLocation type when the query specifies a valid code or name.

Done when:

add name and key properties to GriddedLocation schema type, these are filled if a named location can be match either by location or by code. (See resolution field).
For matching named locations we ALWAYS uses a 0.01 degree resolution regardless of resolution argument.
resolution field dermines how 'fussy' the query spec is going to be for locations that do not match named locations. Typically our grid locations are at 0.1 degree resolution, but may sometimes be at 0.2. NB Expect some change on this feature if/when we offer mixed grids

Feature: Add colour bar feature for gridded map visulazations

When we produce a Gridded Hazard geojson file, we also need to show the users what colours were applied to the geojson features.

Done when:

round input max values to nearest 0.5
return an ordered list of colours using a chosen cmap (e.g. 'jet') on 0.1 intervals from min to max.

Kororaa Fix: vs30 update THS version on Koraraa API

we need THS 0.5.5 for vs30 fix

Feature: basic AWS deployment to DEV works

sls deploy works as expected

K-API feature: more gridded-hazard caching

make main gridded hazard resolver fully cacheable (in memory)

K-API feature: add clipping option for gridded_hazard geojson.

K-API Fix: Resolve poetry install issue on CI/CD chain

getting this error in GHA (see below) ...

as I'm runnig poetry 1.14 locally it's very likely a poetry lock file issue - see snok/install-poetry#89

Run poetry install --with dev
  poetry install --with dev
  poetry add tox-gh-actions
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.9.13/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.13/x64/lib
    VENV: .venv/bin/activate
  
Creating virtualenv kororaa-graphql-api in /home/runner/work/kororaa-graphql-api/kororaa-graphql-api/.venv
Installing dependencies from lock file


Error: Process completed with exit code 1.

Configure cloudwatch metrics

add cloudwatch config
create basic metrics dashboard

Feature: add LogNorm option for gridded_hazard colour scale

K-API Fix: vs30=500

kororaa-graphql-api version:
Python version:
Operating System:

Description

Occurs on map query with vs30=500

Query

query HazardMapsPageQuery($grid_id: RegionGrid, $hazard_model_id: String, $imt: String, $agg: String, $vs30: Int, $poe: Float, $color_scale: String, $color_scale_vmax: Float, $fill_opacity: Float, $stroke_width: Float, $stroke_opacity: Float, $color_scale_normalise: ColourScaleNormalise) {
  gridded_hazard(grid_id: $grid_id, hazard_model_id: $hazard_model_id, imt: $imt, agg: $agg, vs30: $vs30, poe: $poe) {
    ok
    gridded_hazard {
      grid_id
      hazard_model
      imt
      agg
      hazard_map(color_scale: $color_scale, color_scale_vmax: $color_scale_vmax, fill_opacity: $fill_opacity, stroke_width: $stroke_width, stroke_opacity: $stroke_opacity, color_scale_normalise: $color_scale_normalise) {
        geojson
        colour_scale {
          levels
          hexrgbs
        }
      }
    }
  }
}

Variables

{
		"agg": "mean",
		"color_scale": "inferno",
		"color_scale_normalise": "LOG",
		"color_scale_vmax": 0,
		"fill_opacity": 1,
		"grid_id": "NZ_0_1_NB_1_1",
		"hazard_model_id": "SLT_v8_gmm_v2_FINAL",
		"imt": "PGA",
		"poe": 0.1,
		"stroke_opacity": 0,
		"stroke_width": 5,
		"vs30": 500
}

Response

{
  "errors": [
    {
      "message": "Invalid vmin or vmax",
      "locations": [
        {
          "line": 9,
          "column": 7
        }
      ],
      "path": [
        "gridded_hazard",
        "gridded_hazard",
        0,
        "hazard_map"
      ]
    }
  ],
  "data": {
    "gridded_hazard": {
      "ok": true,
      "gridded_hazard": [
        {
          "grid_id": "NZ_0_1_NB_1_1",
          "hazard_model": "SLT_v8_gmm_v2_FINAL",
          "imt": "PGA",
          "agg": "mean",
          "hazard_map": null
        }
      ]
    }
  }
}

KAPI Fix: Whangarei Hazard curves not resolving on FINAL model

kororaa-graphql-api version: current

Description

the query below from Kororaa returns curves for Auckland only...

What I Did

{"query":"query HazardChartsPlotsViewQuery(\n  $hazard_model: String\n  $vs30s: [Int]\n  $imts: [String]\n  $locs: [String]\n  $aggs: [String]\n  $resolution: Float\n) {\n  hazard_curves(hazard_model: $hazard_model, vs30s: $vs30s, imts: $imts, locs: $locs, aggs: $aggs, resolution: $resolution) {\n    ok\n    locations {\n      lat\n      lon\n      resolution\n      code\n      name\n      key\n    }\n    curves {\n      hazard_model\n      imt\n      loc\n      agg\n      vs30\n      curve {\n        levels\n        values\n      }\n    }\n  }\n}\n","variables":{"hazard_model":"SLT_v8_gmm_v2_FINAL","vs30s":[175],"imts":["PGA","SA(0.1)","SA(0.2)","SA(0.3)","SA(0.4)","SA(0.5)","SA(0.7)","SA(1.0)","SA(1.5)","SA(2.0)","SA(3.0)","SA(4.0)","SA(5.0)","SA(6.0)","SA(7.5)","SA(10.0)"],"locs":["-35.72~174.32","-36.87~174.77"],"aggs":["mean","0.05","0.95","0.1","0.9"],"resolution":0.1}}```

FIX: AWS runtime error

runtime error on AWS ImportError: cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs'

is triggered on the first import of boto3 in the cloudwatch module.

ref boto/boto3#3648

FIX: SRG locations

kororaa-graphql-api version: 0.6.4
Python version: 3.9
Operating System: Ubunto

Description

The new location added to nzshm-common for SRG have high precision. But the API is rounding these to nearest 0.01 degrees. Meanwhile in THS, the location precision used is 0.001 degrees. So new locations queried via the API will likely miss.

e.g. Franz Josef (corrected location) ->

{'id': 'srg_164',
 'name': 'Franz Josef',
 'latitude': -43.375755162,
 'longitude': 170.18836548
}

Suggested fix:

hazard curve search with two degrees precision can be safely resampled to three degrees
and with three degrees precision should remain at three degrees
also for the locations resolver, all locations should be returned with three degrees precision.

K-API Fix: disaggs location xx1

Cannot retrieve list of avialble disaggs when disaggs.json contains a location with 'location_key': 'xx1'. As this bug was found via kororaa-ui, I don't have the exact k-api error. Attached are 2 files

disaggs.json contains locations_key xx1 (causes error)
disaggs_noxx1.json same with every entry containing xx1 removed (no error)

disaggs.zip

Feature: Gridded Hazard map layer

We want to produce gridded hazasrd maps in Kororaa. The API should return a geojson gridded hazard map given suitable arguments

Korora API : PROD deployment

Feature: setup the AWS deployment pipeline

deploy-test
main

K-API feature: API listing of NSHM publications

AS Kororaa support team

we want to add, and modify the contents of publications without too much overhead.

Done when

define a schema for publications (author, reviewer, bilblio refs , links etc
API method reads data from a JSON file that we'll store in S3 (Same as used for Disaggregration reports & Markdown content)

K-API feature: text content refinements

when API mapping fails (S3 Not found error) return a placeholder MD content at {PATH} not available
provide sample API calls and add to Kororaa UI ticket.

Feature: return hazard curves for an arbitrary location

I want to get hazard curves from nearest grid location to the point specified. Return the requested hazard data and the actual grid point requested.

Done when...

~~on fail: let the user know if there is no gird point available for the input args~~
automatically resolve off-grid points to the closest grid (per model)
on sucess: return the actual coordinates of chosen grid point
and all the hazard curve detail as requested

Feature: API endpoint for Hazard Result

We want to use the API to retrieve HAZARD curves from toshi-hazard-store.

hazard query returns values for given ID, etc (as per toshi-hazard-store.)
~~metadata lists meta for given VS30~~

K-API Feature: model changelog

provide CHAGELOG.md from S3 on request
request "index" : "CHANGELOG.md"

K-API Feature: IFM Analysis query

We want an API query for logic tree components.

Provide mapping from logic tree parameters to inversion solution ID
provide list of available values for each argument in query
#64 is now standalone

API examples:

note that value and value_options fields are encoded as JSON-String types. This is done because the types vary and this is not easily handled in graphql.

get Model options

using source_logic_tree_spec field we can find the options applicable to each fault system branch

query get_model_by_version {
  nzshm_model (version: "NSHM_1.0.0" ) {
    model {
      version
      title
      source_logic_tree_spec {
          fault_system_branches {
              short_name
              long_name
              branches {
                  name
                 long_name
                 value_options
              }
          }
       }
     }
   }
}

returns

{
  "data": {
    "nzshm_model": {
      "model": {
        "version": "NSHM_1.0.0",
        "title": "Initial version",
        "source_logic_tree_spec": {
          "fault_system_branches": [
            {
              "short_name": "PUY",
              "long_name": "Puysegur",
              "branches": [
                {
                  "name": "dm",
                  "long_name": "deformation model",
                  "value_options": "[\"0.7\"]"
                },
                {
                  "name": "bN",
                  "long_name": "bN pair",
                  "value_options": "[[0.902, 4.6]]"
                },
                {
                  "name": "C",
                  "long_name": "area-magnitude scaling",
                  "value_options": "[4.0]"
                },
                {
                  "name": "s",
                  "long_name": "moment rate scaling",
                  "value_options": "[0.28, 1.0, 1.72]"
                }
              ]
            },

... etc

2) get Source Logic Tree

using source_logic_tree field to obtain the complete SLT

query get_model_version {
  nzshm_model(version: "NSHM_1.0.0") {
    model {
      version
      title
      source_logic_tree {
        fault_system_branches {
          long_name
          short_name
          branches {
            weight
            inversion_solution_id
            inversion_solution_type
            onfault_nrml_id
            distributed_nrml_id
            values {
              long_name
              json_value
            }
          }
        }
      }
    }
  }
}

returns

{
  "data": {
    "nzshm_model": {
      "model": {
        "version": "NSHM_1.0.0",
        "title": "Initial version",
        "source_logic_tree": {
          "fault_system_branches": [
            {
              "long_name": "Puysegur",
              "short_name": "PUY",
              "branches": [
                {
                  "weight": 0.21,
                  "inversion_solution_id": "U2NhbGVkSW52ZXJzaW9uU29sdXRpb246MTE4NTQ2",
                  "inversion_solution_type": "ScaledInversionSolution",
                  "onfault_nrml_id": "SW52ZXJzaW9uU29sdXRpb25Ocm1sOjExODcxNw==",
                  "distributed_nrml_id": "RmlsZToxMzA3NTM=",
                  "values": [
                    {
                      "name": "dm",
                      "long_name": "deformation model",
                      "json_value": "\"0.7\""
                    },
                    {
                      "name": "bN",
                      "long_name": "bN pair",
                      "json_value": "[0.902, 4.6]"
                    },
                    {
                      "name": "C",
                      "long_name": "area-magnitude scaling",
                      "json_value": "4.0"
                    },
                    {
                      "name": "s",
                      "long_name": "moment rate scaling",
                      "json_value": "0.28"
                    }
                  ]
                },

etc...

K-API feature: query to return a single InversionSolution ID given a valid set of branch values

K-API Feature: TEXT_CONTENT for IFM anlaysis page

Add help text for IFM Analysis page

"index": ifm_analysis_help.md"

K-API feature: add Disaggregation reports

AS Kororaa support team

we want to add, and modify the disaggregation reports without too much overhead.

Done when

define a schema for disaggs (loc, vs30 etc)
API method reads data from a JSON file that we'll store in S3 (Same as used for Disaggregration reports & Markdown content)

Feature: basic serverless graphl API about (hello-world) endpoint

We want to see the API deployed via SLS (similar to Toshi API) and have this sit nicely with poetry test etc.

add serverless framework
update installation docs
add flask + graphql +graphene
add test for Kororaa about query

K-API feature: update to include latest nzshm-model 0.2.0 for IFM

FIX: production queries raising error

Description

testing PROD Kororaa none of the dynamic veis (curves & Maps) are working

{
  "errors": [
    {
      "message": "_convert_to_request_dict() missing 1 required positional argument: 'endpoint_url'"
    }
  ],
  "data": {
    "hazard_curves": {
      "ok": true,
      "locations": [
        {
          "lat": -41.3,
          "lon": 174.78,
          "resolution": 0.01,
          "code": "-41.30~174.78",
          "name": "Wellington",
          "key": "WLG"
        }
      ],
      "curves": null
    }
  }
}

K-API Feature: location codes

return location codes to use in hazard curve legend

see GNS-Science/kororaa#147

gns-science / kororaa-graphql-api Goto Github PK

kororaa-graphql-api's People

Contributors

Watchers

kororaa-graphql-api's Issues

Description

Description

What I Did

Description

Suggested fix:

API examples:

get Model options

2) get Source Logic Tree

Description

Recommend Projects

Recommend Topics

Recommend Org