Giter Club home page Giter Club logo

ogcapi-coverages's Introduction

ogcapi-coverages's People

Contributors

bradh avatar cmheazel avatar cnreediii avatar gbuehler avatar ghobona avatar jerstlouis avatar ogcportal avatar pebau avatar schpidi avatar tomkralidis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ogcapi-coverages's Issues

What extensions should OAPI Coverages have?

In addition to the Core functionality, at some time a wider range of operations should be supported, such as further subsetting, scaling, etc. Here a first attempt:

  • Range subsetting (also known as band subsetting, variable extraction)
  • Scaling (change of grid resolution while keeping the space/time extent)
  • CRS transformation (space, time, and possibly other dimension types!) +
  • Transaction (insert, update, and deletion of coverages)
  • Interpolation (whenever interpolation is involved in the request on hand, the method indicated shall be applied)
  • Processing (WCPS queries for extraction, filtering, aggrgation, analytics, etc.)

Items marked with a "+" are of relevance beyond coverage world and might be discussed in a wider context.

Tiled coverages (to populate a GeoPackage or else).

Hi,

It occurred to me that if could be good to specify a way to obtain tiles from a (grid) coverage. A common use case is to populate a GeoPackage. GeoPackage has defined "elevation" tiles that has the data in them (not a pictorical representation but real elevation numbers).

By combining coverages with tiles we could create a path like this:
'/collections/{collectionId}/coverages/{coverageId}/tiles/{tileMatrixSetId}/{tileMatrix}/{tileRow}/{tileCol}':
that allows access to a particular tile in a coverage.

We are using the same mechanims also for tiled features and tiled maps so it might be interesting to have coverages also on board. I know you will say that you already have a "subset" mechanism. WMS also had a subset (BBOX) but i was useful to specify go to get tiles without caring so much in BBOXes (or caring only once; while defining the TileMatrixSet).

"scaling" conformance class

Has there been any discussion of a "scaling" conformance class to allow a client to request a coverage at a resolution other than the coverage's native resolution?

The WCS 2.0 Scaling Extension (OGC 12-039) defines the following parameters on the GetCoverage request: scaleFactor, scaleAxes, scaleSize, and scaleExtent. CubeWerx's test OGC API implementation at https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa has implemented these parameters at the /collections/{collectionId}/coverage endpoint, and they work nicely. So it should be a simple matter of adapting OGC 12-039 lock, stock and barrel as a conformance class of "OGC API - Coverages".

Coverage with common and OpenAPI definition

Please take a look at the work done here:
https://github.com/opengeospatial/OGC-API-Map-Tiles

where we are considering a OGC API common to support maps, tiles, coverages and features.

This is the common OpenAPI "domains" document:
https://github.com/opengeospatial/OGC-API-Map-Tiles/blob/master/standard/openapi/ogc-api-common.yaml
This is the OpenAPI "domains" document for coverages
https://github.com/opengeospatial/OGC-API-Map-Tiles/blob/master/standard/openapi/ogc-api-coverages.yaml
This is an example of coverage service described with OpenAPI and using the last two documents as building blocks
https://github.com/opengeospatial/OGC-API-Map-Tiles/blob/master/standard/openapi/ogc-api-map-tiles-opf-xmp-ct-more-1-collection.yaml

the CIS schemas are still not include as part of the Coverage API part.

Content Negotiation

Requests such as "/collections/{coverageid}/coverage/all" return a mixture of "metadata" and range set data. In most cases this collection cannot be represented using a single format. So multi-part mime-types are required. While API-Common provides two conformance classes to identify the supported encoding formats, that approach will not scale to API-Coverages. The draft of API-Coverages has attempted to address this issue. Please review, comment, and provide suggestions for improvement.

tree structure vs features api

I may have missed this aspect in the preparations of oapi's...

Oapi-cov uses a tree structure of /collection/coverages/... where oapi-ft uses a structure of /collection/items (which are the features).

At the pyGeoApi group we were discussing how a software product can support both oapi-cov as well as oapi-ft. It should either have to identify the relevant service-type by a unique string, or allow a mixture of features and coverages in a single collection.

Some questions that came to my mind...

  • Shouldn't a coverage be considered as a collection of grid-rows (like features in a ft collection)?
  • Is a coverage-series (collection) a very common use case in the coverage domain? I guess so, because satellites produce many observation on a single flight.
  • Do you envision an api supporting both features as well as coverages in a single collection?

Not only gridded coverages

I am unhappy with the statement: "Only gridded coverages" in the first draft document, as it is feasible to retrieve MultiPoint point cloud based coverages. The UK Met Office and partners are implementing 3/4D curve based coverages, admittedly on underlying gridded data initially.
We should at least consider whether the generic coverage API could tackle these approaches to avoid being locked into a 'gridded only or image coverage' mindset.

do not enforce multipart

Req 2.3 says: "A coverage accessed according to requirement 2.2 SHALL be returned as multipart coverage including the DomainSet, RangeType, and RangeSet in the coverage’s Native Format."

Multipart is just one possible encoding, others have their practical relevance (and importance) as well. Also, coverage components enumerated are incomplete. Further, demanding Native Format is contradicting format negotiation.

AFAICS what this req wants to say is: coverage shall be encoded according to the format negotiation - this, however, is already stated elsewhere.

What is open is the (coverage specific) distinction of single-file vs multipart; se WCS 2.0 for its definition.

Bottom line, rather than reinventing a description hastily we might want to rely on CIS 1.1 which defines it. Hence, suggestion:

"A coverage SHALL be returned encoded as defined in OGC CIS 1.1 and WCS 2.0".

Make use of HTTP headers

For example, format should be requested using standard http headers as defined in RFC2616 like Accept and Accept-Encoding and reported with Content-Type.

Users Guide

Several of the API Standards have associated user guides. They serve to capture non-normative content which would be of use to an implementer. Currently the README is filling that role for API-Coverages. This content should be removed from the README and moved into a separate User Guide document.

Subsetting DomainSet

In cases where the domainSet is huge, retrieving only subsets of it in order to formulate meaningful requests against the rangeSet is desired.

This has been explored in Testbed 14 with CloudSat data as ReferenceableGridCoverages.

Definition of Scope

Populate the README file with the information a user will need to understand and participate in the development of this specification. That will include a clear and concise statement of the scope of the current effort.

Create a Guide for OAPI common

Weather on the Web has extended WFS. Where do we capture their best practices? What other guidance do we need for users of this spec.

Coverage and Features from the same /collections/{collectionid|coverageid}?

In 7.4.1 Collections there are a few things that could use some clarification. These questions are intentionally leading some discussion regarding #8 and opengeospatial/ogcapi-common#140.

First, there is the following text:

Each collection on a Coverages API will be a coverage.

Is this precluding a spatial data resource that is both a coverage AND a feature-collection?


Second, in 7.4.2.2 response there is:

itemType:
    description: indicator about the type of the items in the collection (the default value is 'unknown').

What is the expected itemType for a coverage that can be accessed as both a coverage or features?


Spatial data servers typically list datasets available from the server in some kind of catalog. This catalog would typically lists "access types" for each dataset. In the case of a dataset with both features and coverages access, what would the server link to in both cases?

In the case of a feature collection, it's clear. it would link to the collection.

e.g. https://info.geoconnex.us/collections/hu02

But what if that collection is both accessible as a feature collection and a coverage? Would the server link to the same collection resource? Would it link to /collections/{coverageid}/coverage/ ?


The last question is what are we gaining by trying to overload /collections and have one link relation data ? As the questions above illustrate, the basic details of cataloging collections of just two types -- especially if both types are desired at the same time -- are not well established. What would be lost if we used /coverages and a link relation specific to coverage access instead of /collections at the root of the coverages API?

Coverage representation in HTML

Currently discussion of HTML seems to mostly concentrate on encoding coverage metadata in HTML, ie: not the range set. If we talk about encoding a complete coverage we need to establish an HTML encoding including the range set. (Sometimes an image can be rendered: if the coverage is 2D and contains 1 or 3 integer bands.)

SWG Charter

What is the scope of this SWG and does the proposed charter accommodate it?

Scoping Parameters

API-Coverages introduces the Range Set. This is basically an n-dimensional collection of binary measurements. URL query parameters such as bbox or datetime do not work well here. The draft of API-Coverages attempts to provide a scope to query parameters. Has this effort been successful? How could it be done better?

Change coverages to coverage in the API

As a results of this discussions:

I became convinced that coverages needs to be replaced by "coverage" in the API.

/collecions/{collectionId}/coverage
(simetric to: /collecions/{collectionId}/items)

Reason 1:
Features are too atomic and need to be aggregated in collections for convenience: e.g. to cover a particular extent. Coverages are already an aggregation of data into a coverage structure: a boxel, a point cloud etc. They also have several "attributtes" represented by "rangeTypes" like features.

To me "coverage" is equivalent to "items". They both cover an extent, represent several attributes and can be queried. It is possible to move from items to a coverage by rasterization and for a coverage to items by vectorization. A point cloud can be transformed into a collection of points in a much more direct way.

Reason 2:
In OGC API coverages the collectionId becomes the coverage name in the same way that the collectionId is the feature collection id in OGC API features. Both can be transformed into maps and tiles using:
/collecions/{collectionId}/map
without even knowing the collection type. It becomes symmetric.
They can also be processed in the OPC API processes without caring about the collection type.

Please note that the need for collections of collections is a completely different discussion that also affects features and should be solved in OGC API Common.

So I'm requesting to change the current orientation of the OGC API coverage to use coverage (in singular) and use the {collectionId} and the name of the coverage.

So:
/collecions/{collectionId}/coverage gets the whole coverage
/collecions/{collectionId}/coverage/rangeSet gets the actual values
/collecions/{collectionId}/coverage/domainSet gets the dimensions definition.

Renaming of WCS.SWG

A motion has been initiated in the WCS.SWG to rename the SWG. This ticket serves to collect discussion and alternatives. Giving it a start:

  • Alt 1: keep WCS.SWG -- issue: does not reflect OAPI sufficiently for some
  • Alt 2: rename to OAPI-Coverages.SWG -- Issue: does not reflect WCS sufficiently for some
  • Alt 3: have both in name, suggestions:
    • WCSOAPI.SWG
    • Web coverages - APIs and services SWG (James Passmore)
  • pls add further suggestions + discussion below.

(this github place is maybe not appropriate for the discussion, but the WCS.SWG charter change has been put here, too, so why not have all together)

Req 1.4 unclear

Req 1.4 says: "A bounding box query SHALL return a list of all coverages from the specific collection that interescts the area bounded by that bounding box. "

Aside from the typo, it is not always a collection that is addressed, it can also be a single coverage, and then we want to do subsetting.

Recommendation: explicitly state "A bounding box query on a collection...."

Coverage Offering

The API-Coverages draft returns a Coverage Offering from /collections/{coverageid}/coverage. The schema for the resource this URL returns is mostly empty. What should be included in a Coverage Offering?

Req 4.1 unclear

Req 4.1 says: "If an encoding HTTP header (Accept and Accept-Encoding) is provided then a representation of the coverage in the format indicated SHALL be returned."

I believe what is intended is: "If an encoding HTTP header (Accept and Accept-Encoding) is provided in the request then a representation of the coverage in one of the formats indicated SHALL be returned."

Path syntax

From Swath Coverage Engineering Report

In the context of the WCS REST API design experiment, the resources are identified by paths. Each path uniquely identifies a resource. For example, the root path / identifies the entry point which returns a document containing links to other resources - i.e. specifications identified by "/api", conformance declaration (identified by "/conformance"), and list of coverages (identified by "/coverages"). The coverage description of each coverage is identified as "/coverages/{coverageId}" while the actual data of a coverage is identified as "/coverages/{coverageId}/rangeSet". The paths are unique identifiers for different resources.

OpenAPI basics: what does "/" mean?

If in a URL, say "http://....../..../a/b/c", what does "a/b" actually mean? Naturally, I would expect that "b is a sub-resource of a", but that does not always apply. Before we start building a whole framework on this we may want to clarify this.

Subsetting RangeSet

Should we support subsetting of the RangeSet. A subset operation on the RangeSet would only return content from the RangeSet
Related issue: #30

Determine implications of OGC API - Common - Part 2

The SWG today (2020-05-27) discussed whether the development of OGC API - Common - Part 2: Collections might have implications for OGC API - Coverages. The SWG agreed to revise the document, if possible, to be consistent with OGC API - Common - Part 2: Collections.

Need support for asynchronous coverage requests

Coverages are often many terabytes in size, and therefore it can often take minutes or hours to satisfy a rangeset request (even a subsetted one). This causes HTTP user agents and servers to time out long before the request is satisfied.

CubeWerx resolved this issue in our WCS implementation by adding a vendor-specific "responseHandler" parameter to the GetCoverge operation. This allows a client to request a coverage with "REQUEST=GetCoverage&...&responseHandler=mailto:[email protected]". The client gets an immediate response back indicating that the request has been submitted, and then at some future point in time an e-mail message gets sent to the specified e-mail address that provides either a link to the requested coverage or an indication of failure.

There needs to be a way to do something like this in "OGC API - Coverages".

identifier syntax for OAPI Common

in several places we use identifiers (such as coverages, collections). We should define admissible names and their representation, such as: char set, encoding, entities for escaping special characters, etc.

Subsetting by geometry

For geoscience use cases, we may want to subset a coverage by a defined feature/geometry. For example, subsetting a defined region from a continental-scale evapotranspiration coverage dataset.

Could this be offered in the coverages api? e.g.

GET /collections/{coverageid}/coverage/rangeset?subset=hasGeometry(<geometry_url>)

where Geometry URL could be either cached/known to the server or retrieved in real-time. Suggest hasGeometry aligning with the GeoSPARQL property. An example of this is a Watershed/Drainage basin called "Darling river" in Australia. The URL for this could be: http://gds.loci.cat/geometry/geofabric2_1_1_awradrainagedivision/9400206

In the case that it is known/cached on the server, then the coverage api implementation will go ahead and perform subsetting and return the result (rangeset data for input geometry).

Where the geometry would need to be retrieved, the server could resolve the URL and via content negotiation (say HTTP Accept: application/json), the server could retrieve the geometry and go from there...

Alternatively, a client might drop in the geometry as WKT.

GET /collections/{coverageid}/coverage/rangeset?subset=hasGeometryAsWKT(<geometry as WKT>)

Lastly, the client might want to specify whether to return contained or overlapping cells as a separate parameters, e.g.

GET /collections/{coverageid}/coverage/rangeset?subset=hasGeometry(<geometry_URL>)&subset=geoTopology('<topology>')

... where topology could be from Section 7. Topology Vocabulary Extension (relation_family) in the GeoSPARQL spec, such as, 'sfContains'.

Consideration for validating and linting the OAS with IBM's openapi-validator

Unfortunately I was unable to contribute to the development of the coverages OAS at the hackathon in London.
I would however like to propose that we use IBM's openapi-validator to further standardize the way that the coverages OAS looks.
The results of running the validator are as follows

lint-openapi openapi.yaml

[Warning] No .validaterc file found. The validator will run in default mode.
To configure the validator, create a .validaterc file.

errors

  Message :   Property type+format is not well-defined.
  Path    :   components.schemas.extent.properties.temporal.items.type
  Line    :   526

  Message :   Property type+format is not well-defined.
  Path    :   components.schemas.collectionGeoJSON.properties.timeStamp.type
  Line    :   547

warnings

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./.get.operationId
  Line    :   36

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./conformance.get.operationId
  Line    :   54

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections.get.operationId
  Line    :   73

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}.get.operationId
  Line    :   98

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages.get.operationId
  Line    :   136

  Message :   tag is not defined at the global level: Coverages
  Path    :   paths./collections/{collectionId}/coverages.get.tags
  Line    :   131

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}.get.operationId
  Line    :   222

  Message :   tag is not defined at the global level: Coverages
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}.get.tags
  Line    :   219

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/domainset.get.operationId
  Line    :   265

  Message :   tag is not defined at the global level: Coverages
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/domainset.get.tags
  Line    :   262

  Message :   operationIds must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/rangetype.get.operationId
  Line    :   308

  Message :   tag is not defined at the global level: Coverages
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/rangetype.get.tags
  Line    :   305

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}.get.parameters.0
  Line    :   100

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages.get.parameters.0
  Line    :   138

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}.get.parameters.0
  Line    :   224

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}.get.parameters.1
  Line    :   232

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/domainset.get.parameters.0
  Line    :   267

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/domainset.get.parameters.1
  Line    :   275

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/rangetype.get.parameters.0
  Line    :   310

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   paths./collections/{collectionId}/coverages/{coverageId}/rangetype.get.parameters.1
  Line    :   318

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   components.parameters.collectionId
  Line    :   653

  Message :   Parameter names must follow case convention: lower_snake_case
  Path    :   components.parameters.coverageId
  Line    :   662

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.exception
  Line    :   347

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.exception.properties.code.description
  Line    :   352

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.exception.properties.description.description
  Line    :   354

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.root
  Line    :   356

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.root.properties.links.description
  Line    :   361

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.req-classes
  Line    :   382

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.req-classes.properties.conformsTo.description
  Line    :   387

  Message :   Property names must be lower snake case.
  Path    :   components.schemas.req-classes.properties.conformsTo
  Line    :   387

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.link
  Line    :   396

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.link.properties.href.description
  Line    :   401

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.link.properties.rel.description
  Line    :   403

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.link.properties.type.description
  Line    :   406

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.link.properties.hreflang.description
  Line    :   409

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.content
  Line    :   412

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.content.properties.links.description
  Line    :   418

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.content.properties.collections.description
  Line    :   435

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.collectionInfo
  Line    :   439

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionInfo.properties.links.description
  Line    :   457

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.extent
  Line    :   483

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.extent.properties.crs.enum.0
  Line    :   493

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.extent.properties.trs.enum.0
  Line    :   516

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.collectionGeoJSON
  Line    :   528

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.type.description
  Line    :   534

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.features.description
  Line    :   538

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.links.description
  Line    :   542

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.timeStamp.description
  Line    :   546

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.numberMatched.description
  Line    :   549

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.collectionGeoJSON.properties.numberReturned.description
  Line    :   552

  Message :   Property names must be lower snake case.
  Path    :   components.schemas.collectionGeoJSON.properties.timeStamp
  Line    :   546

  Message :   Property names must be lower snake case.
  Path    :   components.schemas.collectionGeoJSON.properties.numberMatched
  Line    :   549

  Message :   Property names must be lower snake case.
  Path    :   components.schemas.collectionGeoJSON.properties.numberReturned
  Line    :   552

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.collectionGeoJSON.properties.type.enum.0
  Line    :   537

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.featureGeoJSON
  Line    :   555

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.featureGeoJSON.properties.type.description
  Line    :   562

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.featureGeoJSON.properties.properties.description
  Line    :   568

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.featureGeoJSON.properties.type.enum.0
  Line    :   565

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.geometryGeoJSON
  Line    :   575

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.geometryGeoJSON.properties.type.description
  Line    :   580

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.0
  Line    :   583

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.1
  Line    :   584

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.2
  Line    :   585

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.3
  Line    :   586

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.4
  Line    :   587

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.5
  Line    :   588

  Message :   Enum values must be lower snake case.
  Path    :   components.schemas.geometryGeoJSON.properties.type.enum.6
  Line    :   589

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.domainSetJSON
  Line    :   590

  Message :   Schema properties must have a description with content in it.
  Path    :   components.schemas.domainSetJSON.properties.type.description
  Line    :   595

  Message :   Schema must have a non-empty description.
  Path    :   components.schemas.rangeTypeJSON
  Line    :   597

As you can see there are some issues here which relate closely to those raised in #23 for example.

Does anyone have preference on this issue? I would like to know whether I should invest the time to address the issues flagged by the linter before I go ahead and do so.
Thanks

Path case-sensitive

According to RFC 3986 the path component is case-sensitive. Do we want to use the camel-case notation from XML or rather mandate all lower case?

I tend to prefer all lower case as it looks more natural in a URL.

How to model resource (sub)selection based on the coverage model?

Coverages are defined as having some canonical components, most importantly:

  • domain set (where can I find values?)
  • range set (what are the values?)
  • range type (what does a value mean? - BTW, defined based on SWE Common so that the semantics is carried over to downstream services seamlessly)
  • metadata (whatever else the coverage wants to tell us)

This could be translated natively into addressing like:

  • /coverages/{id}/ -- give me the whole coverage
  • /coverages/{id}/domainSet -- give me the domain set (ie, a description of where data siet, such as a bbox)
  • /coverages/{id}/rangeSet -- give me the range set (ie, the pure pixel values, without any coordinates, metadata, etc.)
  • /coverages/{id}/rangeType -- give me the semantics of the values, such as uom
  • /coverages/{id}/metadata -- give me the coverage's metadata

add resolution to collection temporal object

Given various datasets with temporal extents provide multiple timesteps within an extent, does it makes sense to add an optional duration property (consistent with RFC3339)? Example below for model run with 3 hour time steps:

        "temporal": {
          "interval": [
            [
              "2020-05-23T00:00:00Z",
              "2020-06-02T00:00:00Z"
            ]
          ],
          "duration": "PT3H"
        }

Do not enumerate coverage components

Req 2.3 says: "A coverage accessed according to requirement 2.2 SHALL be returned as multipart coverage including the DomainSet, RangeType, and RangeSet in the coverage’s Native Format."

I would not enumerate components - a coverage is well defined, we just need to say that we return the complete coverage. Otherwise there is the danger of missing items - in this case: metadata, coverageFunction, etc.

Variables in the URL template are case-sensitive

The https://tools.ietf.org/html/rfc6570 says:
"Variable names are case-sensitive because the name might be expanded within a case-sensitive URI component."
This means that coverageid is not the same as coverageId.
I request that OGC API coverages uses lowerCammelCase for path variables.

I'm not able to find any reference to "case" in the definition of the key in a KVP "query" part (known as application/x-www-form-urlencoded)". Actually, https://www.ietf.org/rfc/rfc3986.txt says that the query string is case-sensitive. "The other generic syntax components are assumed to be case-sensitive unless specifically defined otherwise by the scheme (see Section 6.2.3)." If somebody is able to find a RFC that says that, please tell me.

It seems that OGC introduced in OWS Common 1.0 "11.5.2 Capitalization" : "The capitalization of parameter names when KVP encoded shall be case insensitive, meaning that parameter names may have mixed case or not. EXAMPLES The “request” parameter name could be REQUEST, request, Request, or ReQuEsT."

Protection from large Coverage API requests

For large datasets, such as satellite products covering long periods of time, it is easy to submit a synchronous coverage request that cannot be fulfilled within typical timeout limits. Even if a large request CAN be fulfilled, the user may be impacted by getting much more data volume than expected. There is likely no one solution to solve this for the diversity of users and servers. An assemblage of solutions might include the following capabilities:

    • page through the coverage response by spatial or temporal chunk or tile
    • obtain estimated size of a response before actually requesting the data (like a HEAD request)
    • allow servers to advertise maximum request limit
    • have a specific error response that indicates the request is too big to satisfy
    • support asynchronous response

HTML not optimal for findability/discovery

clause_6_overview.adoc, under "Encodings", claims that search engines can exploit HTML contents. This is true only at the lowest level: HTML describes syntax, not semantics (remember why we introduced XML and RDF?), hence semantic search a the level required today is not possible.

  • The corresponding text should be changed to reflect modern technology and the state of the art.
  • semantic-rich format should be supported.

Suggest using less prescriptive paths in favor of link relations

In some places, the spec seems to use RESTful resource links and relations to allow navigation and seems to explicitly support those relations potentially crossing server boundaries, allowing more flexibility in API implementation and evolution.

In other places, the spec prescribes specific routes and operations relative to the API root (actually, language-wise, it's not 100% clear that it isn't relative to the server root, but it's my understanding that's not the intention).

Example:

The API SHALL support the HTTP GET operation at the path /collections/{coverageid}/coverage.

As written, client developers would would be crazy to not concatenate paths into a server's resources; it's easier to write such a client and requires fewer network requests for interrogation. The result is that many of the benefits of link relations are lost.

I have three examples where this causes real challenges for us and the potential evolution of the spec:

  1. We have metadata catalogs that could expose a features api. They live on different endpoints from our data services that would provide coverages. We further have coverage services that could benefit from being on different web endpoints depending on the collection / request. As written (See spec: Table 2. Coverage API Paths) we will need to find a way to collapse all of these URLs, rather than simply being able to use link relations to direct clients to the proper resource.
  2. (As discussed in the Bethesda sprint with some consensus) We have a notion of groups of collections, so essentially a parent/child relationship. This could be accommodated with the addition of a single relation, "child", to the spec except that the URL structure definition precludes the possibility of having, say, /collections/{parentId}/{childId}(/{childId})*/coverage. My concern would be that this belies a lack of expressiveness in the spec that will require frequent amendments in the future.
  3. The term "collections" clashes with a domain concept for our (NASA EOSDIS) data. We have usability concerns in providing an endpoint which will need to specify IDs for both collections in the URL, where the ID that is actually labeled "/collections" differs from what we refer to as a "collection" everywhere else.

Note that if we did switch some of the prescriptive URL paths to obey link relations, this wouldn't prevent a user of NASA EOSDIS data or ESA data or other data from constructing URLs in a convenient way for their known server endpoints, it would only impact general-purpose clients / libraries. What I'm picturing here is a client specification, where we define how a conformant client needs to interpret information like link relations (actually, I think this is a good idea regardless so maybe a separate issue?).

Processing end-point and integration with OGC API - Processes, and support for modular workflows

Currently, the draft processing spec suggests /collections with a q query parameter for passing e.g. a WCPS requests.

However, /collections is defined in Common - Part 2, and a q parameter is too generic for an extension at this end-point in this manner. A q parameter changing completely the nature of the response to a GET request at this end-point, which normally returns a list of collections, does not seem appropriate either. Moreover, processing may act on coverages spread out across different datasets (or even different services), and may be part of a workflow chain, where /collections would not be an appropriate end-point for this.

I suggest instead a dedicated processing resource at /processes/wcps to receive WCPS processing requests. This could live at multiple end-points within an OGC API tree, for example for a single collection, for a dataset, and for a multi-dataset API (as detailed at https://eratosthenes.pvretano.com/Projects/tb16/ogcapi/datastores.html, with these 3 separate processing end-points). Such an extension would also not specifically require WCPS, and algorithms could be submitted in other languages (e.g. openEO). This is also consistent with a similar approach suggested for OGC API - Maps (opengeospatial/ogcapi-maps#42).

A request would be posted to this end-point following a process execution schema defined by OGC API - Processes and being refined in Modular OGC API Workflow projects ( as detailed in opengeospatial/ogcapi-processes#47 (comment) ). If Processes is fully supported, this would also be a self-describable end-point which explains that this receives WCPS code.

Potentially this end-point may also supporting WCPS directly with a WCPS-specific media type, as opposed to a process execution document in the POST payload. If support for a GET request together with a q parameter is desired to facilitate access, perhaps this could live at a sub-resource e.g. GET /processes/wcps/queryResult?q=....

Change <coverageid> to <collectionid>

The Commons API defines a <collectionid>, used in the path:

/collections/<collectionid>

Whereas the Coverages API, where a Collection IS a Coverage, uses the term

/collections/<coverageid>

However, this requires explanation that coverageid=collectionid and extra documentation that seems unnecessary.

Instead if the Coverage API uses the same field name, , that I think it is more clear that coverage is just an additional endpoint on a Collection that the Coverage API adds to the Commons API.

Consider WCS Core service model

WCS Core already contains a service model along which service facets can be modelled in an intuitive way; see Figure 1 in 09-110r4. Effectively, all current WCS Core operations can be nicely described a extracting this or that component(s) from the overall service model tree.

Advantage: request writers can intuitively construct what they want without learning lots of individual items by heart.

Attached a simplified version of WCS Core Fig 1.
untitled2

Coverage collections

The benefits or not of building hierarchies, have possibly been done to death, but before we decide that we have a hierarchy that is coverages/coverage, how might we address a notion that a provider might want to create one or more sets of coverages ~ I know the idea has been proposed before (where a coverage could be a member of more than one set)?

Subset requirements class definition incomplete and does not match its examples

(Please let me know if you'd like this sort of feedback to be split up in the future. I'm trying to avoid polluting the tracker with tons of related feedback.)

Inconsistencies:

  • The BNF specifies that intervals are colon-separated. Examples are comma-separated. It's not clear to me which is correct, but I prefer colon pretty strongly based on it being easier to parse and understand when accepting unexploded parameters and using a syntax potentially familiar to Python users (though comma is conventional for mathematical intervals). Note also that both comma and colon cause encodeURIComponent to return an encoded value, potentially impacting readability and ability for end-users constructing URLs to generate correct values.
  • The definition uses lowercase subset= and the examples use uppercase SUBSET=. Even if case-insensitive, consistency is preferable.
  • Intervals are defined differently to how datetime defines them, both in how they separate components and their notation for open and closed intervals. FWIW, I prefer the subset notation because it doesn't tend to trip
  • Examples use inconsistent capitalization of axes. I can't tell if this is intentional. Any case sensitivity in axis names should be noted. If it's case sensitive, I'd suggest capitalizing Time for consistency and to suggest case sensitivity. If it's not, I'd suggest lowercasing everything for consistency with the rest of the API.
  • bbox and datetime naming and syntax are not simply carried forward for spatial and temporal subsetting, causing clients to, in very common cases, specify the same information two different ways. "Give me the coverages matching this bounding box, then subset their latitude and longitude using these two ranges" vs "Give me the coverages matching this bounding box, then give me only the points within this bounding box."

Incomplete items:

  • The meaning of asterisk (*) is not explained in this section. I assume it has similar meaning to open / closed interval in datetime
  • The document does not specify whether intervals are inclusive or exclusive. The examples seem to suggest they're inclusive, which would make sense, but it should be spelled out.

Clashing Syntax / Interpretation:

  • Mathematical intervals use ( to denote exclusive intervals and [ to denote inclusive intervals, where subset= (I think) describe inclusive intervals and use (
  • C-style function calls. lat(0, 10) triggers "function call" in developer brains, especially if comma is correct (unclear)
  • Mathematical points. (0, 10) looks a lot more like a point than an interval, if the comma is correct notation
  • The OpenAPI doc includes rangeSubset which seems to be some sort of variable subsetting. If that's intended to be added to the spec, its meaning in relation to this is very confusing without referencing the docs

I'm willing to fix the obviously necessary items above (BNF and examples using different syntax and caps, documenting asterisk) if someone can clarify/confirm the correct interpretation.

Alternative syntax options for consideration, so I'm not just throwing stones:

  • subset=lat[0:10] Advantages: Uses widely recognized array syntax, along with a common slicing syntax. Does not trigger any mathematical interpretation, but if you wanted to look at it as an interval, it's clearly inclusive.
  • subset[lat]=0,10 Advantages: Consistent with bounding box syntax. May be easier for some languages to parse into appropriate structures.
  • bbox=0,0,10,10 Advantages: Can literally carry forward a well-known query parameter that's available in core for querying collections without having to (a) detect which axes correspond to which spatial dimension or (b) provide the alternative syntax subset=lat(0:10)&subset=lon(0:10). Note, I'd propose this as an additional convenience syntax, as I can se the utility in also being able to subset arrays not expressible by bbox or datetime

How to handle feature-coverages?

It is common practice in hydrology to treat a collection of watershed features that cover the landscape as a coverage. This is also common with spatial units in various domains that can be thought of as a coverage.

There is also a very common practice where a grid (rectilinear or otherwise) can be accessed via index position. In this arrangement, the i,j position of a cell can be thought of as a feature identifier.

In both cases, an individual entity can be identified uniquely as a feature, but it can be equally valid to access the "layer" as a coverage. From the perspective of OGC API, it would seem that we shouldn't actually make a distinction between coverages and features as unique kinds of data, but rather, support various operations on a given "layer".

So say we had a feature collection that could also be treated as a coverage, we might do:
https://acme.com/collections/foo_collection/items
as well as:
https://acme.com/collections/foo_collection/rangeset

Using this kind of arrangement, if a provider didn't want to allow access to the collection items, they wouldn't have to. Conversely, if a set of features really shouldn't be treated as a coverage of features, then the coverage functionality wouldn't be supported. But at the end of the day, functions/subsetting capabilities that operate on coverages and features could be applicable to either, right?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.