python module validation of observation_id field - too stringent? about caom2 HOT 5 OPEN

see https://www.opencadc.org/caom2/#ObservationURI and https://www.opencadc.org/caom2/#PlaneURI

Those two types specify the structure of those URIs, but since they do so in reference to other fields they only loosely imply that (i) those field values must be valid to use in URIs and (ii) additional path separators (/) are not allowed.

This is admittedly a poorly documented part of the model. Since URIs are used internally and also services are expected to be able to provide ivoa publisher ID values based on these same strings, the restriction on chars that cannot be used in URIs (eg space) is pretty much impossible to remove.

from caom2.

Since ObservationURI contains {collection} and {observationID} (one /) and PlaneURI contains that plus {productID}, the URIs can currently be validated based on number of path components to make sure s/w is using the right one in the right place... we could think about allowing additional separators (/) but that would weaken the validation of those two types of URI.

from caom2.

@alaity47 Can you provide some concrete use case/example of observationID and/or productID values? Where they come from, who decides what those strings are, etc.

from caom2.

The format/content of observationid for us varies by mission. We only have a couple of collections where a space or a slash can be in there, and it's because we're either using part or all of a filename path, or some kind of target/object name.

Some examples:

allsky/970607n/s043
BD+62 1644
HR 7042

I completely understand enforcing these restrictions on URIs, but the way we've implemented things here there's no direct relationship between observation_id and any URIs. To me the documentation reads as if that field can be any string (https://www.opencadc.org/caom2/2.3/#Observation.observationID)

from caom2.

At MAST, the observation_id is typically the "fileroot" of the primary science file. For example, if the file is ipppssoot_drz.fits, the observation_id would be ipppssoot. I know we use URI parsing logic for the artifact URI and would not be surprised if we also have some for the observation URI. Many times that parsing logic is very simple and just splits by "/" so having additional ones would likely break things on our end.
Regardless, because our observation IDs are derived from filenames, we generally have the restriction that they would be valid filenames in the first place and so "/" would not be allowed. Spaces in principle should have no issues, but for us we use underscore and dashes to split up parts of a fileroot (eg, hlsp_k2sc_k2_llc_211804118-c05_kepler_v2_lc).
Could you consider replacing / or spaces with _ or - to work around these limitations?

from caom2.