opencadc / caom2service Goto Github PK
View Code? Open in Web Editor NEWCommon Archive Observation Model - micro-service implementations
License: GNU Affero General Public License v3.0
Common Archive Observation Model - micro-service implementations
License: GNU Affero General Public License v3.0
If the download URL does not provide a filename via the Content-Disposition header the tar generation fails with a null pointer exception.
The current method of using TAP query processing is too subtle and isDownloadable() is deprecated.
The definitive correct way: get the plane release (dates) and read groups and use AccessUtil (from caom2 core lib) to determine permissions. Viable as of CAOM 2.4 and needed to add support for Artifact.contentRelease and Artifact.contentReadGroups
These should be position_dimension_naxis1[2] instead.
The following caom 2.4 fields are not populated from the ADQL query response by the mappers:
Observation.metaReadGroups
Plane.dataReadGroups
Plane.metaReadGroups
Artifact.contentReadGroups
The PlaneMapper
class throws a NullPointerException when caom2:Plane.energy.bounds.samples
are missing, but caom2:Plane.time.bounds.samples
are present.
trailing QUERY_STATUS = FAIL leads to particularly obscure bugs (not found in meta and datalink)
eg ivo://cadc.nrc.ca/mirror/HST?ocqn3hytq/ocqn3hytq-RAW_STANDARD
This fails in toPlaneURI(PublisherID) because that code tries to extract the collection name from the path but gets mirror/HST instead of HST.
CADC code to generate publisherID values assumes a base URI and appends collection so a quick fix is to use the last path component, but that is not mandated anywhere.
When loading a particular Plane into the caom2ops/meta
service, some observations are throwing errors:
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long at ca.nrc.cadc.caom2ops.Util.getLong(Util.java:86) at ca.nrc.cadc.caom2ops.mapper.PlaneMapper.mapRow(PlaneMapper.java:226) at ca.nrc.cadc.caom2ops.CaomTapQuery.buildObservation(CaomTapQuery.java:359) at ca.nrc.cadc.caom2ops.CaomTapQuery.performQuery(CaomTapQuery.java:165) at ca.nrc.cadc.caom2.meta.MetaQueryRunner.doIt(MetaQueryRunner.java:210) at ca.nrc.cadc.caom2.meta.MetaQueryRunner.run(MetaQueryRunner.java:159) at ca.nrc.cadc.uws.server.AbstractExecutor.executeSync(AbstractExecutor.java:208) at ca.nrc.cadc.uws.server.AbstractExecutor.execute(AbstractExecutor.java:171) at ca.nrc.cadc.uws.server.SimpleJobManager.execute(SimpleJobManager.java:220) at ca.nrc.cadc.uws.server.SyncServlet.processRequest(SyncServlet.java:410) at ca.nrc.cadc.uws.server.SyncServlet.access$000(SyncServlet.java:109) at ca.nrc.cadc.uws.server.SyncServlet$1.run(SyncServlet.java:298) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:421) at ca.nrc.cadc.uws.server.SyncServlet.doit(SyncServlet.java:293) at ca.nrc.cadc.uws.server.SyncServlet.doGet(SyncServlet.java:247) at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:190) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)
The Util.getLong() method
is trying to cast a Long to an Integer.
Version 1.3.1 does not have this problem, and so was introduced in 1.3.2.
In a request with FORMAT=foo the error handling fails to create a TableWriter impl that can output the error message correctly. The bug is in PluginFactory.getTableWriter being used to get the result TableWriter (should throw UnsupportedOperation exception) and to get the error tableWriter (which should always successfully return a writer.
Correct behaviour:
In the getURL()
on line 195, the StorageResolver
class is determined from the URI scheme of the given URI. That resolver does not have the authMethod
from the CaomArtifactResolver
set so that it can provide the appropriate URL (i.e. include /auth
for basic authentication).
Plane.position.bounds is a Shape and can be a circle or simple polygon. The TAP type should be char,*,shape (proposed for DALI-1.2)
Plane.position.bounds.samples is only set when Plane.position.bounds is a Polygon. The TAP type should be char,*,multipolygon (proposed for DALI-1.2)
A pair of Converter/Formatter is needed so that the position_bounds column is used in predicates but the position_bounds_points column should be used for polymorphic output.
Aside: It would have been nicer if the spoly column had the internal-only name (eg. position_bounds_spoly) and the real values were stored in position_bounds; then one would only need to modify predicate column reference in the CaomRegionConverter. Such a change spans caom2-tap-server and caom2persistence and would need to be coordinated
Here is a search for a SITELLE observation:
Using the package service endpoint (https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/pkg?ID=ivo%3A%2F%2Fcadc.nrc.ca%2FCFHT%3F2384125p%2F2384125p) will result in a file containing an error message, but it has a .tar
extension:
unexpected failure: java.lang.RuntimeException: entry size '8606365612' is too big ( > 8589934591 ).
CFHT-2384125p-2384125p/README0100644 0000000 0000000 00000000222 14043065762 013230 0ustar000000000 0000000 OK 2384125z.hdf5 1ca4483aac2b46e6281a80d41b36aa39 https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/data/pub/CFHT/2384125z.hdf5
As the HD5 file is larger than 8GB.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.