gmantele / vollt Goto Github PK

Any reason why this:

select * from publications where mission=xmm

fails with:

unresolved identifiers:
  * [l.1 c.43 - l.1 c.46] Unknown column \"xmm\" !

when my parser is setup like this:

    DefaultDBTable publicationTable = new DefaultDBTable("publications");
    publicationTable.addColumn(new DefaultDBColumn("id", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("doi", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("bibcode", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("title", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("journal", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("pubYear", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("abstract", publicationTable));
    publicationTable.addColumn(new DefaultDBColumn("mission", publicationTable));

but works ok when I pass an int like:

select * from publications where mission=123

A query like this: "SELECT TOP 1 * FROM ivoa.ObsCore -- comment "
without a ending newline returns:

INTERNAL SERVER ERROR! Sorry, this error is grave and unexpected. No explanation can be provided for the moment. Details about this error have been reported in the service log files ; you should try again your request later or notify the administrator(s) by yourself (with the following REQ_ID).

Could you please have a look? Thanks, Alberto

Please consider using Savot 4.0:
https://github.com/aschaaff/savot

Requires Java 1.6 but it is far more efficient and supports row by row processing (faster)

The test fails on Linux with the following error:

org.junit.ComparisonFailure: expected:<...ered at l.1, c.10: "[\u00e9" ('��]'), after : ""> but was:<...ered at l.1, c.10: "[?" ('?]'), after : "">
	at org.junit.Assert.assertEquals(Assert.java:115)
	at org.junit.Assert.assertEquals(Assert.java:144)

It works on MacOS: I think it has to do with the charset. Can't you make the check more relaxed so that it works independently of that?

Apparently the UNION operator is not supported by your grammar. Would it be possible to add it?
Thanks,
Alberto

Hi,

In case of a query failure, I have the following exceptions in the jetty logs:

2015-03-12 11:21:43,579 [eXistThread-18326] ERROR (XQueryURLRewrite.java [service]:382) - Error while processing /exist/tap/sync: Committed
java.lang.IllegalStateException: Committed
at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1144)
at org.eclipse.jetty.server.Response.reset(Response.java:1065)
at javax.servlet.ServletResponseWrapper.reset(ServletResponseWrapper.java:223)
at tap.error.DefaultTAPErrorWriter.writeError(DefaultTAPErrorWriter.java:183)
at tap.error.DefaultTAPErrorWriter.writeError(DefaultTAPErrorWriter.java:169)
at tap.resource.TAP.executeRequest(TAP.java:731)
at fr.jmmc.oidb.tap.OiDBTAPServlet.service(OiDBTAPServlet.java:100)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)

uwd.log reports an error:
2015-03-12 11:21:43.578 INFO TAP END_EXEC S1426155676676A ADQL query execution finished. SYNC,duration=26872ms (upload=-1,parse=2,exec=1911,format[votable]=-1)
2015-03-12 11:21:43.579 ERROR THREAD END S1426155676676A Synchronous thread "S1426155676676A" ended with an error. S1426155676676A (thread ID: 28150) is RUNNABLE in the group UWS_GROUP where 1 threads are active
2015-03-12 11:21:43.579 ERROR TAP SYNC_END S1426155676676A An error occured while executing the query of the synchronous job S1426155676676A.

I could increase the response buffer size but it seems that you write an error whereas the response was already written !

Is it possible to defer writing the query response until the query is sucessfull ?

Laurent

I need to expose a database field whose database datatype is boolean (db values true and false).
The issue is that TAP v1.0 §2.5 (table upload) states that the database column type 'boolean' is not supported. For that reason I guess taplib does not support this either.
TAP1.1 will support booleans, but the ESO TAP will be deployed before TAP1.1 becomes a REC.

Would you have any suggestion on how I could expose this boolean field before TAP1.1?

Many thanks,
Alberto

When I re-generate the parser with javacc adqlGrammar.jj, I get a number of failures when building the package, like:

src/adql/db/exception/UnresolvedJoinException.java:53: error: no suitable constructor found for ParseException(String,TextPosition)
		super(message, errorPosition);
		^
    constructor ParseException.ParseException(Token,int[][],String[]) is not applicable
      (actual and formal argument lists differ in length)
    constructor ParseException.ParseException() is not applicable
      (actual and formal argument lists differ in length)
    constructor ParseException.ParseException(String) is not applicable
      (actual and formal argument lists differ in length)
[...]
src/adql/parser/ADQLParser.java:4755: error: reference to ADQLParser is ambiguous
     this(stream, null);
     ^
  both constructor ADQLParser(InputStream,ADQLQueryFactory) in ADQLParser and constructor ADQLParser(InputStream,String) in ADQLParser match
src/adql/parser/ADQLParser.java:4754: error: recursive constructor invocation
  public ADQLParser(java.io.InputStream stream) {
         ^
src/adql/db/DBChecker.java:721: error: no suitable constructor found for ParseException(String,TextPosition)
				throw new ParseException("Column index out of bounds: " + index + " (must be between 1 and " + select.size() + ") !", colRef.getPosition());
				      ^
    constructor ParseException.ParseException(Token,int[][],String[]) is not applicable
      (actual and formal argument lists differ in length)
    constructor ParseException.ParseException() is not applicable
      (actual and formal argument lists differ in length)
    constructor ParseException.ParseException(String) is not applicable
      (actual and formal argument lists differ in length)
src/adql/db/exception/UnresolvedIdentifiersException.java:70: error: cannot find symbol
					addIdentifierName(colName + " " + pe.getPosition());
					                                    ^
  symbol:   method getPosition()
  location: variable pe of type ParseException
[...]

Javacc version is 5.0.

The following error can be (not often but yet) visible in the TAP log file:

java.util.ConcurrentModificationException
at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394)
at java.util.LinkedHashMap$ValueIterator.next(LinkedHashMap.java:409)
at uws.service.backup.DefaultUWSBackupManager.saveOwner(DefaultUWSBackupManager.java:451)
at uws.service.backup.DefaultUWSBackupManager.saveOwner(DefaultUWSBackupManager.java:415)
at uws.job.JobList.destroyJob(JobList.java:634)
at uws.job.JobList.destroyJob(JobList.java:664)
at uws.service.actions.DestroyJob.apply(DestroyJob.java:105)
at uws.service.UWSService.executeRequest(UWSService.java:1131)
at tap.resource.ASync.executeResource(ASync.java:191)
at tap.resource.TAP.executeRequest(TAP.java:839)
at tap.config.ConfigurableTAPServlet.service(ConfigurableTAPServlet.java:236)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:200)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

Voici un patch pour corriger la regexp: ')' en trop à la fin:

diff --git a/src/adql/db/STCS.java b/src/adql/db/STCS.java
index 2f1f4f3..7228a90 100644
--- a/src/adql/db/STCS.java
+++ b/src/adql/db/STCS.java
@@ -538,9 +538,9 @@ public final class STCS {
}

            // The final regular expression must be reduced to a coordinate system and nothing else after:

•           finalRegExp.append(")\s*");
  

•           finalRegExp.append(")\\s*$");
  

•           return (nbCoordSys > 0) ? finalRegExp.append(")$").toString() : defaultCoordSysRegExp;
  

•           return (nbCoordSys > 0) ? finalRegExp.toString() : defaultCoordSysRegExp;
  }
  

Laurent

Hi Gregory,
A query like:

SELECT top 10 centroid(s_region), distance(centroid(s_region),point('', 187.48, 2.05))
FROM ivoa.ObsCore

returns the error:
Incorrect ADQL query: Encountered "centroid". Was expecting one of: "POINT" """ <REGULAR_IDENTIFIER> "POINT" <REGULAR_IDENTIFIER> """

Though, CENTROID does return a POINT.

Using distance( cast(centroid(s_region) as POINT), point('', 187.48, 2.05) ) does not help.
Hope you could have a look.
Many thanks in advance,
Alberto

At least in SQL Server, the ADQL parser mishandles aliases in GROUP BY statements, causes query failure with the message "The multi-part identifier "rr.resource.ivoid" could not be bound." This occurs when using TOPCAT underlying queries in VO "Find Services" searches. The SQL for one of these, as run through a version 1.4 SQLServer_ADQLQueryFactory / SqlServerTranslator follows:

SELECT res.ivoid AS "ivoid" , res.short_name AS "short_name" , res.res_title AS "res_title" , res.reference_url AS "reference_url" , rr.res_role.base_role AS "base_role" , rr.res_role.role_name AS "role_name" , rr.res_role.email AS "email" , rr.interface.intf_index AS "intf_index" , rr.interface.access_url AS "access_url" , rr.capability.standard_id AS "standard_id" , rr.capability.cap_type AS "cap_type" , rr.capability.cap_description AS "cap_description" , rr.interface.std_version AS "std_version" , sbj.res_subjects AS "res_subjects" FROM rr.resource AS res INNER JOIN rr.interface INNER JOIN rr.capability LEFT OUTER JOIN rr.res_role LEFT OUTER JOIN (SELECT rr.res_subject.ivoid AS "ivoid" , dbo.ivo_string_agg(rr.res_subject.res_subject, ', ') AS res_subjects FROM rr.res_subject GROUP BY rr.res_subject.ivoid) AS sbj ON rr.res_role.ivoid=sbj.ivoid ON rr.capability.ivoid=rr.res_role.ivoid ON rr.interface.ivoid=rr.capability.ivoid AND rr.interface.cap_index=rr.capability.cap_index ON rr.resource.ivoid=rr.interface.ivoid WHERE (rr.res_role.base_role = 'contact' OR rr.res_role.base_role = 'publisher' OR rr.res_role.base_role IS NULL) AND rr.capability.standard_id = 'ivo://ivoa.net/std/conesearch' AND rr.interface.intf_type = 'vs:paramhttp'

Changing ON rr.resource.ivoid=rr.interface.ivoid to ON res.ivoid=rr.interface.ivoid makes the query work.

When getting the result of an ADQL query with the FORMAT 'votable/fits' (i.e. a VOTable with the data part serialized in FITS), the data are truncated. Then, the VOTable is corrupted ; it can not especially be read by TOPCAT/STIL/STILTS.

Hi Grégory,

I am currently preparing a Debian package for adql (as a prequisite for STILTS), and Hendrik pointed me to the github repository. I think now it would be worth to create a common package for all three libraries instead, taking the source directly from here. This would make the maintenance (dependencies and such) much easier, and we would get the other to packages into Debian for free.

The problem with that is, however, that I would need to use a common version number for the whole package. Would you consider to switch to a common version number for all components (tap, adql, uws)? Do you have any release plans for the near future?

Best regards

Ole

The following (completely valid) query returns an error:

SELECT em_min, em_max, raw.dp_id as Originating_RAW, raw.filter_path as filter
FROM ivoa.obscore as myalias, obs_metadata.data_products as raw
WHERE raw.dp_id in
(select source_file from phase3v2.provenance as p, phase3v2.files as f
where p.product_file=f.file_id and archive_id=myalias.dp_id)
and myalias.dp_id='ADP.2016-07-29T13:44:04.217'

ERROR: Unexpected error while executing a SQL query: The multi-part identifier "ivoa.ObsCore.dp_id" could not be bound.

If I remove the table alias (as shown here below), the query works fine.

SELECT em_min, em_max, raw.dp_id as Originating_RAW, raw.filter_path as filter
FROM ivoa.obscore, obs_metadata.data_products as raw
WHERE raw.dp_id in
(select source_file from phase3v2.provenance as p, phase3v2.files as f
where p.product_file=f.file_id and archive_id=ivoa.obscore.dp_id)
and ivoa.obscore.dp_id='ADP.2016-07-29T13:44:04.217'

Please notice that the problem is all in usage of an alias in the subquery (and not in the last WHERE constraint); indeed the following query runs successfully:
SELECT em_min, em_max
FROM ivoa.obscore as obscore
WHERE obscore.dp_id='ADP.2016-07-29T13:44:04.217'

Thanks,
Alberto

This comparison fails, especially on Java 9 (due out in three weeks):

https://github.com/gmantele/taplib/blob/0057e594d8308f1e2613f1d7dcc4703faeffd526/test/adql/translator/TestPgSphereTranslator.java#L318

It's not valid to compare floating point down to an unlimited number of decimal places like that. Can the test be rephrased to only check the first few decimal places?

Build log, of libadql-java, as it was called last time Debian updated it:

    [junit] 
    [junit] Testcase: testTranslateGeometryToDB took 0.447 sec
    [junit] 	FAILED
    [junit] expected:<...6d,0.997763534763054[3]d),(44.1514718625761...> but was:<...6d,0.997763534763054[5]d),(44.1514718625761...>
    [junit] junit.framework.AssertionFailedError: expected:<...6d,0.997763534763054[3]d),(44.1514718625761...> but was:<...6d,0.997763534763054[5]d),(44.1514718625761...>
    [junit] 	at adql.translator.TestPgSphereTranslator.testTranslateGeometryToDB(TestPgSphereTranslator.java:300)
    [junit] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    [junit] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    [junit] 
    [junit] Testcase: testTranslateGeometryFromDB took 0.011 sec

The following query generates a NPE:

SELECT * FROM TAP_SCHEMA.tables where table_type in ('table')

This is the resulting VOTable:
<VOTABLE xmlns="http://www.ivoa.net/xml/VOTable/v1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.3" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3"> <RESOURCE type="results"> <INFO name="QUERY_STATUS" value="ERROR">java.lang.NullPointerException</INFO> <INFO name="PROVIDER" value="ESO">"TAP service for data products (integration)"</INFO> <INFO name="REQ_ID" value="S1496142410790"/> <INFO name="ERROR_TYPE" value="fatal"/> <INFO name="ACTION" value="sync"/> </RESOURCE> </VOTABLE>

From the logs:
Caused by a java.lang.NullPointerException at adql.db.DBChecker$SearchColumnOutsideGroupByHandler.goInto(DBChecker.java:1315) at adql.search.SimpleSearchHandler.search(SimpleSearchHandler.java:206) at adql.db.DBChecker.resolveColumns(DBChecker.java:620) at adql.db.DBChecker.checkDBItems(DBChecker.java:471) at adql.db.DBChecker.check(DBChecker.java:409) at adql.db.DBChecker.check(DBChecker.java:374) at adql.parser.ADQLParser.Query(ADQLParser.java:512) at adql.parser.ADQLParser.parseQuery(ADQLParser.java:316) at tap.ADQLExecutor.parseADQL(ADQLExecutor.java:554) at tap.ADQLExecutor.start(ADQLExecutor.java:366) at tap.ADQLExecutor.start(ADQLExecutor.java:303) at tap.TAPSyncJob$SyncThread.run(TAPSyncJob.java:386)

If I run taplint against the latest version I get this error:
Wrong ObsCore identifier ivo://ivoa.net/std/ObsCore/v1.1 reported, should be ivo://ivoa.net/std/ObsCore#core-1.1 (corrected from PR-ObsCore-20160330)

The following ADQL:

 SELECT ivoid FROM rr.resource 
  RIGHT OUTER JOIN ( 
    SELECT 'ivo://' || detail_value || '%' AS pat 
      FROM rr.res_detail 
    WHERE detail_xpath='/managedAuthority' 
      AND ivoid='ivo://esavo/registry') 
    AS authpatterns 
  ON (resource.ivoid LIKE authpatterns.pat) 

gives the error:

<VOTABLE version="1.3" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<RESOURCE type="results">
<INFO name="QUERY_STATUS" value="ERROR">Unexpected error while executing a SQL query: ERROR: type "concat_str" does not exist  Position: 89</INFO>
<INFO name="PROVIDER" value="ESAC ESAVO">European Space Agency ESAC centre EuroVO / ESAVO Relational Registry</INFO>
<INFO name="REQ_ID" value="S1508763096700"/>
<INFO name="ERROR_TYPE" value="fatal"/>
<INFO name="USER" value="131.176.161.23 (131.176.161.23)"/>
<INFO name="ACTION" value="sync"/>
</RESOURCE>
</VOTABLE>

Where is "concat_str" defined?

The query:
SELECT top 1 * form dbo.SSA
fails probably because one field (mandatory in the SSA protocol) is declared DOUBLE arraysize="2".
The problem I signal here is on the fact the returned document (which should be a votable returning an error) is an invalid XML document.

<?xml version="1.0" encoding="utf-8"?>
<VOTABLE version="1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3">
<RESOURCE type="results">
<INFO name="QUERY_STATUS" value="OK"/>
<INFO name="PROVIDER" value="ESO">"TAP service for data products (integration)"</INFO>
<INFO name="QUERY" value="SELECT top 1 * from dbo.SSA"/>
<?xml version="1.0" encoding="utf-8"?>
<VOTABLE version="1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3">
<RESOURCE type="results">
<INFO name="QUERY_STATUS" value="ERROR">java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number</INFO>
<INFO name="PROVIDER" value="ESO">"TAP service for data products (integration)"</INFO>
<INFO  name="REQ_ID" value="S1505823511420"/>
<INFO  name="ERROR_TYPE" value="fatal"/>
<INFO  name="ACTION" value="sync"/>
</RESOURCE>
</VOTABLE>
'''
Many thanks,
Alberto

FORMAT is a TAP optional parameter. If not provided it defaults to "votable".
The standard does not mandate a specific serialization of the votable format.
The TAPLIB choice for the serialisation of a FORMAT="votable" request is to use base64.
I would really like to change that in our TAP and make TABLEDATA the default serialization.
That is, by default I would like our users to receive a readable votable, and to provide back a slimmer base64 votable only if they know what they are doing and they request it.
How can we achieve that? Is it a configuration paramater that can drive this?
Many thanks,
Alberto

Taplint cannot execute the ObsCore tests because the /capabilities xml does not contain the with the ObsCore ivo id.
Would it be possible to add it, e.g. as a configuration parameter? That would make it compliant with the TAPRegExt.
Thanks Gregory!

Gregory,

For our OiDB service, it is critical to support the OFFSET clause in order to have proper pagination of results: ~ 50 000 rows in our ObsCore table:

I would like to hack the ADQL parser to support:
SELECT TOP n OFFSET m ...

Any advice ?

How do you generate the ADQLParser class ? JavaCC / BNF ??

Whereas it is an extension to ADQL, I would like to collaborate as we need this feature to deploy our service in production.

Laurent

On the Gaia Archive site the following query works

SELECT
    parallax
FROM
    gaiadr1.tgas_source AS tgas
WHERE
    tgas.parallax > (10.0 * tgas.parallax_error)

but adding quotes around the schema name breaks it

SELECT
    parallax
FROM
    "gaiadr1".tgas_source AS tgas
WHERE
    tgas.parallax > (10.0 * tgas.parallax_error)

adding quotes around the table name breaks it

SELECT
    parallax
FROM
    gaiadr1."tgas_source" AS tgas
WHERE
    tgas.parallax > (10.0 * tgas.parallax_error)

If I issue a query containing a CASE statement I get an error.

Example:
SELECT top 10 CASE WHEN calib_level = 2 THEN 'default' ELSE 'higher_level' END as level
FROM ivoa.ObsCore

Error: Incorrect ADQL query: Encountered "calib_level". Was expecting one of: "," "FROM"

Caused by a adql.parser.ParseException at adql.parser.ADQLParser.generateParseException(ADQLParser.java:5917)
Encountered "calib_level". Was expecting one of: "," "FROM"

Is CASE supported?
Thanks,
Alberto

If I execute one of these queries:

SELECT BOX(‘’, 0, 0, 10, 10) from ObsCore
SELECT COORDSYS(POINT(‘’, 0, 0)) from ObsCore

(it's just to test my translator)

I get the following error on the console:

2017-03-01 14:28:15.791	INFO	TAP	PARSING	S1488374895785	Parsing ADQL: SELECT COORDSYS(POINT(‘’, 0, 0)) from ObsCore	SYNC,duration=-1ms (upload=-1,parse=-1,exec=-1,format[text]=-1)
2017-03-01 14:28:15.792	FATAL	THREAD	END	S1488374895785	Synchronous thread "S1488374895785" ended with a FATAL error.	S1488374895785 (thread ID: 160) is RUNNABLE in the group UWS_GROUP where 1 threads are active
2017-03-01 14:28:15.793	FATAL	JOB	END	S1488374895785	The following GRAVE error interrupted the execution of the synchronous job S1488374895785.
Caused by a adql.parser.TokenMgrError at adql.parser.ADQLParserTokenManager.getNextToken(ADQLParserTokenManager.java:1737)
	Lexical error at line 1, column 23.  Encountered: "\u2018" (8216), after : ""
2017-03-01 14:28:15.795	FATAL	HTTP	RESPONSE_SENT	S1488374895785	TAP resource "sync" execution FAILED with a GRAVE error!	HTTP-500 to the user ANONYMOUS as application/xml;charset=UTF-8
Caused by a adql.parser.TokenMgrError at adql.parser.ADQLParserTokenManager.getNextToken(ADQLParserTokenManager.java:1737)
	Lexical error at line 1, column 23.  Encountered: "\u2018" (8216), after : ""
2017-03-01 14:28:15.913	INFO	UWS	STOP		UWS Service "TAP/async" stopped!
2017-03-01 14:28:15.914	INFO	TAP	STOP		TAP Service stopped!

As far as I can see this has nothing to do with my translator, but it's in the ADQL parser itself.

Is there any specific reason why the JSON library is included in the repository, or can it be removed and replaced with this package?

LOW PRIORITY
I have a UWS job in PENDING status;
If I change the destruction date to a future date, the system works as expected.
If I change the destruction date to a past date, the system returns a VOTable with the error message:
Incorrect job ID! The job "1513770746650" does not exist in the jobs list "async".
And that is true, the job is no longer listed in the jobs list; as if got probably destroyed.

Would not be better to then return the jobs list instead of an error?

Thanks!

In the following valid ADQL query, the name of the column m50" is not returned as such but as m50"":

select top 10  Name,Seq , "m50""" as e
from "J/A+A/527/A145/table6"

Adding quotes around an alias should allow it to have special characters.

Using quotes hides a dash '-'

SELECT
    parallax AS "para-lax"
FROM
    gaiadr1.tgas_source AS tgas
WHERE
    tgas.parallax > (10.0 * tgas.parallax_error)
ORDER BY
    "para-lax"

but using quotes does not hide a dot '.'

SELECT
    parallax AS "para.lax"
FROM
    gaiadr1.tgas_source AS tgas
WHERE
    tgas.parallax > (10.0 * tgas.parallax_error)
ORDER BY
    "para.lax"

TAP Error are returned using default VOTableFormat (1.3, binary).

In our OiDB project we expect votable 1.2 defined by our own ServiceConnection implementation in getOutputFormat("votable").

The DefaultTAPErrorWriter instances are created automatically (1.3) by the following classes:
tap.AbstractTAPFactory
    74:  this(service, new DefaultTAPErrorWriter(service));
tap.resource.TAP
  102:  errorWriter = new DefaultTAPErrorWriter(service);

1. I can add a new constructor for DefaultTAPErrorWriter to add a new parameter DataFormat votFormat (version)
2. It is possible to call TAP.setErrorWriter() to give our own error writer but how to modify the AbstractTAPFactory.errorWriter field (final) ? add getter / setter ?

I can propose a patch ...

Columns of type array are not specified as array in the VOTable metadata of the result table. Theoretically, the attribute arraysize of a FIELD should indicate the length of the array. This is never the case when an array is returned.

According to §2.9.1 of: http://www.ivoa.net/documents/TAP/20100327/REC-TAP-1.0.html
the INFO QUERY_STATUS should be written:
<INFO name='QUERY_STATUS' value='OK'/>
and not as currently done:
<INFO name='QUERY_STATUS'>OK</INFO>

Is there a specific reason as to why you don't use a standard logging system, such as SLF4J? I find a bit confusing to have the logs of the server in two different places.

It would also give more flexibility to the users, that would be able to define their own logging format.

If you are interested I can also look into it: it shouldn't be a big change.

A query with a group by on a mathematical expression returns an error.
Example:
select count() as qty, target_name, 0.5(em_max + em_min) as central_lambda from ivoa.ObsCore where obs_collection="PESSTO" and dataproduct_type='spectrum' group by target_name, 0.5*(em_max + em_min)

returns the error:
Incorrect ADQL query: Encountered "0.5". Was expecting one of: """ <REGULAR_IDENTIFIER>

If I group by using the alias central_lambda, I get:
Unexpected error while executing a SQL query: Invalid column name 'central_lambda'.

Thanks!
Alberto

The result of the following query:
SELECT * FROM TAP_SCHEMA.tables;

is

<?xml version="1.0" encoding="utf-8"?>
<VOTABLE version="1.3" xmlns="http://www.ivoa.net/xml/VOTable/v1.3" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ivoa.net/xml/VOTable/v1.3 http://www.ivoa.net/xml/VOTable/v1.3">
<RESOURCE type="results">
<INFO name="QUERY_STATUS" value="OK"/>
<INFO name="PROVIDER" value="">"TAP service"</INFO>
<INFO name="QUERY" value="SELECT count(*)
FROM TAP_SCHEMA.tables;"/>
<TABLE>
<FIELD arraysize="*" datatype="char" name="COUNT_ALL"/>
<DATA>
<BINARY>
<STREAM encoding='base64'>
AAAAATY=
</STREAM>
</BINARY>
</DATA>
</TABLE>
</RESOURCE>
</VOTABLE>

datatype="char" is wrong: the datatype is set to UNKNOWN_NUMERIC in ADQLQUERY.getResultingColumns() and then converted into a char in VotType(final DBType tapType)

I'm working on my fork, but I don't think my code has anything to do with that.

A query like:

    SELECT TOP 10 *
    FROM gaiadr1.gaia_source
    WHERE astrometric_primary_flag = 1

Fails with the following error:

ERROR: operator does not exist: boolean = integer
Hint: No operator matches the given name and argument type(s).
You might need to add explicit type casts.

VOTable lets the ability to specify for pairs of coordinate FIELDs, the used coordinate system. However, TAP metadata does not give much information about that, neither the ADQL query results (in VOTable and FITS). This is getting more and more important with catalogs like Gaia which have several releases, each at a different epoch, and more than one pair of coordinates, each in a different coordinate system.

It would be interesting to support a coordinate system in TAPLib.

Hi,

I was wondering what is the default job retention time in the UWS list, and if and how it can be configured.
Also, what happens if a job is aborted? It looks like it disappears from the list altogether: is that the desired behaviour?

Cheers,
Vincenzo

Is it possible to remove the hidden file? :
taplib-master/src/adql/query/operand/function/geometry/ .#ContainsFunction.java.1.1

Thanx

We are upgrading from uws v4.0 to uws v4.1. Between these two we have found that parameters passed in the body of the POST request are ignored. Parameters in the query string of the POST request are still processed, however it is more typical to include parameters in the body for a POST request, e.g. form submission.

The taplib creates many threads within a web application: use a ThreadPool for performance, better resource handling (max jobs) and also handle properly shutdown:
http://wiki.apache.org/tomcat/MemoryLeakProtection#cclThreadSpawnedByWebApp

If the webapp is stopped or the web server is shutdown, created threads MUST be interrupted ... and the pending jobs should be serialized as backup (to restart later)

I can have a look and see how to do it...

In DBChecker.resolveTables (line 522), the DB name of the table's DBLink was dropped when a table alias was provided:

if (table.hasAlias())
    dbTable = dbTable.copy(null, table.getAlias());

Dear Gregory,

The J2000 is a possible (and for us useful!) reference frame in the STC-S specification. Unfortunately, J2000 is not a supported value in your implementation.
Would it be possible to add support for it?
Many thanks,
Alberto

The definition for Error says :

An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions.

The parser throws a TokenMgrError given the string "@", which is bad input but I would not classify it as an abnormal condition.

The following database error:

Caused by a org.postgresql.util.PSQLException at  [...]
ERROR: column t.Y does not exist

is returned when executing the following ADQL query:

SELECT t.*
FROM (SELECT (ra+ra_error) AS x,
             pmra AS "ProperMotion",
             (dec+dec_error) AS Y
      FROM mytable) AS t

I am trying to limit the duration of TAP requests. I added

max_execution_duration = 1000

(by the way, it is in ms, but it gets rounded down to the second, at least in the capabilities, so anything below 1000 is 0, which is not the expected behaviour).

And if I make a sync call I get the following message:

Time out! The execution of this synchronous TAP query was limited to 1 seconds. You should try again but in asynchronous mode.

How do I limit the queries in async mode?

One more thing: is the timeout applied to the duration of the DB query or the TAP request?

When requesting text output the TextFormat builds the resulting AsciiTable in memory, and this can lead to an OutOfMemoryError in case of big result sets. In our case we hit the limit with less than 1M entries from ivoa.ObsCore.

The class VOTableFormat works properly in streaming mode.

Hi,

I'm trying to define a UDF with the following signature:
INTERSECTION(region1 region, region2 region) -> region
I provided a class that implements it and it works, but if I use it in a select I don't get the desired format because the return value is interpreted as UNKNOWN instead of REGION.

Do I have to define it in my UDF class? IF so, how?