Comments (2)
Steps to reproduce the issue
hive
CREATE TABLE test_from_utc_timestamp_source (source_float float);
CREATE VIEW test_from_utc_timestamp_view AS SELECT CAST(from_utc_timestamp(source_float, 'America/Los_Angeles') AS STRING) ts_float FROM test_from_utc_timestamp_source;
select * from test_from_utc_timestamp_view;
-- 1970-01-30 21:30:00
trino
insert into hive.default.test_from_utc_timestamp_source values (2592000.0);
trino version 420 while using coral version 2.2.9
SELECT * FROM hive.default.test_from_utc_timestamp_view;
Query 20231013_184405_00000_fa5un failed: line 1:15: Failed analyzing stored view 'hive.default.test_from_utc_timestamp_view': line 1:18: Unexpected parameters (varchar, varchar) for function at_timezone. Expected: at_timezone(timestamp(p) with time zone, varchar(x)), at_timezone(timestamp(p) with time zone, interval day to second)
trinoSql
in ViewReaderUtil
SELECT CAST(CAST("at_timezone"("format_datetime"("from_unixtime"(CAST("test_from_utc_timestamp_source"."source_float" AS DOUBLE)), 'yyyy-MM-dd HH:mm:ss'), "$canonicalize_hive_timezone_id"('America/Los_Angeles')) AS TIMESTAMP(3)) AS VARCHAR(65535)) AS "ts_float"
FROM "default"."test_from_utc_timestamp_source" AS "test_from_utc_timestamp_source"
trino version 420
while using coral version 2.1.5
trino> SELECT * FROM hive.default.test_from_utc_timestamp_view;
ts_float
-------------------------
1970-01-30 16:00:00.000
trinoSql in ViewReaderUtil
SELECT CAST(CAST("at_timezone"("from_unixtime"(CAST("test_from_utc_timestamp_source"."source_float" AS DOUBLE)), "$canonicalize_hive_timezone_id"('America/Los_Angeles')) AS TIMESTAMP(3)) AS VARCHAR(65535)) AS "ts_float"
FROM "default"."test_from_utc_timestamp_source" AS "test_from_utc_timestamp_source"
2.1.5
CAST(CAST("at_timezone"("from_unixtime"...
2.2.9
CAST(CAST("at_timezone"("format_datetime"("from_unixtime"
format_datetime
shouldn't be there.
from coral.
FromUtcTimestampOperatorTransformer is playing with both:
- from_unixtime_nanos
- from_unixtime
from_unixtime_nanos
is a Trino specific function https://trino.io/docs/current/functions/datetime.html?highlight=from_unixtime_nanos#from_unixtime_nanos
Translation of from_unixtime_nanos works as expected.
from_unixtime
is:
- a Trino specific function https://trino.io/docs/current/functions/datetime.html?highlight=from_unixtime#from_unixtime returning timestamp(3) with time zone#
- a Hive specific function https://cwiki.apache.org/confluence/display/hive/languagemanual+udf returning string type
string
from_unixtime(bigint unixtime[, string pattern])
Converts a number of seconds since epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current time zone(using config "hive.local.time.zone") using the specified pattern. If the pattern is missing the default is used ('uuuu-MM-dd HH:mm:ss' or yyyy-MM-dd HH:mm:ss'). Example: from_unixtime(0)=1970-01-01 00:00:00 (hive.local.time.zone=Etc/GMT)
Given the fact that from_unixtime
is a Hive function, it gets its own FromUnixtimeOperatorTransformer
and that's where our problem actually occurs.
When we transform from_utc_timestamp
via FromUtcTimestampOperatorTransformer
we're creating (in case of dealing with floats) a nested from_unixtime
call which gets unintentionally transformed as well to "at_timezone"("from_unixtime" .
from coral.
Related Issues (20)
- Is there a packaged docker image available to try it out? HOT 1
- Avoid overriding `public Result visit(Values e)` by backporting fixes to linkedin-calcite
- Trino to Spark conversion is broken HOT 2
- Does not support hive function unix_timestamp()
- Questions about implicit conversion when hiveSQL is converted to trinoSQL HOT 1
- Does Trino to Spark only support conversion syntax differences? Does it not support conversion of function differences? HOT 2
- getException "java.lang.NoSuchFieldError:RULE_SET" HOT 1
- Extract_union doesnt return struct
- parse Hive sql : java.lang.RuntimeException: while converting
- Conversion failed when the table join contains the `in` operator
- Excuse me, does Coral support the conversion of Hive UDF? HOT 1
- Trino's substr has different start index
- Is it possible to do the conversion without metadata, just like sqlglot?
- Feature: Consider supporting flink sql HOT 22
- trino parser decimal type get wrong scala HOT 1
- Package coral-service/frontend HOT 1
- [Coral-Trino] cannot recognize input near 'select' 'max' '(' in expression specification
- "status":404,"error":"Not Found","message":"","path":"/" HOT 1
- connect hms(hive metastore store) sometimes breaks and does not reconnect
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coral.