Comments (2)
But I've requirement of converting pure (LogicalPlan -> String) to re-generate sql.
Could you please clarify what kind of logical plan representation this refers to? If you are referring to Spark logical plan string representation, we have a WIP module that converts Spark's logical plan to a RelNode representation. You can see some test cases here. This is an example where HiveQL is not the input format to Coral. Actually, in this example the metadata source is not necessarily the HMS either.
In general, the way the input logic is expressed can be decoupled from the metadata store/engine. In the current implementation there are two options for sources of metadata: the Hive Metastore, and a local file on the local file system. Other metadata sources can be added as well. All metadata sources, including HMS, can be used with HiveQL or other input languages.
from coral.
Sorry for replying so late.
I think you got my point correctly.
If my requirement is to just convert a spark's logical plan object into any kind of output sql (for example take presto / trino) then HMS or any kind of metastore is redundant.
Spark's logical plan can be analyzed and further schema can be resolved for upstreams.
The tests that you have mentioned still requires HMS (after 3 months).
So wanted to know, can we independently convert spark logical plans into supported output sqls using coral? Or is it like we will always require a metadata store/engine?
from coral.
Related Issues (20)
- Avoid overriding `public Result visit(Values e)` by backporting fixes to linkedin-calcite
- Trino to Spark conversion is broken HOT 2
- Does not support hive function unix_timestamp()
- Questions about implicit conversion when hiveSQL is converted to trinoSQL HOT 1
- Does Trino to Spark only support conversion syntax differences? Does it not support conversion of function differences? HOT 2
- getException "java.lang.NoSuchFieldError:RULE_SET" HOT 1
- Extract_union doesnt return struct
- parse Hive sql : java.lang.RuntimeException: while converting
- Conversion failed when the table join contains the `in` operator
- Excuse me, does Coral support the conversion of Hive UDF? HOT 1
- Trino's substr has different start index
- Is it possible to do the conversion without metadata, just like sqlglot?
- Feature: Consider supporting flink sql HOT 22
- trino parser decimal type get wrong scala HOT 1
- Bug: `FromUnixtimeOperatorTransformer` possibly built on false supposisions. HOT 2
- Package coral-service/frontend HOT 1
- [Coral-Trino] cannot recognize input near 'select' 'max' '(' in expression specification
- "status":404,"error":"Not Found","message":"","path":"/" HOT 1
- connect hms(hive metastore store) sometimes breaks and does not reconnect
- Are there plans to support flink sql? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coral.