Comments (10)
定义了用户自定义函数udf类,并且通过addjar参数指定了路径。是否可以在sql里直接使用?源代码中没有对自定义的函数做注册和声明,可以直接使用吗?
from flinkstreamsql.
用户自定义额函数CREATE (scala|table) FUNCTION CHARACTER_LENGTH WITH com.dtstack.Kun
在sql里面要加上这个sql,就能用了
from flinkstreamsql.
这里的com.dtstack.Kun ,就是自己写的自定义函数类的绝对路径?
from flinkstreamsql.
是的
from flinkstreamsql.
在测试udf过程中出现如下现象:
1.初次通过addjar提交了自定义函数jar包,测试使用udf正常。再次提交新的jar包和新的udf函数,报错提示函数未注册。
2.原先第一次提交的jar包中的udf,在之后启动任务时,不用指定addjar路径参数,也可正常使用。
3.原先第一次提交的jar包中的udf,重新写入新的逻辑,指定addjar参数,提交任务,可以未报错,但是输出的是原先第一次的udf逻辑,没有使用新逻辑处理。也就是新处理逻辑未生效。
请问,提交的addjar文件时flink缓存在了某处吗?在哪里呢?
from flinkstreamsql.
你是用什么模式执行的
from flinkstreamsql.
flink是1.5.4 standalone模式
sh ./submit.sh -sql /home/jinzhiliang/test/flink/sqlTXT/sideSql.txt -name xctest -remoteSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -localSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -addjar ["/home/jinzhiliang/test/flink/addjar/flinkUDFtest-1.0-SNAPSHOT.jar"] -mode standalone -yarnconf /home/jinzhiliang/test/flink/yarnconf -flinkconf /home/dl/projects/flink-1.5.4/conf -confProp {"time.characteristic":"EventTime","sql.checkpoint.interval":10000}
from flinkstreamsql.
我重启了flink1.5.4,不指定addjar,提交任务,原先第一次提交的udf函数任然可以使用。
from flinkstreamsql.
你们测试有预计这个问题吗?
from flinkstreamsql.
问题找到了,我第一次根本就没有读入自定义函数。sql里面写的有问题。现在改了一下,但是在加载自定义函数包的时候报类记载器错误:
Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at com.dtstack.flink.sql.launcher.LauncherMain.main(LauncherMain.java:93)
Caused by: java.lang.NoSuchMethodException: sun.misc.Launcher$AppClassLoader.addURL(java.net.URL)
at java.lang.Class.getDeclaredMethod(Class.java:2130)
at com.dtstack.flink.sql.util.FlinkUtil.urlClassLoaderAddUrl(FlinkUtil.java:271)
at com.dtstack.flink.sql.util.FlinkUtil.loadExtraJar(FlinkUtil.java:262)
at com.dtstack.flink.sql.Main.registerUDF(Main.java:246)
at com.dtstack.flink.sql.Main.main(Main.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
from flinkstreamsql.
Related Issues (20)
- flinkStreamSQL 这个项目是否支持 华为云版本的fi 的flink sql ? 有人试过吗? HOT 1
- .RuntimeException: create table statement requires property of type 报错了?
- Unknown database 'flinktest' ,明明是有这个 库, 为何最好老是说不存在? HOT 1
- 我想放到 centos7上运行 , 发现没有plugin这个组件,是要自己编译吗? HOT 1
- flinkStreamSql更新
- 我win10本地运行 ,读kafka写kafka ,咋就没有这个呢? com.dtstack.flink.sql.sink.kafka.table.KafkaSinkParser HOT 2
- flinkStreamSQL 这个项目,有没有好心人做了web系统,供大家方便使用呀? HOT 1
- kafka source,原生的flink可以读取解析到kafka的元数据比如timestamp。FlinkStreamSql支持读取元数据吗?应该如何操作? HOT 1
- 【2021-04-11】FlinkStreamSQL 更新记录
- Create table 中Array<String> 类型提交报错 HOT 2
- 1.10release版本编译不通过Could not find artifact com.dtstack.flink:sql.core:jar:tests:1.0-SNAPSHOT HOT 1
- 今天发现维表只能支持12个,我勒个去
- flink
- flinkstreamsql kafka任务,开启checkpoint,如果我手动杀死任务,如何从指定的最后一次保存的checkpoint启动任务 HOT 3
- 有没有支持web ui啊? HOT 1
- 有无对sql 语法校验的方法 HOT 1
- 新版FlinkStreamSQL已经融合进FlinkX项目
- kudu sink writemode参数没填写报NULLPointerException HOT 1
- Yarn模式下的sql是在什么地方解析的
- 这个如何引入使用flink-connector-hive啊 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flinkstreamsql.