Giter Club home page Giter Club logo

Comments (10)

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

定义了用户自定义函数udf类,并且通过addjar参数指定了路径。是否可以在sql里直接使用?源代码中没有对自定义的函数做注册和声明,可以直接使用吗?

from flinkstreamsql.

yangsishu avatar yangsishu commented on May 18, 2024

用户自定义额函数CREATE (scala|table) FUNCTION CHARACTER_LENGTH WITH com.dtstack.Kun
在sql里面要加上这个sql,就能用了

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

这里的com.dtstack.Kun ,就是自己写的自定义函数类的绝对路径?

from flinkstreamsql.

zoudaokoulife avatar zoudaokoulife commented on May 18, 2024

是的

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

在测试udf过程中出现如下现象:
1.初次通过addjar提交了自定义函数jar包,测试使用udf正常。再次提交新的jar包和新的udf函数,报错提示函数未注册。
2.原先第一次提交的jar包中的udf,在之后启动任务时,不用指定addjar路径参数,也可正常使用。
3.原先第一次提交的jar包中的udf,重新写入新的逻辑,指定addjar参数,提交任务,可以未报错,但是输出的是原先第一次的udf逻辑,没有使用新逻辑处理。也就是新处理逻辑未生效。
请问,提交的addjar文件时flink缓存在了某处吗?在哪里呢?

from flinkstreamsql.

yangsishu avatar yangsishu commented on May 18, 2024

你是用什么模式执行的

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

flink是1.5.4 standalone模式
sh ./submit.sh -sql /home/jinzhiliang/test/flink/sqlTXT/sideSql.txt -name xctest -remoteSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -localSqlPluginPath /home/dl/projects/flink-1.5.4/plugins -addjar ["/home/jinzhiliang/test/flink/addjar/flinkUDFtest-1.0-SNAPSHOT.jar"] -mode standalone -yarnconf /home/jinzhiliang/test/flink/yarnconf -flinkconf /home/dl/projects/flink-1.5.4/conf -confProp {"time.characteristic":"EventTime","sql.checkpoint.interval":10000}

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

我重启了flink1.5.4,不指定addjar,提交任务,原先第一次提交的udf函数任然可以使用。

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

你们测试有预计这个问题吗?

from flinkstreamsql.

nuaabuaa07 avatar nuaabuaa07 commented on May 18, 2024

问题找到了,我第一次根本就没有读入自定义函数。sql里面写的有问题。现在改了一下,但是在加载自定义函数包的时候报类记载器错误:
Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)
at com.dtstack.flink.sql.launcher.LauncherMain.main(LauncherMain.java:93)
Caused by: java.lang.NoSuchMethodException: sun.misc.Launcher$AppClassLoader.addURL(java.net.URL)
at java.lang.Class.getDeclaredMethod(Class.java:2130)
at com.dtstack.flink.sql.util.FlinkUtil.urlClassLoaderAddUrl(FlinkUtil.java:271)
at com.dtstack.flink.sql.util.FlinkUtil.loadExtraJar(FlinkUtil.java:262)
at com.dtstack.flink.sql.Main.registerUDF(Main.java:246)
at com.dtstack.flink.sql.Main.main(Main.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

from flinkstreamsql.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.