allwefantasy / mlsql-api-console Goto Github PK

View Code? Open in Web Editor NEW

22.0 4.0 25.0 78.87 MB

Java 17.21% Scala 48.35% HTML 3.02% Dockerfile 0.25% Shell 2.29% JavaScript 11.51% CSS 17.36%

mlsql-api-console's Introduction

MLSQL Console

MLSQL Console 主要展示了MLSQL 的能力，对接方式。

本地开发环境设置

软件要求：

JDK 8+
Maven 3.3+
Git
MySQL 5.7

推荐Idea Intellj 使用。

配置文件配置：

将 config/application.docker.yml 改成 config/application.yml.

将如下内容修改为你自己的数据库连接信息：

#mode
mode:
  development
#mode=production

###############datasource config##################
#mysql,mongodb,redis等数据源配置方式
development:
  datasources:
    mysql:
      host: MYSQL_HOST
      port: 3306
      database: mlsql_console
      username: xxxxx
      password: xxxxx
      disable: false
      initialSize: 3
      removeAbandoned: true
      testWhileIdle: true
      removeAbandonedTimeout: 30
      filters: stat,log4j
      maxWait: 100

启动类：

tech.mlsql.MLSQLConsole

右键启动即可，默认端口为 9002.

SQL脚本(找最新的就好)：

mlsql_console_2020-12-24.sql

安装部署

参考官方文档MLSQL Console安装

mlsql-api-console's People

Contributors

Stargazers

Watchers

mlsql-api-console's Issues

Console Notebook support add/remove cell

Related Project Address: https://github.com/allwefantasy/mlsql-web-console

Current implementation lack of Add/Remove cell.

Running with shaded jar throws NPE

Steps to re-produce:

Pull latest code from main branch
Build with shade plugin

mvn package -DskipTests -Pshade

Start with shaded jar
Login API-console and click "scripts from the left panel
NPE thrown

java.lang.NullPointerException
	at net.csdn.jpa.JPA.modifyPersistenceXml(JPA.java:116)
	at net.csdn.jpa.JPA.getJPAConfig(JPA.java:96)
	at net.csdn.modules.http.processor.impl.DefaultHttpFinishProcessor.closeTx(DefaultHttpFinishProcessor.java:48)
	at net.csdn.modules.http.processor.impl.DefaultHttpFinishProcessor.process(DefaultHttpFinishProcessor.java:31)
	at net.csdn.modules.http.HttpServer$DefaultHandler.handle(HttpServer.java:201)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:748)
[2021-07-22 07:13:15,671][INFO ][org.hibernate.jpa.boot.internal.PersistenceXmlParser] HHH000318: Could not find any META-INF/persistence.xml file in the classpath
[2021-07-22 07:13:15,821][INFO ][tech.mlsql.service.RunScript] proxy response time:13042

If you start API-console from IDEA, everything is fine, no NPE. Shading could be the root cause.

[WARNING] javax.persistence-api-2.2.jar, hibernate-jpa-2.1-api-1.0.2.Final.jar define 198 overlapping classes: 
[WARNING]   - javax.persistence.Convert
[WARNING]   - javax.persistence.EntityManager
[WARNING]   - javax.persistence.metamodel.SingularAttribute
[WARNING]   - javax.persistence.StoredProcedureQuery
[WARNING]   - javax.persistence.AccessType
[WARNING]   - javax.persistence.metamodel.IdentifiableType
[WARNING]   - javax.persistence.EmbeddedId
[WARNING]   - javax.persistence.Lob
[WARNING]   - javax.persistence.criteria.CompoundSelection
[WARNING]   - javax.persistence.MapKeyColumn
[WARNING]   - 188 more...

bug in username‘s validation

[email protected]
ng

[email protected]
ok

It seems that the length of company’s name in validation of username which is a email address is not collect.

Start failed with exception ClassNotFoundException

Either debuging MLSQLConsole.scala or running start-default.sh throws exception.

Exception

Caused by: java.lang.ClassNotFoundException: tech.mlsql.common.utils.reflect.ClassPath
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)

Root cause analysis

Recently, mlsql-api-console upgrades ServiceFramework from 2.0.6 to 2.0.8, which upgrades tech.mlsql.common-utils from 0.3.4 to 0.3.9.2. The 0.3.9.2 version contains tech.mlsql.common.utils.reflect.ClassPath
However, mlsql-api-console still uses tech.mlsql.common-utils 0.3.4, and 0.3.4 does not contain tech.mlsql.common.utils.reflect.ClassPath.

Proposed fix:

Bump tech.mlsql.common-utils version to 0.3.9.2.

[2021-07-15 16:30:51,704][ERROR][modules.http             ] System processing error
java.lang.NullPointerException
	at scala.collection.immutable.StringOps$.slice$extension(StringOps.scala:40)
	at scala.collection.immutable.StringOps.slice(StringOps.scala:29)
	at scala.collection.IndexedSeqOptimized$class.take(IndexedSeqOptimized.scala:138)
	at scala.collection.immutable.StringOps.take(StringOps.scala:29)
	at tech.mlsql.quill_model.MlsqlJob.render(quill_model.scala:173)
	at tech.mlsql.api.controller.JobController$$anonfun$jobList$1.apply(JobController.scala:42)
	at tech.mlsql.api.controller.JobController$$anonfun$jobList$1.apply(JobController.scala:42)
	at scala.collection.immutable.List.map(List.scala:277)
	at tech.mlsql.api.controller.JobController.jobList(JobController.scala:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.csdn.modules.http.RestController.filter(RestController.java:139)
	at net.csdn.modules.http.RestController.dispatchRequest(RestController.java:99)
	at net.csdn.modules.http.HttpServer$DefaultHandler.handle(HttpServer.java:182)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:499)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:748)

同学，您这个项目引入了122个开源组件，存在76个漏洞，辛苦升级一下

检测到 allwefantasy/mlsql-api-console 一共引入了122个开源组件，存在76个漏洞

漏洞标题：Oracle MySQL 输入验证错误漏洞
缺陷组件：mysql:[email protected]
漏洞编号：CVE-2021-2471
漏洞描述：Oracle MySQL是美国甲骨文（Oracle）公司的一套开源的关系数据库管理系统。
Oracle MySQL 的 MySQL Connectors 产品中存在输入验证错误漏洞，该漏洞允许高特权攻击者通过多种协议访问网络来破坏 MySQL 连接器。成功攻击此漏洞会导致对关键数据的未授权访问或对所有 MySQL 连接器可访问数据的完全访问，以及导致 MySQL 连接器挂起或频繁重复崩溃。
影响范围：(∞, 8.0.27)
最小修复版本：8.0.27
缺陷组件引入路径：allwefantasy:[email protected]>tech.mlsql:[email protected]>mysql:[email protected]

另外还有76个漏洞，详细报告：https://mofeisec.com/jr?p=iedd7b

Expanded json contains incrorect value

Description

Running the following code snippet to expand json {"key": "value", "key_2":"value_2"}.

SELECT '{"key": "value", "key_2":"value_2"}' AS col_1
, "hangzhou" AS city
AS table_1;
run table_1 as JsonExpandExt.`` where inputCol="col_1" AND samplingRatio="1.0" as result;

The result should be:

key	key_2	city
value	value_2	hangzhou

Actual result is:

key	key_2	city
0	value_2	hangzhou

Note that value becomes 0 in key column.