Giter Club home page Giter Club logo

springboot-datax's Introduction

Java springboot

preparation

  • Language: Java 8
  • Environment: MacOS, 16G RAM
  • Database: Mysql5.7
  • 建议Python2.7

todo list

  • springboot重构项目
  • 集成swagger，方便调试
  • 集成mybatis plus和Mysql数据库存放应用数据
  • 网页端修改并持久化job配置的json到数据库
  • 网页端实时查看抽取日志，类似Jenkins的日志控制台输出功能
  • 实时查看抽取日志BUG功能修复2019-11-07
  • 网页端各种读写插件模板生成，可以在页面组装使用
  • 实现datax分布式作业
  • 实现部分写插件支持自动建表功能

前端项目

github地址

how to run

1. 下载datax打包之后的文件或者github拉取datax代码打包，配置环境变量

 DATAX_HOME=G:\learndemo\springboot-datax\datax\bin

2. 执行datax-web/db下面的sql文件并修改application.yml数据库配置信息

3. application.yml配置数据抽取日志文件保存路径

etlLogDir: D:\temp\logs\datax-web\

4. 终端访问测试作业接口

curl http://localhost:8080/startJob

可以看到成功跑完一个datax作业

5. 打开网页端启动作业

http://localhost:8080/index.html#/datax/job

6. 在线查看作业日志

springboot-datax's People

Contributors

zhouhongfa avatar kyofin avatar binaryworld avatar wuchase avatar weiye-jing avatar sufism avatar lw309637554 avatar

Watchers

James Cloos avatar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.