Giter Club home page Giter Club logo

xmapst / autoexecflow Goto Github PK

View Code? Open in Web Editor NEW
0.0 2.0 0.0 44.94 MB

An API for cross-platform custom orchestration of execution steps without any third-party dependencies. Based on DAG, it implements the scheduling function of sequential execution of dependent steps and concurrent execution of non-dependent steps.

License: GNU General Public License v3.0

Makefile 0.43% Go 97.13% CSS 1.41% HTML 0.41% JavaScript 0.63%
batch-script cicd dag executor golang powershell shell

autoexecflow's Introduction

AutoExecFlow

Go

An API for cross-platform custom orchestration of execution steps without any third-party dependencies. Based on DAG , it implements the scheduling function of sequential execution of dependent steps and concurrent execution of non-dependent steps.

It provides API remote operation mode, batch execution of Shell , Powershell , Python and other commands, and easily completes common management tasks such as running automated operation and maintenance scripts, polling processes, installing or uninstalling software, updating applications, and installing patches.

Operating system remote execution interface

Feature

  • support Windows / Linux / Mac
  • Dynamically adjust the number of workers
  • Orchestrating execution based on directed acyclic graph ( DAG )
  • Supports forced termination of tasks or steps
  • Supports suspension and resumption of tasks or steps
  • Support timeout for tasks or steps
  • Task-level Workspace isolation
  • Browse, upload, and download tasks in Workspace
  • Self-update, use parameter --self_url
  • WebShell
  • Support delayed Task
  • Send events before/after a task or step is executed
  • Task or step plugin implementation

Help

Usage:
  AutoExecFlow_linux_amd64_v1 [command]

Available Commands:
  client      a self-sufficient executor
  help        Help about any command
  server      start server

Flags:
      --help      Print usage
  -v, --version   Print version information and quit

Use "AutoExecFlow_linux_amd64_v1 [command] --help" for more information about a command.

How to use

Windows

Open PowerShell in management mode to add services

New-Service -Name AutoExecFlow -BinaryPathName "C:\AutoExecFlow\bin\AutoExecFlow_windows_amd64_v1.exe server" -DisplayName  "AutoExecFlow " -StartupType Automatic
sc.exe failure AutoExecFlow reset= 0 actions= restart/0/restart/0/restart/0
sc.exe start AutoExecFlow

Linux

echo > /etc/systemd/system/AutoExecFlow.service <<EOF
[Unit]
Description=Operating system remote execution interface
Documentation=https://github.com/busybox-org/AutoExecFlow.git
After=network.target nss-lookup.target

[Service]
NoNewPrivileges=true
ExecStart=/usr/local/AutoExecFlow/bin/AutoExecFlow_linux_amd64_v1 server
Restart=on-failure
RestartSec=10s
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now AutoExecFlow.service

Local compilation (Linux)

  • Depends on the Docker environment
git clone https://github.com/xmapst/AutoExecFlow.git
cd AutoExecFlow
make

Request Example

Create a task

# URL parameter support
name: "Customize task name"
timeout: "Task timeout"
env: "Task global environment variable injection"
async: "Concurrent execution or custom orchestration"

# example:
# http://localhost:2376/api/v1/task?name=test&timeout=10m&env=TEST_SITE=www.google.com&async=true

# By default, the execution is in order.
curl -X POST -H "Content-Type:application/json" -d '[
  {
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "env", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.google.com"
    }
  },
  {
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "curl ${TEST_SITE}", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.baidu.com"
    }
  }
]' 'http://localhost:2376/api/v1/task' 

# Concurrent Execution
curl -X POST -H "Content-Type:application/json" -d '[
  {
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "env", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.google.com"
    }
  },
  {
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "curl ${TEST_SITE}", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.baidu.com"
    }
  }
]' 'http://localhost:2376/api/v1/task?async=true'

# Customized orchestration execution
curl -X POST -H "Content-Type:application/json" -d '[
  {
    "name": "step0",
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "env", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.google.com"
    }
  },
  {
    "name": "step1",
    "type": "bash", # support[python2,python3,bash,sh,cmd,powershell]
    "content": "curl ${TEST_SITE}", # Script content
    "env": { # Environment variable injection
      "TEST_SITE": "www.baidu.com"
    },
    "depends": [
      "step1"
    ]
  }
]' 'http://localhost:2376/api/v1/task?async=true'

Get the task list

curl -X GET -H "Content-Type:application/json" 'http://localhost:2376/api/v1/task'

Get task details

curl -X GET -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}

Get the task working directory

curl -X GET -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/workspace

Task Control

# Task to force kill
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}?action=kill

# Pause task execution [Only pending tasks can be paused]
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}?action=pause

# Pause task execution (pause for 5 minutes) [Only tasks to be run can be paused]
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}?action=pause&duration=5m

# Continue the task
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}?action=resume

Get step console output

curl -X GET -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/step/{step name}

Step Control

# Steps to force kill
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/step/{step name}?action=kill

# Pause step execution [Only pending steps can be paused]
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/step/{step name}?action=pause

# Pause step execution (pause for 5 minutes) [Only steps to be run can be paused]
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/step/{step name}?action=pause&duration=5m

# Continue to step
curl -X PUT -H "Content-Type:application/json" http://localhost:2376/api/v1/task/{task name}/step/{step name}?action=resume

[Notes]

  • code:
    • 0: success
    • 1001: running
    • 1002: failed
    • 1003: not found
    • 1004: pending
    • 1005: paused

autoexecflow's People

Contributors

xmapst avatar dependabot[bot] avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.