Giter Club home page Giter Club logo

krontab's Introduction

krontab

Build Status Current Release

A crontab replacement for Kubernetes.

Create CronJob resources on your Kubernetes cluster in the same way you would on your *nix system. Krontab works by constructing a virtual crontab file from your CronJob resources and communicating changes back to the Kubernetes API. You can create more complex and customised jobs with custom templates and trigger your jobs manually any time from the command line.

Example crontab:

$ krontab -l
# template: default
0 1 * * * echo hello  # name: test

Installation

Quick install

curl -sL https://git.io/krontab | bash

Linux (x86 64-bit)

LATEST_VERSION=$(curl -sL -o /dev/null -w %{url_effective} "https://github.com/jacobtomlinson/krontab/releases/latest" | rev | cut -f1 -d'/'| rev)
curl -L https://github.com/jacobtomlinson/krontab/releases/download/${LATEST_VERSION}/krontab-linux-x86_64 -o /usr/local/bin/krontab
chmod +x /usr/local/bin/krontab

Linux (arm 32-bit)

LATEST_VERSION=$(curl -sL -o /dev/null -w %{url_effective} "https://github.com/jacobtomlinson/krontab/releases/latest" | rev | cut -f1 -d'/'| rev)
curl -L https://github.com/jacobtomlinson/krontab/releases/download/${LATEST_VERSION}/krontab-linux-arm -o /usr/local/bin/krontab
chmod +x /usr/local/bin/krontab

OS X (64-bit)

LATEST_VERSION=$(curl -sL -o /dev/null -w %{url_effective} "https://github.com/jacobtomlinson/krontab/releases/latest" | rev | cut -f1 -d'/'| rev)
curl -L https://github.com/jacobtomlinson/krontab/releases/download/${LATEST_VERSION}/krontab-darwin-x86_64 -o /usr/local/bin/krontab
chmod +x /usr/local/bin/krontab

Configuration

Authentication with your Kubernetes cluster will use your ~/.kube/config credentials or a service account if being used from inside a pod on the cluster. Advanced configuration options are specified with environment variables.

Env var Default Description
KRONTAB_NAMESPACE default The kubernetes namespace to use.
KRONTAB_OWNER N/A The owner of a cronjob. This will be used when creating jobs and will be used as a filter when listing them.

Usage

$ krontab help
Krontab is a crontab replacement for kubernetes.

You can use it to create cron jobs on a kubernetes cluster in a familiar crontab format.
Krontab works by allowing you to create job templates which are used in kubernetes. Then create
specific cron jobs using the crontab. Example krontab:

# Crontab example

# template: default
0 1 * * * echo hello  # name: test

Usage:
  krontab [flags]
  krontab [command]

Available Commands:
  create      Create a krontab resource
  delete      Delete a krontab resource
  edit        Edit a krontab resource
  get         Get krontab resources
  help        Help about any command
  list        List krontab resources
  run         Run a krontab job
  version     Print the version number of krontab

Flags:
  -e, --edit-crontab   Edit the crontab
  -h, --help           help for krontab
  -l, --list-crontab   List the crontab

Use "krontab [command] --help" for more information about a command.

Templates

Krontab uses templates to turn your crontab into Kubernetes CronJob resources. You will get a default template which you can view with krontab get template default. You can create, edit and delete your own templates too.

$ krontab get template default
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: test
spec:
  schedule: 0 */6 * * *
  jobTemplate:
    metadata:
      name: test
      krontabTemplate: default
    spec:
      template:
        metadata:
        spec:
          restartPolicy: Never
          containers:
            - name: test
              image: busybox
              command: ["echo", "hello"]
              resources:
                limits:
                  cpu: "1"
                  memory: 2G
                requests:
                  cpu: "0.25"
                  memory: 0.5G

Crontab

Krontab will automatically generate a crontab from your existing Kubernetes CronJob resources. You can edit them with krontab -e or krontab edit crontab. Adding new lines to your crontab will be mapped into new CrobJob resources. Editing lines will update them and deleteing lines will remove them.

$ krontab -e
# Welcome to krontab, a crontab like editor for Kubernetes cron jobs.
#
# This is a virtual file and was generated from the kubernetes API. Any edits you make here will be
# sent back to the kubernetes API to create/update/delete CronJob resources. Next time you open this crontab
# you may notice different formatting and comments to how you save it. Comments are meaningful and contain
# metadata about the job.
#
# Example job
# -----------
#
# # template: default
# 0 1 * * * echo hello world  # name: hello-world
#
# Templates
# ---------
#
# Krontab uses templates to turn your your crontab into kubernetes compliant CronJob resources. You will find
# a default template by running 'krontab get template default'. This is a minimal CrobJob resource with runs your
# command in a ubuntu container. When krontab creates your jobs it will replace the container command and schedule pattern
# in the template with the command and schedule pattern from the crontab.
#
# All cronjobs following a '# template: <name>' comment will be created using that template. If no template comments exist
# then the default template will be used.
#
# Names
# -----
# Kubernetes requires each job to have a name and is specified at the end of a job with a '# name: <name>' comment.
# If you do not specify a name one will be autogenerated.


# template: default
0 1 * * * echo hello world  # name: test

Examples

Create a new cron job

To create a new cron job file you need to open your krontab file for editing. This will automatically open in vim or whatever text edit you have configured with VISUAL or EDITOR.

krontab -e

Once this file is open you can create a new job in the same way you would when using regular crontab. You must specify your timings as a five column cron time (minute, hour, day of month, month of year, day of week) followed by your shell command. This command will be triggered every time the current time matches the schedule you have specified.

Let's create a job which echo's "hello` every hour.

0 * * * * echo hello

When creating a CronJob resource on the kubernetes cluster krontab will need to give it a name and choose a template to use. If you omit this information like we have in our echo hello example above krontab will use the default template (krontab get template default if you want to see it) and random uuid will be created for the name.

You can optionally choose to specify this information as yaml style comments in your crontab.

# template: default
0 * * * * echo hello  # name: hello-world

Here template has been specified at the document level which will affect all jobs below it until it is redefined. But the name info has been specified in line with the job and will only affect that job.

Edit a job

Once you have some jobs when you run crontab -e you will see a crontab file which contains your existing jobs. They may not look exactly like when you created them as this crontab file is dynamically generated on the fly. You can make changes to any of the information in the crontab and they will be reflected in ths CronJob resources on the kubernetes cluster.

Note: All changes except renaming will perform an update task on the cluster. Renaming will result in a deletion and creation of a new job.

Delete a job

You can delete a job by running crontab -e and removing the line you wish to delete. This will result in the CronJob resource being deleted from the cluster.

Test a job

You can manually take a job for a test drive any time, even if it is not going to be triggered any time soon.

krontab run job <job name>

Protip: You can run one shot jobs without having to create a cron job first. This is useful for submitting one off batch jobs.

krontab run job --template <template name> --command <the command>

Get running jobs

You can list any currently running jobs.

krontab list running

View a template

You can print out any template.

krontab get template <template name>

Edit a template

You can edit templates.

krontab edit template <template name>

Note: You cannot edit the default template. Also editing a template will not result in jobs being recreated with the modified template.

Deleting a template

You can delete templates.

krontab delete template <template name>

Note: You cannot delete the template.

Creating new templates

You can create new templates. These must be valid Kubernetes CronJob resources. The schedule, command and name sections will be overridden.

krontab create template <template name>

Contributing

Environment

This project requires Go to be installed. On OS X with Homebrew you can just run brew install go.

Running it then should be as simple as:

$ make
$ ./bin/krontab

Testing

$ make test

Releasing

Decide the new release version. Check out the current releases and follow SemVer to work out what it will be. Then run make release which will ask for the tag in the form v{major}.{minor}.{patch}. This will create the tag and push it to GitHub, Travis CI will then build the binaries and push them to github along with some autogenerated release notes.

$ make release

krontab's People

Contributors

jacobtomlinson avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

krontab's Issues

Bring flags in line with crontab command

One of the goals of krontab is to replace crontab for systems which live on a Kubernetes cluster (e.g pangeo/z2jh). Therefore ideally I would like it to have a superset of crontab's flags.

Looking at the crontab man page there are some flags which haven't been implemented yet.

Flags

  • -l List the crontab. This should list all kubernetes CronJob resources in a crontab format.
  • -e Edit the crontab. This should open all kubernetes CronJob resources in an editor (either system default or that configured with VISUAL/EDITOR) in a crontab format. Then changes should be propagated back to the Kubernetes API.
  • -u Modify the crontab of another user. This could be used to override the KRONTAB_OWNER config setting.
  • -r Remove the current crontab. This should delete all the CronJob resources, however this is a bit scary ⚠️. We should probably prompt the user to confirm.
  • -i Modifies the -r flag to prompt the user to confirm y/n. This should possibly be the default and use a -y to force without confirmation. Therefore maybe we should just stub this out.
  • -s Changes the SELinux context of the crontab file before editing. Probably not necessary in krontab but we should stub it out.

Cannot find in-cluster config

Krontab is looking for ~/.kube/config as it is assuming that we are not in-cluster.

$ krontab
panic: stat /home/jovyan/.kube/config: no such file or directory

goroutine 1 [running]:
github.com/jacobtomlinson/krontab/crontab.init.0()
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/crontab/crontab.go:358 +0x364

We should check for in-cluster config before checking for out-of-cluster config.

Job name collision

Currently job names can collide if two users (separated with the KRONTAB_OWNER setting) create a job with the same name.

The proposed fix is to append the owner name to job names in the CRUD operations. Then remove them again when listing and getting.

Handle passing command as a string

There are some caveats which need addressing around the krontab run job --command <command> usage.

Currently if you try and pipe output or use logical operators to string multiple commands together they get executed locally.

# Pipes the output of the krontab command into /tmp/test.log on the local machine
krontab run job --command echo hello > /tmp/test.log

# Runs echo hello in job and then uptime locally
krontab run job --command echo hello && uptime

One solution to this would be to allow you to wrap the command in quotes. However this results in problems.

krontab run job  --command 'echo hello > /tmp'
# Output of job is 'hello &gt; /tmp'

The string needs unwrapping to resolve this.

Getting templates from the system path doesn't work

Listing works, but getting and and therefore creating jobs that use them doesn't.

$ krontab get template pangeo
panic: open /home/jovyan/.config/krontab/krontab/templates/pangeo.yaml: no such file or directory

goroutine 1 [running]:
github.com/jacobtomlinson/krontab/template.GetTemplate(0x7fffaa2c53f3, 0x6, 0xc000161cc8, 0x598075, 0x1c52620, 0x59d4f0)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/template/template.go:150 +0x307
github.com/jacobtomlinson/krontab/cmd.glob..func5(0x1c52620, 0xc000283f50, 0x1, 0x1)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/cmd/get.go:25 +0x49
github.com/spf13/cobra.(*Command).execute(0x1c52620, 0xc000283f30, 0x1, 0x1, 0x1c52620, 0xc000283f30)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0x1c53460, 0x0, 0x1c53b80, 0xc000161f58)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0x1c53460, 0x407460, 0xc0000ae058)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:800 +0x2b
github.com/jacobtomlinson/krontab/cmd.Execute()
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/cmd/root.go:44 +0x2d
main.main()
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/main.go:8 +0x20

Change template format

Currently templates are whole Cron Job resources. This should be changed so that they are just the pod spec.

This makes the templates consistent with dask-kubernetes and will simplify batch jobs with #2.

Add list running command

It would be good to be able to list currently running jobs.

krontab list jobs

or

krontab list running

Useful with #2.

Validate templates

We should validate templates to ensure they can be used. This would avoid problems like #29.

Get version info from git

Currently the version number is hard coded as well as being set by git tags. We should remove the hard coding in favor of git tags and set it at build time in the same way we set build date.

Add run command

It could be useful for testing purposes to be able to manually run a job from the command line.

E.g

krontab run myjobname

And maybe even support running one off jobs.

krontab run --template default --command echo hello world

Move templates to ConfigMaps

Storing templates as files in a local config directory means that when changing computers can cause problems.

One solution could be to store templates in kubernetes config maps.

However this may be an edge case that we are not interested in.

Add get logs

Should be able to run krontab get logs <job name> to retrieve job logs.

Catch invalid job names

When trying to create a job with the name runningTest I get the following error.

$ krontab -e
panic: CronJob.batch "runningTest" is invalid: [metadata.name: Invalid value: "runningTest": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.jobTemplate.spec.template.spec.containers[0].name: Invalid value: "runningTest": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')]

goroutine 1 [running]:
github.com/jacobtomlinson/krontab/crontab.KronJob.Create(0xc0000b5c58, 0x7, 0xc0000b5ca0, 0xb, 0xc0000b5e60, 0xc, 0xc0000b5f10, 0xc, 0x0, 0x18)
        /Users/jacob/Projects/pangeo/krontab/crontab/crontab.go:78 +0xf1
github.com/jacobtomlinson/krontab/crontab.EditCrontab()
        /Users/jacob/Projects/pangeo/krontab/crontab/crontab.go:207 +0x5a1
github.com/jacobtomlinson/krontab/cmd.glob..func10(0x2849f20, 0xc00030ee80, 0x0, 0x1)
        /Users/jacob/Projects/pangeo/krontab/cmd/root.go:32 +0x59
github.com/spf13/cobra.(*Command).execute(0x2849f20, 0xc0000c2030, 0x1, 0x1, 0x2849f20, 0xc0000c2030)
        /Users/jacob/go/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0x2849f20, 0x0, 0x284a640, 0xc000147f58)
        /Users/jacob/go/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0x2849f20, 0x1007500, 0xc0000a0058)
        /Users/jacob/go/pkg/mod/github.com/spf13/[email protected]/command.go:800 +0x2b
github.com/jacobtomlinson/krontab/cmd.Execute()
        /Users/jacob/Projects/pangeo/krontab/cmd/root.go:44 +0x2d
main.main()
        /Users/jacob/Projects/pangeo/krontab/main.go:8 +0x20

Creating a template should read from stdin

A nice feature would be for template creation to read from stdin. Currently when you run krontab create template <name> a blank template file will be opened in a text editor.

It would be nice if we could pipe some text into that command to give us a starting point. For example we could effectively duplicate the default template by running

krontab get template default | krontab create template <new template>

This would open the new template for editing but the file would be already populated with the contents of the default template.

We could also include a flag to skip the interactive part which would allow a one line duplication. We should also think about what happens if the user exits from the test editor without saving, should it create the new template or not?

Reduce binary size

Binaries seem to be around 30MB. There must be some ways to reduce this because it is not a complex application.

Using system templates seems to be broken

$ krontab run job --template pangeo --command 'echo hello'
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/jacobtomlinson/krontab/crontab.KronJob.Construct(0x7fffb3d173c9, 0x6, 0x11d2862, 0x8, 0x11d2c0b, 0x9, 0x7fffb3d173da, 0xa, 0x1256200, 0x9, ...)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/crontab/crontab.go:96 +0x5ba
github.com/jacobtomlinson/krontab/crontab.KronJob.Run(0x7fffb3d173c9, 0x6, 0x11d2862, 0x8, 0x11d2c0b, 0x9, 0x7fffb3d173da, 0xa, 0x3, 0x3, ...)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/crontab/crontab.go:115 +0x67
github.com/jacobtomlinson/krontab/crontab.RunJob(0xc00027cb80, 0x1, 0x4, 0x7fffb3d173c9, 0x6, 0xc000159cb0, 0xc000159c80, 0x0, 0x0)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/crontab/crontab.go:417 +0x13a
github.com/jacobtomlinson/krontab/cmd.glob..func11(0x1c423a0, 0xc00027cb80, 0x1, 0x4)
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/cmd/run.go:27 +0xb0
github.com/spf13/cobra.(*Command).execute(0x1c423a0, 0xc00027cb40, 0x4, 0x4, 0x1c423a0, 0xc00027cb40)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x2cc
github.com/spf13/cobra.(*Command).ExecuteC(0x1c41ee0, 0x0, 0x1c42600, 0xc000159f58)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2fd
github.com/spf13/cobra.(*Command).Execute(0x1c41ee0, 0x4056b0, 0xc0000ac058)
        /home/travis/gopath/pkg/mod/github.com/spf13/[email protected]/command.go:800 +0x2b
github.com/jacobtomlinson/krontab/cmd.Execute()
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/cmd/root.go:44 +0x2d
main.main()
        /home/travis/gopath/src/github.com/jacobtomlinson/krontab/main.go:8 +0x20

Add system location from templates

Currently templates are always stored in the users app data directory. A sysadmin may want to provide some system level templates so we should include the system path too.

  • List should list all templates
  • Get should get any template
  • Edit should error nicely when trying to edit a system template without perms (should still work with sudo)
  • Delete should error nicely when trying to delete a system template without perms (should still work with sudo)

Allow overriding images

We could make it possible to override images in the crontab too. Currently if you want to use a different image you need to create a new template. However the rest of the template may be totally fine, you just want a different image.

How would this work?

  • It could be another document level yaml comment like the template. Should it affect every command below? Should it just affect the next command? Should it be consistent with the template specification/
  • It could be another in line yaml comment. How do you add multiple keys in one line? Use JSON notation instead of the yaml notation? 0 1 * * * echo hello # {"name": "test", "image": "myimage"}
  • Should image and tag be specified as two keys or a combined string?

Add namespace flag

Currently the namespace is automatically picked up from the config or service account. We should add a --namespace/-n flag to allow it to be overridden from the command line.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.