Giter Club home page Giter Club logo

jobschedulers.jl's Introduction

JobSchedulers.jl

A Julia-based job scheduler and workload manager inspired by Slurm/PBS and Crontab.

Documentation

Why JobScheduler?

We may find different tasks or programs use different CPU and memory. Some can run simultaneously, but some have to run sequentially. JobScheduler is stable, useful and powerful for task queuing and workload management, inspired by Slurm/PBS and Crontab.

Package Features

  • Job and task scheduler.

  • Local workload manager.

  • Support CPU, memory, run time management.

  • Support running a job at specific time, or a period after creating (schedule).

  • Support recurring/repetitive jobs using Cron-like schedule expressions.

  • Support deferring a job until specific jobs reach specific states (dependency).

  • Support automatic backup and reload.

  • Fancy progress meter in terminal.

    progress meter

Installation

JobSchedulers.jl can be installed using the Julia package manager. From the Julia REPL, type ] to enter the Pkg REPL mode and run

pkg> add JobSchedulers

To use the package, type

using JobSchedulers

Documentation

  • STABLE โ€” documentation of the most recently tagged version.
  • DEVEL โ€” documentation of the in-development version.

jobschedulers.jl's People

Contributors

cihga39871 avatar femtotrader avatar jclugstor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

jobschedulers.jl's Issues

Optional removal of self-dependence for recurring jobs

I have a recurring job whose function can sometimes take longer to finish than the actual period of recurrence. The subsequent call of function would be independent of the previous one, however due to the implicit dependence, it's not started until the previous one is finished. Would it be possible to add some option that removes this dependence?

As an MWE, this is what I have now (the effective function call period becomes 6 seconds):

julia> using JobSchedulers
julia> function f()  # function runs within 4 seconds
           sleep(4)
           println(Dates.format(now(), "HH:MM:SS"))
       end;
julia> submit!(f, cron=Cron("*/2", "*", "*", "*", "*", "*"));  # function call at every 2 second, starting at 15:19:16
15:19:20
15:19:26
15:19:32
15:19:38
...

What would be great to have is something like this (the function call is 2 seconds as specified by cron):

 julia> submit!(f, cron=Cron("*/2", "*", "*", "*", "*", "*"), remove_self_dependence=true);  # function call at every 2 second, starting at 15:19:16
15:19:20
15:19:22
15:19:24
15:19:26
...

progress meter

  1. Only show progress meter when stdout is a TTY. If stdout is not a TTY, only show the latest result with plain text.
  2. Add ESC to stop progress meter when isinteractive().

Recurring job every 5 minutes don't have 5 minutes interval between 1st run and 2nd run

Hello,

The following code

using JobSchedulers

recurring_job = Job(cron = Cron(0, "*/5", *, *, *, *)) do
    println("function job done at $(now())")
end

submit!(recurring_job)

while true
    sleep(0.01)
end

run like this

>julia sched.jl
function job done at 2023-07-13T10:09:32.344
function job done at 2023-07-13T10:10:00.029
function job done at 2023-07-13T10:15:00.020
function job done at 2023-07-13T10:20:00.056

Is there a way to avoid first run ie ensure that first run is done at second=0 and minute as multiple of 5 ?

Kind regards

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Can I see how much memory a Job actually took?

Not sure if this is a feature request or a question: I'm not sure how much memory my Jobs take relative to the inputs they receive. I'd like to update the estimate I provide to the mem keyword based on the amount of memory the last jobs took. Is the high-water mark of memory consumed by a given job recorded by the scheduler? How could I access it?

If it isn't recorded, would it be feasible to add that feature?

To do v0.8

  1. dependency = DONE => job_A: to be simplified to dependency = job_A or dependency = job_A.id

  2. submit!(Job(...)): to be simplified to submit!(...)

  3. Set a lower loop interval.

  4. Move scheduler_start() in __init__.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.