Giter Club home page Giter Club logo

rjm's Introduction

Lightweight Remote Job Management

Manage computational jobs on a cluster from your desktop computer.

This toolkit contains command-line tools to submit batches of jobs, cancel batches of jobs, upload files, download files, clean up job directories. All commands are issued over a single SSH connection to achieve robustness and scalability.

Born out of frustration with notoriously unreliable and non-scaling Grid Middleware stacks.

Documentation: https://wiki.auckland.ac.nz/display/CERES/Lightweight+Remote+Job+Management

Note: This is not a general-purpose tool, but tailored for researchers using the Auckland NeSI cluster. It's also no longer supported. The batch scheduling system in use is Slurm.

rjm's People

Contributors

mondkaefer avatar

Stargazers

Blair Bethwaite avatar

Watchers

avatar James Cloos avatar

rjm's Issues

rjm_batch_submit: create job configuration file after remote job directory creation

Create the job configuration file for each job submitted in the batch after the job directory has been created, and not only after the job has been submitted.
It'll contain only the remote job directory then, initially, but will be augmented with the job id after the submission was successful.

This change will enable clean up of the job directory even if upload/submission fails.

prepare_job: add support for walltime soft limit

If a soft limit is specified, LoadLeveler will send SIGXCPU signal if the soft limit is exceeded. This will enable executable to catch the signal and print message.

Without a soft limit, jobs are killed the hard way, and may not print anything to any file. This makes it hard to understand what happened to the job.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.