dgruber / drmaa2os Goto Github PK

The JobTracker interface is the glue between the backend implementation and the
generic Go DRMAA2 implementation.

One thing missing is returning a more detailed explanation of the current state. This is
called substate in DRMAA2. Like when a JobFailed it would we valuable to access
the reason programmatically.

Hence the JobState signature needs to be changed from:

JobState(jobid string) drmaa2interface.JobState

to

JobState(jobid string) (drmaa2interface.JobState, string, error)

The string is the substate as specified in the standard and the error indicates an internal
error when trying to get the JobState. Usage of the job substate is optional.

As drmaa2os is primarily oriented towards job scheduling and control, I believe, is there a similar project or approach (ore even an alternate usage model for drmaa2os) that could apply to working with accounting data of the sort produced by such schedulers? UGE has a REST API that can be used, for example; is there an equivalent in the other back-end schedulars that can be exposed through a common API?

I'm also intrigued by Gloo and the growing family of tools from solo.io that appear to have features that could be used for API gateways and aggregation, creation of service meshes, etc. Any thoughts along these lines that could supplement or replace the approach of drmaa2os?

When using OS processes with the simpletracker implementation, job related information is made persistent in the DB.
After closing a JobSession the DB should be re-usable by a new JobSession to see past jobs stored in the DB.
Currently the JobSession.Close() method does not call the DB close call.

Docker does not support relative paths but the paths could be resolved in this library. That would make applications more portable.

https://criu.org/Main_Page

Live migration project in go:

https://criu.org/P.Haul

https://github.com/checkpoint-restore/go-criu

While data management is not in focus of DRMAA2 we have stagein and stageout maps in the job template.
Typically data between two jobs should be passed by accessing external storage (buckets, shared filesystem).
Nevertheless it would be helpful on Kubernetes to copy smaller junks of data between jobs through JobTemplate instructions.

If I start a job then my script dies, is there any way to reinstantiate the previously created session manager (as it appears to be serialized in the sqlite database?), or if I must create a new session manager, can I create a new job object with the ID of a running job started through DRMAA so I can call methods like GetState()?

This project looks interesting! Is Slurm included in the supported platforms?