As a developer of a PBS hook, I would like to obtain user-defined options/parameters from qsub about openpbs HOT 6 CLOSED

The parameters passed to qsub (the options including -v) are what get passed as input to the queuejob hook. The hook can then modify those passed values before allowing (or disallowing) the actual submit.

from openpbs.

Another option that I've used when constructing hooks is to specify custom string resources for specialized parameters and then handle parsing/processing in the QUEUEJOB and MODIFYJOB hooks as appropriate this allows users to pass them in via the -l flag to qsub, and they can then be accessed from the hook event().job.Resource_List or the job's select statement if the resource was a per-vnode resource and was defined as a part of a chunk.

The addition of a custom string resource for hook use can be scripted and passed into qmgr, for deployment purposes, along with defaults that may need to be set.

This information should then be accessible from other hooks later, including the execjob hooks that run on the MoM (a server query may be needed depending on the type of resource from some of the execjob_* hooks).

And as bayucan said above, the QUEUEJOB and MODIFYJOB hooks can be used to read, modify, and validate these parameters. This would also have the advantage of keeping these separate from the environment variables for the job.

By way of example, you could do something along the lines of:

create resource my_params type=string
set server resources_default.my_params = "foo=bar"

to create and set a default for a string resource and then do something like:

qsub <other qsub options> -l my_params="foo=bar,baz=qux" my_qsub_script.sh

Hooks could then be used to process that string and react to the parameters on the job in PBS.

Would that help with/cover your use-case?

from openpbs.

Yep! This pretty much provides what we need. I didn't know you could create/define your own resources. This has the advantage that we can leverage qsub to do some validation for us. Any further validation can be performed in the queuejob and modifyjob events as you suggested.

Thanks for the suggestion and detailed information! Really appreciate it.

from openpbs.

@pmrich: I tested your suggestion. I'm able to create a custom resource and I'm able to see it in the job object's Resource_list attribute within my hook when it runs on the server (e.g. if I configure it to run on queuejob). However, when I configure my hook to run on an execution host event (e.g. execjob_prologue), I don't see it.

I did some digging, and I see there is a flag attribute you can set for resources. I see you can set it to h to indicate it's a host-level resource, for example. I gave that a try, but I still can't see it. I'm sure I'm missing something. Do you know how to make a custom resource visible to hooks that run on execution hosts?

from openpbs.

Found the relevant section in the PBS BigBook for 2022.1: for hooks that run from the MoM, you have to get access to custom resources from the server's job object, they don't get sent along with the job event object. (pg HG-54. Hooks Guide section 5.2.4.1). You should be able to get at the server-level resources for the job there, which is what I think you want here.

As for the h-flag to a resource, that makes it a host-level resource (set per-node) and I know that the m-flag will work with that. How you would set the default value would be different from something like the queue level, (you would declare that as default_chunk.foo for a host-level resource as opposed to resources_available.foo for something at the server or queue levels).

from openpbs.

Thanks for looking that up! Yeah, I was just able to verify that the server's job object does contain the custom resource that I define (and pass via qsub), and I'm able to access it from hooks running on execution hosts.

from openpbs.