wjcunningham7 / covalent-nersc-plugin Goto Github PK

Covalent is a Pythonic workflow tool used to execute tasks on advanced computing hardware.

This executor plugin interfaces Covalent with the NERSC Superfacility which allows users to offload workflow tasks securely to the Perlmutter supercomputer. Usage of this plugin is restricted to authorized users and is subject to NERSC's appropriate use policies. To request an account, refer to the documentation here.

To use this plugin with Covalent, install it using pip:

pip install covalent-nersc-plugin

This is an example of how a workflow can be adapted to use the NERSC executor plugin.

There are many configuration options that can be passed into the class ct.executor.NERSCExecutor or by modifying the Covalent config file under the section [executors.nersc].

Further detail about this plugin's configuration will be added to the core Covalent documentation in the near future. Check back soon!

In order to run Covalent workflows with this plugin, users will first need to have an account with NERSC. Users will then need to create a Superfacility API Client in the Iris portal using the Red (highest) security level, with ingress allowed only from the Covalent server. Save the client ID and the private PEM key in a file on the Covalent server in the following format:

<client_id>
-----BEGIN RSA PRIVATE KEY-----
<private key contents>
-----END RSA PRIVATE KEY-----

This file will be ingested by the plugin when the Covalent server offloads workflow tasks to it.

For more information on how to get started with Covalent, check out the project homepage and the official documentation.

Release notes for this plugin are available in the Changelog.

Please use the following citation in any publications:

W. J. Cunningham, S. K. Radha, F. Hasan, J. Kanem, S. W. Neagle, and S. Sanand. Covalent. Zenodo, 2022. https://doi.org/10.5281/zenodo.5903364

Covalent is licensed under the GNU Affero GPL 3.0 License. Covalent may be distributed under other licenses upon request. See the LICENSE file or contact the support team for more details.