pyslurm / pyslurm Goto Github PK

View Code? Open in Web Editor NEW

458.0 23.0 115.0 5.41 MB

Python Interface to Slurm

Home Page: https://pyslurm.github.io

License: GNU General Public License v2.0

Python 13.77% Shell 0.34% C 0.51% Cython 85.38%

slurm cython python hpc cluster

pyslurm's Introduction

pyslurm is the Python client library for the Slurm Workload Manager

Requirements

Slurm - Slurm shared library and header files
Python - >= 3.6
Cython - >= 0.29.36

This Version is for Slurm 23.11.x

Versioning

In pyslurm, the versioning scheme follows the official Slurm versioning. The first two numbers (MAJOR.MINOR) always correspond to Slurms Major-Release, for example 23.11. The last number (MICRO) is however not tied in any way to Slurms MICRO version, but is instead PySlurm's internal Patch-Level. For example, any pyslurm 23.11.X version should work with any Slurm 23.11.X release.

Installation

By default, it is searched inside /usr/include for the Header files and in /usr/lib64 for Slurms shared-library (libslurm.so) during Installation. For Slurm installations in different locations, you will need to provide the corresponding paths to the necessary files.

You can specify those with environment variables (recommended), for example:

export SLURM_INCLUDE_DIR=/opt/slurm/23.11/include
export SLURM_LIB_DIR=/opt/slurm/23.11/lib

Then you can proceed to install pyslurm, for example by cloning the Repository:

git clone https://github.com/PySlurm/pyslurm.git && cd pyslurm
scripts/build.sh

# Or simply with pip
pip install .

Also see python setup.py --help

Contributors

pyslurm is made by contributors like you.

Support

Feel free to ask questions in the GitHub Discussions

Found a bug or you are missing a feature? Feel free to open an Issue!

pyslurm's People

Contributors

Stargazers

Watchers

Forkers

livenson baoboa waxie aeigis88 arcapix ciemat-tic mrobbert anderbubble edf-hpc treydock alexeyg qytz tcooper fasrc rcthomas nealepetrillo wilshire461 q-leap-networks dev-alex-alex2006hw webbres vreuter rjms-bull jstaf delgadom basvandervlies ddcr kmanalo cci-moc diagnijmegen kmhernan hrk2109 paran1 lgro pkcakeout nmccollum lrodrigo79 hpcugent slurmrelated hpc-highperformancecomputing mayidudu mattaezell jianshuli jbd mcaubet matthdan arsvision cacheboxinc fbeneventi ahmedhany93 sepiidae swillner niejn monash-merc vkantchev buysdb databill86 enfo14 bikerdanny laanstragj acidburn0zzz davidchoi76 lslowmotion cola20150301 eurocfd cicerolneto jaeohwoo suhuihuang csetraynor wahello andrewtsao trendscenter prehensilecode wpoely86 granquet jochym mercury00 0xstarfox mask891 stxinsite saboritoscar tazend jackguyver li-positive-one abadem ahmedfathymahmoud kcl-eresearch oxfordfun flatironinstitute sk8boy xandm leihaibo0621 schluenz pllopis steenlysgaard nih-hpc multimeric elelayan youkee-first robgics ricardojacomini

pyslurm's Issues

Complete class based method

The original pyslurm was a 30 minute hack to access the C API in a rush but meant I had to wrap C pointer in order to pass being python functions. By moving to a class based access to the C functions we keep the C data within the class and we can remove the wrapper functions. However I will lose the exact python to C function name mapping, I feel that is not required as generally we only want to get at the SLURM data via native python data types.

Python3 compatibility

Have to do this at some point but gradually removing any deprecated usage :

Dictionary has_key() calls removed.
Dictionary iteritems() calls removed.
Unicode strings (in progress).

Completed porting of the following for Py3k support :

Updated "init.py" so the new absolute import is used via from future
setup.py minimum requirement is now Python2.6 as that has back ported support for some Py3k enhancements

Pyslurm-2.5.0 and Cython-0.17.2 raises API issue

Whilst completing PySlurm-2.5.0 for release the compile was tested against Cython-0.17.2 which has raised the following code error against the Slurm API. Cython is correct and the code below is indeed incorrect ! This needs to be fixed and retested before formal release and previous versions should be verified as well at some point to see where the API changed.

makalu:/home/mark/pyslurm # python setup.py build_ext -inplace
INFO:root:Info:
INFO:root:Info: Building PySlurm (2.5.0-1)
INFO:root:Info: ------------------------------
INFO:root:Info:
INFO:root:Info: Cython version 0.17.2 installed

running build_ext
cythoning pyslurm/pyslurm.pyx to pyslurm/pyslurm.c

Error compiling Cython file:
------------------------------------------------------------
...
        resv_msg.duration = uint32_value

        if reservation_dict[u'node_cnt'] != -1:
                int_value = reservation_dict[u'node_cnt']
                resv_msg.node_cnt = <uint32_t>slurm.xmalloc(sizeof(uint32_t) * 2)
                *resv_msg.node_cnt = int_value
 ^
------------------------------------------------------------

pyslurm/pyslurm.pyx:2893:2: a starred assignment target must be in a list or tuple - maybe you meant to use an index assignment: var              [0] = ...

ImportError: No module named pyslurm

This may be me just being stupid, but after successfully building and installing pyslurm, when I run import pyslurm from an interactive python shell, I get the following error:

Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyslurm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyslurm/__init__.py", line 17, in <module>
    from .pyslurm import *
ImportError: No module named pyslurm
>>>

Any ideas what could be going on there?

Cannot install, build error

python setup.py build --slurm-lib=/usr/lib64/ --slurm-inc=/usr/include fails with the following error:

pyslurm/pyslurm.c: At top level:
pyslurm/pyslurm.c:68541:65: error: unknown type name ‘connection_type’
 static CYTHON_INLINE PyObject* __Pyx_PyInt_From_connection_type(connection_type value) {
                                                             ^
pyslurm/pyslurm.c:68567:63: error: unknown type name ‘node_use_type’
 static CYTHON_INLINE PyObject* __Pyx_PyInt_From_node_use_type(node_use_type value) {
                                                               ^
error: command 'gcc' failed with exit status 1

System is CentOS 6.6 with the following packages:

slurm-devel-14.11.8-1.el6.x86_64
slurm-pam_slurm-14.11.8-1.el6.x86_64
slurm-plugins-14.11.8-1.el6.x86_64
slurm-torque-14.11.8-1.el6.x86_64
slurm-spank-x11-0.2.5-1.x86_64
slurm-perlapi-14.11.8-1.el6.x86_64
slurm-blcr-14.11.8-1.el6.x86_64
slurm-munge-14.11.8-1.el6.x86_64
slurm-14.11.8-1.el6.x86_64

rpm -ql slurm-devel | head -n 15 returns:

/usr/include/slurm
/usr/include/slurm/pmi.h
/usr/include/slurm/pmi2.h
/usr/include/slurm/slurm.h
/usr/include/slurm/slurm_errno.h
/usr/include/slurm/slurmdb.h
/usr/include/slurm/smd_ns.h
/usr/include/slurm/spank.h
/usr/lib64/libpmi.la
/usr/lib64/libpmi2.la
/usr/lib64/libslurm.la
/usr/lib64/libslurmdb.la
/usr/lib64/pkgconfig
/usr/lib64/pkgconfig/slurm.pc
/usr/share/man/man3/slurm_allocate_resources.3.gz

/usr/include/slurm/slurm.h includes the following lines:

enum node_use_type {
        SELECT_COPROCESSOR_MODE,/* use extra processor for communications */
        SELECT_VIRTUAL_NODE_MODE,/* application uses both processors */
        SELECT_NAV_MODE         /* either mode is acceptable */
};

enum connection_type {
        SELECT_MESH,            /* nodes wired in mesh */
        SELECT_TORUS,           /* nodes wired in torus */
        SELECT_NAV,             /* nodes wired in torus else mesh */
        SELECT_SMALL,           /* nodes in a small partition */
        SELECT_HTC_S,           /* nodes in a htc running SMP mode */
        SELECT_HTC_D,           /* nodes in a htc running Dual mode */
        SELECT_HTC_V,           /* nodes in a htc running VN mode */
        SELECT_HTC_L            /* nodes in a htc running in Linux mode */
};

I have no idea how to get it to install.

No symbols in python module

After compiling with cython 0.19 and 2.5.7 my module has nothing in it.

dir(pyslurm)
['DLFCN', 'builtins', 'doc', 'file', 'name', 'package', 'path', 'version', 'absolute_import', 'old_dlopen_flags', 'os', 'sys', 'version']

any ideas?

Create a PySlurm 2.4.5 branch

No changes in the 2.4.5 API compared to 2.4.4 API

Incorporate Python-hostlist into pyslurm

Question: can pyslurm access per-partition CPU info like sinfo?

Does pyslurm have access to the number of allocated/idle/other/total CPUs per partition? I could only find the total_cpus in pyslurm.partition().

$ sinfo -o '%P %C'
PARTITION CPUS(A/I/O/T)
norm* 9094/794/32/9920

Thanks,
Giovanni

pyslurm.node().get() doesn't work as expected anymore. Regression?

I have a script that does the following:

pyslurmnode = pyslurm.node()
nodeinfo = pyslurmnode.find_id(node)

nodeinfo is now always returning an empty {}. If I revert to an older version of pyslurm, nodeinfo is populated correctly. I think the problem may be in this commit: 6110534. self.get() was taken out of the node constructor. It is there, however, for the job class constructor (https://github.com/gingergeeks/pyslurm/blob/slurm-14.11.5/pyslurm/pyslurm.pyx#L1678)

I can get around this by manually calling get() after initializing the pyslurm.node object:

pyslurmnode = pyslurm.node()
pyslurmnode.get()
nodeinfo = pyslurmnode.find_id(node)

Stringing together a method call after object initialization also no longer works in the interpreter console for node():

>>> pprint(pyslurm.node().find_id("cn0001"))
{}

Thoughts?

Add .gitignore file to repo

Need to identify files that should not be considered candidates for commit

feature request: add alloc_mem to pyslurm.node()

pyslurm.node().get() doesn't give allocated memory for any nodes. I'm looking for this value:

scontrol show node cn0001 | grep AllocMem
   OS=Linux RealMemory=124926 AllocMem=90112 Sockets=2 Boards=1

pyslurm does report real_memory for each node.

Thanks,
Giovanni

Problem while importing pyslurm on ubuntu 12.04

Hello,

I just set up slurm 2.5, and then pyslurm on an ubuntu 12.04 computer.
Slurm is working fine (in /usr/local), and I was able to compile pyslurm and install it (python setup.py build --slurm=/usr/local; sudo python setup.py install).
However, when I am just importing pyslurm, I get :

Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyslurm
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/pyslurm/init.py", line 17, in
from .pyslurm import *
File "slurm_defines.pxi", line 84, in init pyslurm.pyslurm (pyslurm/pyslurm.c:51839)
NameError: slurm

I uninstalled cython from ubuntu, and pip installed a fresh version of it, but I am getting the same message.

Does somebody have an idea of what is going on ?

Best regards,

Arnaud Laprévote

Use of licenses (-L) makes pyslurm job.get() crashing

In starting the following job (after having created a license test in slurm.conf) :
srun -L test -N 1 ./wait-arg.sh 240

I get the following error with pyslurm :
python
Python 2.7.3 (default, Apr 10 2013, 06:20:15)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyslurm
job=pyslurm.job()
a=job.get()
Traceback (most recent call last):
File "", line 1, in
File "pyslurm.pyx", line 1588, in pyslurm.pyslurm.job.get (pyslurm/pyslurm.c:20271)
File "pyslurm.pyx", line 1638, in pyslurm.pyslurm.job.__get (pyslurm/pyslurm.c:20709)
File "pyslurm.pyx", line 3670, in pyslurm.pyslurm.__get_licenses (pyslurm/pyslurm.c:41420)
ValueError: need more than 1 value to unpack

This occured both on Ubuntu 12.04 (slurm 2.5.0) and on centos 6.3 (slurm 2.5.something) with pyslurm compiled from the trunk.

If I add in pyslurm.pyx, the following line :
print(("debug __get_licenses %s",alist[i]))
on line 3669 of pyslurm.pyx, then I have the resulting output :
arnaud@D3550:/usr/local$ python
Python 2.7.3 (default, Apr 10 2013, 06:20:15)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import sys
sys.path.append("/usr/local/lib/python2.7/dist-packages/pyslurm-2.5.6_1")
import pyslurm
a=pyslurm.job()
b=a.get()
('debug __get_licenses %s', 'test')
Traceback (most recent call last):
File "", line 1, in
File "pyslurm.pyx", line 1588, in pyslurm.pyslurm.job.get (pyslurm/pyslurm.c:20273)
File "pyslurm.pyx", line 1638, in pyslurm.pyslurm.job.__get (pyslurm/pyslurm.c:20711)
File "pyslurm.pyx", line 3670, in pyslurm.pyslurm.__get_licenses (pyslurm/pyslurm.c:41430)
ValueError: need more than 1 value to unpack

So we see that pyslurm waits for a string formatted as xxxx_number . And in my configuration file, I have : Licenses=test_2,test2
so at least the test*2 should be ok. So it is the reading of this string that is buggy.

slurm_job_terminate core dump

Hello to all,

I am trying on slurm 2.5 slurm_job_terminate. What I am doing :

import pyslurm
jobs=pyslurm.job()
dod=jobs.get()
dod
{174: {u'comment': None, u'time_limit': 26L, u'cnode_cnt': None, u'alloc_node': u'D3550', u'features': [], u'eligible_time': 1357308185, u'contiguous': False, u'resv_id': None, u'ramdisk_image': None, u'block_id': None, u'sockets_per_node': 65534, u'req_switch': 0L, u'resv_name': None, u'licenses': {}, u'qos': None, u'submit_time': 1357308185, u'mloader_image': None, u'num_cpus': 1L, u'conn_type': (None, 'None'), u'show_flags': 0, u'user_id': 1001L, u'network': None, u'restart_cnt': 0, u'work_dir': u'/home/arnaud/src/slurmjob', u'pn_min_tmp_disk': 0L, u'max_nodes': 0L, u'job_state': (1, 'RUNNING'), u'assoc_id': 0L, u'exit_code': 0L, u'num_nodes': 1L, u'priority': 4294901721L, u'batch_script': None, u'boards_per_node': 0, u'ntasks_per_socket': 65535, u'batch_flag': 0, u'derived_ec': 0L, u'nodes': None, u'preempt_time': 0, u'pn_min_cpus': 1, u'nice': 10000, u'ntasks_per_node': 0, u'linux_image': None, u'altered': None, u'sockets_per_board': 0, u'alloc_sid': 10014L, u'start_time': 1357308185, u'pre_sus_time': 0, u'ionodes': None, u'state_reason': (0, 'None'), u'pn_min_memory': 0L, u'rotate': False, u'reboot': None, u'blrts_image': None, u'shared': 0, u'time_min': 10L, u'wait4switch': 0L, u'ntasks_per_core': 65535, u'wckey': None, u'account': None, u'requeue': True, u'name': u'wait-arg.sh', u'req_nodes': [], u'gres': [], u'suspend_time': 0, u'partition': 'debug', u'cores_per_socket': 65534, u'batch_host': u'D3550', u'dependency': None, u'max_cpus': 0L, u'state_desc': None, u'command': u'/home/arnaud/src/slurmjob/./wait-arg.sh', u'end_time': 1357309745, u'cpus_per_task': 1, u'resize_time': 0, u'group_id': 1001L, u'exc_nodes': [], u'threads_per_core': 65534}}
pyslurm.pyslurm.slurm_terminate_job(174)
Erreur de segmentation (core dumped)

Strange as at the same time notify, kill, suspend and resume are perfectly working.

Support for newer versions of SLURM

Hi,

I wanted to try to your python module and see if we can use it to build a dynamic queue manager for the iPython parallel cluster. When trying to build the package, I noticed that the version of SLURM we use (15.8.6) is not supported by the package:

INFO:root:Info: Build - Detected Slurm include file version - 0x0f0806 (15.8.6)
ERROR:root:Fatal: Build - Incorrect slurm version detected, require Slurm-14.11.5 to slurm-14.11.8

I removed the version check from setup.py to see if it would still build despite the version mismatch, but I hit some compilation errors:

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include -I. -I/opt/insy/env.el7/sys_enhance/include/python2.7 -c pyslurm/pyslurm.c -o build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o
pyslurm/pyslurm.c: In function ‘__pyx_f_7pyslurm_7pyslurm_6config___get’:
pyslurm/pyslurm.c:8850:75: error: ‘slurm_ctl_conf_t’ has no member named ‘dynalloc_port’
     __pyx_t_1 = __Pyx_PyInt_From_uint16_t(__pyx_v_self->__pyx___Config_ptr->dynalloc_port); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 413; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                                           ^
pyslurm/pyslurm.c: In function ‘__pyx_f_7pyslurm_7pyslurm_3qos___get’:
pyslurm/pyslurm.c:49488:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘grp_cpu_mins’
         __pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_qos->grp_cpu_mins); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4136; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49500:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘grp_cpu_run_mins’
         __pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_qos->grp_cpu_run_mins); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4137; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49512:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘grp_cpus’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->grp_cpus); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4138; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49536:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘grp_mem’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->grp_mem); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4140; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49548:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘grp_nodes’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->grp_nodes); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4141; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49584:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_cpu_mins_pj’
         __pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_qos->max_cpu_mins_pj); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4144; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49596:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_cpu_mins_pj’
         __pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_qos->max_cpu_mins_pj); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4145; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49608:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_cpu_run_mins_pu’
         __pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_qos->max_cpu_run_mins_pu); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4146; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49620:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_cpus_pj’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->max_cpus_pj); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4147; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49632:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_cpus_pu’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->max_cpus_pu); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4148; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49656:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_nodes_pj’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->max_nodes_pj); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4150; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
pyslurm/pyslurm.c:49668:58: error: ‘slurmdb_qos_rec_t’ has no member named ‘max_nodes_pu’
         __pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_qos->max_nodes_pu); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 4151; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                          ^
error: command 'gcc' failed with exit status 1

I was wondering if there are any plans to update the package for newer versions of SLURM or whether you had looked into it at all to see whether this is a lot of work or not.

Add in energy reporting for node

Adding into node class the gathering of info from acct_gather_energy structure. Just need to build kernel with msr module and test with userspace rapl.

Have implemented an energy dictionary within the node dictionary to hold the metrics -

Host_dict['energy'] = {}
Host_dict['energy']['base_watts'] = self._Node_ptr.node_array[i].energy.base_watts
Host_dict['energy']['current_watts'] = self._Node_ptr.node_array[i].energy.current_watts
Host_dict['energy']['consumed_energy'] = self._Node_ptr.node_array[i].energy.consumed_energy
Host_dict['energy']['base_consumed_energy'] = self._Node_ptr.node_array[i].energy.base_consumed_energy
Host_dict['energy']['previous_consumed_energy'] = self._Node_ptr.node_array[i].energy.previous_consumed_energy

Testing will need to be done to account for non-msr installs/architectures.

Doesn't build with slurm-14.03.8 and Cython-0.20

Even when patching setup.py to build with newer python version, the build fails.
building 'pyslurm.pyslurm' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/pyslurm
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include -I. -I/usr/include/python2.7 -c pyslurm/pyslurm.c -o build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o
pyslurm/pyslurm.c: In function ‘pyx_f_7pyslurm_7pyslurm_6config___get’:
pyslurm/pyslurm.c:8641:5: attention : passing argument 1 of ‘__Pyx_PyInt_From_uint16_t’ makes integer from pointer without a cast [enabled by default]
__pyx_t_1 = __Pyx_PyInt_From_uint16_t(__pyx_v_self->__pyx___Config_ptr->job_acct_gather_freq); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 407; __pyx_clineno = __LINE; goto __pyx_L1_error;}
^
pyslurm/pyslurm.c:2145:29: note: expected ‘uint16_t’ but argument is of type ‘char ’
static CYTHON_INLINE PyObject __Pyx_PyInt_From_uint16_t(uint16_t value);
^
pyslurm/pyslurm.c: In function ‘__pyx_f_7pyslurm_7pyslurm_slurm_requeue’:
pyslurm/pyslurm.c:15177:3: erreur: too few arguments to function ‘slurm_requeue’
__pyx_v_errCode = slurm_requeue(__pyx_v_JobID);
^
In file included from pyslurm/pyslurm.c:345:0:
/usr/include/slurm/slurm.h:3758:12: note: declared here
extern int slurm_requeue PARAMS((uint32_t job_id, uint32_t state));
^
pyslurm/pyslurm.c: In function ‘__pyx_f_7pyslurm_7pyslurm_slurm_terminate_job’:
pyslurm/pyslurm.c:16682:3: attention : implicit declaration of function ‘slurm_terminate_job’ [-Wimplicit-function-declaration]
__pyx_v_errCode = slurm_terminate_job(__pyx_v_JobID);
^
error: command 'gcc' failed with exit status 1

Is this project dead?

"ImportError: No module named pyslurm" despite successful install

Hi guys,
I was able to successfully (I believe) compile the pyslurm module, despite a few warnings:

$ python setup.py build --slurm-inc=/cm/shared/apps/slurm/14.11.6/include --slurm-lib=/cm/shared/apps/slurm/14.11.6/lib64
INFO:root:Info: 
INFO:root:Info: Building PySlurm (14.11.5)
INFO:root:Info: ------------------------------
INFO:root:Info: 
INFO:root:Info: Cython version 0.22 installed

INFO:root:Info: Clean - checking for objects to clean
INFO:root:Info: Clean - completed
INFO:root:Info: Build - Detected Slurm include file version - 0x0e0b06 (14.11.6)
INFO:root:Info: Build - Writing Slurm version to pyslurm/slurm_version.pxi
INFO:root:Info: Build - Generating pyslurm/bluegene.pxi file
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/pyslurm
copying pyslurm/__init__.py -> build/lib.linux-x86_64-2.7/pyslurm
running build_ext
cythoning pyslurm/pyslurm.pyx to pyslurm/pyslurm.c
building 'pyslurm.pyslurm' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/pyslurm
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/cm/shared/apps/slurm/14.11.6/include -I. -I/home/aurelien.mazurie/.pyenv/versions/2.7.9/include/python2.7 -c pyslurm/pyslurm.c -o build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o
In file included from pyslurm/pyslurm.c:255:0:
/cm/shared/apps/slurm/14.11.6/include/slurm/slurm.h:1605:2: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
  void (*acct_full)();
  ^
/cm/shared/apps/slurm/14.11.6/include/slurm/slurm.h:1606:2: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
  void (*dbd_fail)();
  ^
/cm/shared/apps/slurm/14.11.6/include/slurm/slurm.h:1607:2: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
  void (*dbd_resumed)();
  ^
/cm/shared/apps/slurm/14.11.6/include/slurm/slurm.h:1608:2: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
  void (*db_fail)();
  ^
/cm/shared/apps/slurm/14.11.6/include/slurm/slurm.h:1609:2: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
  void (*db_resumed)();
  ^
In file included from pyslurm/pyslurm.c:256:0:
/cm/shared/apps/slurm/14.11.6/include/slurm/slurmdb.h:1193:1: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
 extern void *slurmdb_connection_get();
 ^
gcc -pthread -shared -L/home/aurelien.mazurie/.pyenv/versions/2.7.9/lib build/temp.linux-x86_64-2.7/pyslurm/pyslurm.o -L/cm/shared/apps/slurm/14.11.6/lib64 -L/cm/shared/apps/slurm/14.11.6/lib64/slurm -Wl,-R/cm/shared/apps/slurm/14.11.6/lib64/ -Wl,-R/cm/shared/apps/slurm/14.11.6/lib64/slurm -lslurmdb -o build/lib.linux-x86_64-2.7/pyslurm/pyslurm.so

I was then able to install the module with python setup.py install:

INFO:root:Info: 
INFO:root:Info: Building PySlurm (14.11.5)
INFO:root:Info: ------------------------------
INFO:root:Info: 
INFO:root:Info: Cython version 0.22 installed

running install
running build
running build_py
running build_ext
skipping 'pyslurm/pyslurm.c' Cython extension (up-to-date)
running install_lib
creating /home/aurelien.mazurie/.pyenv/versions/pyslurm/lib/python2.7/site-packages/pyslurm
copying build/lib.linux-x86_64-2.7/pyslurm/pyslurm.so -> /home/aurelien.mazurie/.pyenv/versions/pyslurm/lib/python2.7/site-packages/pyslurm
copying build/lib.linux-x86_64-2.7/pyslurm/__init__.py -> /home/aurelien.mazurie/.pyenv/versions/pyslurm/lib/python2.7/site-packages/pyslurm
byte-compiling /home/aurelien.mazurie/.pyenv/versions/pyslurm/lib/python2.7/site-packages/pyslurm/__init__.py to __init__.pyc
running install_egg_info
Writing /home/aurelien.mazurie/.pyenv/versions/pyslurm/lib/python2.7/site-packages/pyslurm-14.11.5-py2.7.egg-info

But when attempting to import it in Python, I get an ImportError:

Python 2.7.9 (default, Feb 23 2015, 14:53:24) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyslurm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyslurm/__init__.py", line 17, in <module>
    from .pyslurm import *
ImportError: No module named pyslurm
>>>

Any idea about why I get this error message?

Best,
Aurélien

Representation of partitions in pyslurm

While investigating the representation of partitions in pyslurm, I end up with the following results :
Informations available for example with scontrol show partition 👍
PartitionName=debug
AllocNodes=ALL AllowGroups=ALL Default=YES
DefaultTime=NONE DisableRootJobs=NO GraceTime=0 Hidden=NO
MaxNodes=UNLIMITED MaxTime=UNLIMITED MinNodes=1
Nodes=snode1,D3550
Priority=1 RootOnly=NO ReqResv=NO Shared=NO PreemptMode=OFF
State=UP TotalCPUs=2 TotalNodes=2 DefMemPerNode=UNLIMITED
MaxMemPerNode=UNLIMITED
Which can be represented this way (soap interface)
String name;
String AllocNodes; // ',' separator <-> allow_alloc_nodes ?
String AllowGroups; <-> allow_groups
Boolean Default;
Long defaultTime; // seconds <-> default_time
Boolean DisableRootJobs;
Long GraceTime; // seconds <-> grace_time
Boolean Hidden;
Integer MaxNodes; <-> max_nodes
Long MaxTime; // seconds <-> max_time
Integer MinNodes; <-> min_nodes
String Nodes; // ',' separator <-> nodes
Integer Priority; // max 65533 <-> priority
Boolean RootOnly;
ReqResv
Boolean Shared;
String preemptMode; // OFF, CANCEL, CHECKPOINT, REQUEUE, SUSPEND <-> preempt_mode (0,'OFF')
String state; // UP, DOWN, DRAIN, INACTIVE <-> state_up 3
Integer TotalCpus; <-> total_cpus
Integer TotalNodes; <-> total_nodes
Long DefMemPerNode; // megabytes <-> def_mem_per_cpu
Long MaxMemPerNode; // megabytes <-> max_mem_per_cpu
Pyslurm representation
>>> import pyslurm
>>> a=pyslurm.partition()
>>> a.get()
{u'debug': {
'max_mem_per_cpu': 0L, *
'total_cpus': 2L, *
'alterncdate': None, ---
'def_mem_per_cpu': 0L, *
'grace_time': 0L, *
'max_nodes': 4294967295L, *
'default_time': 4294967294L, *
'max_share': 1, ---------------
'last_update': 1356695530, *
'priority': 1, *
'allow_groups': [], *
'allow_alloc_nodes': [], *?
'flags': 1, -------------------
'state_up': 3, *
'max_time': 4294967295L, *
'total_nodes': 2L, *
'nodes': ['snode1', 'D3550'], *
'min_nodes': 1L, *
'preempt_mode': (0, 'OFF')}} *

There are some discrepancies :
DefMemPerNode <=> def_mem_per_cpu
MaxMemPerNode <=> max_mem_per_cpu

The representation of DEFAULT is unclear (flags ?) if available.
DisableRootJobs, ReqResv, Hidde, Shared are also unclear or basically not present.

Check for all types from Slurm where it could be None

There appears to be situations where it was assumed that an integer is expected by when no value is received then it is translated as None by Python. We need to check that the value is valid before passing onto to conversion routines ie get_conn_type_string.

Number of running jobs showed in example/jobs_list.py does not correspond to the real number of jobs

While running the example/jobs_list.py and having one running job, I have :
arnaud@D3550:/src/pyslurm/examples$ ./jobs_list.py
No jobs found !
arnaud@D3550:/src/pyslurm/examples$ ./jobs_list.py
JobID 136 :
account : None
alloc_node : D3550
alloc_sid : 2849
altered : None
assoc_id : 0
batch_flag : 0
batch_host : D3550
batch_script : None
block_id : None
blrts_image : None
boards_per_node : 0
cnode_cnt : None
command : /home/arnaud/src/slurmjob/./wait-arg.sh
comment : None
conn_type : (None, 'None')
contiguous : False
cores_per_socket : 65534
cpus_per_task : 1
dependency : None
derived_ec : 0
eligible_time : Fri Dec 28 14:51:01 2012
end_time : Sat Dec 28 14:51:01 2013
exc_nodes : []
exit_code : 0
features : []
gres : []
group_id : 1001
ionodes : None
job_state : (1, 'RUNNING')
licenses : {}
linux_image : None
max_cpus : 0
max_nodes : 0
mloader_image : None
name : wait-arg.sh
network : None
nice : 10000
nodes : None
ntasks_per_core : 65535
ntasks_per_node : 0
ntasks_per_socket : 65535
num_cpus : 1
num_nodes : 1
partition : debug
pn_min_cpus : 1
pn_min_memory : 0
pn_min_tmp_disk : 0
pre_sus_time : 0
preempt_time : 0
priority : 4294901759
qos : None
ramdisk_image : None
reboot : None
req_nodes : []
req_switch : 0
requeue : True
resize_time : N/A
restart_cnt : 0
resv_id : None
resv_name : None
rotate : False
shared : 0
show_flags : 0
sockets_per_board : 0
sockets_per_node : 65534
start_time : Fri Dec 28 14:51:01 2012
state_desc : None
state_reason : (0, 'None')
submit_time : Fri Dec 28 14:51:01 2012
suspend_time : 0
threads_per_core : 65534
time_limit : Infinite
time_min : 0
user_id : 1001
wait4switch : 0
wckey : None

work_dir : /home/arnaud/src/slurmjob

Number of Jobs - 1

[]
Number of pending jobs - 0
Number of running jobs - 0

JobIDs in Running state - []

This is fine. But the 2 last lines : Number of running jobs and JobIDs in Running state are wrong.

In jobs_list.py, it corresponds to the call to : a.find('job_state', pyslurm.JOB_RUNNING). It seems that the representation of the running jobs has changed in pyslurm, which result in this non critical bug.

Add cleanup of build directory and temp files to setup.py

Add Sphinx auto build documentation

pyslurm.slurm_api_version() doesn't work

Dear,

When I call the pyslurm.slurm_api_version(), I got the follwoing error message.

print "SLURM\t%s-%s-%s\n" % (pyslurm.slurm_api_version())
Traceback (most recent call last):
File "", line 1, in
File "pyslurm.pyx", line 163, in pyslurm.pyslurm.slurm_api_version (pyslurm/pyslurm.c:6679)
NameError: name 'slurm' is not defined

What is the problem?

My installed slurm version is slurm-2.5.0 and pyslurm version is PySLURM 2.5.0-1.

print "PySLURM\t%s" % (pyslurm.version())
PySLURM 2.5.0-1

Thank you in advance.

segfault in node.get

I am getting a segmentation fault in nodes.get() on my production slurm cluster (2.6.2, using pyslurm master branch).

When I run the example:

[~/Source/pyslurm/examples]: python ./node_list.py 
python: error: Unsupported option 4 for get_nodeinfo.
Segmentation fault (core dumped)

It works just fine on the (almost) identical test cluster I have. I did get the core- backtrace shows:

[~/Source/pyslurm/examples]: gdb `which python` ./core 
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /usr/bin/python...Reading symbols from /usr/lib/debug/usr/bin/python2.7...done.
done.
[New LWP 21151]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python ./node_list.py'.
Program terminated with signal 11, Segmentation fault.
#0  __strlen_sse2_pminub () at ../sysdeps/x86_64/multiarch/strlen-sse2-pminub.S:39
39  ../sysdeps/x86_64/multiarch/strlen-sse2-pminub.S: No such file or directory.
(gdb) 
(gdb) bt
#0  __strlen_sse2_pminub () at ../sysdeps/x86_64/multiarch/strlen-sse2-pminub.S:39
#1  0x000000000051d60b in PyString_FromString ()
#2  0x00007f2848b41798 in __pyx_f_7pyslurm_7pyslurm_4node___get (__pyx_v_self=0x21900b0, __pyx_skip_dispatch=<optimized out>)
    at pyslurm/pyslurm.c:23989
#3  0x00007f2848b0c7f3 in __pyx_f_7pyslurm_7pyslurm_4node_get (__pyx_v_self=0x21900b0, __pyx_skip_dispatch=<optimized out>)
    at pyslurm/pyslurm.c:23133
#4  0x00007f2848b0a210 in __pyx_pf_7pyslurm_7pyslurm_4node_10get (__pyx_v_self=<optimized out>, unused=<optimized out>)
    at pyslurm/pyslurm.c:23181
#5  0x00000000004870ca in PyEval_EvalFrameEx ()
#6  0x000000000048d930 in PyEval_EvalCodeEx ()
#7  0x00000000004246a1 in PyRun_FileExFlags ()
#8  0x000000000042492e in PyRun_SimpleFileExFlags ()
#9  0x0000000000425cb6 in Py_Main ()
#10 0x00007f284928676d in __libc_start_main (main=0x41bb00 <main>, argc=2, ubp_av=0x7fff063a3998, init=<optimized out>, 
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff063a3988) at libc-start.c:226
#11 0x000000000041bb31 in _start ()
(gdb)

In the cython-built pyslurm.c, line 23989:

    /* "pyslurm/pyslurm.pyx":2128
 *          Host_dict['total_cpus'] = total_used
 * 
 *          Hosts[u'%s' % name] = Host_dict             # <<<<<<<<<<<<<<
 * 
 *      self._NodeDict = Hosts
 */
    __pyx_t_2 = PyBytes_FromString(__pyx_v_name); if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2128; __pyx_clineno = __LINE__; goto __pyx_L1_error;}

Some debugging revealed that name is being set to null- line 2055 in pyslurm.pyx:

name = self._Node_ptr.node_array[i].name

It appears that self._Node_ptr.node_array[i].name is returning a None type for some reason. This seems to work just fine in my test cluster (which has different host names, but otherwise should be very similar to the production slurm install). All hostnames are pretty plain (no extended characters).

Thanks... I'd appreciate any hints on debugging or tracking down the fault.

Segmentation fault on pyslurm.node()

Here is an example of session with Pyslurm triggering the error message:

Python 2.7.9 (default, Feb 23 2015, 14:53:24) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyslurm
>>> pyslurm.version()
'14.11.5'
>>> pyslurm.slurm_api_version()
(14, 11, 6)
>>> pyslurm.node()
Segmentation fault

Best,
Aurélien

Test and validate PySlurm with Cython-0.17.1

refuses to build on 0x0e0b04 (14.11.4) but appears to work

I changed setup.py to allow building against 14.11.4, and it built successfully.

diff --git a/setup.py b/setup.py
index fb1aa70..1ed7e5b 100644
--- a/setup.py
+++ b/setup.py
@@ -24,7 +24,7 @@ logging.basicConfig(level=20)
 #VERSION = imp.load_source("/tmp", "pyslurm/__init__.py").__version__
 __version__ = "14.11.0-0"
 __min_slurm_hex_version__ = "0x0e0b00"
-__max_slurm_hex_version__ = "0x0e0b02"
+__max_slurm_hex_version__ = "0x0e0b04"

 def fatal(logstring, code=1):
        logger.error("Fatal: " + logstring)

Running the "Query Node List" example worked, too.

Can setup.py be updated to allow building against 14.11.4?

MIXED state is misreported as ALLOCATED

pyslurm is not reporting the correct node state for nodes in the MIXED state. In this example, pyslurm reports the node is ALLOCATED...

pyslurm.node().get()['cn0313']['node_state']
'ALLOCATED'

...while the slurm commands report the node is in the MIXED state:

$ scontrol show node cn0313 | grep -i state
State=MIXED ThreadsPerCore=2 TmpDisk=6451 Weight=40

sinfo also reports the node in the MIXED state. Tested wrong in both pyslurm version 14.11.0-0 and 14.11.5.

I am using SLURM 14.11.7, python 2.6.6 and Cython 0.22.1.

Design decisions for Pyslurm update

Hi all,

This is the thread where I ask you obvious stuff about pyslurm architecture until you get bored of me. I am newbie both with pyslurm and with Cython, so probably some of the questions are kind of obvious at first. Sorry for that.

This said,

I have the impression that all the methods from slurm.h are defined in slurm.pdx. Is this correct?
... but not all the constants and parameters, like

/* Open stdout/err file mode, 0 for system default (JobFileAppend) */
#define OPEN_MODE_APPEND    1
#define OPEN_MODE_TRUNCATE  2

Why?

Then, pyslurm.pyx does not implement all methods from slurm.pdx, only a subset. Why? Is there any criteria on that?
This applies for example to

#
    # JobSteps
    #

    cdef extern int slurm_get_job_steps (time_t, uint32_t, uint32_t, job_step_info_response_msg_t **, uint16_t)
    cdef extern void slurm_free_job_step_info_response_msg (job_step_info_response_msg_t *)
    cdef extern slurm_step_layout_t *slurm_job_step_layout_get (uint32_t, uint32_t)
    cdef extern void slurm_job_step_layout_free (slurm_step_layout *)
    cdef extern int slurm_job_step_stat (uint32_t, uint32_t, char *, job_step_stat_response_msg_t **)
    cdef extern int slurm_job_step_get_pids (uint32_t, uint32_t, char *, job_step_pids_response_msg_t **)
    cdef extern void slurm_job_step_pids_free (job_step_pids_t *)
    cdef extern void slurm_job_step_pids_response_msg_free (void *)
    cdef extern void slurm_job_step_stat_free (job_step_stat_t *)
    cdef extern void slurm_job_step_stat_response_msg_free (void *)
    cdef extern int slurm_update_step (step_update_request_msg_t * step_msg)

Some of the methods from pyslurm.pyx keep the naming from the original C library. Others, however, have been renamed. It seems that slurm.pdx keeps the naming from slurm.h, but pyslurm.pyx creates a new one. For example

slurm_job_cpus_allocated_on_node to __cpus_allocated_on_node
slurm_job_cpus_allocated_on_node_id to __cpus_allocated_on_node_id

The tests tend to fail. I am correcting them to make them work, but I have the impression that they are kind of old, so they are not expected to work with the lastest version of your library. Is this correct, or am I doing something wrong?

I think this is all for now. Thanks for your help,

Manuel

hostList not set in jobDict with SLURM 2.5.5

ramonb@r7n17:$ squeue -o '%N' -j 37
NODELIST
r7n[18,20]
ramonb@r7n17:$

and with pyslurm:


>>> j.get()[37]
{u'comment': None, u'time_limit': 120L, u'cnode_cnt': None, u'alloc_node': u'r7n17', u'features': [], u'eligible_time': 1366992590, u'contiguous': False, u'resv_id': None, u'ramdisk_image': None, u'block_id': None, u'sockets_per_node': 65534, u'req_switch': 0L, u'resv_name': None, u'licenses': {}, u'qos': None, u'submit_time': 1366992590, u'mloader_image': None, u'num_cpus': 2L, u'conn_type': (None, 'None'), u'show_flags': 0, u'user_id': 31005L, u'network': None, u'restart_cnt': 0, u'work_dir': u'/home/ramonb', u'pn_min_tmp_disk': 0L, u'max_nodes': 0L, u'job_state': (1, 'RUNNING'), u'assoc_id': 0L, u'exit_code': 0L, u'num_nodes': 2L, u'priority': 4294901747L, u'batch_script': None, u'boards_per_node': 0, u'ntasks_per_socket': 65535, u'batch_flag': 1, u'derived_ec': 0L, u'nodes': None, u'preempt_time': 0, u'pn_min_cpus': 1, u'nice': 10000, u'ntasks_per_node': 0, u'linux_image': None, u'altered': None, u'sockets_per_board': 0, u'alloc_sid': 7956L, u'start_time': 1366992590, u'pre_sus_time': 0, u'ionodes': None, u'state_reason': (0, 'None'), u'pn_min_memory': 0L, u'rotate': False, u'reboot': None, u'blrts_image': None, u'shared': 2, u'time_min': 0L, u'wait4switch': 0L, u'ntasks_per_core': 65535, u'wckey': None, u'account': None, u'requeue': True, u'name': u'test.slurm', u'req_nodes': [], u'gres': [], u'suspend_time': 0, u'partition': 'batch', u'cores_per_socket': 65534, u'batch_host': u'r7n18', u'dependency': None, u'max_cpus': 0L, u'state_desc': None, u'command': u'/home/ramonb/test.slurm', u'end_time': 1366999790, u'cpus_per_task': 1, u'resize_time': 0, u'group_id': 31016L, u'exc_nodes': [], u'threads_per_core': 65534}
>>> print [ j.get()[37]['nodes'] ]
[None]
>>>

It is always: None

nodes attribute of job_info changed type

I have been tracking the recent changes in order to use this library with another project (job monarch) and today I got it close to working, but found a potential bug. It may just be that a change is needed on the use of the library, but I wanted to check. The nodes attribute in the job_info structure used to be a string and it changed to a list. The code I'm using grabs this attribute and then calls the create function of the hostlist class, which takes a string. This apparently worked in the past without any conversion in the calling application and I've changed it back and it still works just fine. So, I guess my question is this, should that attribute be changed back to a string to maintain compatibility of hostlist.create() function. Or should the hostlist.create() function be changed. Or do users of the pyslurm library need to account for the fact that the two are different?

PySlurm fails to build against Slurm 2.3.4

Hi,

Some of the elements declared in both pyslurm.pyx and slurm.pxd don't longer exist. Compilation output:

pyslurm/pyslurm.c:33212:104: error: ‘block_info_t’ has no member named ‘ionodes’
pyslurm/pyslurm.c:33266:104: error: ‘block_info_t’ has no member named ‘nodes’
pyslurm/pyslurm.c:33278:96: error: ‘block_info_t’ has no member named ‘node_cnt’
pyslurm/pyslurm.c: In function ‘__pyx_f_7pyslurm_7pyslurm_8topology___load’:
pyslurm/pyslurm.c:35122:29: warning: variable ‘__pyx_v_new_topo_info_ptr’ set but not used [-Wunused-but-set-variable]
pyslurm/pyslurm.c: In function ‘initpyslurm’:
pyslurm/pyslurm.c:47091:30: error: ‘WAIT_TBD2’ undeclared (first use in this function)
pyslurm/pyslurm.c:47091:30: note: each undeclared identifier is reported only once for each function it appears in
error: command 'gcc' failed with exit status 1

nacho@ndnd: (-) [0|0]                                                                                                  
~/tmp $ find -maxdepth 1 -type d                                                                                       
.                                                                                                                      
./slurm-2.3.3                                                                                                          
./slurm-2.3.4                                                                                                          
./slurm-2.2.7                                                                                                          
./slurm-2.3.2                                                                                                          
nacho@ndnd: (-) [0|0]                                                                                                  
~/tmp $ find . -name *.h* | xargs grep WAIT_TBD2                                                                       
./slurm-2.3.3/slurm/slurm.h.in: WAIT_TBD2,                                                                             
./slurm-2.2.7/slurm/slurm.h.in: WAIT_TBD2,                                                                             
./slurm-2.3.2/slurm/slurm.h.in: WAIT_TBD2,

Some other things are also missing:

This diff compares datatype block_info_t as it was declared in versions 2.2.7 and 2.3.4 (sorry, couldn't paste here because I was unable to make GTM highlight a diff). Some of the removed fields are still present in slurm.pxd (ionodes, nodes, node_cnt):

  ctypedef struct block_info_t:                                                                                        
    char *bg_block_id                                                                                                  
    char *blrtsimage                                                                                                   
    int *bp_inx                                                                                                        
    uint16_t conn_type                                                                                                 
    char *ionodes                                                                                                      
    int *ionode_inx                                                                                                    
    uint32_t job_running                                                                                               
    char *linuximage                                                                                                   
    char *mloaderimage                                                                                                 
    char *nodes                                                                                                        
    uint32_t node_cnt                                                                                                  
    uint16_t node_use                                                                                                  
    char *owner_name                                                                                                   
    char *ramdiskimage                                                                                                 
    char *reason                                                                                                       
    uint16_t state

I barely have experience with Cython, so maybe I'm missing something. Let me know if so.

I'm on commit da95d1e.

Hope this helps.

Python3: More string issues

In python3, the dictionary returned by job().get() (and all other similar functions) returns plain strings where binary strings are expected, except formatted to include the b'' syntax. I know this is confusing, so here is what I mean:

python2::

from pyslurm import job
job().find_id(5397444)['name']
Out[2]: u'LD0091.part1.wasp1.sbatch'
type(job().find_id(5397444)['name'])
Out[3]: unicode

python3::

from pyslurm import job
job().find_id(5397444)['name']
Out[5]: "b'LD0091.part1.wasp1.sbatch'" 
type(job().find_id(5397444)['name'])
Out[6]: str

Any ideas? Right now I am handling it by just stripping out the b'' characters at either end of the string. That isn't very elegant (or sensible) though, so I think a fix could be worth it.

undefined symbol in select_cray.so

As described here:
https://github.com/SchedMD/slurm/issues/48

While playing around with pyslurm (and SLURM 2.5.6) I get:

python: error: plugin_load_from_file: dlopen(/usr/lib/slurm/select_cray.so): /usr/lib/slurm/select_cray.so: undefined symbol: default_slurm_config_file
python: error: Couldn't load specified plugin name for select/cray: Dlopen of plugin file failed

This is on a 'normal' Linux amd64 system.

The select_cray plugin is only installed because it is in the Debian package: slurm-llnl-basic-plugins

pyslurm is build like this:


gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include -I. -I/usr/include/python2.6 -c pyslurm/pyslurm.c -o build/temp.linux-x86_64-2.6/pyslurm/pyslurm.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.6/pyslurm/pyslurm.o -Wl,-R/usr/lib/lib -lslurm -o build/lib.linux-x86_64-2.6/pyslurm/pyslurm.so

This is with SLURM 2.5.6, Debian 6.0.7, Cython 0.15.1

Detect cluster mode in Slurm

Unsupported option 4 for get_nodeinfo.

This is with:

Debian 6.0.7
SLURM 2.5.6
Cython 0.15.1
latest pyslurm

This is on a test cluster with 14 slurm nodes.

When I get the nodes, the error is printed 14 times once for each node.


# python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyslurm
>>> n = pyslurm.node()
>>> n.get()
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.
{u'r7n28': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217129, 'real_memory': 1L, 'node_hostname': u'r7n28', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539625, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n28', 'arch': u'x86_64', 'boards': 1361539625, 'name': u'r7n28', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 1L}, u'r7n29': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217126, 'real_memory': 1L, 'node_hostname': u'r7n29', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539639, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n29', 'arch': u'x86_64', 'boards': 1361539639, 'name': u'r7n29', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 1L}, u'r7n22': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217124, 'real_memory': 1L, 'node_hostname': u'r7n22', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539516, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n22', 'arch': u'x86_64', 'boards': 1361539516, 'name': u'r7n22', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 2L}, u'r7n23': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217127, 'real_memory': 1L, 'node_hostname': u'r7n23', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539509, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n23', 'arch': u'x86_64', 'boards': 1361539509, 'name': u'r7n23', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 2L}, u'r7n20': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217122, 'real_memory': 1L, 'node_hostname': u'r7n20', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539492, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n20', 'arch': u'x86_64', 'boards': 1361539492, 'name': u'r7n20', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 4L}, u'r7n21': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217124, 'real_memory': 1L, 'node_hostname': u'r7n21', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539567, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n21', 'arch': u'x86_64', 'boards': 1361539567, 'name': u'r7n21', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 4L}, u'r7n26': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217126, 'real_memory': 1L, 'node_hostname': u'r7n26', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539578, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n26', 'arch': u'x86_64', 'boards': 1361539578, 'name': u'r7n26', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 3L}, u'r7n27': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1366966932, 'real_memory': 1L, 'node_hostname': u'r7n27', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539632, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n27', 'arch': u'x86_64', 'boards': 1361539632, 'name': u'r7n27', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 5L}, u'r7n24': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217130, 'real_memory': 1L, 'node_hostname': u'r7n24', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539559, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n24', 'arch': u'x86_64', 'boards': 1361539559, 'name': u'r7n24', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 3L}, u'r7n25': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217127, 'real_memory': 1L, 'node_hostname': u'r7n25', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539559, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n25', 'arch': u'x86_64', 'boards': 1361539559, 'name': u'r7n25', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 3L}, u'r7n31': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217125, 'real_memory': 1L, 'node_hostname': u'r7n31', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539506, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n31', 'arch': u'x86_64', 'boards': 1361539506, 'name': u'r7n31', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 3L}, u'r7n30': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217126, 'real_memory': 1L, 'node_hostname': u'r7n30', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539635, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n30', 'arch': u'x86_64', 'boards': 1361539635, 'name': u'r7n30', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 1L}, u'r7n32': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367217127, 'real_memory': 1L, 'node_hostname': u'r7n32', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361539583, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n32', 'arch': u'x86_64', 'boards': 1361539583, 'name': u'r7n32', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 3L}, u'r7n18': {'features': [], 'weight': 1L, 'energy': {'current_watts': 0L, 'consumed_energy': 0L, 'base_consumed_energy': 0L, 'previous_consumed_energy': 0L, 'base_watts': 0L}, 'cpus': 8, 'sockets': 8, 'tmp_disk': 0L, 'slurmd_start_time': 1367262323, 'real_memory': 1L, 'node_hostname': u'r7n18', 'total_cpus': 8, 'err_cpus': 0, 'reason': None, 'alloc_cpus': 0, 'threads': 1, 'boot_time': 1361518886, 'cores': 1, 'reason_uid': 4294967294L, 'node_addr': u'r7n18', 'arch': u'x86_64', 'boards': 1361518886, 'name': u'r7n18', 'gres': [], 'node_state': (4098, 'IDLE~'), 'os': u'Linux', 'cpu_load': 8L}}
>>>

As you can see regardless of the error, the node().get() does work

Add Cray (Alps) Support

Just a place holder to remind me to add Cray XT series (ALPS/BASIL) support, this will developed on a Cray XK6 and supported with one or more setup.py flags.

Add Unicode Support

Don't really want to but if we add unicode support to the Python2.2.x version then porting to Python3 will be easier

Question: Recommended slurm / pyslurm version for simple usage and stability?

Hi guys,
excuse the abuse of the issue tracker, but I figured if I ask a question here it might be of use to others.

Anyway, I'm trying to set up slurm for a pretty run-of-the-mill cluster, fairly simple FIFO queue with a few high priority jobs etc - the point is, I don't need the latest and greatest.

Out of the box, the slurm apt-package on Ubuntu 12.04 is 2.3.2, but I see no branches matching that - do you know if the 2.3.3 branch would work, and is it likely to have bugs which have since been fixed?

Second question - if 2.3.2 is too ancient, is there a recommended 'stable and blessed' slurm/pyslurm version number? I'm wondering if building 2.4.4 might be sensible.

Cheers!

Blue Gene detection type

Need to code in some way of detecting the cluster type slurm was built for, specifically BG/L, BG/P and BG/Q. Must chat to Slurm developers about this, maybe a slurm function to return build info would be possible.

Strange python non blocking error message while using pyslurm.node() nodes.get()

When using pyslurm.node, I get the following behaviour :

Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyslurm
nodes = pyslurm.node()
nodes_dict = nodes.get()
python: error: Unsupported option 4 for get_nodeinfo.
python: error: Unsupported option 4 for get_nodeinfo.

There are useful information in nodes_dict after that. However, the error message is disturbing. This is a totally non critical error. I use slurm2.5 and a trunk version of pyslurm.

Problem with pyslurm on example jobs_list.py

So now import pyslurm is working (ubuntu 12.04, slurm2.5.0, trunk of pyslurm, cython from pip install). So I am going through the different examples.

I see the partition, blocks_list.py send back No Blocks.

However :
python jobs_list.py
Traceback (most recent call last):
File "jobs_list.py", line 47, in
jobs = a.get()
File "pyslurm.pyx", line 1573, in pyslurm.pyslurm.job.get (pyslurm/pyslurm.c:20204)
File "pyslurm.pyx", line 1669, in pyslurm.pyslurm.job.__get (pyslurm/pyslurm.c:21206)
File "pyslurm.pyx", line 4119, in pyslurm.pyslurm.get_conn_type_string (pyslurm/pyslurm.c:45363)
TypeError: an integer is required

Same thing for sjobs.py (normal, exactly the same error).

And with node_list.py :
arnaud@D3550:~/src/pyslurm/examples$ python node_list.py
python: error: Unsupported option 4 for get_nodeinfo.

python: error: Unsupported option 4 for get_nodeinfo.

D3550 :
alloc_cpus : 0
arch : x86_64
boards : 1355999036
boot_time : Thu Dec 20 10:23:56 2012
cores : 1
cpu_load : 116
cpus : 1
err_cpus : 0
features : []
gres : []
name : D3550
node_addr : 127.0.0.1
node_hostname : D3550
Traceback (most recent call last):
File "node_list.py", line 41, in
display(node_dict)
File "node_list.py", line 25, in display
print "\t%-17s : %s" % (part_key, pyslurm.get_node_state(value[part_key]))
File "pyslurm.pyx", line 3944, in pyslurm.pyslurm.get_node_state (pyslurm/pyslurm.c:43705)
TypeError: an integer is required

Regards,

Arnaud Laprévote

Create a PySlurm 2.4.3 branch

Problem with unicode characters in Slurm job names

Hello to all,

When jobs are submitted with names including unicode characters (perfectly allowed by slurm), then when doing a job.get on them, I get 👍

JobId=90 Name=dédé job
UserId=admin(9999) GroupId=admin(9999)
Priority=4294901746 Account=(null) QOS=(null)
JobState=COMPLETED Reason=None Dependency=(null)
Requeue=1 Restarts=0 BatchFlag=1 ExitCode=0:0
RunTime=00:00:01 TimeLimit=12:00:00 TimeMin=N/A
SubmitTime=2013-05-06T15:53:38 EligibleTime=2013-05-06T15:53:38
StartTime=2013-05-06T15:53:38 EndTime=2013-05-06T15:53:39
PreemptTime=None SuspendTime=None SecsPreSuspend=0
Partition=computes AllocNode:Sid=vpdev-master:24449
ReqNodeList=(null) ExcNodeList=(null)
NodeList=workstation-1-1
BatchHost=workstation-1-1
NumNodes=1 NumCPUs=1 CPUs/Task=1 ReqS:C:T=::*
MinCPUsNode=1 MinMemoryCPU=1024M MinTmpDiskNode=0
Features=(null) Gres=(null) Reservation=(null)
Shared=0 Contiguous=0 Licenses=(null) Network=(null)
Command=/tmp/tmppKlgGg
WorkDir=/tmp

import pyslurm
jobs=pyslurm.job()
jobs.get()
Traceback (most recent call last):
File "", line 1, in
File "pyslurm.pyx", line 1588, in pyslurm.pyslurm.job.get (pyslurm/pyslurm.c:20266)
File "pyslurm.pyx", line 1705, in pyslurm.pyslurm.job.__get (pyslurm/pyslurm.c:21489)
File "slurm.pxd", line 56, in pyslurm.slurm.stringOrNone (pyslurm/pyslurm.c:46456)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

Clearly, this is something that pyslurm does not like much. I have not tested, but I suppose that unicode character in the workdir will have the same consequences.

Add Slurm version check to setup.py

In order to make sure that the version of PySlurm matches the version of Slurm we need to add a version check in setup.py. One possible way is to check the version hex string from slurm.h.

Correct preempt mode decode

The following example is different from the output of scontrol - should be simple to resolve

mark@makalu:~/pyslurm/examples> python partition_list.py
debug :
    allow_alloc_nodes    : []
    allow_groups         : []
    alternate            : None
    def_mem_per_cpu      : 0
    default_time         : (4294967294L, 'no_value')
    flags                : (1, {u'Default': 1, u'Shared': u'EXCLUSIVE', u'Hidden': 0, u'DisableRootJobs': 0, u'RootOnly': 0})
    grace_time           : 0
    last_update          : Wed Dec 26 20:09:42 2012
    max_mem_per_cpu      : 0
    max_nodes            : Unlimited
    max_share            : 1
    max_time             : Unlimited
    min_nodes            : 1
    nodes                : ['makalu', 'shivling']
    preempt_mode         : (65534, 'GANG,UNKNOWN')
    priority             : 1
    state_up             : (3, u'Allocated')
    total_cpus           : 2
    total_nodes          : 2
--------------------------------------------------------------------------------

Partition IDs - [u'debug']

mark@makalu:~/pyslurm/examples> scontrol show partition
PartitionName=debug
AllocNodes=ALL AllowGroups=ALL Default=YES
DefaultTime=NONE DisableRootJobs=NO GraceTime=0 Hidden=NO
MaxNodes=UNLIMITED MaxTime=UNLIMITED MinNodes=1
Nodes=makalu,shivling
Priority=1 RootOnly=NO ReqResv=NO Shared=NO PreemptMode=GANG,SUSPEND
State=UP TotalCPUs=2 TotalNodes=2 DefMemPerNode=UNLIMITED
MaxMemPerNode=UNLIMITED

pyslurm / pyslurm Goto Github PK

pyslurm's Introduction

Requirements

Versioning

Installation

Contributors

Support

pyslurm's People

Contributors

Stargazers

Watchers

Forkers

pyslurm's Issues

work_dir : /home/arnaud/src/slurmjob

python: error: Unsupported option 4 for get_nodeinfo.

Recommend Projects

Recommend Topics

Recommend Org