Jump to content
  • Announcements

    • admin

      PBS Forum Has Closed   06/12/17

      The PBS Works Support Forum is no longer active.  For PBS community-oriented questions and support, please join the discussion at http://community.pbspro.org.  Any new security advisories related to commercially-licensed products will be posted in the PBS User Area (https://secure.altair.com/UserArea/). 
Sign in to follow this  
Chiara Latini

Python APIs?

Recommended Posts

Hi,

I'm developing a plug-in for PBS-Pro jobs' submission, control and monitor, including the advance reservations functionality.

My application is written in python. I could use the C API' s but I see in the programmer's guide there's also a python pbs_module, offering many functionalities. It seems it can only be used inside the "hook" scripts. Can I directly import that module in my application and use it to connect to the Pbs Server?

Thanks in advance for your help,

Chiara

Share this post


Link to post
Share on other sites

Hello,

I've found that SWIG-PBS Professional interface is working pretty well for me, thought I post some notes here on how it can be used.

What's SWIG?

SWIG is an acronym for Simplified Wrapper and Interface Generator. It's a tool for providing various scripting languages with interfaces to code written in C or C++.

How Do I Use It?

Thanks to the interface information included in PBS Professional it's now very simple to use SWIG with PBS Professional. Ignoring for the moment any complex interface (caused by C language constructs in the PBS Professional APIs that do not have exact translations to Python), one can build a Python module with interfaces to PBS Professional using this interface file

/* File : pbs.i */
%module pbs
%{
#define SWIG_FILE_WITH_INIT
#include "pbs_ifl.h"
%}
%include "pbs_ifl.h"

and compiling it using

pypath="/usr/local/bin"            # path to python binary
pyincludepath="/usr/local/include/python2.5" # Python include path
conf=${PBS_CONF_FILE:-/etc/pbs.conf} # PBS configuration file

. $conf
swig -I$PBS_EXEC/include -python pbs.i
gcc -shared -fPIC -I$pyincludepath -I $PBS_EXEC/include pbs_wrap.c $PBS_EXEC/lib/libpbs.a -o _pbs.so -L/lib -lcrypto -lssl

If using an SELinux-configured version of Linux, you may also need to

chcon -t texrel_shlib_t _pbs.so

Example: a simple pbsnodes

Let's apply the above to build a very simple version of the pbsnodes utility, call it pbs_statnode.py. This one will support only the -a, -s, and -v flags.

from pbs import pbs_connect, pbs_stathost, pbs_statvnode
from optparse import OptionParser
import sys

def print_nodes(batch_status_list):
while batch_status_list != None:
print "%s" % batch_status_list.name
print_node(batch_status_list.attribs)
batch_status_list = batch_status_list.next

def print_node(attrlist):
while attrlist != None:
if attrlist.resource != None:
print " %s.%s = %s" % (attrlist.name, attrlist.resource,
attrlist.value)
else:
print " %s = %s" % (attrlist.name, attrlist.value)
attrlist = attrlist.next

if __name__ == '__main__':
parser = OptionParser()
parser.add_option("-a", "--all", action="store_true", dest="allnodes",
help="report all nodes")
parser.add_option("-s", "--server", action="store", dest="server",
default = "localhost",
help="server to query (default \"localhost\")")
parser.add_option("-v", "--vnodes", action="store_true", dest="do_vnodes",
help="report vnodes")
(options, progargs) = parser.parse_args()

conn = pbs_connect(options.server)
if conn < 0:
print "Error connecting to %s" % options.server
sys.exit(1)

if options.do_vnodes:
statfunc = pbs_statvnode
else:
statfunc = pbs_stathost
if options.allnodes:
print_nodes(statfunc(conn, "", None, None))
else:
for node in progargs:
print_nodes(statfunc(conn, node, None, None))

To use it, simply do

python pbs_statnode.py -as PBShost
PBShost
Mom = PBShost.example.com
ntype = PBS
state = free
license = u
pcpus = 1
resources_available.arch = hpux10
resources_available.host = PBShost.example.com
resources_available.mem = 1310720kb
resources_available.ncpus = 2
resources_available.vnode = PBShost.example.com
resources_assigned.mem = 0kb
resources_assigned.ncpus = 0
resources_assigned.vmem = 0kb
resv_enable = True
sharing = default_shared

Example: a simpler pbsnodes

import pbs 

conn = pbs.pbs_connect(<servername>)
b = pbs.pbs_statvnode(conn,<nodename>,None,None)
print b.name
attribs = b.attribs
while attribs!=None:
if attribs.resource != None:
print attribs.value
attribs = attribs.next

Complex interfaces: pbs_selectjob

Here is one example about generating complex interfaces using <pbs_ifl.h>:

extern char **pbs_selectjob (int, struct attropl *, char *);

We'll use the SWIG interface file and the Python C interfaces to teach SWIG how to treat the special char ** return type,

// This tells SWIG to treat char ** as a special case (for pbs_selectjob())
%typemap(out) char ** {
int size;
int i;
char **r;

if ($1 == NULL) {
PyRun_SimpleString("print \"%s%s is NULL\" % ( '$', '1')");
PyErr_SetString(PyExc_TypeError, "NULL $1");
}

for (size = 0, r = (char **) $1; *r != NULL; r++)
size++;
$result = PyTuple_New(size);
if ($result == NULL) {
PyRun_SimpleString("print \"PyTuple_New failure\"");
PyErr_SetString(PyExc_TypeError,"PyTuple_New() failure");
return NULL;
}
for (i = 0, r = (char **) $1; *r != NULL; i++, r++) {
PyObject *o;
o = PyString_FromString(*r);
if (PyTuple_SetItem($result, i, o) != 0) {
PyRun_SimpleString("print \"PyTuple_SetItem failure\"");
PyErr_SetString(PyExc_TypeError,"PyTuple_SetItem() failure");
return NULL;
}
}
return $result;
}

// This cleans up the char ** array we malloc'd before the function call
%typemap(freearg) char ** {
free((char *) $1);
}

/* File : pbs.i */
%module pbs
%{
#define SWIG_FILE_WITH_INIT
#include "pbs_ifl.h"
%}
%include "pbs_ifl.h"

After building as above, try it out using this code,

import pbs
import sys

def find_jobs(fd, state):
ao = pbs.attropl()
ao.next = None
ao.name = pbs.ATTR_state
ao.resource = None
ao.value = state
ao.op = pbs.EQ
return pbs.pbs_selectjob(fd, ao, None)

if __name__ == '__main__':
conn = pbs.pbs_connect("localhost")
if conn < 0:
print "Error connecting to localhost"
sys.exit(1)
print "job list: %s" % find_jobs(conn, 'R')

which we'll call pbs_selectjob.py and use like this,

echo sleep 60 | qsub
<jobID>
python pbs_selectjob.py

More information can be found into the PBS Professional Programmer's Guide:
/>http://www.pbsworks.com/documentation/support/PBSProProgramGuide11.1.pdf

Hope this helps,

Dario

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
Sign in to follow this  

×