Jump to content
  • Announcements

    • admin

      PBS Forum Has Closed   06/12/17

      The PBS Works Support Forum is no longer active.  For PBS community-oriented questions and support, please join the discussion at http://community.pbspro.org.  Any new security advisories related to commercially-licensed products will be posted in the PBS User Area (https://secure.altair.com/UserArea/). 

CSCFCEM

Members
  • Content count

    4
  • Joined

  • Last visited

  1. So while trying to configure some queues for another problem (http://forum.pbsworks.com/index.php?/topic/631-pbs-queue-for-limited-walltime-and-exclusive-nodes/), I seem to have hit some unexpected behavior. I was attempting to assign exclusive nodes to a queue. The system I use has 32 cores, on 4 nodes. I want our "small" queue to use 2 nodes. Our server has the nodes listed as server[0], server[1], server[2], and server[3]. We have a route queue (as default queue) set up as well to use the small queue. What we tried was this (following from examples seen in the linked problem and elsewhere): set node server[0] queue = small set node server[1] queue = small I assigned a walltime limit (max of 8 hours) for this queue as well. I then submitted 3 jobs without specifying the queue (only walltime) and the following happened: job1 (<8 hours requested walltime) ---> immediately runs on small queue on node server[3] job2 (>8 hours requested walltime) ---> immediately runs on other queue on node server[2] job3 (<8 hours requested walltime) ---> not enough resources, job is queued I was expecting job1 to go to small queue on node server[0] or server[1], and I'd expect job3 to as well. It is as if the set node command has blocked me from assigning any jobs to those nodes. Any idea as to what went wrong?
  2. Thanks again BrianL, As usual, new solutions present new special cases. Now we are starting to wonder what happens when our q2 fills up, and q2 "qualified" jobs get put into q3 (let's say behind some really big job). Eventually q2 clears up, there is a job in q3 that is qualified to run in q2, is there a mechanism for PBS to automatically bring it over to q2 once it has room? In general, is there a way for PBS to re-sort between queues (not within itself). I am aware of qalter, but was thinking something that PBS would do automatically (perhaps based on an exit event from q2, although I don't think an epilogue can change other jobs in queues)? I am familiar with the job_sort_formula, but I'm pretty sure that is only for jobs in a queue, not for sorting between queues (correct me if I'm wrong). I'm guessing this will have to involve "peer scheduling". Also, it appears that queued_jobs_threshold is PBSPro 13.0 and later, so I think it's a good time to mention I'm using 11.3 (better late than never I suppose). However I think max_queued should suffice. Thank you.
  3. Looks great @BrianL, it looks like you addressed any concerns I had. I think the part I was missing was having multiple queues assigned to the reroute queue (sounds like a strange thing to worry about in hindsight). I do however have another question now. Let's say I submit multiple jobs that would get rerouted to q2 in your example. As a result, q2 is now full. If another job that would get redirected to q2 is submitted, and say q3 is empty, would that job sit and wait for q2? Or would it go to where it can run (in q3). I would like to implement that second scenario. The example we were using here in the office... imagine you own a grocery store... you want there to be an express lane for shoppers with a few items, but if that line was too long, and a regular lane was open, a shopper with few items should be able to go into the regular lane. I don't want to FORCE all the jobs into q2, just like express lanes shouldn't force all shoppers with a few items to use it. Hope that makes sense. Is that possible? Will PBS reroute small jobs to another queue that has room if it was originally rerouted to a full queue? Thanks again.
  4. I had a question about setting up a queue on exclusive nodes and accepting jobs with limited walltime. I am more interseted in "what happens when..." for this scenario. I saw this example (http://forum.pbsworks.com/index.php?/topic/251-assign-exclusive-nodes-to-a-queue/) for setting exclusive nodes to a queue, and it is very clear to me. I read through the documentation for assigning a max walltime for a queue as well using a route queue. This also makes sense to me. It's combining these that has me scratching my head. So let's say I want a queue with 4 exclusive nodes (but my system has plenty other nodes), but to only accept jobs with a walltime less than 6 hours. If I set up the queue to with the exclusive nodes I want, and set up a routing queue to send all jobs with less than 6 hours of requested walltime to that same queue, what will happen if I submit a job requesting more than 4 nodes but less than 6 hours of walltime. Will the job get rejected for requesting too many resources? Will PBS be smart enough to just send it to a queue with non exclusive nodes (which would be most desirable)? Or will I need to create a hook to handle this? In other words, I want a queue with N exclusive nodes than only accepts jobs with requested walltime less than M, but if a jobs has a requested walltime less than M, but requests more than N nodes, I don't want to to get rejected for there not being enough resources. Is this a legitamit concern, or does PBS handle this appropriately without hooks (or any other intervention).
×