[Rocks-Discuss] Stop Interaction between different queues
bbrashers at geomatrix.com
Mon Jan 29 09:51:51 PST 2007
Hi Roy and brat,
I'm sorry if it discussed earlier.
I created new queues(long, short and interactive). Each queue
having individual nodes. when I submitted using qsub -q <queue name>
<job file>, jobs are going to individual nodes correctly. But i am
facing some difficulties.
Assume following queues have the particular nodes
queue long = node01, node02
queue short = node03
queue interactive = node04, node05
1. i submitted job interactively and job went to particular
node04. i ran the job interactively using following command
mpirun -machinefile hosts -np 2 ./jobfile.
hosts file having node04 & node05. Jobs are running fine. If i
change the host file to node01 & node02, jobs are running on node01 &
node02. But node01 & node02 are assigned for queue long.
I want to run particular queue to particular nodes. I dont want
people to hack other nodes.
You should be using mpiexec, not mpirun. mpiexec knows about which
nodes have been assigned (by PBS) to the job, and uses those
automatically without the need to specify "-machinefile hosts". Just
use "mpiexec -np 2 jobfile" in your submission script.
2. I want to define the interactive queue only for interactive
mode. I should not submit job, through interactive mode.
for example qsub -q interactive jobfile, now jobs started
3. if i submit job using, qsub -q long -I -l nodes=1:ppn=2,
then jobs started in interactive mode using queue long. I want to use
long and short queue for submitting jobs(non-interactive mode) and
interactive queue for only interactive jobs.
Since you requested an interactive job (by using the -I flag in qsub)
PBS did what you asked. If you want to prevent interactive jobs on a
particular set of nodes, you should be able to set some resources_max
http://clusterresources.com/torquedocs20/4.1queueconfig.shtml, or node
However, it's very hard to prevent all crafty users from circumventing
your rules. They'll find ways to work around almost all the rules you
put in place. Better to cultivate good relationships with your users,
and make them want to follow the rules.
I also don't see a routing queue in your set-up. Build a routing queue,
and make the queues long, short and interactive not be available for
direct submission. You may be able to do this by using acl_users on
your 3 execution queues -- set them to empty and no one can submit jobs
directly to those queues. Then have your users submit only to the
default routing queue, and it will sort (based on resources_max and
resources_min) where to send the job.
Roy can probably suggest a better way to accomplish what you want, or
try to the Torque and/or Maui mailing lists:
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the npaci-rocks-discussion