[Rocks-Discuss]Compile node and serial nodes?

Roy Dragseth Roy.Dragseth at cc.uit.no
Wed Aug 31 04:25:35 PDT 2005


Hi.  See my post:

https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/2005-February/010345.html

on how to pipe ssh requests through the frontends firewall.  You don't even 
need any extra physical network connections to do this.

To exclude a node from running any batch jobs you just create a standing 
reservation on it.  put this is /opt/maui/maui.cfg

SRCFG[compile] PERIOD=INFINITY HOSTLIST=compute-0-0

To reserve some nodes for serial jobs the easiest way is to let the default 
queue be a routing queue in pbs and set up other queues that have max limits 
on the number of cpus in consecutive order, then you assign nodes to these 
queues with standing reservations in maui:

Put this in a text file and send it through qmgr:

#
# Create and define queue default
#
create queue default
set queue default queue_type = Route
set queue default route_destinations = serial
set queue default route_destinations += parallel
set queue default enabled = True
set queue default started = True
#
# Create and define queue serial
#
create queue parallel
set queue parallel queue_type = Execution
set queue express resources_max.nodes = 1
set queue parallel enabled = True
set queue parallel started = True
#
# Create and define queue parallel
#
create queue parallel
set queue parallel queue_type = Execution
set queue parallel enabled = True
set queue parallel started = True

(you can put this instead of the definition of the default queue 
in /opt/torque/pbs.default)

Then the standing reservations in maui is done like this

SRCFG[serial] PERIOD=INFINITY HOSTLIST=compute-0-[123] CLASSLIST=serial
SRCFG[parallel] PERIOD=INFINITY HOSTLIST=compute-0-[4567] CLASSLIST=parallel

this will reserve compute-0-1, ..,compute-0-3 to serial and 
compute-0-4,...,compute-0-6 to parallel.

I haven't tested this specific setup myself, it is an excerpt from one of the 
systems I'm responsible for.  You don't need to specify which queue you want 
to run in, the routing will take of the selection for you.

Hope this is what you were looking for,
r.

-- 

  The Computer Center, University of Tromsø, N-9037 TROMSØ Norway.
	      phone:+47 77 64 41 07, fax:+47 77 64 41 00
     Roy Dragseth, High Performance Computing System Administrator
	 Direct call: +47 77 64 62 56. email: royd at cc.uit.no



More information about the npaci-rocks-discussion mailing list