[Rocks-Discuss] question about environment modules on Rocks

Steve Jones stevejones at stanford.edu
Wed Feb 10 14:14:37 PST 2010


----- "Steve Jones" <stevejones at stanford.edu> wrote:

> > >>> I have installed the environment-modules roll from
> > rocks.stanford.edu
> > >>> and am trying to learn how to make good use of it. I have a
> > question
> > >>> about how the environment is passed along for jobs. Let's say
> that
> > I
> > >>> have two versions of a program and that I want to run jobs for
> > each
> > >>> version. The program expects to have a $PROGRAMHOME environment
> > >>> variable
> > >>> as well as a $PATH variable so the same variables have to point
> > to
> > >>> two
> > >>> different locations for the respective versions of the program.
> > >>> Furthermore, they each use MPI and of course will be controlled
> > by
> > >>> SGE.
> > >>> What is the best way to handle the environment for this? Would
> I
> > call
> > >>> module to load the appropriate environment in the SGE job
> scripts
> > or
> > >>> would I to tell SGE to pass along the required environment
> > variables
> > >>> via
> > >>> either the -v variable=value or -V options to qsub? Of course,
> > the
> > >>> environment has to be passed to the hosts of the MPI processes,
> > using
> > >>> OpenMPI.
> > >>>
> > >>> Thanks.
> > >>>
> > >>> -- 
> > >>> Glenn Johnson <glenn.johnson at ars.usda.gov>
> > >>>       
> > >> Hi.
> > >>
> > >> Here's a link to the modulefile man page:
> > http://modules.sourceforge.net/man/modulefile.html
> > >>
> > >> Let's say I have two versions of a program, foo-v1 and foo-v2. I
> > could do something like this:
> > >>
> > >> # mkdir /share/apps/modules/modulefiles/foo
> > >>
> > >> In the foo directory I would create two module files, v1 and v2.
> > Here is what v1 would look like:
> > >>
> > >>
> >
> #%Module1.0#####################################################################
> > >> ##
> > >> ## foo-v1 modulefile
> > >> ##
> > >> proc ModulesHelp { } {
> > >>         global version
> > >>
> > >>         puts stderr "\tThis module provides support for the"
> > >>         puts stderr "\tfoo-v1 program."
> > >> }
> > >>
> > >> conflict vasp
> > >>
> > >> module-whatis   "Provides foo-v1 program"
> > >> # for Tcl script use only
> > >> set     version      "3.2.6"
> > >>
> > >> conflict foo
> > >>
> > >> setenv PROGRAMHOME /share/apps/foo/v1
> > >> prepend-path PATH /share/apps/mvapich/custom-version/bin
> > >>
> > >> The example above sets the $PROGRAMHOME variable, loads my
> custom
> > version of MVAPICH, and has a conflict allowing only one version of
> > foo to be loaded at a time.
> > >>
> > >> Modules is very flexible.
> > >>
> > >> Hope this helps.
> > >>
> > >> Steve
> > >>     
> > >
> > > Thanks, but I was actually asking about how to handle the
> > environment
> > > with the module files in hand. So when I want to run calculations
> > with
> > > foo-v1 and foo-v2, do I load the respective module in the SGE job
> > script
> > > so that the compute host gets the environment or load the module
> I
> > want,
> > > and then use qsub -V to pass the environment to the host that
> will
> > > actually run the job? Perhaps it does not really matter but my
> main
> > > concern is having the environment properly passed to the MPI
> > processes.
> > >
> > > Thanks.
> > >
> > >   
> > I have never had much luck running the module command within SGE
> > scripts 
> > because it is actually an alias and causes some strange behavior.  I
> 
> > always do it this way.  Add the -V to the job script, load the
> version
> > 1 
> > module,  run the job, unload the module, load the version 2 module,
> > and 
> > run the job.  You can actually use shell scripts to do this  load
> the
> > 
> > module and qsub the job script from within a shell script.
> > 
> > Scott
> 
> Hi.
> 
> This issue with modules alias in interactive and non-interactive batch
> sessions has been resolved some time ago, though I probably never made
> an announcement. I added code to the logon scripts fixing it shown
> here for 5.3:
> 
> http://rocks.stanford.edu/trac/ENVIRONMENT-modules-5.3/browser/nodes/modules-base.xml
> 
> I've tested with Torque/Maui and Torque/Moab, but not with SGE as we
> don't use it. The person who came up with the fix is an SGE user and
> stated he's tested on SGE 6.2u4.
> 
> Steve

Hi.

An additional note, you might try something like this in your script:

module purge
module load intel/intel-11 mpi/mpi-ver foo/v1

This way you have complete control over which modules are loaded by first purging.

Hope this helps.

Steve


More information about the npaci-rocks-discussion mailing list