Using MPICH with Gigabit Ethernet

MPICH 1.2.6 is only intended for use if there are problems with ChaMPIon/Pro. MPICH is built with the (default) Intel 9 compilers, so mpicc/mpif77 will accept the Intel compiler options (see the man pages for ifort, icc and icpc for details). This version of MPICH runs on TCP/IP over Tungsten's Gigabit Ethernet network.

To use MPICH, add it to your environment with the softenv key:

  soft add +mpich-tcp-1.2.6-intel9

If you use MPICH to compile a program, you also need to add the above command in your batch job script before the program is run.

See SoftEnv for more information about changing the user environment.

To compile your program with MPICH, use mpicc, mpicxx, mpif77 or mpif90 (see the man pages for details).

To run an MPI program with MPICH, use the mpirun command in a batch job. Note that two environment variables are automatically set to simplify the mpirun command:

$NPROCS

is the number of processors requested by the batch job (#BSUB -n command).

$LSB_NODEFILE

The name of the machinefile used by the mpirun command.

For more information about running jobs, see Running Jobs and see the sample batch script below.

#!/bin/csh
#BSUB -n 4              # Use 4 processors
#BSUB -W 1:00           # Specify job run time limit of 1 hour

#BSUB -o testjob.%J.o   # Store the standard output and standard error of the
                        # job in file testjob.jobid.o (optional)

#BSUB -J testjob        # Specify job name (optional)
# End of embedded BSUB options
#

cd $SCR                                 # Change to job scratch directory
msscmd cd test,get a.out,get test.input # Get the files a.out and test.input
                                        # from directory test in UniTree

chmod u+x a.out                         # Since UniTree gives you the same
                                        # permissions as what your umask is
                                        # set to (generally only read and
                                        # write), you need to give yourself
                                        # executable permission to run a.out

soft add +mpich-tcp-1.2.6-intel9      # add mpich to your environment

mpirun -np $NPROCS -machinefile $LSB_NODEFILE a.out  # Run executable

msscmd cd test, put test.output         # Put output file generated by run in
                                        # UniTree