IBM LoadLeveler for AIX 5L: Using and Administering

Additional examples of building job command files

Serial job command file gives you an example of a simple job command file. This section contains examples of building and submitting more complex job command files.

Example 1: Generating multiple jobs with varying outputs

To run a program several times, varying the initial conditions each time, you could can multiple LoadLeveler scripts, each specifying a different input and output file as described in Figure 43. It would probably be more convenient to prepare different input files and submit the job only once, letting LoadLeveler generate the output files and do the multiple submissions for you.

Figure 42 illustrates the following:

• You can refer to the LoadLeveler name of your job symbolically, using $(jobid) and $(stepid) in the LoadLeveler script file.
• $(jobid) refers to the job identifier.
• $(stepid) refers to the job step identifier and increases after each queue command. Therefore, you only need to specify input, output, and error statements once to have LoadLeveler name these files correctly.

Assume that you created five input files and each input file has different initial conditions for the program. The names of the input files are in the form longjob.in.x, where x is 0-4.

Submitting the LoadLeveler script shown in Figure 42 results in your program running five times, each time with a different input file. LoadLeveler generates the output file from the LoadLeveler job step IDs. This ensures that the results from the different submissions are not merged.

Figure 42. Job command file with varying input statements

# @ executable = longjob
# @ input = longjob.in.$(stepid)
# @ output = longjob.out.$(jobid).$(stepid)
# @ error = longjob.err.$(jobid).$(stepid)
# @ queue
# @ queue
# @ queue
# @ queue
# @ queue

To submit the job, type the command:

llsubmit longjob.cmd

LoadLeveler responds by issuing the following:

submit: The job "ll6.23" with 5 job steps has been submitted.

The following table shows you the standard input files, standard output files, and standard error files for the five job steps:

Example 2: Using LoadLeveler variables in a job command file

Figure 43 shows how you can use LoadLeveler variables in a job command file to assign different names to input and output files. This example assumes the following:

• The name of the machine from which the job is submitted is lltest1
• The user's home directory is /u/rhclark and the current working directory is /u/rhclark/OSL
• LoadLeveler assigns a value of 122 to $(jobid).

In Job Step 0:

• LoadLeveler creates the subdirectories oslsslv_out and oslsslv_err if they do not exist at the time the job step is started.

In Job Step 1:

• The character string rhclark denotes the home directory of user rhclark in input, output, error, and executable statements.
• The $(base_executable) variable is set to be the "base" portion of the executable, which is oslsslv.
• The $(host) variable is equivalent to $(hostname). Similarly, $(jobid) and $(stepid) are equivalent to $(cluster) and $(process), respectively.

In Job Step 2:

• This job step is executed only if the return codes from Step 0 and Step 1 are both equal to zero.
• The initial working directory for Step 2 is explicitly specified.

Figure 43. Using LoadLeveler variables in a job command file

# Job step 0 ============================================================
#   The names of the output and error files created by this job step are:
#
#     output: /u/rhclark/OSL/oslsslv_out/lltest1.122.0.out
#     error : /u/rhclark/OSL/oslsslv_err/lltest1_122_0_err
#
# @ job_name = OSL
# @ step_name = step_0
# @ executable = oslsslv
# @ arguments = -maxmin=min -scale=yes -alg=dual
# @ environment = OSL_ENV1=20000; OSL_ENV2=500000
# @ requirements = (Arch == "R6000") && (OpSys == "AIX43")
# @ input  = test01.mps.$(stepid)
# @ output = $(executable)_out/$(host).$(jobid).$(stepid).out
# @ error  = $(executable)_err/$(host)_$(jobid)_$(stepid)_err
# @ queue
#
# Job step 1 ============================================================
#   The names of the output and error files created by this job step are:
#
#     output: /u/rhclark/OSL/oslsslv_out/lltest1.122.1.out
#     error : /u/rhclark/OSL/oslsslv_err/lltest1_122_1_err
#
# @ step_name = step_1
# @ executable = rhclark/$(job_name)/oslsslv
# @ arguments = -maxmin=max -scale=no -alg=primal
# @ environment = OSL_ENV1=60000; OSL_ENV2=500000; \
                  OSL_ENV3=70000; OSL_ENV4=800000;
# @ input  = rhclark/$(job_name)/test01.mps.$(stepid)
# @ output = rhclark/$(job_name)/$(base_executable)_out/$(hostname).$(cluster).$(process).out
# @ error  = rhclark/$(job_name)/$(base_executable)_err/$(hostname)_$(cluster)_$(process)_err
# @ queue
#
# Job step 2 ============================================================
#   The names of the output and error files created by this job step are:
#
#     output: /u/rhclark/OSL/oslsslv_out/lltest1.122.2.out
#     error : /u/rhclark/OSL/oslsslv_err/lltest1_122_2_err
#
# @ step_name = OSL
# @ dependency = (step_0 == 0) && (step_1 == 0)
# @ comment = oslsslv
# @ initialdir = /u/rhclark/$(step_name)
# @ arguments = -maxmin=min -scale=yes -alg=dual
# @ environment = OSL_ENV1=300000; OSL_ENV2=500000
# @ input  = test01.mps.$(stepid)
# @ output = $(comment)_out/$(host).$(jobid).$(stepid).out
# @ error  = $(comment)_err/$(host)_$(jobid)_$(stepid)_err
# @ queue

Example 3: Using the job command file as the executable

The name of the sample script shown in Figure 44 is run_spice_job. This script illustrates the following:

• The script does not contain the executable keyword. When you do not use this keyword, LoadLeveler assumes that the script is the executable. (Since the name of the script is run_spice_job, you can add the executable = run_spice_job statement to the script, but it is not necessary.)
• The job consists of four job steps (there are 4 queue statements). The spice3f5 and spice2g6 programs are invoked at each job step using different input data files:
  • spice3f5: Input for this program is from the file spice3f5_input_x where x has a value of 0, 1, and 2 for job steps 0, 1, and 2, respectively. The name of this file is passed as the first argument to the script. Standard output and standard error data generated by spice3f5 are directed to the file spice3f5_output_x. The name of this file is passed as second argument to the script. In job step 3, the names of the input and output files are spice3f5_input_benchmark1 and spice3f5_output_benchmark1, respectively.
  • spice2g6: Input for this program is from the file spice2g6_input_x. Standard output and standard error data generated by spice2g6 together with all other standard output and standard error data generated by this script are directed to the files spice_test_output_x and spice_test_error_x, respectively. In job step 3, the name of the input file is spice2g6_input_benchmark1. The standard output and standard error files are spice_test_output_benchmark1 and spice_test_error_benchmark1.

  All file names that are not fully qualified are relative to the initial working directory /home/loadl/spice. LoadLeveler will send the job steps 0 and 1 of this job to a machine for that has a real memory of 64 MB or more for execution. Job step 2 most likely will be sent to a machine that has more that 128 MB of real memory and has the ESSL library installed since these preferences have been stated using the LoadLeveler preferences keyword. LoadLeveler will send job step 3 to the machine ll5.pok.ibm.com for execution because of the explicit requirement for this machine in the requirements statement.

Figure 44. Job command file used as the executable

#!/bin/ksh
# @ job_name = spice_test
# @ account_no = 99999
# @ class = small
# @ arguments = spice3f5_input_$(stepid) spice3f5_output_$(stepid)
# @ input  = spice2g6_input_$(stepid)
# @ output = $(job_name)_output_$(stepid)
# @ error  = $(job_name)_error_$(stepid)
# @ initialdir = /home/loadl/spice
# @ requirements = ((Arch == "R6000") && \
#           (OpSys == "AIX43") && (Memory > 64))
# @ queue
# @ queue
# @ preferences  = ((Memory > 128)  && (Feature == "ESSL"))
# @ queue
# @ class = large
# @ arguments = spice3f5_input_benchmark1 spice3f5_output_benchmark1
# @ requirements = (Machine == "ll5.pok.ibm.com") 
# @ input  = spice2g6_input_benchmark1
# @ output = $(job_name)_output_benchmark1
# @ error  = $(job_name)_error_benchmark1
# @ queue
OS_NAME=  `uname` 
 
case $OS_NAME in
   AIX)
      echo "Running $OS_NAME version of spice3f5" > $2
      AIX_bin/spice3f5 < $1  >> $2  2>&1
      echo "Running $OS_NAME version of spice2g6"
      AIX_bin/spice2g6
      ;;
   *)
      echo "spice3f5 for $OS_NAME is not available" > $2
      echo "spice2g6 for $OS_NAME is not available"
      ;;
esac