IBM LoadLeveler for AIX 5L: Using and Administering

The LoadLeveler job cycle

Figure 4 illustrates the information flow through the LoadLeveler cluster:

Figure 4. High-level job flow

View figure.

The managing machine in a LoadLeveler cluster is known as the central manager. There are also machines that act as schedulers, and machines that serve as the executing machines. The arrows in Figure 4 illustrate the following:

• Arrow 1 indicates that a job has been submitted to LoadLeveler.
• Arrow 2 indicates that the scheduling machine contacts the central manager to inform it that a job has been submitted, and to find out if a machine exists that matches the job requirements.
• Arrow 3 indicates that the central manager checks to determine if a machine exists that is capable of running the job. Once a machine is found, the central manager informs the scheduling machine which machine is available.
• Arrow 4 indicates that the scheduling machine contacts the executing machine and provides it with information regarding the job.

Figure 4 is broken down into the following more detailed diagrams illustrating how LoadLeveler processes a job.

1. Submit a LoadLeveler job:

   Figure 5. Job is submitted to LoadLeveler
   
   View figure.
   

   Figure 5 illustrates that the schedd daemon runs on the scheduling machine. This machine can also have the startd daemon running on it. The negotiator daemon resides on the central manager machine. The arrows in Figure 5 illustrate the following:

   • Arrow 1 indicates that a job has been submitted to the scheduling machine.
   • Arrow 2 indicates that the schedd daemon, on the scheduling machine, stores all of the relevant job information on local disk.
   • Arrow 3 indicates that the schedd daemon sends job description information to the negotiator daemon.
2. Permit to run:

   Figure 6. LoadLeveler authorizes the job
   
   View figure.
   

   In Figure 6, arrow 4 indicates that the negotiator daemon authorizes the schedd daemon to begin taking steps to run the job. This authorization is called a permit to run. Once this is done, the job is considered Pending or Starting. (See LoadLeveler job states for more information.)

3. Prepare to run:

   Figure 7. LoadLeveler prepares to run the job
   

   View figure.

   In Figure 7, arrow 5 illustrates that the schedd daemon contacts the startd daemon on the executing machine and requests that it start the job. The executing machine can either be a local machine (the machine from which the job was submitted) or a remote machine (another machine in the cluster).

4. Initiate job:

   Figure 8. LoadLeveler starts the job
   
   View figure.
   

   The arrows in Figure 8 illustrate the following:

   • The two arrows numbered 6 indicate that the startd daemon on the executing machine, spawns a starter process and awaits more work.
   • The two arrows numbered 7 indicate that the schedd daemon sends the starter process the job information and the executable.
   • Arrow 8 indicates that the schedd daemon notifies the negotiator daemon that the job has been started and the negotiator daemon marks the job as Running. (See LoadLeveler job states for more information.)

   The starter forks and executes the user's job, and the starter parent waits for the child to complete.

5. Complete job:

   Figure 9. LoadLeveler completes the job
   
   View figure.
   

   The arrows in Figure 9 illustrate the following:

   • The arrows numbered 9 indicate that when the job completes, the starter process notifies the startd daemon, and the startd daemon notifies the schedd daemon.
   • Arrow 10 indicates that the schedd daemon examines the information it has received and forwards it to the negotiator daemon.

LoadLeveler job states

As LoadLeveler processes a job, the job moves through various states. Possible job states are listed in Table 2 and detailed in the appendix under Job states. For more information about the daemons controlling these job states see Daemons.

Table 2. Job states