IBM LoadLeveler for AIX 5L: Using and Administering

llq - Query job status

Purpose

Returns information about job steps in the LoadLeveler queues.

Syntax

llq [-?] [-H] [-v] [-x] [-s] [ -l] [-w] [joblist] [-u userlist] [-h hostlist] [-c classlist] [-f category_list] [-r category_list]

Flags

-?

Provides a short usage message.

-H

Provides extended help information.

-v

Outputs the name of the command, release number, service level, service level date, and operating system used to build the command.

-x

Provides extended information about the selected job. If the -x flag is used with the -r, -s, or -f flag, an error message is generated.

CPU usage and other resource consumption information on active jobs can only be reported using the -x flag if the LoadLeveler administrator has enabled it by specifying A_ON and A_DETAIL for the ACCT keyword in the LoadLeveler configuration file.

Normally, llq connects with the central manager to obtain job information. When you specify -x, llq connects to the schedd machine that received the specified job to get extended job information. However, some statistics, including those corresponding to System Priority and q_sysprio, are available only from the central manager. Do not use the -x option if you need these statistics.

When specified without -l, CPU usage for active jobs is reported in the short format.

Note:

Using both the -l and -x options without a joblist specification can produce a very long report and excessive network traffic.

-s

Provides information on why a selected list of jobs remain in the NotQueued, Idle or Deferred state. Along with this flag, users must specify a list of jobs. The user can also optionally supply a list of machines to be considered when determining why the jobs cannot run. If a list of machines is not provided, the default is the list of machines in the LoadLeveler cluster. For each job, llq determines why the job remains in one of the given states instead of Running.

-l

Specifies that a long listing be generated for each job for which status is requested. Fields included in the long listing are shown in Results.

If -l is not specified, then the standard listing is generated as shown in Results.

-w

Provides AIX Workload Manager (WLM) CPU and real memory statistics for jobs in the running state. This flag can be used with a joblist, steplist or a single stepid. All other flags except -h will result in an error message.

When the -w flag is augmented with a single stepid, the -h flag can be used in conjunction with -w to specify a single hostname.

The following statistics are displayed for every node the job is running on:

Current CPU resource consumption as a percentage of the total resources available
Total CPU time consumed in milliseconds
Current real memory consumption as a percentage of the total resources available
The highest number of resident memory pages used

joblist

Is a blank-delimited list of jobs of the form host.jobid.stepid where:

host is the name of the schedd machine to which the job was submitted (delimited by dot). The default is the local machine.
If the job was submitted from a submit-only machine, this is the name of the machine where the schedd daemon that sent the job to the negotiator resides.
jobid is the job id assigned to the job when it was submitted using the llsubmit command.
stepid (delimited by dot) Is the step id assigned to the job when it was submitted using the llsubmit command. The default is to include all the job steps of the job associated with the jobid.

-u userlist

Is a blank-delimited list of users. When used with the -h option, only the user's jobs monitored on the machines in the hostlist are queried. When used alone, only the user's jobs monitored on the schedd machine are queried.

-h hostlist

Is a blank-delimited list of machines. If the -s flag is not specified, all jobs monitored on machines in this list are queried. If the -s flag is specified, the list of machines is considered when determining why a job remains in Idle state. When issued with the -u option, the userlist is used to further select jobs for querying.

-c classlist

Is a blank-delimited list of classes. When used with -h, only those jobs monitored on the machines in the hostlist are queried.

-f category_list

Is a blank-delimited list of categories you want to query. Each category you specify must be preceded by a percent sign. The category_list cannot contain duplicate entries. This flag allows you to create a customized version of the standard llq listing. You cannot use this flag with the -l flag. The output fields produced by this flag all have a fixed length. The output is displayed in the order in which you specify the categories. category_list can be one or more of the following:

%a: Account number
%c: Class
%cc: Completion code
%dc: Completion date
%dd: Dispatch Date
%dh: Hold date
%dq: Queue date
%gl: LoadLeveler group
%gu: UNIX group
%h: Hostname (first hostname if more than one machine is allocated to the job step)
%id: Step ID
%is: Virtual image size
%jn: Job name
%jt: Job type
%nh: Number of hosts allocated to the job step
%o: Job owner
%p: User priority
%sn: Step name
%st: Status

-r category_list: Is a blank-delimited list of formats (categories) you want to query. Each category you specify must be preceded by a percent sign. The category_list cannot contain duplicate entries. This flag allows you to create a customized version of the standard llq listing. You cannot use this flag with the -l flag. The output produced by this flag is considered raw, in that the fields can be variable in length. Output fields are separated by an exclamation point (!). The output is displayed in the order in which you specify the formats. category_list can be one or more of the formats listed under the -f flag.

If the -u or -h options are not specified, and if no jobid is specified, then all jobs are queried.

The -u and -h options override the jobid parameters.

Examples

This example generates a long listing for job 8, job step 2 submitted to machine gold:

llq -l gold.8.2

This example generates a standard listing for all job steps of job name 12 submitted to the local machine:

llq 12

Results

Standard listing: The standard listing is generated when you do not specify the -l option with the llq command. The following is sample output from the llq -h mars command, where the machine mars has two jobs running and one job waiting:

+--------------------------------------------------------------------------------+
|Id                       Owner      Submitted   ST PRI Class        Running On  |
|------------------------ ---------- ----------- -- --- ------------ ----------- |
|mars.498.0               brownap     5/20 11:31 R  100 silver       mars        |
|mars.499.0               brownap     5/20 11:31 R  50  No_Class     mars        |
|mars.501.0               brownap     5/20 11:31 I  50  silver                   |
|                                                                                |
|3 job step(s) in query, 1 waiting, 0 pending, 2 running, 0 held, 0 preempted    |
+--------------------------------------------------------------------------------+

The standard listing includes the following fields:

Id: Job identifier presented in the format: host.jobid.stepid. If the llq command returns information about a job owned by a schedd in the same domain, then the domain of the hostname will not appear in the output. However, when the llq command reports information about a job owned by a schedd in a different domain, the fully qualified hostname is always included. Due to space limitations, the domain of the host may be truncated to fit in the space allocated to the Id field. If the domain is truncated, a dash (-) will appear at the end to indicate that characters have been left out. To see the full job ID, run llq with the -l flag.
Owner: Userid of the job submitter.
Submitted: Date and time of job submission.
ST: Current job status (state). Job status can be:

C: Completed
CA: Canceled
CK: Checkpointing
CP: Complete Pending
D: Deferred
E: Preempted
EP: Preempt Pending
H: User Hold
HS: User Hold and System Hold
I: Idle
MP: Resume Pending
NR: Not Run
NQ: Not Queued
P: Pending
R: Running
RM: Removed
RP: Remove Pending
S: System Hold
ST: Starting
SX: Submission Error
TX: Terminated
V: Vacated
VP: Vacate Pending
X: Rejected
XP: Reject Pending

For a detailed explanation of job states, see Job states.

PRI: User priority of the job step, where the values are defined with the user_priority keyword in the job command file or changed by the llprio command. See llprio - Change the user priority of submitted job steps
Class: Job class.
Running On: If running, the name of the machine the job step is running on. This is blank when the job is not running. For a parallel job step, only the first machine is shown.

Customized, formatted standard listing: A customized and formatted standard listing is generated when you specify llq with the -f flag. The following is sample output from this command:

   llq -f %id %c %dq %dd %gl %h

+--------------------------------------------------------------------------------+
|Step Id           Class      Queue Date  Disp. Date  LL Group   Running On      |
|----------------- ---------- ----------- ----------- ---------- --------------- |
|ll6.2.0           No_Class   04/08 09:19 04/08 09:21 No_Group   ll6.pok.ibm.com |
|ll6.1.0           No_Class   04/08 09:19 04/08 09:21 No_Group   ll6.pok.ibm.com |
|ll6.3.0           No_Class   04/08 09:19 04/08 09:21 No_Group   ll5.pok.ibm.com |
|                                                                                |
|3 job step(s) in queue, 0 waiting, 0 pending, 3 running, 0 held, 0 preempted    |
|                                                                                |
+--------------------------------------------------------------------------------+

Customized, unformatted standard listing: A customized and unformatted (raw) standard listing is generated when you specify llq with the -r flag. Output fields are separated by an exclamation point (!). The following is sample output from this command:

   llq -r %id %c %dq %dd %gl %h

+--------------------------------------------------------------------------------+
|ll6.pok.ibm.com.2.0!No_Class!04/08/2001 09:19!04/08/2001 09:21!No_Group!ll6.pok.ibm&|
|ll6.pok.ibm.com.1.0!No_Class!04/08/2001 09:19!04/08/2001 09:21!No_Group!ll6.pok.ibm&|
|ll6.pok.ibm.com.3.0!No_Class!04/08/2001 09:19!04/08/2001 09:21!No_Group!ll5.pok.ibm&|
+--------------------------------------------------------------------------------+

WLM CPU and real memory statistics listing: If the LoadLeveler interface to AIX Workload Manager (WLM) is enabled, then the -w option can be used to obtain CPU and real memory statistics of job steps in running state. The following is the output of "llq -w c209f1n05.13.0" where c209f1n05.13.0 is a CPU intensive parallel job step currently running on the 2 nodes c209f1n05 and c209f1n01:

+--------------------------------------------------------------------------------+
|   =============== Job Step c209f1n05.ppd.pok.ibm.com.13.0 ===============      |
|   c209f1n05.ppd.pok.ibm.com:                                                   |
|           Resource: CPU                                                        |
|                   snapshot: 99                                                 |
|                   total: 80172                                                 |
|           Resource: Real Memory                                                |
|                   snapshot: 1                                                  |
|                   high water: 2561                                             |
|                                                                                |
|   c209f1n01.ppd.pok.ibm.com:                                                   |
|           Resource: CPU                                                        |
|                   snapshot: 100                                                |
|                   total: 79303                                                 |
|           Resource: Real Memory                                                |
|                   snapshot: 1                                                  |
|                   high water: 1919                                             |
|                                                                                |
|                                                                                |
+--------------------------------------------------------------------------------+

The output listing associated with the -w option includes these fields:

Resource: The resource being enforced by WLM. This is either CPU or Real Memory.
snapshot: Current CPU or Real Memory consumption as a percentage of the total resources available.
total: Total CPU time consumed in milliseconds. CPU resource only.
high water: The highest number of resident memory pages used. Real Memory resource only.

The long listing: The long listing is generated when you specify the -l option with the llq command. This section contains sample output for two llq commands: one querying a serial job and one querying a parallel job. Following the sample output is an explanation of all possible fields displayed by the llq command.

The following is sample output for the llq -l command for the serial job step c209f1n01.ppd.pok.ibm.com.2.0:

Figure 21. llq -l output for a serial job step

+--------------------------------------------------------------------------------+
|=============== Job Step c209f1n01.ppd.pok.ibm.com.2.0 ===============          |
|        Job Step Id: c209f1n01.ppd.pok.ibm.com.2.0                              |
|           Job Name: c209f1n01.ppd.pok.ibm.com.2                                |
|          Step Name: job_step_1                                                 |
|  Structure Version: 10                                                         |
|              Owner: loadl                                                      |
|         Queue Date: Wed Jul 25 15:49:17 EDT 2001                               |
|             Status: Running                                                    |
|   Execution Factor: 1                                                          |
|      Dispatch Time: Wed Jul 25 15:49:17 EDT 2001                               |
|    Completion Date:                                                            |
|    Completion Code:                                                            |
|      User Priority: 50                                                         |
|       user_sysprio: 0                                                          |
|      class_sysprio: 35                                                         |
|      group_sysprio: 70                                                         |
|    System Priority: -33                                                        |
|          q_sysprio: -33                                                        |
|      Notifications: Complete                                                   |
| Virtual Image Size: 24 kb                                                      |
|     Checkpointable: no                                                         |
|    Ckpt Start Time:                                                            |
|Good Ckpt Time/Date:                                                            |
|   Ckpt Elapse Time: 0 seconds                                                  |
|Fail Ckpt Time/Date:                                                            |
|    Ckpt Accum Time: 0 seconds                                                  |
|    Checkpoint File:                                                            |
|  Restart From Ckpt: no                                                         |
| Restart Same Nodes: no                                                         |
|            Restart: yes                                                        |
|     Hold Job Until:                                                            |
|                Cmd: /tmp/LL_V2/TEST/serial_90_60                               |
|               Args: arg_01 arg_02 arg_3                                        |
|                Env:                                                            |
|                 In: /dev/null                                                  |
|                Out: job1.c209f1n01.2.0.out                                     |
|                Err: job1.c209f1n01.2.0.err                                     |
|Initial Working Dir: /tmp/LL_V2/TEST                                            |
|         Dependency:                                                            |
|          Resources: ConsumableMemory(50.000 mb) ConsumableVirtualMemory(85.000 |
|       Requirements: (Arch == "R6000") && (OpSys == "AIX51") && (Memory > 128)  |
|        Preferences: (Machine == { "c209f1n01.ppd.pok.ibm.com" "c209f1n05.ppd.pok.ib|
|                                                                                |
+--------------------------------------------------------------------------------+

+--------------------------------------------------------------------------------+
|                     && (Feature == "ESSL")                                     |
|          Step Type: Serial                                                     |
|     Min Processors:                                                            |
|     Max Processors:                                                            |
|     Allocated Host: c209f1n01.ppd.pok.ibm.com                                  |
|         Node Usage: shared                                                     |
|    Submitting Host: c209f1n01.ppd.pok.ibm.com                                  |
|        Notify User: loadl@c209f1n01.ppd.pok.ibm.com                            |
|              Shell: /bin/ksh                                                   |
|  LoadLeveler Group: chemistry                                                  |
|              Class: large                                                      |
|    Ckpt Hard Limit: undefined                                                  |
|    Ckpt Soft Limit: undefined                                                  |
|     Cpu Hard Limit: 02:30:00 (9000 seconds)                                    |
|     Cpu Soft Limit: 02:30:00 (9000 seconds)                                    |
|    Data Hard Limit: 5.500 gb (5905580032 bytes)                                |
|    Data Soft Limit: 4.400 gb (4724464025 bytes)                                |
|    Core Hard Limit: 8.000 gb (8589934592 bytes)                                |
|    Core Soft Limit: 8.000 gb (8589934592 bytes)                                |
|    File Hard Limit: 1.500 tb (1649267441664 bytes)                             |
|    File Soft Limit: 1.200 tb (1319413953331 bytes)                             |
|   Stack Hard Limit: 400.000 mb (419430400 bytes)                               |
|   Stack Soft Limit: 300.000 mb (314572800 bytes)                               |
|     Rss Hard Limit: 3.000 pb (3377699720527872 bytes)                          |
|     Rss Soft Limit: 2.000 pb (2251799813685248 bytes)                          |
|Step Cpu Hard Limit: 04:00:30 (14430 seconds)                                   |
|Step Cpu Soft Limit: 04:00:30 (14430 seconds)                                   |
|Wall Clk Hard Limit: 00:11:40 (700 seconds)                                     |
|Wall Clk Soft Limit: 00:10:00 (600 seconds)                                     |
|            Comment: Test job 1 of Serial test suite 3.                         |
|            Account: 99999                                                      |
|         Unix Group: loadl                                                      |
|   NQS Submit Queue:                                                            |
|   NQS Query Queues:                                                            |
|Negotiator Messages:                                                            |
|Adapter Requirement:                                                            |
|          Step Cpus: 1                                                          |
|Step Virtual Memory: 85.000 mb                                                  |
|   Step Real Memory: 50.000 mb                                                  |
|Step Adapter Memory: 0 bytes                                                    |
|                                                                                |
+--------------------------------------------------------------------------------+

The following listing is sample output for the llq -l -x c209f1n01.1.0 command, where c209f1n01.1.0 is a parallel, non-checkpointing job step:

Figure 22. llq -l -x output for a parallel, non-checkpointing job step

+--------------------------------------------------------------------------------+
|=============== Job Step c209f1n05.ppd.pok.ibm.com.1.0 ===============          |
|        Job Step Id: c209f1n05.ppd.pok.ibm.com.1.0                              |
|           Job Name: c209f1n05.ppd.pok.ibm.com.1                                |
|          Step Name: parallel_job_step_1                                        |
|  Structure Version: 10                                                         |
|              Owner: loadl                                                      |
|         Queue Date: Wed Jul 25 17:49:51 EDT 2001                               |
|             Status: Running                                                    |
|   Execution Factor: 1                                                          |
|      Dispatch Time: Wed Jul 25 17:49:51 EDT 2001                               |
|    Completion Date:                                                            |
|    Completion Code:                                                            |
|      User Priority: 50                                                         |
|       user_sysprio: 0                                                          |
|      class_sysprio: 45                                                         |
|      group_sysprio: 0                                                          |
|    System Priority:                                                            |
|          q_sysprio:                                                            |
|      Notifications: Complete                                                   |
| Virtual Image Size: 387 kb                                                     |
|     Checkpointable: no                                                         |
|    Ckpt Start Time:                                                            |
|Good Ckpt Time/Date:                                                            |
|   Ckpt Elapse Time: 0 seconds                                                  |
|Fail Ckpt Time/Date:                                                            |
|    Ckpt Accum Time: 0 seconds                                                  |
|    Checkpoint File:                                                            |
|  Restart From Ckpt: no                                                         |
| Restart Same Nodes: no                                                         |
|            Restart: yes                                                        |
|     Hold Job Until:                                                            |
|                Env: MANPATH=/usr/local/man:/usr/share/man: ...                 |
|                 In: /dev/null                                                  |
|                Out: poe5_1.c209f1n05.1.0.out                                   |
|                Err: poe5_1.c209f1n05.1.0.err                                   |
|Initial Working Dir: /tmp/TEST/PARA                                             |
|                                                                                |
+--------------------------------------------------------------------------------+

+--------------------------------------------------------------------------------+
|         Dependency:                                                            |
|          Resources: ConsumableMemory(75.000 mb) ConsumableVirtualMemory(125.000|
|          Step Type: General Parallel                                           |
|         Node Usage: shared                                                     |
|    Submitting Host: c209f1n05.ppd.pok.ibm.com                                  |
|        Notify User: loadl                                                      |
|              Shell: /bin/ksh                                                   |
|  LoadLeveler Group: No_Group                                                   |
|              Class: Parallel                                                   |
|    Ckpt Hard Limit: undefined                                                  |
|    Ckpt Soft Limit: undefined                                                  |
|     Cpu Hard Limit: 00:30:00 (1800 seconds)                                    |
|     Cpu Soft Limit: 00:25:00 (1500 seconds)                                    |
|    Data Hard Limit: 4.250 pb (4785074604081152 bytes)                          |
|    Data Soft Limit: 1.500 tb (1649267441664 bytes)                             |
|    Core Hard Limit: 2.250 tb (2473901162496 bytes)                             |
|    Core Soft Limit: 1.250 tb (1374389534720 bytes)                             |
|    File Hard Limit: 1.200 eb (1383505805528216384 bytes)                       |
|    File Soft Limit: 1.100 eb (1268213655067531680 bytes)                       |
|   Stack Hard Limit: 40.000 mb (41943040 bytes)                                 |
|   Stack Soft Limit: 30.000 mb (31457280 bytes)                                 |
|     Rss Hard Limit: 1.200 eb (1383505805528216384 bytes)                       |
|     Rss Soft Limit: 5.500 pb (6192449487634432 bytes)                          |
|Step Cpu Hard Limit: 3+08:00:00 (288000 seconds)                                |
|Step Cpu Soft Limit: 23:59:59 (86399 seconds)                                   |
|Wall Clk Hard Limit: 01:40:00 (6000 seconds)                                    |
|Wall Clk Soft Limit: 01:40:00 (6000 seconds)                                    |
|            Comment: Test job 1 of Parallel test suite 5.                       |
|            Account: 99999                                                      |
|         Unix Group: loadl                                                      |
| User Space Windows: 8                                                          |
|   NQS Submit Queue:                                                            |
|   NQS Query Queues:                                                            |
|Negotiator Messages:                                                            |
|Adapter Requirement: (css0,LAPI,shared,US),(css0,MPI,shared,US)                 |
|          Step Cpus: 4                                                          |
|Step Virtual Memory: 500.000 mb                                                 |
|   Step Real Memory: 300.000 mb                                                 |
|Step Adapter Memory: 8.000 mb (8388608 bytes)                                   |
+--------------------------------------------------------------------------------+

+--------------------------------------------------------------------------------+
|--------------- Detail for c209f1n05.ppd.pok.ibm.com.1.0 ---------------        |
|       Running Host: c209f1n05.ppd.pok.ibm.com                                  |
|      Machine Speed: 1.000000                                                   |
|  Starter User Time: 00:00:00.230000                                            |
|Starter System Time: 00:00:00.190000                                            |
| Starter Total Time: 00:00:00.420000                                            |
|     Starter maxrss: 1972                                                       |
|      Starter ixrss: 8788                                                       |
|      Starter idrss: 13468                                                      |
|      Starter isrss: 0                                                          |
|     Starter minflt: 0                                                          |
|     Starter majflt: 0                                                          |
|      Starter nswap: 0                                                          |
|    Starter inblock: 0                                                          |
|    Starter oublock: 0                                                          |
|     Starter msgsnd: 0                                                          |
|     Starter msgrcv: 0                                                          |
|   Starter nsignals: 3                                                          |
|      Starter nvcsw: 82                                                         |
|     Starter nivcsw: 56                                                         |
|     Step User Time: 00:01:20.460000                                            |
|   Step System Time: 00:00:00.790000                                            |
|    Step Total Time: 00:01:21.250000                                            |
|        Step maxrss: 4312                                                       |
|         Step ixrss: 52544                                                      |
|         Step idrss: 9308828                                                    |
|         Step isrss: 0                                                          |
|        Step minflt: 6941                                                       |
|        Step majflt: 0                                                          |
|         Step nswap: 0                                                          |
|       Step inblock: 0                                                          |
|       Step oublock: 0                                                          |
|        Step msgsnd: 0                                                          |
|        Step msgrcv: 0                                                          |
|      Step nsignals: 0                                                          |
|         Step nvcsw: 507                                                        |
|                                                                                |
+--------------------------------------------------------------------------------+

+--------------------------------------------------------------------------------+
|        Step nivcsw: 8515                                                       |
|--------------------------------------------------------------------------------|
|Node                                                                            |
|----                                                                            |
|                                                                                |
|   Name            :                                                            |
|   Requirements    : (Arch == "R6000") && (OpSys == "AIX51") && (Memory > 128)  |
|   Preferences     : (Machine == { "c209f1n01.ppd.pok.ibm.com" "c209f1n05.ppd.pok.ib|
|                     && (Feature == "ESSL")                                     |
|   Node minimum    : 2                                                          |
|   Node maximum    : 2                                                          |
|   Node actual     : 2                                                          |
|   Allocated Hosts : c209f1n05.ppd.pok.ibm.com:RUNNING:css0(1,LAPI,US,1M),css0(2|
|                     css0(3,LAPI,US,1M),css0(4,MPI,US,1M)                       |
|                   + c209f1n01.ppd.pok.ibm.com:RUNNING:css0(1,LAPI,US,1M),css0(2|
|                     css0(3,LAPI,US,1M),css0(4,MPI,US,1M)                       |
|                                                                                |
|   Master Task                                                                  |
|   -----------                                                                  |
|                                                                                |
|      Executable   : /bin/poe                                                   |
|      Exec Args    : /tmp/TEST/PARA/ivp_60 -euilib us -ilevel 6 -labelio yes -pm|
|      Num Task Inst: 1                                                          |
|      Task Instance: c209f1n05:-1                                               |
|                                                                                |
|   Task                                                                         |
|   ----                                                                         |
|                                                                                |
|      Num Task Inst: 4                                                          |
|      Task Instance: c209f1n05:0:css0(1,LAPI,US,1M),css0(2,MPI,US,1M)           |
|      Task Instance: c209f1n05:1:css0(3,LAPI,US,1M),css0(4,MPI,US,1M)           |
|      Task Instance: c209f1n01:2:css0(1,LAPI,US,1M),css0(2,MPI,US,1M)           |
|      Task Instance: c209f1n01:3:css0(3,LAPI,US,1M),css0(4,MPI,US,1M)           |
|                                                                                |
+--------------------------------------------------------------------------------+

The long listing includes these fields:

Job Step ID

The job step identifier.

Job Name

The name of the job.

Step Name

The name of the job step

Structure Version

An internal version identifier.

Owner

The userid of the user submitting the job.

Queue Date

The date and time that LoadLeveler received the job.

Status

The status (state) of the job. A job's status can be:

Canceled

Checkpointing

Completed

Complete Pending

Deferred

Idle

Not Queued

Not Run

Pending

Preempted (user-initiated)

Preempted (system-initiated)

Preempt Pending (user-initiated)

Preempt Pending (system-initiated)

Rejected

Reject Pending

Removed

Remove Pending

Resume Pending

Running

Starting

Submission Error

System Hold

System and User Hold

Terminated

User Hold

Vacated

Vacate Pending

Note:

For a detailed explanation of these job states, see Job states.

Execution Factor

The weight factor of the relative processing time when using Gang scheduling.

Dispatch Time

The time the job was dispatched.

Completion Date

Date and time job completed or exited.

Completion Code

The status returned by the wait3 UNIX system call.

User Priority

The priority of the job step, as specified by the user in the job command, or changed by the llprio command.

user_sysprio

The user system priority of the job step, where the value is defined in the administration file.

class_sysprio

The class priority of the job step, where the value is defined in the administration file.

group_sysprio

The group priority of the job step, where the value is defined in the administration file.

System Priority

The overall system priority of the job step, where the value is defined by the SYSPRIO expression in the configuration file.

q_sysprio

The adjusted system priority of the job step (see How does a job's priority affect dispatching order?).

Notifications

The notification status for the job step, where:

always: Indicates notification is sent through the mail for all four notification categories below.
complete: Indicates notification is sent through the mail only when the job step completes.
error: Indicates notification is sent through the mail only when the job step terminates abnormally.
never: Indicates notification is never sent.
start: Indicates notification is sent through the mail only when starting or restarting the job step.

Virtual Image Size

The value of the image_size keyword (if specified) or the size of the executable associated with the executable keyword (if specified) or the size of the job command file.

Restart

Restart status (yes or no)

Checkpointable

Indicates if LoadLeveler considers the job step checkpointable (yes, no, or interval).

Ckpt Start Time

The start time of the current checkpoint in progress. Blank if no checkpoint running.

Good Ckpt Time/Date

Time and date stamp of the last successful checkpoint.

Ckpt Elapse Time

Amount of time taken to perform the last successful checkpoint.

Fail Ckpt Time/Date

Time and date stamp of the last failed checkpoint.

Ckpt Accum Time

Accumulated time, in seconds, the job step has spent checkpointing.

Checkpoint File

Location of the directory and file name to be used for checkpoint data.

Restart From Ckpt

Indicates if a job has been restarted from an existing checkpoint (yes or no).

Restart Same Nodes

Indicates if a job step should be restarted on the same nodes after vacate (yes or no).

Hold Job Until

Job step is deferred until this date and time.

Cmd

The name of the executable associated with the executable keyword (if specified) or the name of the job command file.

Args

Arguments that were passed to the executable.

Env

Environment variables to be set before executable runs. Appears only when the -x option is specified.

In

The file to be used for stdin.

Out

The file to be used for stdout.

Err

The file to be used for stderr.

Initial Working Dir

The directory from which the job step is run. The relative directory from which the stdio files are accessed, if appropriate.

Dependency

Job step dependencies as specified at job submission.

Requirements

Job step requirements as specified at job submission.

Preferences

Job step preferences as specified at job submission.

Task_geometry

Reflects the settings for the task_geometry keyword in the job command file.

Resources

Reflects the settings for the resources keyword in the job command file.

Blocking

Reflects the settings for the blocking keyword in the job command file.

Step Type

Type of job step:

Serial
General parallel
PVM3

Min Processors

The minimum number of processors needed for this job step.

Max Processors

The maximum number of processors that can be used for this job step.

Allocated Hosts

The machines that have been allocated for this job step.

Node Usage

A request that a node be shared or not shared or that a time-slice is not shared. The user specifies this request while submitting the job.

Submitting Host

The name of the machine to which the job is submitted.

Notify User

The user to be notified by mail of a job's status.

Shell

The shell to be used when the job step runs.

LoadLeveler Group

The LoadLeveler group associated with the job step.

Class

The class of the job step as specified at job submission.

Ckpt Hard Limit

Checkpoint hard limit as specified at job step submission.

Ckpt Soft Limit

Checkpoint soft limit as specified at job step submission.

Cpu Hard Limit

CPU hard limit as specified at job submission.

Cpu Soft Limit

CPU soft limit as specified at job submission.

Data Hard Limit

Data hard limit as specified at job submission.

Data Soft Limit

Data soft limit as specified at job submission.

Core Hard Limit

Core hard limit as specified at job submission.

Core Soft Limit

Core soft limit as specified at job submission.

File Hard Limit

File hard limits as specified at job submission.

File Soft Limit

File soft limit as specified at job submission.

Stack Hard Limit

Stack hard limit as specified at job submission.

Stack Soft Limit

Stack soft limit as specified at job submission.

Rss Hard Limit

RSS hard limit as specified at job step submission.

Rss Soft Limit

RSS soft limit as specified at job step submission.

Step Cpu Hard Limit

Job step CPU hard limit as specified at job submission.

Step Cpu Soft Limit

Job step CPU soft limit as specified at job submission.

Wall Clk Hard Limit

Wall clock hard limit as specified at job submission.

Wall Clk Soft Limit

Wall clock soft limit as specified at job submission.

NQS Submit Queue

The name of the NQS pipe queue to which the NQS job will be routed.

NQS Query Queues

The NQS queue names you can use to monitor the job.

Comment

The comment specified by the comment keyword in the job command file.

Account

The account number specified in the job command file.

Unix Group

The effective UNIX group name.

DCE Principal

The DCE principal name associated with the process that submitted the job to LoadLeveler.

User Space Windows

The number of switch adapter windows assigned to the job step.

Negotiator Messages

Informational messages for the job step if it is in the Idle or NotQueued state.

Adapter Requirement

Reflects the settings of the network keyword in the job command file.

Step Cpus

The total ConsumableCpus for the job step.

Step Virtual Memory

The total ConsumableVirtualMemory for the job step.

Step Real Memory

The total ConsumableMemory for the job step.

Step Adapter Memory

The total adapter pinned memory for the job step.

When -x and -l options are specified, llq also displays the information listed below. If several LoadL_starter processes are used for running this job step, then the values reported are either cumulative totals or the maximum values. The same is true for the processes of the job step.

Starter maxrss/Step maxrss: Maximum resident set size utilized. Maximum value.
Starter ixrss/Step ixrss: Size of the text segment. Maximum value.
Starter idrss/Step Starter idrss: Size of the data segment. Maximum value.
Starter isrss/Step isrss: Integral unshared stack used. Maximum value.
Starter minflt/Step minflt: Number of page faults (reclaimed). Cumulative total.
Starter majflt/Step majflt: Number of page faults (I/O required). Cumulative total.
Starter nswap/Step nswap: Number of times swapped out. Cumulative total.
Starter inblock/Step inblock: Number of times file system performed input. Cumulative total.
Starter oublock/Step oublock: Number of times file system performed output. Cumulative total.
Starter msgsnd/Step msgsnd: Number of IPC messages sent. Cumulative total.
Starter msgrcv/Step msgrcv: Number of IPC messages received. Cumulative total.
Starter nsignals/Step nsignals: Number of signals delivered. Cumulative total.
Starter nvcsw/Step nvcsw: Number of context switches due to voluntarily giving up processor. Cumulative total.
Starter nivcsw/Step nivcsw: Number of involuntary context switches. Cumulative total.
Starter User Time/Step User Time: CPU user time of Starter/Step processes. Cumulative total.
Starter System Time/Step System Time: CPU system time of Starter/Step processes. Cumulative total.
Starter Total Time/Step Total Time: CPU total time of Starter/Step processes. Cumulative total.
Running Host: For a serial job step, the machine that is running this job step. For a parallel job step, the first machine that has been allocated for this job step.
Machine Speed: For a serial job step, the value associated with the "speed" keyword of the machine that is running this job step. For a parallel job step, the value associated with the "speed" keyword of the first machine that has been allocated for this job step.

Other fields displayed for parallel jobs are:

(Node) Name

Blank value. Reserved for future use.

(Node) Requirements

Job step requirements as specified at job submission.

(Node) Preferences

Job step preferences as specified at job submission.

(Node) Node minimum

Minimum number of machines of this Node type required to run this job step.

(Node) Node maximum

Maximum number of machines of this Node type that can be used to run this job step.

(Node) Node actual

Actual number of machines of this Node type that are used in the running of this job step.

(Node) Allocated Hosts

The machines of this Node type that have been allocated for this job step. The format is:

hostname:task status:adapter usage, ... ,adapter usage + ... +
hostname:task status:adapter usage, ... ,adapter usage

The adapter usage information has the format: adapter name(adapter window ID, network protocol, mode, adapter window memory)

(Node/Master Task) Executable

The executable associated with the master task.

(Node/Master Task) Exec Args

The arguments passed to the master task executable.

(Node/Master Task) Num Task Inst

The number of task instances of the master task.

(Node/Master Task) Task Instance

Task instance information has the format: hostname:task ID:adapter usage, ... ,adapter usage
Adapter usage information has the format: adapter name(adapter window ID, network protocol, mode, adapter window memory)

(Node/Task) Num Task Inst

The number of task instances.

(Node/Task) Task Instance

Task instance information has the format: hostname:task ID:adapter usage, ... ,adapter usage
Adapter usage information has the format: adapter name(adapter window ID, network protocol, mode, adapter window memory)

[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]