SGI ProPack 4 to ProPack 5 Transition Notes

The NCSA SGI Altix system will go into production under SGI ProPack 5 on Wednesday August 6, 2008.

User impact

The access node will now be a 32 processor SGI Altix 3700 with 64 GB of memory. The interactive limits remain the same as before.
After the upgrade you will be placed in the default ProPack 5 environment when you log in. You will need to copy over any customized softenv settings that you had in ProPack 4; your ProPack 4 .soft file will be located at $HOME/.soft-pp4.
Production jobs that won't be able to complete before the upgrade will remain in the queue, and will be moved to the new ProPack 5 environment. All jobs in the friendly user environment will be deleted.
In general, programs built under ProPack 4 should run without problems on ProPack 5. However, due to various upgrades in the software environment we strongly recommend that you recompile your programs.
If your programs link to the Intel MKL libraries, you should recompile. Linking information is available at Migrating to ProPack 5 Math libraries.
As of ProPack 5, support for the SGI Cray Scientific Library (BLAS, LAPACK, FFT, and associated libraries for BLACS and ScaLAPACK) has been dropped by SGI. You can use Intel MKL 10 to provide high-performance BLAS, LAPACK, BLACS, SCALAPACK, FFT, etc. Intel MKL 10 supports the SGI MPT library for BLACS. Documentation describing the ProPack 5 transition is available at Migrating to ProPack 5 Math libraries.
For assistance in porting off of SCSL please contact mathsoft@ncsa.uiuc.edu.
A resource named altix4700 will be available to specify that jobs run on the new SGI Altix 4700 (co-compute3) when desired. Add the following qsub option to your batch script:
```
    #PBS -l altix4700=true
```
The Intel compiler provides the following option to optimize for the dual-core Itanium 2 processor on the Altix 4700 (co-compute3):
```
     mtune=itanium2-p9000
```
The charging algorithm will change for timeshared jobs.
The scheduler allocates whole node boards (CPUs and associated memory) to jobs so different jobs don't share resources on the same node board. Charging will be based on the node boards required to accommodate the CPU and memory specifications of the job:
```
 # SUs =  ServiceLevel * Time

where

ServiceLevel = 1.0  for Normal Interactive
                        Normal Batch
                        Normal Dedicated Batch
             = 1.25 for Industrial Batch
                        Industrial Dedicated Batch

Timeshared jobs:
 Time = # CPUs allocated  * Total Wall Clock
            to the job           Hours

Dedicated jobs:
 Time = # CPUs on the host * Total Wall Clock
                                  Hours
  
```
The Altix 3700 systems have 2 CPUs per node board and the Altix 4700 system has 4 cores per node board. Since the memory per CPU varies on the compute nodes, the scheduler will make every attempt to run each job on the resource that represents the most efficient use of resources while also attempting to start the job as soon as possible.

ProPack 4 vs ProPack 5

                                   ProPack 4       ProPack 5
                                   ---------       ---------
Distribution                       SLES9           SLES10
Linux Kernel                       2.6.5           2.6.16
glibc                              2.3.3           2.4.31
batch system                       pbs-7.1         pbs-9.2
Default Intel Compiler             9.0             10.1
Default Intel MKL                  7.2             10.0    
SGI Message Passing Toolkit(MPI)   1.14            1.16

Please report any problems or issues to the NCSA Consulting Office via electronic mail at consult@ncsa.uiuc.edu or by telephone at (217) 244-1144.