Users with jobs in queue after PP5 transition
The SCSL (and supporting SDSM library) is no longer supported by SGI under ProPack5.
It is recommended that users move to using Intel MKL 10 to replace the
functionality provided by SCSL.
It is also recommended that you recompile/relink with MKL 10. Make sure you do not have hard-codes paths to the older versions of MKL and refer instead to ${MKL_HOME}.
If you have linked to older MKL versions but keep MKL 10 in your environment you will see the following error message:
error while loading shared libraries: /usr/local/intel/mkl/10.0.3.020/lib/64/libmkl.so: invalid ELF header
If you see the following message:
: error while loading shared libraries: libscs.so: cannot open shared object
then you can add the PP4 SCS library while getting familar with MKL.
Current MKL 10 documentation can be found here.
The software database entry for MKL10 on Cobalt has additional information.
*Note:* the dynamic linking schemes presented below may be subject to runtime numerical failures. For examples on statically linking libraries, consult MKL10 on Cobalt and SGI Altix: Programming Environment.
Note: Please do not use the Intel compiler option -fast as it adds -static and there are not static libraries for the SGI MPT (MPI) libraries. If you would like to link to the Intel compiler libraries statically please add -static-intel -static-libgcc.
BLAS, LAPACK
Use MKL 10
- recompile with
-I ${MKL_HOME}/include if needed
- relink with
-Wl,-Bstatic -Wl,--start-group -L ${MKL_HOME}/lib/64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -Wl,-Bdynamic
If you want to use the threaded versions of the routines:
-Wl,-Bstatic -Wl,--start-group -L ${MKL_HOME}/lib/64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -Wl,-Bdynamic -lguide -lpthread
Use SCSL while porting to MKL
- add
+sgi-mathlibs-pp4 to your ${HOME}/.soft file before @teragrid-basic
`resoft`- recompile (include search paths have been updated)
- relink with
-lscs for serial routines
and
-lscsl_mp -lpthread for threaded routines
Note: There is an issue with the ETIME() routine provided in -lscs and -lscs_mp. Please add -lifport before -lscs or -lscs_mp when linking to make sure you get the Intel Fortran Portability library version of ETIME().
BLACS, PBLAS and ScaLAPACK
Use MKL 10
- recompile with
-I ${MKL_HOME}/include if needed
- relink with
-L ${MKL_HOME}/lib/64 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -Wl,-Bstatic -Wl,--start-group -L ${MKL_HOME}/lib/64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl,--end-group -Wl,-Bdynamic -lmpi
If you want to use the threaded BLAS versions of the routines:
-L ${MKL_HOME}/lib/64 -lmkl_scalapack_lp64 -lmkl_blacs_sgimpt_lp64 -Wl,-Bstatic -Wl,--start-group -L ${MKL_HOME}/lib/64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -Wl,-Bdynamic -lguide -lmpi
Use SCSL and SDSM while porting to MKL
- add
+sgi-mathlibs-pp4 to your ${HOME}/.soft file before @default or @teragrid-basic
`resoft`- recompile (include search paths have been updated)
- relink with
-lsdsm -lscs -lmpi for serial routines with MPI
and
-lsdsm_mp -lscsl_mp -lpthread for threaded routines
Use NETLIB versions of ScaLAPACK 1.8 and Intel MKL BLACS
- relink your code:
-L /usr/apps/math/scalapack/scalapack-1.8.0/lib -lscalapack-INTEL10-PP5 -L {MKL_HOME}/lib/64 -lmkl_blacs_sgimpt_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmpi
Use NETLIB versions of ScaLAPACK 1.8 and BLACS
- relink your code:
-L /usr/apps/math/scalapack/scalapack-1.8.0/lib -lscalapack-INTEL10 -L /usr/apps/math/scalapack/BLACS/LIB -lblacsF77init_MPI-INTEL10-0 -lblacs_MPI-INTEL10-0 -lblacsF77init_MPI-INTEL10-0 -L {MKL_HOME}lib/64 -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmpi
FFT
Use MKL 10
You will need to convert calls to the SCSL FFT routines to MKL's FFT routines. More documentation on this to follow.
Use SCSL while porting to MKL
- add
+sgi-mathlibs-pp4 to your ${HOME}/.soft file before @teragrid-basic
`resoft`- recompile (include search paths have been updated)
- relink with
-lscs for serial routines
and
-lscsl_mp -lpthread for threaded routines
For users who have jobs in the queue on Cobalt and are the SGI libscs (-lscs) and/or libsdsm (-lsdm) or the older MKL libraries you will need to either:
- kill their job, rebuild, test and resubmit
- rebuild, test, and release their existing jobs if the new binary is put in the same place as the one expected in the batch job
- add the appropriate soft key to their .soft, test ,and release existing job
For SCSL put
+sgi-mathlibs-pp4
For MKL, put
@remove +intel-mkl10.0
and then one of
+intel-mkl-7.2.1-r1 (was the default for PP4)
+intel-mkl-8.1.014-r1
+intel-mkl-9.1.023-r1
in .soft.
The key is to test before releasing the job in the queue.
