NERSCPowering Scientific Discovery Since 1974

Math Kernel Library (MKL)

Description

The Intel Math Kernel Library (Intel MKL) contains highly optimized, extensively threaded math routines for science, engineering, and financial applications. Core math functions include BLAS, LAPACK, ScaLAPACK, Sparse Solvers, Fast Fourier Transforms, Vector Math, and more.

NERSC provides MKL on the Carver system for all compilers.  MKL is available on the Hopper system for Intel compiler only.

How to Access MKL on Carver

module load mkl

How to Use MKL on Carver

Linking Your Code with MKL on Carver

Basic usage, with (default) PGI compiler:

mpif90 my_code.f $MKL

Using ScaLAPACK:

mpif90 my_code.f $MKL -L$MKL_LIBDIR -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64

Basic usage, with (default) Intel or gcc compilers (after appropriate module swap commands):

mpicc -c -I$MKL_INC test.c $MKL

Running Your Code with MKL on Carver

The above procedures create static executables. When running your code you may see an error message such as this:

./a.out: error while loading shared libraries: libmkl_intel_lp64.so: 
cannot open shared object file: No such file or directory

There are several possible solutions:

1. The simplest solution is, in your batch script, include the statement "module load mkl". You need to load the same modules that were used for building your executable.

2. You can also type "module load mkl" from the shell and then include the statement "#PBS -V" in your batch script.

3. This is a two-step procedure. First, type "module load mkl". Then, when linking your code you can include the "-rpath $MKL_LIBDIR" option. Note that in a Makefile the syntax would be something like:

FC   = mpif90
BTLIBS = -L$(MKL_LIBDIR) -lmkl_core -lmkl_blacs_openmpi_lp64
TESTOBJ = hello.o
all: hello.x
hello.x: $(TESTOBJ)
$(FC) -o $@ $(TESTOBJ) $(BTLIBS) -rpath $(MKL_LIBDIR)
.f.o : ; $(FC) -c $(FCFLAGS) $*.f

How to Use MKL on Hopper (Intel compiler only)

Linking Your Code with MKL

Just use the "-mkl" flag with the compiler wrappers under the Intel programming environment. The default is to use the threaded libraries ("-mkl" is equivalent to "-mkl=parallel") .  If sequential libraries are needed, use the "-mkl=sequential" flag instead. If ScaLapack routines are needed, use the -mkl=cluster option. Note, the -mkl=cluster flag links to the sequential routines of the libraries, if the scalapack with threaded libraries are needed, you need to provide the correct combinations of the libraries to the link line. Please refer to the Intel's  MKL Advisor page.

module swap PrgEnv-pgi PrgEnv-intel
ftn my_code.f -mkl
cc test.c -mkl 

How to Use MKL on Edison (Intel compiler only) 

On Edison, the Intel programming environment is the default, so you don't need to swap the programming environment to Intel first.  However, here are some important notes.

On Edison, the default library searched for math routines such as BLAS is Cray's SciLib.  This library is loaded into your environment by default in the cray-libsci modulefile.  With Fortran, even if you use the "-mkl" compiler flag Cray's SciLib will probably be used to resolve BLAS and other similar symbols.  (Remember, you can always check which library is used with the -Wl,-yNAME compiler option, where NAME is the actual symbol you want to check.)  To preferentially use an MKL routine from Fortran you should unload the cray-libsci modulefile and then compile and link, using the "-mkl" compiler flag.  Example: 

% ftn -Wl,-ycgbbrd_ cgbbrdx.f
/scratch2/scratchdirs/hjw/ifortqfXPFP.o: reference to cgbbrd_
/opt/cray/libsci/12.1.01/intel/130/sandybridge/lib/libsci_intel_mp.a(cgbbrd.o): definition of cgbbrd_
% module unload cray-libsci/12.1.01
% ftn -Wl,-ycgbbrd_ cgbbrdx.f -mkl
/scratch2/scratchdirs/hjw/ifortbkYcxX.o: reference to cgbbrd_
/opt/intel/composer_xe_2013.5.192/mkl/lib/intel64/libmkl_intel_lp64.a(_cgbbrd_lp64.o): definition of cgbbrd_

To use MKL with C or C++ on Edison, the following is necessary:

% module load mkl
% CC test.cc -I$MKL_INC $MKL 


Documentation

The release notes, user's guide, and reference manual are available in PDF and HTML format on the Intel web site.

Intel also has an "MKL Advisor" that can help with choosing linking options.  To see which libraries are recommended for a particular use case view the Intel interactive form.

Availability

PackagePlatformCategoryVersionModuleInstall DateDate Made Default
MKL carver libraries/ math 10.2 mkl/10.2 2010-02-10 2012-03-14
MKL carver libraries/ math 10.3 mkl/10.3 2012-04-23
MKL carver libraries/ math 13.0.1 mkl/13.0.1 2013-02-04 2013-02-14
MKL edison libraries/ math 13.0.1 mkl/13.0.1 2013-02-11 2013-02-11
MKL edison libraries/ math 13.0.3 mkl/13.0.3 2013-10-21 2013-10-21
mkl genepool libraries/ general 2013.1.117 mkl/2013.1.117 2013-06-13 2013-06-13
MKL hopper libraries/ math 10.2 mkl_ccm/10.2 2012-04-04 2012-04-04
MKL hopper libraries/ math 12.0.5.220 mkl/12.0.5.220 2012-03-14 2012-03-14
MKL hopper libraries/ math 12.1.2.273 mkl/12.1.2.273 2012-03-14 2012-03-20
MKL hopper libraries/ math 12.1.3.293 mkl/12.1.3.293 2012-06-12
mkl phoebe libraries/ general 2013.1.117 mkl/2013.1.117 2013-06-18 2013-06-26