Python Tools

Description and Overview

Python is an interpreted, general-purpose high-level programming language. Various of versions of Python are installed on most of the NERSC systems, usually accompanied with computational tool such as numpy and scipy.

Using Python on NERSC Systems

To use the python installed as module on NERSC systems, you need do

module load python

Remember, to run a python script on the Hopper compute nodes you must set an environment variable:
setenv CRAY_ROOTFS DSL.  To execute a script on the Hopper compute nodes dedicated to your job you need to use aprun and add the option -a xt, e.g.,:

aprun -n 1 -a xt ./hello_world.py

Using Numpy, Scipy and Matplotlib

Several scientific computing tools for Python are installed with each Python release. You can get the version of these packages from the "Availability" table.

Numpy is a language extension that defines the numerical array and matrix type and basic operations on them.

Scipy is another language extension that uses numpy to do advanced math, signal processing, optimization, statistics and much more.

Matplotlib is a language extension to facilitate plotting.

For details about these packages, please refer to their official documentations.

Installing Your Own Python Package

Users can install their own python packages in their home directories. Here is an example installing a python package called setuptools:

• Un-tar the package and cd into the package directory

% tar xvf setuptools-0.6c11.tar.gz
% cd setuptools-0.6c11/

• Create a directory hierarchy for your python module libraries. Example: $HOME/python_modules/machine_name/lib/python

• Set the PYTHONPATH environment variable

% setenv PYTHONPATH $HOME/python_modules/machine_name/lib/python

• Install your python package with the default version of python in /usr/bin/python

% python setup.py install --home=~/python_modules/machine_name

• OR Install your python package with alternate version of python. (There could be a more recent version of python as a module)

% module load python
% python setup.py install --home=~/python_modules/machine_name

• Test your installation

% python
>> import setuptools
>> ...

Using the DLCache library tools for large scale python applications on Hopper

Large scale Python applications may spend huge time at startup on Hopper. The DLCache library is a set of functions that can be incorporated into a dynamically-linked application to provide improved performance during the loading of dynamic libraries when running the application at large scale on Cray XE6. To access this library, do

module load dlcache

Please read the user guide to use the tools provided by this library. This library is still in experimental stage, please report any problems, comments you see with this library to "consult at nersc dot gov".

Availability

You can check the availability of python by using the following command:

module avail python

You can also refer to the table below for the version of installed computational tools along with python.