NERSCPowering Scientific Discovery Since 1974

Python on Genepool

Using Python on Genepool

There are two major branches of Python supported on genepool: python 2.7.3 and python 3.2.3. These packages are installed in /usr/common/usg, and not in /jgi/tools. To access these python installations, you must use the environment modules system. Python 2.7.3 is the default version when loading python.

Loading the python module

 genepool01:~$ module load python
genepool01:~$ which python
/usr/common/usg/languages/python/2.7.3_1/bin/python
genepool01:~$ python
Python 2.7.3 (default, Jul 20 2012, 00:55:25)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

To load the python 3.2.3 module, simply specify the version: module load python/3.2.3

Loading either python module will reset the PYTHONPATH environment variable to an empty string. This is done because python packages installed in a user-defined directory for python 2.7.3 are unlikely to work for python 3.2.3 and vice versa. Therefore, if you are setting PYTHONPATH in your login scripts (e.g. in .bashrc.ext), please load the python module before setting your PYTHONPATH.

 

Example .bashrc.ext loading python and setting PYTHONPATH for genepool

 ...
if [ $NERSC_HOST == "genepool" ]; then
    # load python/2.7.3
    module load python
    # add ~someuser/python/genepool/python2.7 to PYTHONPATH; maintaining existing
    # PYTHONPATH in case module does set PYTHONPATH to something useful
    export PYTHONPATH="${HOME}/python/${NERSC_HOST}/python2.7:${PYTHONPATH}"
fi
...

 

Installed Python 2.7.3 Packages

These are the python packages installed as part of the python/2.7.3 module on genepool.

Package NameVersionOptional ModulesDescription
Cython 0.16   Cython is a language that makes writing C extensions for the Python language as easy as Python itself.
docutils 0.9.1   Documentation Utilities: Written in Python, for General- and Special-Purpose Use
drmaa 0.5   Provides DRMAA-compliant access to resource management systems like SGE and torque.
gdata 2.0.17   The Google Data APIs (Google Data) provide a simple protocol for reading and writing data on the web.
h5py 2.0.1 hdf5 HDF5 for Python (h5py) is a general-purpose Python interface to the Hierarchical Data Format library, version 5.
ipython 0.13 qt, zeromq, atlas Provides a powerful, user-friendly shell to python. ipython may load numpy, scipy, matplotlib, pyzmq among other python packages, thus the additional environment modules may need to be loaded depending on your usage.
matplotlib 1.1.1 qt, atlas matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib may utilize numpy or PyQt4, thus additional environment modules may need to be loaded depending on your usage.
MySQL-Connector 0.3.2   Pure python MySQL driver.
nose 1.1.2   nose extends unittest to make testing easier.
Numpy 1.6.2 atlas Numpy is the fundamental package for scientific computing within Python.
pip 1.1   pip installs packages. Python packages. An easy_install replacement
pygments 1.5   Pygments is a general syntax highlighter for general use.
pymysql 0.5   Pure python MySQL driver.
pexpect 2.4   Pexpect is a pure Python module for spawning child applications; controlling them; and responding to expected patterns in their output.
PyQt4 4.9.4 qt PyQt is a set of Python bindings for Nokia's Qt application framework and runs on all platforms supported by Qt including Windows, MacOS/X and Linux.
reportlab 2.5   The ReportLab Toolkit is the time-proven, ultra-robust open-source engine for programatically creating PDF documents.
ruffus 2.2   Ruffus provides support for managing computational pipelines.
Scipy 0.10.1 atlas Scipy is open-source software for mathematics, science, and engineering.
setuptools 0.6c11   Download, build, install, upgrade, and uninstall Python packages -- easily!
sip 4.13.3   SIP is a tool that makes it very easy to create Python bindings for C and C++ libraries.
sphinx 1.1.3   Sphinx is a tool that makes it easy to create intelligent and beautiful documentation
SQLAlchemy 0.7.8   SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.
tornado 2.3   Tornado is an open source scalable, non-blocking web server.
virtualenv 1.7.2   virtualenv is a tool to create isolated Python environments.
pyzmq 2.0.8 zeromq Python bindings for ZeroMQ

 

Other environment modules for Python 2.7.3

Some python packages for the python/2.7.3 module are installed in other environment module. Each of these modules will load the python/2.7.3 module if it is not already loaded.

Module NameSoftware PackageVersionDescription
biopython Biopython 1.60* Biopython is a set of freely available tools for biological computation written in Python.

Note: Asterisk indicates default version.

 

Installed Python 3.2.3 Packages

These are the python packages installed as part of the python/2.7.3 module on genepool.

Package NameVersionOptional ModulesDescription
Cython 0.16   Cython is a language that makes writing C extensions for the Python language as easy as Python itself.
distribute 0.6.27   Distribute is intended to replace Setuptools as the standard method for working with Python module distributions.
docutils 0.9.1   Documentation Utilities: Written in Python, for General- and Special-Purpose Use
MySQL-Connector 0.3.2   Pure python MySQL driver.
Numpy 1.6.2 atlas Numpy is the fundamental package for scientific computing within Python.
pip 1.1   pip installs packages. Python packages. An easy_install replacement
PyQt4 4.9.4 qt PyQt is a set of Python bindings for Nokia's Qt application framework and runs on all platforms supported by Qt including Windows, MacOS/X and Linux.
Scipy 0.10.1 atlas Scipy is open-source software for mathematics, science, and engineering.
sip 4.13.3   SIP is a tool that makes it very easy to create Python bindings for C and C++ libraries.
SQLAlchemy 0.7.8   SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL. Note: The python/3.2.3 version of SQLAlchemy does not have compiled C-bindings to accelerate performance. Please use the python/2.7.3 version for high performance applications.
virtualenv 1.7.2   virtualenv is a tool to create isolated Python environments.

Other environment modules for Python 3.2.3

Some python packages for the python/3.2.3 module are installed in other environment module. Each of these modules will load the python/3.2.3 module if it is not already loaded.

Module NameSoftware PackageVersionDescription
No external python/3.2.3 modules installed yet

Note: Asterisk indicates default version.