NERSCPowering Scientific Discovery Since 1974

HDF5

Description and Overview

Hierarchical Data Format version 5 (HDF5) is a set of file formats, libraries, and tools for storing and managing large scientific datasets. Originally developed at the National Center for Supercomputing Applications, it is currently supported by the non-profit HDF Group.

HDF5 is different product from previous versions of software named HDF, representing a complete redesign of the format and library.  It also includes improved support for parallel I/O. The HDF5 file format is not compatible with HDF 4.x versions. You can use the 'h5toh4' and 'h4toh5' converters that are available on all NERSC machines. 

If you are not familiar with parallel I/O, please refer to the tutorial below. 

Scientific I/O Tutorial

Using HDF5 On Cray Systems

There are HDF5 libraries provided by Cray and by NERSC. Use the command "module avail cray-hdf5" and "module avail hdf5" to see the available Cray and NERSC versions. The default HDF5 library for use on Hopper or Edison is provided by Cray.

Cray serial HDF5 on Hopper and Edison

% module swap PrgEnv-pgi PrgEnv-gnu (or similar command to use another compiler)
% module load hdf5
% ftn ... (for Fortran code)
% cc ... (for C code)
% CC ... (for C++ code)

NERSC serial HDF5 library with PGI Compiler (on Hopper only)

% module load hdf5/1.8.7
% h5cc ... (for C code)
% h5fc ... (for Fortran code)
% h5c++ ... (for C++ code)

NERSC serial HDF5 with Intel Compiler (on Hopper only)

% module swap PrgEnv-pgi PrgEnv-intel 
% module load hdf5/1.8.7
% h5cc ... (for C code)
% h5fc ... (for Fortran code)
% h5c++ ... (for C++ code)

Cray parallel HDF5 on Hopper and Edison

% module swap PrgEnv-pgi PrgEnv-cray (or similar commands to use another compiler)
% module load hdf5-parallel
% ftn ... (for Fortran code)
% cc ... (for C code)

Using HDF5 On Carver

Serial HDF5

% module load hdf5 
% h5fc ...
% h5cc ...
or
% mpif90 ... $HDF5
% mpicc ... $HDF5

Parallel HDF5

% module load hdf5_parallel
% h5pfc ...
% h5pcc ...
or
% mpif90 ... $HDF5
% mpicc ... $HDF5

Documentation

For questions about HDF5 on any NERSC systems send e-mail to consult@nersc.gov.

Additional information is available at The HDF Group.

Availability at NERSC

PackagePlatformCategoryVersionModuleInstall DateDate Made Default
HDF carver libraries/ I/O 1.8.11 hdf5/1.8.11 2013-06-11
HDF carver libraries/ I/O 1.8.3 hdf5/1.8.3 2010-02-26 2010-02-26
HDF carver libraries/ I/O 1.8.6 hdf5/1.8.6 2012-01-13 2013-07-02
HDF5 babbage libraries/ I/O host-1.8.10-p1 hdf5/host-1.8.10-p1 2014-01-08
HDF5 babbage libraries/ I/O mic-1.8.10-p1 hdf5/mic-1.8.10-p1 2014-01-08 2014-01-08
hdf5 genepool pe_libraries/ general 1.8.10-patch1 hdf5/1.8.10-patch1 2013-03-28 2013-04-08
hdf5 genepool pe_libraries/ general 1.8.11 hdf5/1.8.11 2013-06-11
hdf5 genepool pe_libraries/ general 1.8.12 hdf5/1.8.12 2014-01-14
hdf5 genepool pe_libraries/ general 1.8.4-patch1 hdf5/1.8.4-patch1 2013-04-17
HDF5 genepool applications/ bioinformatics 1.8.7 hdf5/1.8.7 2012-04-04 2012-04-04
hdf5 genepool pe_libraries/ general 1.8.9 hdf5/1.8.9 2012-07-19 2012-07-19
hdf5 genepool libraries/ general 4.10.2 rpm/4.10.2 2013-01-24 2013-01-24
HDF5 hopper libraries/ I/O 1.8.10.743 cray-hdf5-parallel/1.8.10.743 2013-12-17
HDF5 hopper libraries/ I/O 1.8.11 cray-hdf5/1.8.11 2013-09-25 2013-12-11
HDF5 hopper libraries/ I/O 1.8.4.1 hdf5/1.8.4.1 2013-07-28
HDF5 hopper libraries/ I/O 1.8.4.1 hdf5-parallel/1.8.4.1 2013-07-28
hdf5 hopper libraries/ I/O 1.8.4.1 (Cray version) hdf5/1.8.4.1 2010-08-18 2010-08-18
HDF5 hopper libraries/ I/O 1.8.5.0 hdf5/1.8.5.0 2013-07-28
HDF5 hopper libraries/ I/O 1.8.5.0 hdf5-parallel/1.8.5.0 2013-07-28
hdf5 hopper libraries/ I/O 1.8.5.0 (Cray version) hdf5/1.8.5.0 2010-09-01 2010-09-01
HDF5 hopper libraries/ I/O 1.8.6 hdf5/1.8.6 2013-07-28
HDF5 hopper libraries/ I/O 1.8.6 hdf5-parallel/1.8.6 2013-07-28
hdf5 hopper libraries/ I/O 1.8.6 (Cray version) hdf5/1.8.6 2011-11-07
HDF5 hopper libraries/ I/O 1.8.7 hdf5/1.8.7 2012-01-14 2012-01-25
HDF5 hopper libraries/ I/O 1.8.7 cray-hdf5/1.8.7 2013-07-25
HDF5 hopper libraries/ I/O 1.8.7 hdf5-parallel/1.8.7 2013-07-28
HDF5 hopper libraries/ I/O 1.8.8 hdf5/1.8.8 2012-06-13 2012-08-15
HDF5 hopper libraries/ I/O 1.8.8 cray-hdf5/1.8.8 2013-07-25
HDF5 hopper libraries/ I/O 1.8.8 hdf5-parallel/1.8.8 2013-12-04
HDF5 hopper libraries/ I/O 1.8.9 cray-hdf5/1.8.9 2013-01-15 2013-02-27
hdf5 hopper libraries/ I/O Parallel 1.8.4.1 (Cray version) hdf5-parallel/1.8.4.1 2010-08-18 2010-08-18
hdf5 hopper libraries/ I/O Parallel 1.8.5.0 (Cray version) hdf5-parallel/1.8.5.0 2010-09-01 2010-09-01
hdf5 phoebe pe_libraries/ general 1.8.10-patch1 hdf5/1.8.10-patch1 2013-03-29 2013-06-26
hdf5 phoebe pe_libraries/ general 1.8.11 hdf5/1.8.11 2013-06-18
hdf5 phoebe pe_libraries/ general 1.8.4-patch1 hdf5/1.8.4-patch1 2013-05-15
hdf5 phoebe pe_libraries/ general 1.8.9 hdf5/1.8.9 2012-07-26 2012-07-26
hdf5 phoebe libraries/ general 4.10.2 rpm/4.10.2 2013-05-15 2013-06-26