NERSCPowering Scientific Discovery Since 1974

Burst Buffer

The NERSC Burst Buffer is based on Cray DataWarp that uses flash or SSD (solid-state drive) technology to significantly increase the I/O performance on Cori for all file sizes and all access patterns. 

 

 

Burst Buffer FAQ

The Burst Buffer is available for all to use - and is easy to use! This page is intended as a quick source of information for NERSC users interested in getting started with the Burst Buffer.  Burst Buffer Basics What is a Burst Buffer? The Burst Buffer is a layer of SSD storage in Cori that sits within the high-speed network, between the on-node storage and the Lustre and GPFS file systems. It offers high-performance I/O on a per-job or short-term basis.  Can I use the Burst Buffer? Yes!… Read More »

Burst Buffer Architecture

The Burst Buffer on Cori is a layer of non-volatile storage that sits between the a processors' memory and the parallel file system. The burst buffer will serve to accelerate I/O performance of application on Cori. Read More »

How to use the Burst Buffer

Burst Buffer usage documentation, including example batch scripts and links to interactive tutorials/ Read More »

Known Issues

There are a number of known issues to be aware of when using the Burst Buffer on Cori. This page will be updated as problems are discovered, and as they are fixed.  General Issues  Do not use a decimal point when you specify the burst buffer capacity - slurm does not parse this correctly and will allocate you one grain of space instead of the full request. This is easy to work around - request 3500GB instead of 3.5TB, etc.   Data is at risk in a Persistent Reservation if an SSD fails - there… Read More »

Performance Tuning

Hints and tips on how to optimize your Burst Buffer performance Note: this only applies to the Cori Burst Buffer and should not be taken as general Burst Buffer advice (i.e. your mileage will vary on other systems). This page will be updated as the DataWarp software is updated and performance continues to improve.  For larger files, ensure your Burst Buffer allocation will be striped over multiple nodes Currently, the Burst Buffer granularity is 82GiB in the wlm_pool, and 20.14GiB in the… Read More »