NERSCPowering Scientific Discovery Since 1974

Damian Hazen

Damian Hazen
Group Lead
Storage Systems Group
National Energy Research Scientific Computing Center
Phone: (510) 495-2599
Fax: (510) 486-6459
Lawrence Berkeley National Laboratory
1 Cyclotron Road
Mailstop: 59R4010A
Berkeley, CA 94720 US


Kirill Lozinskiy, Glenn K. Lockwood, Lisa Gerhardt, Ravi Cheema, Damian Hazen, Nicholas J. Wright, A Quantitative Approach to Architecting All‐Flash Lustre File Systems, Lustre User Group (LUG) 2019, May 15, 2019,

Kirill Lozinskiy, Lisa Gerhardt, Annette Greiner, Ravi Cheema, Damian Hazen, Kristy Kallback-Rose, Rei Lee, User-Friendly Data Management for Scientific Computing Users, Cray User Group (CUG) 2019, May 9, 2019,

Wrangling data at a scientific computing center can be a major challenge for users, particularly when quotas may impact their ability to utilize resources. In such an environment, a task as simple as listing space usage for one's files can take hours. The National Energy Research Scientific Computing Center (NERSC) has roughly 50 PBs of shared storage utilizing more than 4.6B inodes, and a 146 PB high-performance tape archive, all accessible from two supercomputers. As data volumes increase exponentially, managing data is becoming a larger burden on scientists. To ease the pain, we have designed and built a “Data Dashboard”. Here, in a web-enabled visual application, our 7,000 users can easily review their usage against quotas, discover patterns, and identify candidate files for archiving or deletion. We describe this system, the framework supporting it, and the challenges for such a framework moving into the exascale age.

Kirill Lozinskiy, Glenn K. Lockwood, Lisa Gerhardt, Ravi Cheema, Damian Hazen, Nicholas J. Wright, Designing an All-Flash Lustre File System for the 2020 NERSC Perlmutter System, Cray User Group (CUG) 2019, May 7, 2019,

New experimental and AI-driven workloads are moving into the realm of extreme-scale HPC systems at the same time that high-performance flash is becoming cost-effective to deploy at scale. This confluence poses a number of new technical and economic challenges and opportunities in designing the next generation of HPC storage and I/O subsystems to achieve the right balance of bandwidth, latency, endurance, and cost. In this paper, we present the quantitative approach to requirements definition that resulted in the 30 PB all-flash Lustre file system that will be deployed with NERSC's upcoming Perlmutter system in 2020. By integrating analysis of current workloads and projections of future performance and throughput, we were able to constrain many critical design space parameters and quantitatively demonstrate that Perlmutter will not only deliver optimal performance, but effectively balance cost with capacity, endurance, and many modern features of Lustre.

D. Hazen, J. Hick, W. Hurlbert, M. Welcome, Media Information Record (MIR) Analysis, LTUG 2011, April 19, 2011,

Presentation of Storage Systems Group findings from a year-long effort to collect and analyze Media Information Record (MIR) statistics from our in-production Oracle enterprise tape drives at NERSC.  We provide information on the data collected, and some highlights from our analysis. The presentation is primarily intended to declare that the information in the MIR is important to users or customers to better operating and managing their tape environments.

D. Hazen, J. Hick, HPSS v8 Metadata Conversion, HPSS 8.1 Pre-Design Meeting, April 7, 2010,

Provided information about the HPSS metadata conversion software to other developers of HPSS.  Input was important to establishing a design for the version 8 HPSS metadata conversions.


Glenn K. Lockwood, Damian Hazen, Quincey Koziol, Shane Canon, Katie Antypas, Jan Balewski, Nicholas Balthaser, Wahid Bhimji, James Botts, Jeff Broughton, Tina L. Butler, Gregory F. Butler, Ravi Cheema, Christopher Daley, Tina Declerck, Lisa Gerhardt, Wayne E. Hurlbert, Kristy A. Kallback-
Rose, Stephen Leak, Jason Lee, Rei Lee, Jialin Liu, Kirill Lozinskiy, David Paul, Prabhat, Cory Snavely, Jay Srinivasan, Tavia Stone Gibbins, Nicholas J. Wright,
"Storage 2020: A Vision for the Future of HPC Storage", October 20, 2017, LBNL LBNL-2001072,

As the DOE Office of Science's mission computing facility, NERSC will follow this roadmap and deploy these new storage technologies to continue delivering storage resources that meet the needs of its broad user community. NERSC's diversity of workflows encompass significant portions of open science workloads as well, and the findings presented in this report are also intended to be a blueprint for how the evolving storage landscape can be best utilized by the greater HPC community. Executing the strategy presented here will ensure that emerging I/O technologies will be both applicable to and effective in enabling scientific discovery through extreme-scale simulation and data analysis in the coming decade.

Damian Hazen, Jason Hick, "MIR Performance Analysis", June 12, 2012, LBNL LBNL-5896E,


We provide analysis of Oracle StorageTek T10000 Generation B (T10KB) Media Information Record (MIR) Per- formance Data gathered over the course of a year from our production High Performance Storage System (HPSS). The analysis shows information in the MIR may be used to improve tape subsystem operations. Most notably, we found the MIR information to be helpful in determining whether the drive or tape was most suspect given a read or write error, and for helping identify which tapes should not be reused given their history of read or write errors. We also explored using the MIR Assisted Search to order file retrieval requests. We found that MIR Assisted Search may be used to reduce the time needed to retrieve collections of files from a tape volume. 


N. Balthaser, D. Hazen, "HSI Best Practices for NERSC Users", May 2, 2011, LBNL 4745E,


In this paper we explain how to obtain and install HSI, create a NERSC authentication token, and transfer data to and from the system. Additionally we describe methods to optimize data transfers and avoid common pitfalls that can degrade data transfers and storage system performance.