NERSCPowering Scientific Discovery for 50 Years

NERSC History

Advancing High Performance Computing and Data

As a leading HPC center, NERSC is developing technologies that advance the state of the art.

In the emerging field of deep learning for science, training performance on the Cori supercomputer at NERSC was enabled at 15 petaflops in 2017, giving climate scientists the ability to use machine learning to identify extreme weather events in the output of huge climate simulations. Analyzing these datasets is challenging, so researchers from NERSC, Intel, the Montreal Institute for Learning Algorithms, and Microsoft Research teamed up to create a novel, semi-supervised convolutional deep neural network (DDN). Predictive accuracies ranging from 89.4% to as high as 99.1% showed that DDNs can identify weather fronts, tropical cyclones, and atmospheric rivers.

In 2015 NERSC developed and released “Shifter,” a container technology that allows users to bring their custom compute environment to NERSC’s supercomputers. Shifter is based on the Docker container technology, extending its use to HPC systems. Shifter was originally inspired by the need to improve the flexibility and usability of HPC systems for data-intensive workloads, but use cases are expanding to include general HPC workloads. Soon after its initial deployment numerous experimental facilities and academic institutions found that Shifter makes it much easier for them to run their data-centric workloads in an HPC environment. In 2016 the supercomputing company Cray adopted Shifter as an official product. In 2016, NERSC demonstrated that Shifter can be used to run complex, scientific, Python-based codes in parallel on more than 9,000 nodes on Cori using its Intel Xeon Phi processors. Shifter was a finalist for a prized 2018 R&D 100 Award.

When a Cray system based on power-efficient Intel Xeon Phi “Knights Landing (KNL)” cores was selected for “NERSC-8” (aka the Cori system), NERSC knew its large user base would need help porting their codes to run efficiently on that architecture. In 2014 it started the NERSC Exascale Science Application Program (NESAP to enable leading teams and their codes to run efficiently at scale on Cori. NERSC worked collaboratively with 20 application teams, Cray and Intel to prepare key applications for Cori. By the time the system went into production in 2017, NESAP applications had improved their performance on KNL by 350%. At the SC14 conference in New Orleans, NERSC received HPCWire’s 2014 Editors’ Choice Award for Best HPC Collaboration Between Government & Industry. The award recognized NERSC’s partnership with Intel and Cray in preparation for Cori.

NERSC introduced the world’s first HPC all-FLASH file system or “burst buffer” on Cori in 2015. Based on Cray’s DataWarp technology, Cori’s burst buffer achieved a world-best 1.7 TB/second of peak I/O performance with 28M IO operations per second and about 1.8PB of storage. The burst buffer greatly improves I/O performance, particularly for codes that are I/O heavy but can use streaming large-block techniques. Many data analytics applications fall into this category, as their data can often be highly complex and unstructured. The paper “Accelerating Science with the NERSC Burst Buffer,” won the “Best Paper” at the 2016 Cray User Group meeting.