Container Images for the NERSC Programming Environment

Science/CS domains

High performance computing, container technology

Project description

Containers provide major benefits for HPC applications, including portability across systems, reproducibility of software stacks, and faster onboarding for users who need consistent build and runtime environments. These benefits are especially important as HPC workflows become more complex, combining simulation, data analysis, and AI components.

At the same time, HPC programming environments are inherently complex, with multiple compiler families, MPI variants, math libraries, and toolchain version combinations that users currently access through module stacks.

This project aims to build a solid set of container images that package the NERSC programming environment directly inside the image, with software primarily built using Spack. The goal is for users to run these images as a practical alternative to loading programming environment modules interactively, while preserving performance, compatibility, and usability expectations on NERSC systems.

Project tasks

The project will focus on the following tasks:

  • Defining target container variants that map to common NERSC programming environment choices.
  • Building and validating base images with compiler, MPI, and core HPC library stacks primarily via Spack.
  • Testing compatibility across representative build and runtime workflows used by NERSC users.
  • Measuring usability and performance tradeoffs versus traditional module-based workflows.
  • Publishing documentation and usage guidance so users can adopt image-based programming environments confidently.

Desired skills/background

  • Familiarity with Linux, containers, and image build workflows.
  • Experience with HPC software stacks (compilers, MPI, math/science libraries).
  • Experience with Spack package management and reproducible environment construction.
  • Interest in developer experience, software reproducibility, and platform operations.

Apply to join this project

To apply or ask a question about this project:

Email Pengfei Ding.

Project mentors

Adam Lavely

HPC User Environment Architect

National Energy Research Scientific Computing Center (NERSC)

HPC Technology Dept.

Programming Environments & Models Group

Meet Adam

Justin Cook

Computer Systems Engineer 3

National Energy Research Scientific Computing Center (NERSC)

HPC Technology Dept.

Programming Environments & Models Group

Meet Justin

Pengfei Ding

Data Science Workflows Architect

National Energy Research Scientific Computing Center (NERSC)

Science Engagement & Workflows Dept.

Data Science Engagement Group

Meet Pengfei