Container Images for the NERSC Programming Environment
Science/CS domains
High performance computing, container technology
Project description
Containers provide major benefits for HPC applications, including portability across systems, reproducibility of software stacks, and faster onboarding for users who need consistent build and runtime environments. These benefits are especially important as HPC workflows become more complex, combining simulation, data analysis, and AI components.
At the same time, HPC programming environments are inherently complex, with multiple compiler families, MPI variants, math libraries, and toolchain version combinations that users currently access through module stacks.
This project aims to build a solid set of container images that package the NERSC programming environment directly inside the image, with software primarily built using Spack. The goal is for users to run these images as a practical alternative to loading programming environment modules interactively, while preserving performance, compatibility, and usability expectations on NERSC systems.
Project tasks
The project will focus on the following tasks:
- Defining target container variants that map to common NERSC programming environment choices.
- Building and validating base images with compiler, MPI, and core HPC library stacks primarily via Spack.
- Testing compatibility across representative build and runtime workflows used by NERSC users.
- Measuring usability and performance tradeoffs versus traditional module-based workflows.
- Publishing documentation and usage guidance so users can adopt image-based programming environments confidently.
Desired skills/background
- Familiarity with Linux, containers, and image build workflows.
- Experience with HPC software stacks (compilers, MPI, math/science libraries).
- Experience with Spack package management and reproducible environment construction.
- Interest in developer experience, software reproducibility, and platform operations.
Apply to join this project
To apply or ask a question about this project:
Project mentors
Adam Lavely
HPC User Environment Architect
National Energy Research Scientific Computing Center (NERSC)
HPC Technology Dept.
Programming Environments & Models Group
Justin Cook
Computer Systems Engineer 3
National Energy Research Scientific Computing Center (NERSC)
HPC Technology Dept.
Programming Environments & Models Group
Pengfei Ding
Data Science Workflows Architect
National Energy Research Scientific Computing Center (NERSC)
Science Engagement & Workflows Dept.
Data Science Engagement Group