Douglas Jacobsen , Ph.D.
Senior Computing Engineer
Phone: (510) 495-2886
Fax: (510) 486-6459
1 Cyclotron Road
Mailstop: 59R4010ABerkeley, CA 94720 US
Doug Jacobsen completed his Ph.D. in Bioinformatics from the University of Michigan in May 2012. Doug's thesis research was both computational and experimental, integrating biophysics, chemistry, and bioinformatics. He was previously in the NERSC User Services Group working with scientists from the Joint Genome Institute. He moved to the NERSC Computational Systems group where he will be spending part of his time helping deliver services developed for JGI data scientists to the larger NERSC community.
Shane Canon, Doug Jacobsen, "Shifter: Containers for HPC", Cray User Group, London, England, May 13, 2016,
Container-based computed is rapidly changing the way software is developed, tested, and deployed. This paper builds on previously presented work on a prototype framework for running containers on HPC platforms. We will present a detailed overview of the design and implementation of Shifter, which in partnership with Cray has extended on the early prototype concepts and is now in production at NERSC. Shifter enables end users to execute containers using images constructed from various methods including the popular Docker-based ecosystem. We will discuss some of the improvements over the initial prototype including an improved image manager, integration with SLURM, integration with the burst buffer, and user controllable volume mounts. In addition, we will discuss lessons learned, performance results, and real-world use cases of Shifter in action. We will also discuss the potential role of containers in scientific and technical computing including how they complement the scientific process. We will conclude with a discussion about the future directions of Shifter.
Tina Declerck, Katie Antypas, Deborah Bard, Wahid Bhimji, Shane Canon, Shreyas Cholia, Helen (Yun) He, Douglas Jacobsen, Prabhat, Nicholas J. Wright, "Cori - A System to Support Data-Intensive Computing", Cray User Group Meeting 2016, London, England, May 2016,
- Download File: Cori-CUG2016.pdf (pdf: 4.4 MB)
Douglas M. Jacobsen, James F. Botts, and Yun (Helen) He, "SLURM. Our Way.", Cray User Group Meeting 2016, London, England, May 2016,
- Download File: slurm-cug2016-paper.pdf (pdf: 121 KB)
Doug Jacobsen, Shane Canon, "Contain This, Unleashing Docker for HPC", Cray User Group 2015, April 23, 2015,
Doug Jacobsen, "procmon: Real-time process monitoring on the Cray XC-30", Cray User Group 2015, April 21, 2015,
Sudip Dosanjh, Shane Canon, Jack Deslippe, Kjiersten Fagnan, Richard Gerber, Lisa Gerhardt, Jason Hick, Douglas Jacobsen, David Skinner, Nicholas J. Wright, "Extreme Data Science at the National Energy Research Scientific Computing (NERSC) Center", Proceedings of International Conference on Parallel Programming – ParCo 2013, ( March 26, 2014)
Doug Jacobsen, Taylor Groves, Global Aries Counter Collection and Analysis, Cray Quarterly Meeting, July 25, 2017,
Douglas Jacobsen, and Zhengji Zhao, Instrumenting Slurm User Commands to Gain Workload Insight, Proceeding of the Cray User Group Meeting (CUG18), Stockholm, Sweden, May 20, 2017,
Tina Declerck, Katie Antypas, Deborah Bard, Wahid Bhimji, Shane Canon, Shreyas Cholia, Helen (Yun) He, Douglas Jacobsen, Prabhat, Nicholas J. Wright, Cori - A System to Support Data-Intensive Computing, Cray User Group Meeting 2016, London, England, May 12, 2016,
Douglas M. Jacobsen, James F. Botts, and Yun (Helen) He, SLURM. Our Way., Cray User Group Meeting 2016. London, England., May 12, 2016,
- Download File: slurm-cug2016-slides.pdf (pdf: 309 KB)
Doug Jacobsen, Shane Canon, Contain This, Unleashing Docker for HPC, NERSC Webcast, May 15, 2015,
Yushu Yao, NERSC; Douglas Jacobsen, NERSC, Connecting to NERSC, NUG 2014, February 3, 2014,
- Download File: 04-ConnectingToNERSC-NUG2014.pdf (pdf: 2.5 MB)