John Shalf
John
M.
Shalf
Group Lead,
Advanced Technologies Group,
Phone: (510) 486-4508 | (510) 316-9427,
Fax: (510) 486-4316
1 Cyclotron Road
Mail Stop 943-256
Berkeley,
CA
94720
US
Biographical Sketch
John Shalf is Group leader for the NERSC Advanced Technology Group, which tries to understand the NERSC scientific computing workload and how it affects computer architecture of future HPC systems. His background is in electrical engineering: he spent time in graduate school at Virginia Tech working on a C-compiler for the SPLASH-2 FPGA-based computing system, and at Spatial Positioning Systems Inc. (now ArcSecond) he worked on embedded computer systems. John first got started in HPC at the National Center for Supercomputing Applications (NCSA) in 1994, where he provided software engineering support for a number of scientific applications groups. While working for the General Relativity Group at the Albert Einstein Institute in Potsdam Germany, he helped develop the first implementation of the Cactus Computational Toolkit, which is used for numerical solutions to Einstein's equations for General Relativity and which enables modeling of black holes, neutron stars, and boson stars. He also developed the I/O infrastructure for Cactus, including a high performance self-describing file format for storing Adaptive Mesh Refinement data called FlexIO. John joined Berkeley Lab in 2000 and has worked in the Visualization Group, on the RAGE robot, which won an R&D100 Award in 2001, and led international high performance networking teams to win to consecutive SciNET bandwidth challenges in 2001-2002. He is a member of the DOE Exascale Steering committee, and is a co-author of the landmark "View from Berkeley" paper as well as the DARPA Exascale Software Report. He currently leads the NERSC Advanced Technology Group (ATG) that leads projects in exascale technology research such as CoDEx (CoDesign for Exascale), and the LBNL Green Flash project ( http://www.lbl.gov/cs/html/greenflash.html ) that seeks to develop energy efficient scientific computing systems using manycore and embedded technologies.
Journal Articles
J. Dongarra, P. Beckman, T. Moore, P. Aerts, G. Aloisio, J.C. Andre, D. Barkai, J.Y. Berthou, T. Boku, B. Braunschweig, others, “The international exascale software project roadmap”, International Journal of High Performance Computing Applications, January 2011, 25:3--60,
John Shalf, Quinlan, Curtis Janssen, “Rethinking Hardware-Software Codesign for Exascale Systems”, IEEE Computer, January 1, 2011, 44:22-30,
K. Datta, S. Williams, V. Volkov, J. Carter, L. Oliker, J. Shalf, K. Yelick, “Auto-Tuning Stencil Computations on Diverse Multicore Architectures”, Scientific Computing with Multicore and Accelerators, edited by Jakub Kurzak, David A. Bader, Jack Dongarra, 2010,
Shoaib Kamil, Oliker, Pinar, John Shalf, “Communication Requirements and Interconnect Optimization for High-End Scientific Applications”, IEEE Trans. Parallel Distrib. Syst., 2010, 21:188-202,
Julian Borrill, Oliker, Shalf, Shan, Andrew Uselton, “HPC global file system performance analysis using a scientific-application derived benchmark”, Parallel Computing, January 1, 2009, 35:358-373,
David Donofrio, Oliker, Shalf, F. Wehner, Rowen, Krueger, Kamil, Marghoob Mohiyuddin, “Energy-Efficient Computing for Extreme-Scale Science”, IEEE Computer, January 1, 2009, 42:62-71,
Samuel Williams, Carter, Oliker, Shalf, Katherine A. Yelick, “Optimization of a lattice Boltzmann computation on state-of-the-art multicore platforms”, Journal of Parallel and Distributed Computing, January 1, 2009, 69:762-777,
Kaushik Datta, Kamil, Williams, Oliker, Shalf, Katherine A. Yelick, “Optimization and Performance Modeling of Stencil Computations on Modern Microprocessors”, SIAM Review, January 1, 2009, 51:129-159,
S. Williams, L. Oliker, R. Vuduc, J. Shalf, K. Yelick, J. Demmel, “Optimization of sparse matrix-vector multiplication on emerging multicore platforms”, Parallel Computing, January 1, 2009, 35:178--194,
Sam Williams, Kaushik Datta, Jonathan Carter, Leonid Oliker, John Shalf, Katherine Yelick, David Bailey, “PERI - Auto-tuning Memory Intensive Kernels for Multicore”, SciDAC: Scientific Discovery Through Advanced Computing, Journal of Physics: Conference Series, January 1, 2008,
- Download File: scidac08peri.pdf (pdf: 889 KB)
Leonid Oliker, Andrew Canning, Jonathan Carter, John Shalf, Stephane Ethier, “Scientific Application Performance On Leading Scalar and Vector Supercomputering Platforms”, IJHPCA, January 1, 2008, 22:5-20,
Michael F. Wehner, Oliker, John Shalf, “Towards Ultra-High Resolution Models of Climate and Weather”, IJHPCA, January 1, 2008, 22:149-165,
John Shalf, “The New Landscape of Parallel Computer Architecture”, Journal of Physics: Conference Series, IOP Electronics Journals, January 1, 2007,
J. Levesque, J. Larkin, M. Foster, J. Glenski, G. Geissler, S. Whalen, B. Waldecker, J. Carter, D. Skinner, H. He, others, “Understanding and mitigating multicore performance issues on the amd opteron architecture”, January 1, 2007,
Erik Schnetter, Ott, Allen, Diener, Goodale, Radke, Seidel, John Shalf, “Cactus Framework: Black Holes to Gamma Ray Bursts”, CoRR, January 1, 2007, abs/0707,
S. Williams, J. Shalf, L. Oliker, S. Kamil, P. Husbands, K. Yelick, “Scientific computing kernels on the Cell processor”, International Journal of Parallel Programming, January 1, 2007, 35:263--298,
Tom Goodale, Shantenu Jha, Hartmut Kaiser, Thilo Kielmann, Pascal Kleijer, Gregor von Laszewski, Craig Lee, Andre Merzky, Hrabri Rajic, Hrabri, John Shalf, “SAGA: A Simple API for Grid Applications -- High-Level Application Programming on the Grid”, Computational Methods in Science and Technology, January 1, 2006, 12,
Ruxandra Bondarescu, Allen, Daues, Kelley, Russell, Seidel, Shalf, Malcolm Tobias, “The Astrophysics Simulation Collaboratory Portal: a framework for effective distributed research”, Future Generation Comp. Syst., January 1, 2005, 21:259-270,
Leonid Oliker, Andrew Canning, Jonathan Carter, John Shalf, David Skinner, Stephane Ethier, Rupak Biswas, Jahed Djomehri, Rob F. Van der Wijngaart, “Performance evaluation of the SX-6 vector architecture for scientific computations”, Concurrency - Practice and Experience, January 1, 2005, 17:69-93,
Horst Simon, William Kramer, William Saphir, John Shalf, David Bailey, Leonid Oliker, Michael Banda, C. William McCurdy, John Hules, Andrew Canning, Marc Day, Philip Colella, David Serafini, Michael Wehner, Peter Nugent, “Science-Driven System Architecture: A New Process for Leadership Class Computing”, Journal of the Earth Simulator, January 1, 2005, 2,
Gabrielle Allen, Goodale, Radke, Russell, Seidel, Davis, Dolkas, D. Doulamis, Kielmann, Merzky, Nabrzyski, Pukacki, Shalf, Ian J. Taylor, “Enabling Applications on the Grid: A Gridlab Overview”, IJHPCA, January 1, 2003, 17:449-466,
John Shalf, E. Wes Bethel, “The Grid and Future Visualization System Architectures”, IEEE Computer Graphics and Applications, January 1, 2003, 23:6-9,
E. Wes Bethel, John Shalf, “Grid-Distributed Visualizations Using Connectionless Protocols”, IEEE Computer Graphics and Applications, January 1, 2003, 23:51-59,
T. J. Jankun-Kelly, Kreylos, Ma, Hamann, I. Joy, Shalf, E. Wes Bethel, “Deploying Web-Based Visual Exploration Tools on the Grid”, IEEE Computer Graphics and Applications, January 1, 2003, 23:40-50,
Gregor von Laszewski, Russell, T. Foster, Shalf, Allen, Daues, Novotny, Edward Seidel, “Community software development with the Astrophysics Simulation Collaboratory”, Concurrency and Computation: Practice and Experience, January 1, 2002, 14:1289-1301,
Michael Russell, Allen, Daues, T. Foster, Seidel, Novotny, Shalf, Gregor von Laszewski, “The Astrophysics Simulation Collaboratory: A Science Portal Enabling Community Software Development”, Cluster Computing, January 1, 2002, 5:297-304,
Gabrielle Allen, Angulo, T. Foster, Lanfermann, Liu, Radke, Seidel, John Shalf, “The Cactus Worm: Experiments with Dynamic Resource Discovery and Allocation in a Grid Environment”, IJHPCA, January 1, 2001, 15:345-358,
Gabrielle Allen, Benger, Dramlitsch, Goodale, Hege, Lanfermann, Merzky, Radke, Seidel, John Shalf, “Cactus Tools for Grid Applications”, Cluster Computing, January 1, 2001, 4:179-188,
Gabrielle Allen, Tom Goodale, Gerd Lanfermann, Thomas Radke, Ed Seidel, Werner Benger, Hans Christian Hege, Andre Merzky, Juan Masso, John Shalf, “Solving Einstein's Equations on Supercomputers”, IEEE Computer, January 1, 1999, 32:52-58,
Michael L. Norman, John Shalf, Stuart Levy, Greg Daues, “Diving Deep: Data-Management and Visualization Strategies for Adaptive Mesh Refinement Simulations”, Computing in Science and Engineering, January 1, 1999, 1:36-47,
M. L. Norman, P. Beckman, G. L. Bryan, J. Dubinski, D. Gannon, L. Hernquist, K. Keahey, J. P. Ostriker, J. Shalf, J. Welling, and S. Yang, “Galaxies Collide on the I-WAY: An Example of Heterogeneous Wide-Area Collaborative Supercomputing”, The International Journal of Supercomputer Applications and High Performance Computing, January 1, 1996, 10:132-144,
Conference Papers
Robert Preissl, Wichmann, Long, Shalf, Ethier, Alice E. Koniges, “Multithreaded global address space communication techniques for gyrokinetic fusion applications on ultra-scale platforms”, SC, November 1, 2011, 15,
Kamesh Madduri, Z. Ibrahim, Williams, Im, Ethier, Shalf, Leonid Oliker, “Gyrokinetic toroidal simulations on leading multi- and manycore HPC systems”, SC, November 1, 2011, 23,
Samuel Williams, Oliker, Carter, John Shalf, “Extracting ultra-scale Lattice Boltzmann performance via hierarchical and distributed auto-tuning”, SC, November 1, 2011, 55,
Jens Krueger, Donofrio, Shalf, Mohiyuddin, Williams, Oliker, Franz-Josef Pfreund, “Hardware/software co-design for energy-efficient seismic modeling”, SC, November 1, 2011, 73,
Keren Bergman, Hendry, Hargrove, Shalf, Jacob, Scott Hemmert, Rodrigues, David Resnick, “Let there be light!: the future of memory systems is photonics and 3D stacking”, MSPC, January 1, 2011, 43-48,
Jack Dongarra, John Shalf, David Skinner, Kathy Yelick, “International Exascale Software Project (IESP) Roadmap, version 1.1”, October 18, 2010,
- Download File: IESP-roadmap-1.1.pdf (pdf: 2.4 MB)
Gilbert Hendry, Johnnie Chan, Shoaib Kamil, Lenny Oliker, John Shalf, Luca P. Carloni, and Keren Bergman, “Silicon Nanophotonic Network-On-Chip Using TDM Arbitration”, IEEE Symposium on High Performance Interconnects (HOTI) 5.1, August 2010,
J. A. Colmenares, S. Bird, H. Cook, P. Pearce, D. Zhu, J. Shalf, K. Asanovic, J. Kubiatowicz, “Resource Management in the Tessellation Manycore OS”, Proceedings of the Second USENIX Workshop on Hot Topics in Parallelism (HotPar'10), Berkeley, California, June 2010,
Alice Koniges, Robert Preissl, Stephan Ethier, John Shalf, “What’s Ahead for Fusion Computing?”, International Sherwood Fusion Theory Conference, Seattle, Washington, April 2010,
John Shalf, S. Dosanjh, John Morrison, “Exascale Computing Technology Challenges”, VECPAR, Berlin, Heidelberg, Springer-Verlag, 2010, 1-25,
- Download File: ShalfVecpar2010.pdf (pdf: 1.1 MB)
High Performance Computing architectures are expected to change dramatically in the next decade as power and cooling constraints limit increases in microprocessor clock speeds. Consequently computer companies are dramatically increasing on-chip parallelism to improve performance. The traditional doubling of clock speeds every 18-24 months is being replaced by a doubling of cores or other parallelism mechanisms. During the next decade the amount of parallelism on a single microprocessor will rival the number of nodes in early massively parallel supercomputers that were built in the 1980s. Applications and algorithms will need to change and adapt as node architectures evolve. In particular, they will need to manage locality to achieve performance. A key element of the strategy as we move forward is the co-design of applications, architectures and programming environments. There is an unprecedented opportunity for application and algorithm developers to influence the direction of future architectures so that they meet DOE mission needs. This article will describe the technology challenges on the road to exascale, their underlying causes, and their effect on the future of HPC system design.
Keith R. Jackson, Ramakrishnan, Muriki, Canon, Cholia, Shalf, J. Wasserman, Nicholas J. Wright, “Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud”, CloudCom, Bloomington, Indiana, January 1, 2010, 159-168,
Shoaib Kamil, Chan, Oliker, Shalf, Samuel Williams, “An auto-tuning framework for parallel multicore stencil computations”, IPDPS, January 1, 2010, 1-12,
Lavanya Ramakrishnan, R. Jackson, Canon, Cholia, John Shalf, “Defining future platform requirements for e-Science clouds”, SoCC, New York, NY, USA, 2010, 101-106,
Andrew Uselton, Howison, J. Wright, Skinner, Keen, Shalf, L. Karavanic, Leonid Oliker, “Parallel I/O performance: From events to ensembles”, IPDPS, Atlanta, Georgia, 2010, 1-11,
Kamesh Madduri, Williams, Ethier, Oliker, Shalf, Strohmaier, Katherine A. Yelick, “Memory-efficient optimization of Gyrokinetic particle-to-grid interpolation for multicore processors”, SC, January 1, 2009,
Marghoob Mohiyuddin, Murphy, Oliker, Shalf, Wawrzynek, Samuel Williams, “A design methodology for domain-optimized power-efficient supercomputing”, SC, January 1, 2009,
Joseph Gebis, Oliker, Shalf, Williams, Katherine A. Yelick, “Improving Memory Subsystem Performance Using ViVA: Virtual Vector Architecture”, ARCS, January 1, 2009, 146-158,
Gilbert Hendry, Kamil, Biberman, Chan, G. Lee, Mohiyuddin, Jain, Bergman, P. Carloni, Kubiatowicz, Oliker, John Shalf, “Analysis of photonic networks for a chip multiprocessor using scientific applications”, NOCS, January 1, 2009, 104-113,
Brian van Straalen, Shalf, J. Ligocki, Keen, Woo-Sun Yang, “Scalability challenges for massively parallel AMR applications”, IPDPS, January 1, 2009, 1-12,
H. Shan, K. Antypas, J.Shalf., “Characterizing and Predicting the I/O Performance of HPC Applications Using a Parameterized Synthetic Benchmark.”, Supercomputing, Reno, NV, November 17, 2008,
K. Datta, M. Murphy, V. Volkov, S. Williams, J. Carter, L. Oliker, D. Patterson, J. Shalf, K. Yelick, “Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures”, Proceedings of the 2008 ACM/IEEE Conference on High Performance Computing and Networking ("Supercomputing", SC08), November 15, 2008,
- Download File: sc08final2.pdf (pdf: 598 KB)
Samuel Williams, Carter, Oliker, Shalf, Katherine A. Yelick, “Lattice Boltzmann simulation optimization on leading multicore platforms”, International Parallel and Distributed Processing Symposium (IPDPS), April 14, 2008, 1-14,
- Download File: LBMHDipdps08final.pdf (pdf: 560 KB)
