David Bruhwiler

1.1. Project Information - supporting several NERSC repositories

2. Project Summary & Scientific Objectives for the Next 5 Years

Please give a brief description of your project - highlighting its computational aspect - and outline its scientific objectives for the next 3-5 years. Please list one or two specific goals you hope to reach in 5 years.

Finite-difference time domain (FDTD) electromagnetic simulations, using the parallel VORPAL framework, are being used for rapid high-fidelity simulation and design of a wide range of accelerating structures. 
 
Superconducting radio-frequency (SRF) accelerating cavities are a key technology for existing and future colliders. Present simulation activities include calculation of frequencies, Q’s, modes (fundamental & high-order) and the resulting surface heating, as well as multipacting studies for specific cavity designs. Relevant DOE/HEP applications include LHC, Project X, and the ILC. 
 
Normal conducting (warm) RF cavities and waveguides are also key technologies for present and future accelerator facilities. Present simulations are being used to help understand the difficult and long unsolved problem of RF breakdown. 
Simulations of "magnetic insulation" of novel RF cavities for muon acceleration are underway. Relevant DOE/HEP applications include muon collider and neutrino factor concepts, RF power transport in waveguides, and CLIC-like concepts. 
 
Dielectric structures are an important advanced concept for next generation high-gradient accelerating structures. Two examples include laser-driven photonic band gap (PBG) accelerating cavities and novel, larger-scale RF structures with ultra-high Q and ultra-low wakefields. 
 
In all of the cases described above, multi-physics capabilities are required in the simulations, especially surface physics, such as coupling electromagnetics in vacuum to surface heating and heat transport within the metal, as well as various electron-wall interactions. 
 
 

3. Current HPC Usage and Methods

3a. Please list your current primary codes and their main mathematical methods and/or algorithms. Include quantities that characterize the size or scale of your simulations or numerical experiments; e.g., size of grid, number of particles, basis sets, etc. Also indicate how parallelism is expressed (e.g., MPI, OpenMP, MPI/OpenMP hybrid)

The primary code being used is VORPAL, which is using finite-difference time domain (FDTD) techniques to solve Maxwell's equations on 1D, 2D and 3D Cartesian meshes, and 2D cylindrical (logically Cartesian) meshes. The largest 3D mesh sizes to date have been ~10 million cells, with <100,000 time steps. Parallelism is expressed only through MPI. Parallel I/O capabilities of HDF5 are used. 

3b. Please list known limitations, obstacles, and/or bottlenecks that currently limit your ability to perform simulations you would like to run. Is there anything specific to NERSC?

3c. Please fill out the following table to the best of your ability. This table provides baseline data to help extrapolate to requirements for future years. If you are uncertain about any item, please use your best estimate to use as a starting point for discussions.

Please list any required or important software, services, or infrastructure (beyond supercomputing and standard storage infrastructure) provided by HPC centers or system vendors.

4. HPC Requirements in 5 Years

4a. We are formulating the requirements for NERSC that will enable you to meet the goals you outlined in Section 2 above. Please fill out the following table to the best of your ability. If you are uncertain about any item, please use your best estimate to use as a starting point for discussions at the workshop.

4b. What changes to codes, mathematical methods and/or algorithms do you anticipate will be needed to achieve this project's scientific objectives over the next 5 years.

4c. Please list any known or anticipated architectural requirements (e.g., 2 GB memory/core, interconnect latency < 3 #s).

4d. Please list any new software, services, or infrastructure support you will need over the next 5 years.

4e. It is believed that the dominant HPC architecture in the next 3-5 years will incorporate processing elements composed of 10s-1,000s of individual cores, perhaps GPUs or other accelerators. It is unlikely that a programming model based solely on MPI will be effective, or even supported, on these machines. Do you have a strategy for computing in such an environment? If so, please briefly describe it.

New Science With New Resources

To help us get a better understanding of the quantitative requirements we've asked for above, please tell us: What significant scientific progress could you achieve over the next 5 years with access to 50X the HPC resources you currently have access to at NERSC? What would be the benefits to your research field if you were given access to these kinds of resources?

Please explain what aspects of "expanded HPC resources" are important for your project (e.g., more CPU hours, more memory, more storage, more throughput for small jobs, ability to handle very large jobs).