NERSCPowering Scientific Discovery for 50 Years

NUG Community Call June 20, 2024

June 20, 2024
Date: Thursday, June 20, 2024
Time: 11:00 AM PST


The Monthly NUG Community Call is a regular opportunity for our users to show off what they've done, for NERSC to get feedback from users, and for users to exchange ideas.

Zoom: Join from PC, Mac, Linux, iOS or Android:
(full information below)

We'll also use the NERSC Users Slack #webinars channel for discussion before, during and after the meeting.

 Add meeting series to calendar


  • Announcements and Calls for Participation: Upcoming conferences, workshops, or other events, that you think might interest or benefit the NERSC user community.
  • NERSC & U: Find out about the latest programmatic efforts by NERSC and the Department of Energy to accelerate and electrify scientific advancements.
  • Topic-of-the-day: Mukul Dave, PhD. will present a NUG Science Talk on accelerating wind energy simulations by reducing communication overheads and then dig deeper into the  CUDA-aware MPI on Perlmutter.

NUG Talk Abstracts:

  • Accelerating wind energy simulations by reducing communication costs

Wind farm simulations require data from mesoscale atmospheric simulations as initial and boundary conditions for the microscale turbine environments. The Energy Research and Forecasting (ERF) code bridges this scale gap and provides an efficient GPU-enabled parallel implementation with adaptive mesh refinement through the underlying AMReX framework. I will present my takeaways from performance engineering for ERF and outline strategies that reduce the overhead of communication among parallel processes.

  • Enabling CUDA-aware MPI on Perlmutter to accelerate scientific applications

High throughput computation on GPUs reveals communication as the primary bottleneck. CUDA-aware MPI can accelerate applications by enabling direct GPU-GPU transfers on Perlmutter. Currently, this requires manually specifying a CPU-GPU-NIC binding. I will explain these settings and demonstrate the performance improvement for the ERF atmosphere modeling code.


As a NESAP for Simulation postdoc at NERSC, Mukul contributes to research software development, enabling and porting codes to make use of the next-generation compute platforms. He works on the Energy Research and Forecasting (ERF) package, which couples mesoscale atmospheric flows with microscale wind farms, and is built upon the AMReX framework for block-structured simulations with adaptive mesh refinement. Mukul's doctoral research at the University of Wisconsin–Madison involved simulating cross-flow turbines that rotate on an axis perpendicular to the flow to harvest energy from flowing wind or water. Prior to that, he worked as an HPC consultant at the Brown University Compute Center.

Video Recording:

Presentation Slides:

NUG Community Call Introduction Slides

Accelerating wind energy simulations by reducing communication costs Slides by Mukul Dave, PhD

CUDA-AWARE MPI Communication Improvement Slides by Mukul Dave, PhD