NERSCPowering Scientific Discovery Since 1974

Christopher Kerr

Case Study Worksheet

Project Information - Coupled High-Resolution Modeling of the Earth System

Document Prepared By Christopher Kerr
Project Title Coupled High-Resolution Modeling of the Earth System
Principal Investigator V Balaji
Participating Organizations GFDL (NOAA)
Science Category Climate Environmental Science Biological Sciences
Funding Agencies DOE SC DOE NSA NSF NOAA NIH Other:

Project Summary (Scientific Objectives)

Please give a brief description of your project and its scientific objectives for the next 3-5 years.

Current approaches to generating consensus and uncertainty estimates 
of climate change rely on statistical methods comparing results from 
many models. In particular, the CCSP calls out the NOAA/GFDL and NCAR 
models as the flagship models of the US, and central to such 
assessments. The IPCC is a key example of such a process. The design 
of model comparison studies is based in part on the understanding of 
the behaviour of a known suite of models at some target resolution. 
The proposed research addresses key scientific issues that centers 
will each have to tackle independently before making the leap to 
higher resolutions. In short, the significance of this research is 
that it gives us an early look at scientific issues associated with 
becoming able to resolve mesoscale features in the atmospheric and 
ocean circulations, and its implications for understanding of forced 
and natural variability of the climate system. Results from such 
simulations will provide us with insight as to what to expect in the 
near future in terms of understanding regional climate change, and may 
in fact inform the design of international modeling campaigns aimed at 
addressing those questions.

Current HPC Usage and Methods

Facilities Used NERSC NCCS ACLF NSF Centers Other: NOAA/GFDL
Architectures Used Cray XT IBM Power BlueGene Linux Cluster Other: SGI Altix
Total Computational Hours Used per Year 2.5M Core-Hours NERSC Hours Used per Year 1.05M Core-Hours
Number of Cores Used in Typical Production Run 864 Wallclock Hours of Single Typical Production Run 4
Total Memory Used per Run GB Minimum Memory Required per Core GB
Total Data Read & Written per Run GB Size of Checkpoint File(s) GB
Amount of Data Moved In/Out of NERSC GB How Often
On-Line File Storage Required (Directly Accesible from a Running Job) GB Files
Off-Line Archival Storage Required GB Files

Please list any required or important software, services, or infrastructure (beyond supercomputing and standard storage infrastructure) provided by HPC centers or system vendors.

netCDF

Please list your current primary codes and their main mathematical methods and/or algorithms. Include quantities that characterize the size or scale of your simulations or numerical experiments; e.g., size of grid, number of particles, basis sets, etc. Also indicate how parallelism is expressed (e.g., MPI, OpenMP, MPI/OpenMP hybrid)

FMS is a software framework for supporting the efficient development, construction, execution, and scientific interpretation of atmospheric, oceanic, and climate system models. FMS comprises the following: 1. A software infrastructure for constructing and running atmospheric, oceanic, and climate system models. This infrastructure includes software to handle parallelization, input and output, data exchange between various model grids, orchestration of the time stepping, makefiles, and simple sample run scripts. This infrastructure should largely insulate FMS users from machine-specific details. 2. A standardization of the interfaces between various component models. 3. Software for standardizing, coordinating, and improving diagnostic calculations of FMS-based models, and input data preparation for such models. Common preprocessing and post-processing software are included to the extent that the needed functionality cannot be adequately provided by available third-party software. 4. Contributed component models that are subjected to a rigorous software quality review and improvement process. The development and initial testing of these component models is largely a scientific question, and would not fall under FMS. The quality review and improvement process includes consideration of (A) compliance with FMS interface and documentation standards to ensure portability and inter-operability, (B) understandability (clarity and consistency of documentation, comments, interfaces, and code), and (C) general computational efficiency without algorithmic changes. 5. A standardized technique for version control and dissemination of the software and documentation.

Please list the known limitations/obstacles/bottleneck of resources currently available HPC systems, and in particular, those at NERSC.

HPC Usage and Methods for the Next 3-5 Years

Anticipated changes to codes, mathematical methods and/or algorithms needed to achieve this project's scientific objectives.

Computational Hours Required per Year
Anticipated Number of Cores to be Used in a Typical Production Run
Anticipated Wallclock to be Used in a Typical Production Run Using the Number of Cores Given Above
Anticipated Total Memory Used per Run GB
Anticipated Minimum Memory Required per Core GB
Anticipated total data read & written per run GB
Anticipated size of checkpoint file(s) GB
Anticipated On-Line File Storage Required (Directly Accesible from a Running Job) GB Files
Anticipated Off-Line Archival Storage Required GB Files

Known or Anticipated architectural requirements (e.g., 2 GB memory/core).

Please list any additional required or important software, services, or infrastructure beyond those listed in the previous section.

It is believed that the dominant HPC architecture in the next 3-5 years will incorporate processing elements composed of 10s-1,000s of individual cores. It is unlikely that a programming model based solely on MPI will be effective, or even supported, on these machines. Do you have a strategy for computing in such an environment? If so, please briefly describe it.

What Do You Need from NERSC?

Please tell us what you need from NERSC to meet your project's computing needs over the next 3-5 years. Also please feel free to make any general comments.