SciDAC Projects at NERSC

• 1. Biological Sciences
  • 1.1 Molecular Dynamics Simulations in Bioenergy, Bioremediation and Protein Dynamics
• 2. Climate Change
  • 2.1 River Transport and Hydrology in CCSM
  • 2.2 Dynamics of Arctic and sub-Arctic climate and atmospheric circulation: diagnosis of mechanisms and biases using data assimilation
  • 2.3 Interaction of Atmospheric Chemistry and Aerosols with Climate
  • 2.4 Simulation of 20th-century and 21st-century climate and composition with interactive chemistry in CAM3: identifying the role of climate and climate composition in simulated trends and air quality
  • 2.5 Decadal Climate Studies with Enhanced Variable and Uniform Resolution GCMs Using Advanced Numerical Techniques
  • 2.6 Towards the prediction of decadal to mult-century processes in a high-throughput climate system model
  • 2.7 The Role of Eddies in the Meridional Overturning Circulation
  • 2.8 Continuous Dynamic Grid Adaptation in a Global Atmospheric Model
  • 2.9 Three-dimensional Global Atmospheric Aerosol and Chemistry Modeling
  • 2.10 Carbon Data Assimilation with a coupled Ensemble Kalman Filter
  • 2.11 Global Cloud Modeling
• 3. Environmental Science
  • 3.1 Hybrid Numerical Methods for Multiscale Simulations of Subsurface Biogeochemical Processes
• 4. Fusion Energy
  • 4.1 Magnetic Fusion Plasma Microturbulence Project
  • 4.2 Fluid and Hybrid Modeling of Electromagnetic Activity in MFE Plasmas
  • 4.3 Magnetic Reconnection Studies and Pellet Injection/ELMs in Tokamaks using AMR
  • 4.4 Center for Simulation of Wave-Plasma Interactions: SciDAC Project
  • 4.5 Turbulent Transport and Multiscale Gyrokinetic Simulation
  • 4.6 Project Title Application of Parallel Kinetic Closures in NIMROD for Fusion Plasma Simulations
  • 4.7 SciDAC GSEP: Gyrokinetic Simulation of Energetic Particle Turbulence and Transport
• 5. See also

1. Biological Sciences

1.1 Molecular Dynamics Simulations in Bioenergy, Bioremediation and Protein Dynamics

Lignin in Aqueous Solution: Hydrolysis of cell-wall cellulose is the critical rate-limiting step in cellulosic ethanol biofuel production (Himmel et al. 2007). Characterization of the structure of lignocellulosic biomass before and after pretreatment is fundamental to overcoming cell-wall recalcitrance to hydrolysis. Molecular simulation is required to obtain an understanding of the structure, dynamics and degradation pathways of extended cellulosic and lignocellulosic materials. The physical properties of lignocellulosic biomass thus derived will serve as a basis for interpreting an array of biophysical experiments, and, in particular, the simulation models derived will be used to calculate and interpret a variety of neutron-scattering properties, a development responding to the improvements in sensitivity expected with currently-operating and planned neutron scattering instruments at Oak Ridge National Laboratory. This combination of simulation and experiment will eventually lead to a description of the physicochemical mechanisms of biomass recalcitrance to hydrolysis, and thus will aid in developing a strategy as to how rationally to overcome the resistance.

We have embarked on a program of simulation of lignocellulosic biomass that partly involves high-performance simulation of ~1-2M atom systems using a 2008 DOE INCITE award on the ORNL Cray XT4 computer. Here, we wish to investigate lignin in aqueous solution. Simulations on a representative variety of lignin topologies must be performed: 26 systems, with differing chemical composition and branching properties, are required to statistically sample typical softwood lignins.

Many microorganisms are known to directly biotransform contaminants into innocuous or immobile forms and have great potential for restoring contaminated environments inexpensively and effectively. However, much remains to be learned about the molecular scale function of these subsurface microbial communities as well as the detailed contaminant biotransformation processes. Mercury is a key contaminant, and understanding the biotransformation affecting the mobility and chemical forms of mercury is of fundamental importance for developing more realistic models for contaminant transformation. Methyl mercury in particular is a regulatory concern because of bioaccumulation in fish. Thus, the identification of mechanisms that can limit the existence of methyl mercury in water as well as its production is urgently needed. MerA, the enzyme to be studied in the proposed research, is a mercuric reductase that has been a subject of extensive experimental investigations in the past. The key problems to be addressed in this project comprises how Hg(II) is transferred from to the inner active site of the catalytic core domain of MerA, what the reductive and catalytic mechanisms used by MerA and MerB are, and what detailed conformational changes of the NmerA domain of MerA are involved during its function. In a first set of calculations, the pathways and mechanisms for the binding of Hg(CN)2, Hg(SCN)2, HgBr2 and Hg(Cys)2 to the inner pair of Cys residues through the C-terminal pathway in MerA and its CCAA mutant will be explored. In a second set of calculations the pathways for the binding of Hg(CN)2, HgBr2 and Hg(Cys)2 to the inner pair of Cys residues through a narrow pathway identified in the crystal structure will be explored. Finally, we will examine the mechanisms for the direct transfer of Hg(II) from NmerA to the C-terminal cysteine pair. The knowledge obtained from this research may also provide important information for understanding other Hg(II) transfer processes between proteins (e.g., between NmerA and MerB or MerT).

2. Climate Change

2.1 River Transport and Hydrology in CCSM

The purpose of the project is to improve the hydrology in the Community Climate System Model. The main focus in on the river transport component. One of the primary goals is to change the framework of the component so that it can carry chemical species and sediments along with the water in the channel flow. Another goal is to test the new catchment framework that has been implemented. Evaluation of the hydrology in light of the new biogeochemical components added to CCSM is one of the goals, as are high resolution hydrology studies of climate resulting from increased use of biofuels with a new fine-mesh framework version of CLM. Another goal is to increase the capability of the component with regard to water storage, such as lakes, dams, and wetlands, adding the ability to more accurately model floods. Increasing the resolution of the existing RTM will be explored with the new parallel framework that has been implemented. The results of the new hydrology will be evaluated in the fully coupled CCSM as will the effects of various land use change and emissions scenarios with regard to the hydrology and model changes.

2.2 Dynamics of Arctic and sub-Arctic climate and atmospheric circulation: diagnosis of mechanisms and biases using data assimilation

The goal of this research is to enhance the representation of the climate and circulation of the Arctic in models used to predict and assess climate change. One important issue in the Arctic is the extent to which poor surface wind simulation leads to errors in the thickness and distribution of sea ice, which in turn affects the loss of Arctic sea ice in future climate simulations. Papers like Rind et al. (JCLIM 1995) find that a substantial fraction of anthropogenically induced warming (40% in their model) comes from feedbacks involving Arctic sea ice. This claim seems particularly significant now, given the unprecedented loss of Arctic sea ice in recent years. A second issue is the extent to which the spread in simulations of present and future sea ice extent, concentration, and thickness can be understood in terms of other aspects of the simulation, such as the surface energy budget of the Arctic. Further research conducted under this project seeks to understand the dynamics and thermodynamics which lead to robust changes in the hydrological cycle of the global in climate change simulations prepared for the IPCC Fourth Assessment Report.

2.3 Interaction of Atmospheric Chemistry and Aerosols with Climate

We have developed multiple interactive atmospheric chemistry and aerosol capabilities for the Community Climate System Model (CCSM) that can stand alone or couple to interactive land and ocean biogeochemistry (eg, DMS and halocarbons from the ocean, and nitrogen deposition plus ozone damage on the land) to create a cutting edge Earth System Model. This is enabling studies of the interactions between chemistry and sulfur aerosols with the climate and biosphere, and thereby improve our understanding of the climate's response to anthropogenic forcing. This is part of the SciDAC CCSM consortium project.

Our atmospheric chemistry capabilities are based on the capabilities of the CCSM coupled climate model plus the IMPACT and MOZART atmospheric chemistry models developed at LLNL and NCAR respectively. The chemistry capabilities now includes one fast chemical mechanism with the capability to simulate the stratosphere as well as the troposphere, and an even faster mechanism for tropospheric only simulations.

With this tool we are examining: (1) the impact of interactivity between chemistry and climate, (2) the interannual and interdecadal variability in chemical species and climate variables, and (3) the interaction between the sulfur cycles in the ocean and the atmosphere.

We are also continuing to work with SciDAC colleagues to: (1) increase the interactions between atmospheric chemistry and the biosphere, and (2) improve the scaling and performance of our code.

Collectively, this work is improving our understanding of, and capability to model, feedbacks within the atmospheric chemistry-aerosols-biosphere-climate system. These feedbacks could significantly alter climate sensitivity to anthropogenic forcings, and even provide modes that result in abrupt climate change.

2.4 Simulation of 20th-century and 21st-century climate and composition with interactive chemistry in CAM3: identifying the role of climate and climate composition in simulated trends and air quality

During the last 30 years of the 20-th century some very large changes have been observed in the distribution of chemical species in the atmosphere; additional and different changes are expected to occur in the 21st century. In particular, the release of CFCs has had a very detrimental impact on the stratospheric ozone layer. The model that we will use is capable of taking into account those changes in CFCs and reproduce with very high skill (it is as skilled as the equivalent simulation with WACCM) level. It also has the avantage of being relatively inexpensive (compared to WACCM) and sensitivity experiments can then be tackled.

We propose to continue the studies of present-day and 21st-century climate we have performed this year; these simulations use a modified version of the CAM4 model that interactively simulates the distribution of atmospheric chemical species (valid for tropospheric and stratospheric chemistry), including aerosols. During FY 2008, we have have extensively analyzed the performance of this model and shown that it is capable of reproducing many of the observed trends in the lower stratosphere. We have then performed a set of sensitivity simulations to identify the role of specific agents (methane, carbon dioxide, ozone-depleting substances) in driving the observed changes in temperature, stratospheric circulation and ozone distribution. Results from those simulations are published in the Journal of Geophysical Research (Atmospheres) in June 2008. The new simulations to be performed will address specific questions about the longer-term trends in composition, and its evolution into the 21st century and beyond; the stratospheric component of the proposed activity is the main topic of the CCMval intercomparison exercise to which we will participate. The tropospheric counterpart is of direct relevance to climate forcing for IPCC.

As WACCM will perform the same CCMval simulations with the same model as the one we use but with a much higher model top (and the associated processes relevant only in the mid to upper stratosphere and lower mesosphere), we will be able to clearly identify the role and importance (or not) of including a comprehensive representation of the stratosphere.

In addition, it is expected that emissions and atmospheric conditions in the lower atmosphere will considerably change during the 21st century. Due to the available representation of tropospheric chemistry in our model, we will be able to tackle air quality changes using the simulations mentioned at the top of this section. These tropospheric studies will be complementary to the IPCC simulations of the 21st century climate performed using CCSM and also prepare ourselves for tackling interesting new questions for IPCC AR-5.

2.5 Decadal Climate Studies with Enhanced Variable and Uniform Resolution GCMs Using Advanced Numerical Techniques

The joint U.S.-Canadian effort is devoted to further producing more accurate and computationally efficient decadal regional and global climate simulations at meso- and larger scales. The U.S. and international GCMs with enhanced to mesoscales variable and uniform resolution are used for studying decadal global and regional (U.S./Canadian and other regions) climate variability and predictability. The emphasis (consistent with the SciDAC and CCPP goals) is on exploring the advanced numerical techniques, multi-model ensembles, and computationally efficient codes for the over 10 teraOPS SciDAC supercomputers. The study includes methodological and experimental comparisons of stretched and nested grid approaches. The international SGMIP-2 (Stretched-Grid Model Intercomparison Project, phase-2), using state-of-the-art variable and uniform resolution GCMs developed at major centers/groups in the U.S., Canada, France, and Australia, has been initiated and successfully conducted using the SciDAC NERSC terra-scale supercomputers. The results for the U.S. multiyear climate are available at the SGMIP web site: http://essic.umd.edu/~foxrab/sgmip.html. The international SGMIP-2 effort, , with the accompanying comparisons of enhanced (0.5 degree or finer) uniform and variable resolution GCMs, puts us in a favorable position for a comprehensive investigation on the diversified impacts on climate simulations due to enhanced global and/or regional model resolution, including the multi-model ensemble results. Maintaining the developed and evolving SGMIP web site allows us to: disseminate the SGMIP data/products and analysis results to the climate modeling community; to provide consultations on demand to potential users of SGMIP-2 data; to make SGMIP-2 data/products and analysis results available on demand to national and international programs and groups such as WMO/WCR/WGNE, CLIVAR, and IPCC.

The study is a joint coordinated effort of the U.S. and Canadian teams combining their significant expertise on climate modeling using advanced numerical and parallelization techniques. We are also collaborating with F. Baer and J. Tribbia on implementation of their version of NCAR CAM-SEAM with the spectral-element dynamics for SGMIP-2. Participation of Michel Deque (Meteo-France) and John McGregor (CSIRO, Australia) brings their outstanding personal expertise on climate modeling and climate change as well as potential collaborations with their centers. The U.S. component of the research is conducted at the ESSIC (Earth System Sciences Interdisciplinary Center), University of Maryland. The Canadian component of the study is conducted mostly at the UQAM/Ouranos and partly at RPN/CMC. Our international project reflects a growing interest and trend in climate modeling and broader communities to move towards more detailed regional and global climate and climate change assessments important for the U.S. public, business and policy decision-makers, and for productive international collaborations on climate-related issues.

2.6 Towards the prediction of decadal to mult-century processes in a high-throughput climate system model

The prediction of decadal to multi-century climate processes and their impact on the global environment provides a great challenge to the climate community. Observational data are usually of short duration and imperfect, making it difficult to extract interdecadal climate signals with high statistical significance. Coupled ocean-atmosphere models therefore provide an essential tool for the understanding and prediction of these long-term changes of climate processes. So far, however, our modeling effort to understand climate variability has been hampered by several factors. First, this long-term climate variability involves complex processes of ocean-atmosphere feedbacks and teleconnections, while our means of understanding these processes in a climate model are very limited. Furthermore, most previous coupled simulations are insufficient for extracting statistically significant signals for these long-term climate changes, constrained by the limited model output or the high computational cost of a coupled model. Finally, these climate changes could be affected substantially by processes outside the ocean-atmosphere system, such as terrestrial ecosystem feedbacks, because of the overlap of the inherent time scales with the terrestrial ecosystem. Unfortunately, our understanding of these feedbacks, however, are extremely poor.

We propose to apply a fast climate system model to the understanding and prediction of decadal to multi-century climate processes and their interaction with the terrestrial ecosystem. To obtain robust climate signals, we will perform multi-century ensemble experiments with a high- throughput climate system model, initially the FOAM, but with the planned transition to a high throughput CCSM. To help understand the coupled feedbacks and teleconnections, we will apply several new modeling strategies, such as partial coupling (PC), partial blocking (PB) and Equilibrium Asynchronous Coupling (EAC). To understand the feedback of terrestrial vegetation systems, we will further study the coupled climate-land ecosystem system by coupling a dynamic land vegetation model to our climate model. All the knowledge gained is aimed at improvements that can be applied to the future use of the CCSM model. We will study climate processes in a coupled atmosphere-ocean model, including both the interdecadal natural climate variability and the forced response to increased atmospheric CO2. We will further study the interaction between climate processes and terrestrial ecosystems using a coupled atmosphere-ocean-land vegetation model. To perform this complex task, we have assembled a multi-institution, interdisciplinary science team with expertise in climate, computational optimization and ecosystem modeling (Z. Liu, J. Kutzbach at UW-Madison, R. Jacob at Argonne Nat. Lab., C. Prentice at the Max Planck Institute for Biogeochemistry in Germany.) It is expected that this work will enhance our understanding and prediction capability of decadal to multi-century climate changes, and contribute significantly to a more efficient application of these studies to CCSM in the future.

These experiments are important initial steps to understanding these complex processes. The significance of this work is not only to better understand these processes but to also develop new strategies for investigating these relationships. We are working on building new modeling techniques to better expose the mechanisms of interdecadal climate variability.

2.7 The Role of Eddies in the Meridional Overturning Circulation

The primary goal of the proposed research is to determine the dependence of the deep stratification and the strength of the MOC on the external parameters of the system---most importantly the diffusivity, K, the pole-to-pole temperature gradient DT, and the wind stress---by systematically varying these parameters in a series of simulations using the two well-equilibrated two-hemisphere runs At the moment, there has only been one effort (beside ours) to examine systematically the role of eddies in the maintenance of the stratification on the basin scale. The group involved is located at the Geophysical Fluid Dynamics Laboratory, Princeton, NJ. However, this group has examined a restricted set of parameters and a small range of values. Furthermore, the GFDL group is not studying the issue of heat transport, which is the fundamental one for climate research.

The dependence of the MOC on high-latitude processes is highly uncertain: different climate models show very different response of the MOC to global warming scenarios ranging from no change to a 50% reduction in transport (IPCC-Fourth Assessment Report). Most studies have focussed on changes due to North Atlantic warming and freshening, but previous results indicate that changes in the North Atlantic and the Southern Ocean circulations are strongly connected. Our preliminary results support this global connection and show that the Southern Ocean properties depend critically on eddy-processes.

2.8 Continuous Dynamic Grid Adaptation in a Global Atmospheric Model

This research couples the computational fluid mechanics expertise of PI Prusa with the climate modeling expertise of PI Gutowski to address topical areas 1 and 3 in the "Request for Grant Applications", Program Notice DE-FG01-04ER04-08 of the DOE CCPP program. It will significantly advance and enhance general coordinate transformation techniques for continuous dynamic grid adaptation (CDGA) in a global atmospheric climate model. This model, CEU, has resulted from the coupling of a new dynamic core, EULAG (Smolarkiewicz et al. 2001), with CDGA technology built in through efforts of previous DOE funding, to the NCAR CAM3. CDGA allows the new global climate model CEU to continuously adapt its grid by deforming in time and space to focus resolution in selected regions of interest. In particular, this capability for grid adaptation readily allows local enhancements of resolution permitting up to an order of magnitude reduction in truncation errors when compared to uniform grids with the same overall number of nodes; and may offer the best hope for the climate modeling and forecasting community to reach kilometer, cloud resolving resolution (in selected mesoscale to regionally sized regions) in the next 5-10 years. We have achieved grid stretching in global simulation with highest resolutions so far of about 50 km. We are poised to extend computations to yet higher resolution, based on our experience to date.

2.9 Three-dimensional Global Atmospheric Aerosol and Chemistry Modeling

The goal of the ARM Program is to improve general circulation models and, in particular, the representation of clouds in GCMs. The purpose of our ARM Program grant is to examine the relationship between observed aerosol concentrations and droplet concentrations, to examine precipitation development and macrophysical cloud changes in a cloud resolving model and a climate model, and to quantify the effect of anthropogenic aerosols on the amount of reflected radiation by clouds. We use the IMPACT global aerosol model with a dynamical module to predict aerosol number. We have coupled the IMPACT model with the CAM climate model to predict droplet number and clouds. The predicted relationship between aerosol number and cloud drop number are compared with ARM data, satellite data and results from our cloud-resolving model, currently the Goddard Cumulus Ensemble Model. We will inform the climate community on how to improve the cloud/aerosol parameterizations in climate models by examining the relationship between predicted cloud drop number and aerosol number in a cloud resolving model and comparing it to that in the IMPACT/CAM model. The impact of any improved parameterization is evaluated by calculating its effect on climate and climate forcing and by comparing model predictions with observations.

Our second project is part of the Climate Prediction Program and SciDac. It aims to develop a climate model that self-adjusts the grid resolution and the complexity of the physics model to the actual atmospheric flow conditions. To accomplish this, we have implemented a fully 3-dimensional non-hydrostatic model within a hydrostatic code while using a block-structured grid that allows for the implementation of smaller grid resolution within both the hydrostatic and non-hydrostatic portions of the grid. The significance of this project is that hope to be able to avoid the cloud parameterizations that are used in most climate models.

2.10 Carbon Data Assimilation with a coupled Ensemble Kalman Filter

The contemporary increase in CO2 in the atmosphere is approximately half the CO2 emitted by fossil fuel combustion. The land and oceans have acted as repositories (sinks) for the remainder of the fossil fuel CO2, plus the CO2 released as a result of land use modification. The key to predicting future levels of atmospheric CO2 and the timing and magnitude of climate change is the prediction not only of the anthropogenic carbon sources, but also of the biogeochemical processes that determine the changing magnitudes and locations of the carbon sinks. These processes determine the rate of carbon exchange between the atmosphere, land and oceans, as well as the stability and longevity of carbon storage in each of these reservoirs in a changing environment.

The objectives of the project are (1) to derive geographically-resolved estimates of the contemporary carbon sources and sinks and their uncertainties that are consistent with all atmospheric, terrestrial and oceanic observations of the carbon system as well as with contemporaneous observations of the varying meteorology, and (2) to improve the representation of terrestrial carbon processes in coupled carbon-climate models.

2.11 Global Cloud Modeling

We will test our new global cloud resolving model, i.e., a global atmospheric circulation model with a grid-cell spacing of approximately 3 km, capable of simulating the circulations associated with large convective clouds.

The GCRM uses a geodesic grid and a non-hydrostatic dynamical core. We will propose several innovations in the GCRM dynamical core, including the choice of the continuous governing equations themselves, the grid structure, and the time-differencing methods. The GCRM will include parameterizations of cloud microphysics, turbulence, and radiation, based on those already in use in regional cloud-resolving models at CSU. It has been coupled to a land-surface model that is currently in use at CSU, and undergoing further development by S. Denning and colleagues.

The GCRM will be tested in numerical weather prediction, following protocols developed under the DOE-sponsored CAPT project. We will also test the model in free-running simulations of up to about one month in duration.

Our NSF sponsored research will involve running a super-parameterized version of version 4 of the Community Atmosphere Model. This code should be able to use 8,000 or more processors efficiently.

3. Environmental Science

3.1 Hybrid Numerical Methods for Multiscale Simulations of Subsurface Biogeochemical Processes

In this SciDAC Science Application, we are developing an integrated multiscale modeling framework with the capability of directly linking different subsurface flow, transport, and reaction process models at continuum, pore, and sub-pore scales. These codes will be modified and/or developed using advanced high-performance component architectures and efficient parallel solvers, and will be integrated into a component-based workflow environment to facilitate seamless integration of codes operating at multiple scales with different physical, biological, and chemical conceptualizations appropriate to the needs of specific simulation problems.

4. Fusion Energy

4.1 Magnetic Fusion Plasma Microturbulence Project

Plasma microturbulence, the dominant mechanism for heat loss from tokamaks, will determine the fusion gain in ITER (see http://www.iter.org/). While plasma microturbulence has been studied since the 1960s, the magnetic fusion community has yet to develop a complete predictive understanding of the turbulent transport of heat, momentum and particles. The development of such a predictive understanding has been identified as a major goal for the US fusion program [Baker, 2002], and achieving it requires the development and use of new, more powerful computational tools. The most significant recent developments in this context are (1) the evolving capacity to perform high-fidelity computer simulations of plasma microturbulence with comprehensive physics models, and (2) the advances in computer power. Since the early 1990s, nonlinear turbulence simulations have provided important insights into the characteristics of plasma transport, such as the importance of linear marginal stability of the plasma to ion temperature gradient modes in determining the ion temperature profile [Kotschenreuther, 1995]. The realization of a predictive description of plasma transport requires that we develop more accurate, comprehensive models of microturbulence through increased fidelity of gyrokinetic simulation. Effects which need to be included in simulations to yield a predictive understanding of transport are kinetic electrons, multiple gyrokinetic ion species, a full electromagnetic description of the turbulent fields, and a sufficiently accurate model of flux-surface shape for actual (or projected) tokamak discharges. The three highest-fidelity gyrokinetic turbulence codes available within the world magnetic fusion program are GYRO, GS2, and GEM. In addition to their outstanding physics fidelity, these codes are computationally efficient and able to effectively utilize the largest parallel computers supported by the Office of Science. Gyrokinetic turbulence is an ideal application for petascale computing, as the 5-dimensional nature of the problem guarantees that high-resolution simulations involve a huge number of grid points, with the potential for excellent scaling to a very large number of processors.

4.2 Fluid and Hybrid Modeling of Electromagnetic Activity in MFE Plasmas

1. Edge Localized Modes: The goal of this effort is a better understanding of the nonlinear ELM cycle in tokamaks to avoid damage to plasma-facing components in ITER. This project is being restarted following recent improvements in our preconditioner for solving the nonsymmetric algebraic systems that arise from the two-fluid model. Our goal for the coming year is to extend a full two-fluid simulation through the late nonlinear crash phase of the ELM and to compare the result with predictions of simpler models.

2. Nonlinear ballooning-interchange study: The goal of this work is to explore the most relevant nonlinear dynamics of the ballooning-interchange processes in prototype configurations. We focus on the nonlinear development of ballooning-interchange instabilities in systems that are close to the stability threshold. Numerical computations for this study are performed with three different codes and are compared with analytical predictions. The results are relevant to ELMs in tokamaks, magnetic substorms in the Earth's magnetotail, and other situations where MHD ballooning modes arise.

3. Sawtooth oscillations in tokamaks: Sawtooth events in tokamaks involve magnetic reconnection, and the fast crash observed in experiment is expected to result from two-fluid effects. With the new preconditioner, NIMROD simulations have reproduced two-fluid effects in helically symmetric configurations, and the code is ready for 3D sawtooth computations in toroidal geometry. We have also been involved in nonlinear benchmarking with the M3D code, and this effort will progress to more realistic cases where transport effects are included in the simulations.

4. Wave interaction with MHD: The Wisconsin group is part of the SciDAC-supported Center for Wave Interaction with Magnetohydrodynamics. The goal is the capability to model the interaction of RF waves with low-frequency macroscopic dynamics. We are developing a theoretical framework and numerical capability for this, and significant computational resources will be devoted to validation with experiment.

5. Non-inductive startup in low-aspect-ratio tokamaks: We are modeling electrostatic current injection in the HIT-II experiment at the Univ. of Washington and in the Pegasus spherical torus at the Univ. of Wisconsin. This current-drive scheme saves geometrically limited inductive energy for Ohmic current drive after the plasma is established. The study of HIT-II will progress to cases involving strong magnetic relaxation and modeling with two-fluid effects, and we will investigate the evolution of helical current filaments in Pegasus. Both are relevant to current-drive experiments in the NSTX device at PPPL.

6. Reversed-field pinch (RFP) dynamics: We are applying the new two-fluid modeling capability to consider RFP relaxation with fast reconnection and drift effects. We work in close collaboration with the Madison Symmetric Torus experimental group, which has measured signatures of two-fluid relaxation in the laboratory. Our simulations will provide a comprehensive physical description including the interaction among magnetic fluctuations and the interactions of fluctuations with the large-scale fields.

7. Investigation of spheromak operation: This effort has proven very successful for understanding the interaction of externally imposed transients with MHD activity and energy transport in SSPX. Our present emphasis is to include two-fluid effects, which may explain the spontaneous rotation of the spheromak plasma and drift stabilization of modes that otherwise arise during the decay phase.

8. Rotational stabilization of magnetic arcades: A CSGF-supported student is studying magnetic arcades in configurations relevant to astrophysical jets. He is investigating the stabilization effect of rotation on MHD kink, which may be important for collimation.

4.3 Magnetic Reconnection Studies and Pellet Injection/ELMs in Tokamaks using AMR

This is a joint project between the SciDAC Applied Partial Differential Equation Center (APDEC), with Dr. Phillip Colella PI at LBNL, and the SciDAC Center for Extended MHD Modeling (CEMM), with S. Jardin PI at PPPL. The focus of the project is to apply the Chombo adaptive mesh refinement framework to MHD problems of importance to fusion energy. We are targeting magnetic reconnection and tokamak refueling with pellet injection, and edge localized modes (ELMs) as the main applications.

4.4 Center for Simulation of Wave-Plasma Interactions: SciDAC Project

The next step toward fusion as a practical energy source is to develop a device capable of producing and controlling the high performance plasma required for self-sustaining fusion reactions, i.e. "burning" plasma. High power electromagnetic waves in the radio frequency (RF) range have great potential to heat fusion plasmas into the burning regime, and to control plasma behavior through localized energy deposition, driven current, and driven plasma flows. Theoretical understanding and accurate modeling of these processes and their coupling to other plasma processes such as plasma transport, MHD stability, microstability, and turbulence are essential to realize this potential.

A team of plasma scientists, computer scientists and applied mathematicians, working under a Scientific Discovery through Advanced Computing (SciDAC) project on wave-plasma interactions has laid the foundations for this understanding by developing and applying advanced wave solvers, optimizing these solvers on the most powerful computers, and demonstrating the feasibility of coupling to Fokker-Planck solvers. We propose to build these codes into a comprehensive wave-plasma simulation capability that can accurately determine the spectrum of waves launched into the plasma from antenna structures, self-consistently treat the nonlinear interaction of the waves with the plasma kinetics, and accurately calculate power, current, and flow deposition for plasma control. These models will be validated by application to experiments, and applied to support the burning plasma effort through RF system design and analysis of operating scenarios. It is also proposed to take the first steps toward an integrated plasma simulation by linking to other SciDAC efforts in plasma transport, MHD, and turbulence.

4.5 Turbulent Transport and Multiscale Gyrokinetic Simulation

This project uses first-principles physics based gyrokinetic Vlasov-Maxwell equations to study plasma confinement and transport issues in tokamaks. Most of the time requested will be utilized to investigate new physics regimes with our global toroidal gyrokinetic particle simulation code, GTS, which has been upgraded to interface with the present US fusion experiments, e.g., DIIID and NSTX via TRANSP, JSOLVER and others. These new regimes involve the study of fast moving electrons in high-temperature plasmas, which were not accessible before because of the lack of suitable algorithms, computer memory and computer time. we are also collaborating with the applied mathematics community associated with the TOPS program of the Integrated Software Infrastructure Centers (ISICs) by utilizing their tookits such as PETSc, HYPRE, Prometheus and SuperLU for solving elliptic-type equations in GTS. The aim here is to substantially improve the timing of the existing elliptic solver in the presence of the new electron physics based on the new double split-weight scheme. The improved GTS code will be used for studying neoclassical and turbulent transport in tokamaks under real experimental conditions. One of the immediate tasks is to study ETG and TEM modes in the NSTX spherical tokamak, where these modes have been the focus of the latest experimental campaign. This type of validation work against the experiments will be the center piece of our work scope in the coming year. The focus of the validation effort will be on our local NSTX experiment and on DIIID, and will lead to predictive simulations of the ITER experiment. The mutiscale mathematics project emphasizes the algorithms design for gyrokinetic-MHD and RF wave heating with mesh refinement. The newly developed numerical schemes will be implemented in GTS as a initial step toward understanding the effects of wave-heating and MHD in the presence of turbulence, and turbulence in the presence of MHD. Simulations of the experimental conditions for ITER will also play a big role. The goal here is to prepare ourselves for the impending Fusion Simulation Project (FSP).

4.6 Project Title Application of Parallel Kinetic Closures in NIMROD for Fusion Plasma Simulations

This project incorporates parallel kinetic physics such as particle trapping, free-streaming and collisional effects in fluid models of fusion plasmas. Specifically, it has lead to the implementation of integral, parallel electron and ion heat flow and ion stress closures in the plasma fluid code NIMROD. Running NIMROD with these closures will lead to improved agreement between simulations and experimental observations of magnetic confinement devices. This is precisely the goal of the NIMROD, CEMM and PSI-Center projects. The goal in terms of neoclassical tearing modes (NTMs) is to numerically simulate them for direct comparison with experimental observations. These simulations will couple in RF current drive which can have a stabilizing effect on NTM magnetic islands. The addition of a continuum calculation (see below) of the parallel electron stress in 2009 will complete the closure scheme thus permitting the NTM simulations. This includes the incorporation of the RF quasi-linear diffusion operator which appears as a source term in the drift kinetic equation. Calculations of heat transport in the SSPX device continue on Franklin with the goal of observing core electron temperatures comparable to those in the experiment. The nature of the integral, parallel heat flow closure is that it is valid both in the moderately collisional core as well as the collisional edge plasma. Because of this general applicability, it is anticipated that use of the closure will bring NIMROD simulations into quantitative agreement with SSPX core electron temperatures. Another effort is underway on Franklin to understand the effects of parallel heat transport in the edge of tokamak plasmas This work is being performed in conjunction with Dr. Scott Kruger. Ongoing comparisons with experimental observations of electron and ion heat confinement will aid in understanding the edge physics of existing experiments as well as ITER. Finally, in the interest of moving toward a more general closure formalism, a continuum solution to the time-dependent drift kinetic equation has been implemented in NIMROD. This implementation will be compared to the previous implementation of the semi-analytic, integral parallel closures for time independent problems as well as tested on the time-dependent problem of Landau damping. All of these efforts are inline with DOE's misson of developing magnetic fusion as a viable energy source.

4.7 SciDAC GSEP: Gyrokinetic Simulation of Energetic Particle Turbulence and Transport

We propose to develop GTC and GYRO codes for comprehensive simulations of energetic particle turbulence and transport in ITER plasmas. Physics research on energetic particle turbulence and transport pertinent to the ITER burning plasma experiments will progress successively at three levels of sophistication:

(1) Toroidal Alfven gap and the lower-frequency kinetic thermal ion gap.

(2) Meso-scale energetic particle turbulence of interacting multi-n modes within and across the spectral gaps.

(3) Meso-micro cross-scale couplings between energetic particle turbulence and microturbulence driven by thermal particles.

We will perform linear and nonlinear benchmark amongst the two gyrokinetic codes GTC and GYRO, hybrid nonlinear codes HMGC and TAEFL, and linear eigenvalue codes (AWECS and NOVA-K). Controlled simulations will also be compared to nonlinear dynamical models. By cross-benchmarking PIC GTC and continuum GYRO, possible effects of particle noise in a PIC code and velocity resolution in a continuum code will be identified and addressed. Simulation-experiment comparison will address all levels of the primacy hierarchy for validation of simulation codes: the primacy fluctuations of mode polarization, spatial structure, frequency, and threshold; the secondary fluctuations of spectral intensity, bispectral analysis, and zonal flows; the transport level of energetic particle diffusivity and re-distribution in phase space (radial and energy); the dimensionless scaling in similarity experiments between DIIID and NSTX; and the inter-machine comparisons using ITPA database.

Comprehensive gyrokinetic simulations of energetic particle turbulence in ITER plasmas are truly grand computational challenge due to the vast dynamical separations of time scales and of spatial scales. Fully resolving the 5D phase space with this dimension in a nonlinear simulation will certainly require petascale computing and beyond. Both GTC and GYRO have demonstrated nearly perfect scalability and high single-node efficiency using more than ten thousand processors. As increasingly more complete physics models are implemented, we need to make sure that codes maintain scalability and efficiency, to further extend the scalability up to hundreds of thousands of processors, and to optimize for the emerging multi-core architecture. Petascale computing produces petascale data. A GTC or GYRO simulation of an ITER plasma could produce terabytes of fluid data in 3D real space and petabytes of particle data in 5D phase space. We will work with computational scientists and applied mathematician to deploy and to develop advanced visualization, data analysis, and data management tools in partnership with SicDAC Centers and fusion simulation project (FSP) prototype Centers.

5. See also