Structure of the hammerhead ribozyme, with the RNA backbone represented by a white ribbon. More details may be found here.

Our anniversary year was marked by significant scientific and mathematical achievements by our nationwide clients and our staff, as well as a new policy that makes NERSC accessible to even more researchers. This section of the Annual Report describes some of those achievements as well as the peer review policy that will help ensure the quality of computational science in the future.

Figure 1: NERSC FY99 massively parallel processing (MPP) allocations by site.

Research by NERSC Clients
As a national facility for scientific research funded by the Department of Energy, Office of Science (DOE SC), NERSC annually serves about 2,400 scientists throughout the United States (Figure 1). These researchers work in DOE laboratories, universities, industry, and other Federal agencies (Figure 2). Computational science conducted at NERSC covers the entire range of scientific disciplines, but is focused on research that supports the DOE's mission and scientific goals (Figure 3).

Advanced Scientific Computing Research

	Figure 2: Percentage of NERSC users and usage (computing time) by institution type.

DOE's Office of Advanced Scientific Computing Research supports a number of projects in computer science and applied mathematics, including work on turbulent flows by NERSC's Center for Computational Sciences and Engineering. Another research group is developing increasingly accurate simulations of acceleration-driven fluid interface instabilities, a field with applications ranging from astrophysics to climatology to mechanical engineering. Still other researchers are applying new numerical methods to simulate the unusual magnetic and electrical properties of condensed matter; one of their goals is to identify experimental probes that would signal the appearance of novel phenomena.

Development of improved scalable linear algebra algorithms (such as a sparse direct linear system solver) for MPP systems is another major focus of OASCR-funded research. Early versions of a variety of scalable algorithms developed on NERSC systems are already in use for computational chemistry, earthquake modeling, electronics engineering, and astrophysics data analysis. Experimental software for information retrieval has successfully passed preliminary tests on a collection of more than half a million full-text documents; future applications will test image retrieval as well.

Figure 3: NERSC users and usage by scientific discipline.

Basic Energy Sciences
NERSC provides computational support for a large number of materials sciences, chemical sciences, geosciences, and engineering research projects sponsored by DOE's Office of Basic Energy Sciences.

Two computational chemists who use NERSC resources-Peter Cummings of the University of Tennessee and Oak Ridge National Laboratory, and David Dixon of Pacific Northwest National Laboratory-were among the scientists featured in a Chemical & Engineering News cover story, "Bridging Chemistry and Engineering" (April 26, 1999). The article demonstrated that, with the help of high performance computers, computational chemistry has become sophisticated enough to have real-world engineering applications, especially in the early stages of process design.

Cummings's research includes the search for surfactants that can be used along with supercritical carbon dioxide as an environmentally friendly replacement for organic solvents. To work successfully with supercritical CO₂, the surfactant molecules would have to display the unusual behavior of organizing themselves as reverse micelles (polar heads in, organic tails out) around water molecules. Cummings and his colleagues achieved the first atomistically detailed molecular dynamics simulation of micellization, in which all the water molecules were scavenged within a nanosecond. The researchers hope their efforts will help chemists understand how these surfactants work and aid in the selection of possible new candidates.

Dixon, according to the C&EN cover story, is "pushing the accuracy envelope. His concern it to try to get the accuracy of his predications down to less than 1 kcal per mole, which is needed in many engineering applications. Accuracy is particularly critical in calculating values for quantities that have never been measured experimentally." Dixon and his research group are developing and testing methods for reliably predicting the energetics and kinetics of chemical processes without recourse to empirical parameters, thus minimizing the expensive experimental measurements needed to model complex systems such as the combustion of hydrocarbon fuels.

In other research, the replacement of electronics by faster optical devices is being furthered by the computational design of photonic band gap materials, whose applications include waveguides that can bend electromagnetic waves with bending radii of only a single wavelength. Advances in semiconductor quantum dot research this year included the first ab initio calculation of the capacitance of nanocrystal quantum dots, and calculation of the electronic and optical structures of million-atom quantum dots. This research may usher in a new generation of nanoscale devices such as lasers, sensors, photovaltaics, and data storage media.

Basic research under way in chemistry and materials science includes incorporating quantum mechanical effects into simulations of chemical reaction dynamics, calculating the electronic structures of reactive chemical systems, and investigating the atomic and electronic structures of ceramic/metal interfaces. With an eye to energy efficiency, computational scientists are simulating reaction pathways in the process of soot formation and conducting scaling studies to improve the fidelity of combustion simulations. Studies of particulate dynamics in filtration and granular flow are relevant to the extraction of water or oil from underground reservoirs as well as commercial filtration processes used in purification and manufacturing. And the successful imaging of subducting tectonic plates, ocean ridges, and velocity anomalies at the base of the Earth's liquid core, based on computational analysis of global seismic data, is offering new insights into the inner structure of our planet.

Biological and Environmental Research
DOE's Office of Biological and Environmental Research is a major supporter of global climate studies as well as computational medical and biological research. These fields are claiming a growing share of NERSC resources.

The highly respected climate research group led by Warren Washington of the National Center for Atmospheric Research has developed a new, state-of-the-art global climate model. Their 300-year control experiment simulates El Niño and La Niña events that are remarkably close to observed patterns. They are now conducting historical simulations to calibrate the model. Results of their simulations of future climate change scenarios will be provided to the Intergovernmental Panel on Climate Change and the U.S. National Assessment of the Potential Consequences of Climate Variability and Change.

The DOE Center for Research on Ocean Carbon Sequestration, which is jointly managed by Lawrence Berkeley and Lawrence Livermore national laboratories, has begun using NERSC resources to provide the scientific basis for understanding the efficacy and environmental impacts of various strategies for ocean carbon sequestration, including biological fertilization and direct injection of CO₂ into the deep ocean.

Other climate research at NERSC includes continuing work on the comparison and diagnosis of climate models; the first three-dimensional, high-resolution simulations of decaying stratified turbulence on a rotating sphere; an improved model for three-dimensional global atmospheric chemistry simulations; and preliminary simulations of Southern Ocean eddy dynamics to understand their contribution to global ocean systems. (Also see here for a discussion of NERSC staff involvement in climate research.)

Medical and biological researchers are using NERSC resources to develop a new technique for recognition and classification of protein folds in genomes; to simulate enzyme reactions relevant to the development of new antibacterial agents; to conduct atomic-scale studies of enzyme catalysis, which may eventually lead to ribozyme-based therapies for certain viruses and cancers; and to search for new scintillator crystals to be used in medical imaging and physics research.

Fusion Energy Sciences
Research funded by the Office of Fusion Energy Sciences is flourishing on massively parallel systems. Steady improvements in computational techniques and codes have resulted in simulations realistic enough to compare with experiments. These simulations have elucidated a variety of experimental results and have helped improve the performance of experimental devices.

The theoretical picture of toroidal ion temperature gradient-driven turbulence was simplified considerably this year by the discovery of a simple fit for the dependence of the thermal flux on the temperature gradient. Researchers also demonstrated a new scheme to reduce heat transport by slightly rippling the equilibrium temperature profile, which generates short-scale zonal flows and thus reduces the heat transport.

Another significant accomplishment was the development of a practical formulation for real tokamak geometry that could be simply incorporated into both gyrokinetic and gyrofluid linear and nonlinear codes. In addition, non-adiabatic electron physics and electromagnetic effects were added to several tokamak turbulence codes. Including electrons extends the applicability of the model to describe particle transport as well as heat transport, and provides a more relevant description of the physics of ion temperature gradient driven modes, since trapped electrons are known to enhance the growth rate of the underlying instability. Another milestone is a new computational approach that explains the previously apparent discrepancy between experimental observations and theoretical calculations using magnetohydro-dynamics equations.

The NIMROD fusion plasma code, which provides for flexibility in both physics and geometry, is maturing rapidly and has more than doubled its performance in the last year. It is being applied to a wide variety of challenging simulations. Significant progress is also being made in simulations of beam dynamics for heavy-ion fusion. And stellarator simulations have helped develop feasible experimental designs with more compact plasmas; if successful, these designs could significantly improve the economics of fusion power.

High Energy and Nuclear Physics
High performance computing continues to have a major impact on the field of high energy and nuclear physics, ranging from theoretical studies, to the design of next-generation experimental facilities, to the storage and analysis of massive data sets from experiments. In nuclear physics, quantum Monte Carlo methods have made it possible to computationally study nuclear systems with realistic nuclear interactions, taking into account the vast amount of nucleon-nucleon scattering data. These interactions produce large spatial, spin, and isospin correlations between the nucleons. These correlations can play an important role in a variety of intriguing processes, ranging from the scattering of electrons by nuclei to the reactions that produce solar neutrinos. Quantum Monte Carlo studies of nuclear structure have produced results that follow the experimental data more closely than do empirical formulas.

In high energy physics, lattice QCD (quantum chromodynamics) researchers provide theoretical calculations that are useful in interpreting experimental results and that suggest improved experiments to test the Standard Model of elementary particles. Due to steady improvements in algorithms and computational techniques, and rapid increases in computing resources, QCD calculations are now having an important impact on high energy physics. For example, this year physicists computed the weak matrix elements responsible for the longstanding puzzle of weak kaon decays, in which two seemingly similar decay processes proceed at very different rates.

The Computational Accelerator Physics Grand Challenge has had a significant impact on the design of several accelerators, including the Next Linear Collider (NLC), the Accelerator Production of Tritium, and the Spallation Neutron Source. Simulations of the NLC resulted in an improved linac design with a higher acceleration gradient, saving $100 million over the original design. Another group of accelerator researchers is testing the feasibility of various plasma-based accelerator concepts. If plasma-based accelerator technology is successfully developed, it could lead to miniaturized tabletop accelerators, which could have an impact as widespread as miniature lasers.

Over the past two years, 6 terabytes of data for the STAR detector at Brookhaven National Laboratory were simulated on NERSC's T3E. These data were invaluable for understanding the detector response of STAR and for developing analysis algorithms. Mechanisms were developed to efficiently transport large volumes of STAR data over the network between computing facilities spread across the country. As a result of these efforts, STAR is now confident that the first data can be reliably handled and efficiently processed to extract the physics.

NERSC is also playing a major role in the storage and analysis of cutting-edge astrophysics data. The successful analysis of the massive set of cosmic microwave background (CMB) data collected during the 1997 BOOMERANG test flight has raised high expectations for the results from the 1999 BOOMERANG long-duration flight, which are currently being analyzed. The CMB contains detailed cosmological information and may answer many fundamental questions about the universe, such as its geometry and expansion rate. In addition, the Supernova Cosmology Project is using NERSC resources to analyze data from the most successful search for nearby supernovas in history. They discovered 35 supernovas, 20 before or at maximum light. Analysis of these Type Ia supernovas will help calibrate high-redshift supernovas and ascertain possible systemic biases in the extraction of cosmological parameters from redshift data.