NERSCPowering Scientific Discovery for 50 Years

Novel Calculation Sheds New Light on Matter/Anti-Matter Mystery

Next-generation HPC Supports Next-generation Studies of the Universe

December 14, 2020

By Kathy Kincade


An international collaboration of theoretical physicists has published a novel calculation that provides new insights into the relationship between matter and antimatter in the universe.

The collaboration, known as RBC-UKQCD, includes scientists from Brookhaven National Laboratory, the RIKEN-BNL Research Center, CERN, Columbia University, the University of Connecticut, the University of Edinburgh, the Massachusetts Institute of Technology, the University of Regensburg, and the University of Southampton.

Their work, highlighted as an editor’s suggestion in a recent issue of Physical Review D, included extensive use of supercomputing resources over six years at the National Energy Research Scientific Computing Center (NERSC), a U.S Department of Energy Office (DOE) of Science user facility located at Lawrence Berkeley National Laboratory.

Color visualization depicting the likelihood of two kaon decay pathways

A new calculation performed using the world's fastest supercomputers allows scientists to more accurately predict the likelihood of two kaon decay pathways and compare those predictions with experimental measurements. The comparison tests for tiny differences between matter and antimatter that could, with even more computing power and other refinements, point to physics phenomena not explained by the Standard Model. Credit: Brookhaven National Laboratory


The new calculation gives a more accurate prediction of the likelihood with which subatomic particles known as kaons decay into a pair of electrically charged pions vs. a pair of neutral pions. Understanding these decays and comparing the prediction with the most recent state-of-the-art experimental measurements at CERN and the Fermi National Accelerator Laboratory gives scientists a way to test for tiny differences between matter and antimatter – known as a violation of CP symmetry – and search for effects that cannot be explained by the Standard Model, the prevailing theory of how basic building blocks of matter interact.

“At the level of precision achieved by this new calculation, we find good agreement between our prediction from the Standard Model and experiment,” explained Christopher Kelly of the Computational Science Institute at Brookhaven and a co-author on the Physical Review paper. “Thus, possible discrepancies between the Standard Model’s predictions and the results of experiment, if they exist, will require even more precise calculations for their discovery.”

Scientists’ current understanding is that the present universe was created with nearly equal amounts of matter and antimatter, “so the present imbalance of matter over anti-matter must result from subsequent physical processes which we should be able to determine,” added Norman Christ, professor of computational theoretical physics at Columbia University, a co-author on the Physical Review paper, and a NERSC PI on this research. “Finding a significant discrepancy between an experimental observation and predictions based on the Standard Model would potentially point the way to new mechanisms of particle interactions that could explain why we are made of matter instead of antimatter.”

Lattice QCD to the Rescue

To date, experiments that show a difference between matter and antimatter involve particles made of quarks and gluons, the subatomic building blocks for larger particles such as protons, neutrons, atomic nuclei, kaons, and pions. Standard Model-based calculations of how these particles behave must therefore include all possible interactions of quarks and gluons.

"Because of the huge number of variables involved, these are some of the most complicated calculations in all of physics," said co-author Tianle Wang, of Columbia University.

To conquer the challenge, the team used a computing approach called lattice quantum chromodynamics (lattice QCD), a version of the modern theory of strong interactions that places the particles on a four-dimensional space-time lattice (three spatial dimensions plus time). This involved customizing a high-performance physics code dubbed “Grid” to enable the research team to use lattice QCD to treat the quarks in a very robust way, Christ explained.

“These are all specific application codes written for lattice QCD and are tailored for our collaboration’s method for the numerical treatment of quarks,” Christ said. “It is very accurate, and the calculations are very difficult.”

The calculations were run at multiple supercomputing centers, including NERSC, which handled the majority of the measurements and analysis; the Hokusai machine at the Advanced Center for Computing and Communication at Japan’s RIKEN Laboratory; the IBM BlueGene/Q installation at Brookhaven; the Mira supercomputer at the Argonne Leadership Computing Facility (ALCF); the DiRAC machine at the University of Edinburgh; and the National Center for Supercomputing Applications Blue Waters machine at the University of Illinois.

The first part of the work involved generating samples of snapshots of the most likely quark and gluon fields; the second and most complex step – extracting the actual kaon decay amplitudes – was performed on NERSC’s Cori supercomputer.

Ongoing NESAP Support

Being affiliated with the NERSC Exascale Science Application Program (NESAP) has been essential to laying the foundation for this research, Christ noted. He was the PI on one of the first NESAP teams, a joint lattice QCD project between the DOE’s High Energy and Nuclear Physics programs that examined both the two-pion decay of the kaon and the properties of QCD at extremely high temperatures.

“We began our NESAP work in 2014, and it is this initial project whose completion we are now reporting with the publication of the Physical Review article,” he said.

That first NESAP experience led to additional projects at NERSC, including that of the RBC-UKQCD collaboration. The team is now part of a new NESAP project that will focus on optimizing lattice QCD codes to take advantage of the incoming Perlmutter system’s high performance features, which will support the increasing data analysis and simulation demands of experimental and observational science.

Jack Deslippe, who leads NERSC’s NESAP program, said, “We’ve really enjoyed working with the QCD collaboration. They are always on the leading edge of scientific computing – excited and able to make use of new technologies. We learn a lot about the capabilities of new technologies working together, and the science output is world-leading.”

Looking ahead, Kelly emphasized, “In order to tighten our test of the Standard Model we must now overcome a number of more fundamental theoretical challenges. Our collaboration has already made significant strides in resolving these issues and, coupled with improvements in computational techniques and the power of near-future DOE supercomputers, we expect to achieve much improved results within the next three to five years.”

ALCF is a DOE Office of Science user facility.

This article uses materials provided by Brookhaven National Laboratory.

About NERSC and Berkeley Lab
The National Energy Research Scientific Computing Center (NERSC) is a U.S. Department of Energy Office of Science User Facility that serves as the primary high performance computing center for scientific research sponsored by the Office of Science. Located at Lawrence Berkeley National Laboratory, NERSC serves almost 10,000 scientists at national laboratories and universities researching a wide range of problems in climate, fusion energy, materials science, physics, chemistry, computational biology, and other disciplines. Berkeley Lab is a DOE national laboratory located in Berkeley, California. It conducts unclassified scientific research and is managed by the University of California for the U.S. Department of Energy. »Learn more about computing sciences at Berkeley Lab.