NERSCPowering Scientific Discovery Since 1974

Edison is Back and Faster

January 16, 2015 by Richard Gerber

Edison is back, now with all 28,000 memory DIMs replaced and upclocked from 1600 MHz to 1866 MHz. (This is the memory speed, not the processor speed.) So, what will this mean to you? It's hard to predict exactly, but some codes will see a noticeable performance increase, which is good news for everyone. If you know that your code is memory bandwidth limited then your code could run up to 16 percent faster. If memory bandwidth doesn't matter to you, then you may see no improvement. Most codes will probably fall somewhere in between.

Streams Benchmark Performance

"Streams" is an application that measures memory bandwidth and NERSC uses streams to monitor Edison performance over time. The plot below shows the difference in streams performance using all cores on an Edison node. At over 100 GB/s, Edison has one of the best memory bandwidths around.

NERSC has a standard set of application benchmark it uses to monitor system performance, and they hint that the average runtime improvement will be on the order of 5 percent. While that may not seem like much, Edison delivers about 2 billion MPP hours per year, so a 5 percent performance increase translates to an additional 100 million MPP hours of computation. (Not in terms of available MPP hours, there are still 24 hours in the day and 365 days, 6 hours in the year, we can't change that - although 2015 will have an extra "leap" second, which actually works out to about 70 additional MPP hours on Edison. Instead NERSC users will get additional work done because it will cost less to run a given job.)

One of the codes we use for monitoring is called "GTC." Here's what the runtime of GTC looks like before and after the memory clock speed was increased (smaller is better in the plot). It's not an insignificant difference.

Of course, this all assumes that the memory will perform reliably. That was the original problem: too many memory errors were ocurring at 1866 MHz. NERSC and all the vendors involved with Edison think it's been figured out and the upgraded configuration will perform as expected. We're pretty confident that it will, but we'll be keeping close watch. 

User Group Meeting

The next face-to-face NERSC User Group (NUG) meeting, NUG 2015, will be Feb. 23-26, 2015, at the Berkeley Lab main campus and at NERSC's Oakland Scientific Facility. Registration (free!) and the agenda are available on the meeting web site. We're putting together an interesting day of talks for the "Science and Technology" day at Berkeley Lab on Tuesday, Feb. 23 (keep checking the web site for update; we'll send an email to all users with more details soon).

Hack-a-Thon, New User & Data Training, and Business Meeting

The next day, Wednesday Feb. 24, we're trying something new with an advanced code optimization "Hack-a-Thon" where users will be able work with NERSC and other experts to tune a code kernel of their choosing and/or a sample kernel provided by NERSC. The somewhat misnamed "Business Day" will be on Thursday, Feb. 26 (it should be more appropriately termed the "NERSC update and discussion of hot topics proposed by users day") in Oakland. The week will start with a morning of "New User" training on Monday, Feb. 23, followed by an afternoon of more in depth training on data-related topics. All events are available in person and live on the web. 

User Group Executive Committee Elections

 The NERSC User Group Executive Committee elections are over, and 232 NERSC users selected a group to represent them for the next three year. I'd like to welcome the new NUGEX members; we look forward to working with you over the next three years. To retiring members, I'd like to convey NERSC's sincere thanks and appreciation. You helped us keep NERSC the most usable and scientifically productive HPC center in the world.

A very special thanks goes to Stephane Ethier, who is the retiring NUGEX chairperson. Stephane has been an invaluable asset to NERSC and its user community for years, serving as chair for so long that I've lost count (six years?). A new chair will be elected by NERSC users very soon. To Stephane and all retiring members, I hope you will continue to participate in NUG and other NERSC activities. Always feel free to contact us with any comments or concerns.

Newly Elected NUGEX Members

Fusion Energy Science
Chris Holland, University of California, San Diego
David Green, Oak Ridge National Laboratory
High Energy Physics
Ted Kisner, Berkeley Lab
Frank Tsung, UCLA
Zarija Lukic, Berkeley Lab
Nuclear Physics
Balint Joo, Jefferson National Accelerator Laboratory
Nicholas Schunck, Lawrence Livermore National Laboratory
At Large Member
James Amundson, Fermi National Accelerator Laboratory

User Survey 

One of our most important activities each year is the NERSC User Survey. User reponse is crucial for NERSC to maintance its world-leading role of supporting and enabling scientific discovery. The results tell us what we are doing well and points out where we need to get better. It's also perhaps the most important way we communicate our value to DOE headquarters. So please, if you have not already done so, fill out the survey. We need 600 responses and we're at 550 with just a few days to go, so help us out if you can. Thanks.