NERSCPowering Scientific Discovery Since 1974

Move to CRT

During 2015, NERSC will be moving from the Oakland Scientific Facility (OSF) in downtown Oakland to a brand new building:  the Computational Research and Theory (CRT) facility, located at Lawrence Berkeley National Laboratory (LBNL).  Groundbreaking for the building took place on February 1, 2012, and construction is continuing.

The information on this page is still very preliminary, and subject to change as planning continues.

User Impact

NERSC will make every effort to minimize the user impact of this move.  We do not expect to shut down the Center completely at any point during the move and NERSC will continue to deliver computing hours to the Department of Energy Office of Science.  However, there will be periods when some systems will be unavailable for several days and there will be significantly fewer computing hours available when the Edison system moves to the CRT building in December.  Edison is expected to be down for 6 weeks.  For this reason, we encourage users to use their allocations early in the year.  There will also be times when available I/O bandwidth to various global file systems will be reduced for extended periods. 

The move will be performed in a series of phases spread over several months.  During the move, there will be extended periods where some resources are at OSF, and others are at CRT.  Communication between the two sites will be provided by a single 400 Gb/s network connection.  Thus, if you are running on a compute platform located at OSF, but performing I/O on a file system located at CRT, the available bandwidth may be reduced.

Summary

Compute Systems

Edison will begin its move to CRT in December 2015; this will result in about 6 weeks of downtime.  Hopper and Carver will not move; both will be retired at OSF.  Carver will be retired at the end of September; Hopper will most likely be retired in mid-to-late December after Cori Phase 1 is installed and stable.  The full Cori, system, will be installed at CRT in mid-2016.  The Cori Phase 1 system arriving in the Fall of 2015 will bridge the gap between the retirement of Hopper and the arrival of Cori.

Some old Genepool hardware will be retired; some current Genepool hardware will be moved; some new Genepool hardware will be acquired.  The same is true for PDSF hardware.

Storage Systems

Global scratch will be retired shortly after Carver retires at the end of September.  All other global file systems (homes, common, project, projectb, dna, seqfs) will be moved to CRT, in phases.  Both HPSS systems (user archive and system backup) will remain at OSF for at least one additional year, accessed via the 400 Gb/s network link.

Schedule

At this time, exact dates of system outages are not known.  Once the building contractors have committed to a particular date when NERSC personnel can begin installing our network infrastructure, we will be able to provide more accurate estimates.  What is presented here is a month-by-month overview of move activities that will impact user access to NERSC systems and services.  We expect these activities to begin in August, 2015.

August 2015

/global/homes will be replicated and installed at CRT.  During the replication, available bandwidth will be reduced.  After installation, bandwidth between systems at OSF and /global/homes at CRT will be limited.  We do not expect any outage of /global/homes during this process.

/global/projectb, /global/dna/, and /global/seqfs (JGI file systems) will be replicated and installed at CRT.  During the replication, available bandwidth will be reduced.  After installation, bandwidth between systems at OSF and JGI file systems at CRT will be limited.  We expect a 2-day outage of JGI file systems during this process.

Some PDSF file systems will be relocated to CRT.  This will result in about a week of outage for those file systems.

September 2015

A cluster with the internal name Mendel (provides resources to PDSF, JGI, and the Materials Project) will be moved.  There will be new compute components for PDSF and JGI already in place at CRT, so this will not be an outage; however, the total amount of available compute resources will be reduced during the move.  For the Materials Project, there will be an outage while Mendel is moved.

Cori Phase 1 and associated file systems will be installed at CRT.

Carver will retire on September 30, 2015.  On the same day, the Global Scratch file system will become read-only.

October 2015

Global Scratch will be retired on October 14, 2015.

Cori Phase 1 will become available to early users.

The Global Project file system will be replicated and installed at CRT.  Following this, a file system outage lasting several days will be necessary to complete the migration of project data.

November 2015

Cori Phase 1 will become available to all users.

December 2015

Edison will begin move to CRT; relocation will result in about 6 weeks of downtime.

Hopper will retire after Cori Phase 1 is installed, stable and has users running on it.  The exact date is TBD, but will likely be in December.

January 2016

Edison will return  to service in its new home in CRT.

June 2016

Cori Phase 2 will be delivered and installed at CRT.

September 2016

Cori Phase 2 will be available to all users.