NERSCPowering Scientific Discovery Since 1974

Move to CRT

During Fall 2015, NERSC will be moving from the Oakland Scientific Facility (OSF) in downtown Oakland to a brand new building:  the Computational Research and Theory (CRT) facility, located at Lawrence Berkeley National Laboratory (LBNL). 

User Impact

NERSC will make every effort to minimize the user impact of this move. We do not expect to shut down the Center completely at any point during the move and NERSC will continue to deliver computing hours to the Department of Energy Office of Science. However, there will be periods when some systems will be unavailable for several days and there will be significantly fewer computing hours available when the Edison system moves to the CRT building in December. Edison is expected to be down for 6 weeks. For this reason, we encourage users to use their allocations early in the year. There will also be times when available I/O bandwidth to various global file systems will be reduced for extended periods. 

The move will be performed in a series of phases spread over several months. During the move, there will be extended periods where some resources are at OSF, and others are at CRT. Communication between the two sites will be provided by a single 400 Gb/s network connection. Thus, if you are running on a compute platform located at OSF, but performing I/O on a file system located at CRT, the available bandwidth may be reduced.


Compute Systems

Edison will begin its move to CRT in December 2015; this will result in about 6 weeks of downtime. Hopper and Carver will not move; both will be retired at OSF.  Carver has been retired on September 30; Hopper will likely be retired December 15, 2015, after Cori Phase 1 is installed and stable. The full Cori, system, will be installed at CRT in mid-2016. The Cori Phase 1 system arriving in the Fall of 2015 will bridge the gap between the retirement of Hopper and the arrival of Cori.

Some old Genepool hardware will be retired; some current Genepool hardware will be moved; some new Genepool hardware will be acquired. The same is true for PDSF hardware.  There will be no outage for Genepool and PDSF.

Storage Systems

Global scratch will be retired shortly after Carver retires at the end of September. All other global file systems (homes, common, project, projectb, dna, seqfs) will be moved to CRT, in phases. Both HPSS systems (user archive and system backup) will remain at OSF until other moves complete, accessed via the 400 Gb/s network link.


Key Dates

Event Date
Carver Retires September 30, 2015 (at noon).  Retired
Jesup Testbed Retires September 30, 2015 (at noon).  Retired
Global Scratch Retires October 14, 2015  at noon (Becomes read-only September 30, 2015 at noon)
Cori Phase 1 becomes available to all users November 2015 (estimated)
Edison offline for move to new facility December 2015
Hopper retires December 15, 2015 (after Cori Phase 1 is stable)


Detailed Schedule

September 2015

Cori Phase 1 and associated file systems were installed at CRT.

Carver has retired on September 30, 2015 at noon. On the same day (and same time), the Global Scratch file system has become read-only and the Jesup testbed system has been retired.

October 2015

Global Scratch will be retired on October 14, 2015 at noon.

Cori Phase 1 will become available to early users.

November 2015

Cori Phase 1 will become available to all users.

December 2015

Edison will begin move to CRT; relocation will result in about 6 weeks of downtime.

Hopper will retire after Cori Phase 1 is installed, stable and has users running on it. The planned date is December 15, 2015.

Global homes will be replicated and installed at CRT. During the replication, available bandwidth will be reduced. After installation, bandwidth between systems at OSF and /global/homes at CRT will be limited. We do not expect any outage of /global/homes during this process.

The Global Project file system will be replicated and installed at CRT. Following this, a file system outage up to five days will be necessary to complete the migration of project data.

January 2016

Edison will return to service in its new home in CRT. It will be brought online with Slurm as the batch system.

February 2016

/global/projectb, /global/dna/, and /global/seqfs (JGI file systems) will be replicated and installed at CRT. During the replication, available bandwidth will be reduced.  After installation, bandwidth between systems at OSF and JGI file systems at CRT will be limited. We expect up to a 7-day outage of JGI file systems during this process.

Some PDSF file systems will be relocated to CRT. This will result in about a two week outage for those file systems.

A cluster providing resources to PDSF, JGI, the Materials Project and Babbage will be moved. There will be new compute components for PDSF and JGI already in place at CRT, so this will not be an outage; however, the total amount of available compute resources will be reduced during the move. For the Materials Project there will be a three week outage while it is moved. Babbage will be offline for up to a month while it is moved.

Summer 2016

Cori Phase 2 will be delivered and installed at CRT.

Fall 2016

Cori Phase 2 will be available to all users.