NERSCPowering Scientific Discovery Since 1974

Move to CRT

During 2015, NERSC will be moving from the Oakland Scientific Facility (OSF) in downtown Oakland to a brand new building:  the Computational Research and Theory (CRT) facility, located at Lawrence Berkeley National Laboratory (LBNL). Groundbreaking for the building took place on February 1, 2012, and construction is continuing.

User Impact

NERSC will make every effort to minimize the user impact of this move. We do not expect to shut down the Center completely at any point during the move and NERSC will continue to deliver computing hours to the Department of Energy Office of Science. However, there will be periods when some systems will be unavailable for several days and there will be significantly fewer computing hours available when the Edison system moves to the CRT building in December. Edison is expected to be down for 6 weeks. For this reason, we encourage users to use their allocations early in the year. There will also be times when available I/O bandwidth to various global file systems will be reduced for extended periods. 

The move will be performed in a series of phases spread over several months. During the move, there will be extended periods where some resources are at OSF, and others are at CRT. Communication between the two sites will be provided by a single 400 Gb/s network connection. Thus, if you are running on a compute platform located at OSF, but performing I/O on a file system located at CRT, the available bandwidth may be reduced.


Compute Systems

Edison will begin its move to CRT in December 2015; this will result in about 6 weeks of downtime. Hopper and Carver will not move; both will be retired at OSF.  Carver will be retired at the end of September; Hopper will likely be retired December 15, 2015, after Cori Phase 1 is installed and stable. The full Cori, system, will be installed at CRT in mid-2016. The Cori Phase 1 system arriving in the Fall of 2015 will bridge the gap between the retirement of Hopper and the arrival of Cori.

Some old Genepool hardware will be retired; some current Genepool hardware will be moved; some new Genepool hardware will be acquired. The same is true for PDSF hardware.

Storage Systems

Global scratch will be retired shortly after Carver retires at the end of September. All other global file systems (homes, common, project, projectb, dna, seqfs) will be moved to CRT, in phases. Both HPSS systems (user archive and system backup) will remain at OSF for at least one additional year, accessed via the 400 Gb/s network link.


Key Dates

Event Date
Carver Retires September 30, 2015
Global Scratch Retires October 15, 2015   (Becomes read-only September 30, 2015)
Cori Phase 1 becomes available to all users November 2015 (estimated)
Edison offline for move to new facility December 2015
Hopper retires December 15, 2015


Detailed Schedule

September 2015

Cori Phase 1 and associated file systems will be installed at CRT.

Carver will retire on September 30, 2015. On the same day, the Global Scratch file system will become read-only.

October 2015

Global Scratch will be retired on October 14, 2015.

Cori Phase 1 will become available to early users.

November 2015

Cori Phase 1 will become available to all users.

December 2015

Edison will begin move to CRT; relocation will result in about 6 weeks of downtime.

Hopper will retire after Cori Phase 1 is installed, stable and has users running on it. The planned date is December 15, 2015.

Global homes will be replicated and installed at CRT. During the replication, available bandwidth will be reduced. After installation, bandwidth between systems at OSF and /global/homes at CRT will be limited. We do not expect any outage of /global/homes during this process.

The Global Project file system will be replicated and installed at CRT. Following this, a file system outage lasting several days will be necessary to complete the migration of project data.

January 2016

Edison will return to service in its new home in CRT.

February 2016

/global/projectb, /global/dna/, and /global/seqfs (JGI file systems) will be replicated and installed at CRT. During the replication, available bandwidth will be reduced.  After installation, bandwidth between systems at OSF and JGI file systems at CRT will be limited. We expect a 2-day outage of JGI file systems during this process.

Some PDSF file systems will be relocated to CRT. This will result in about a week of outage for those file systems.

A cluster with the internal name Mendel (provides resources to PDSF, JGI, and the Materials Project) will be moved. There will be new compute components for PDSF and JGI already in place at CRT, so this will not be an outage; however, the total amount of available compute resources will be reduced during the move. For the Materials Project, there will be an outage while Mendel is moved.

Summer 2016

Cori Phase 2 will be delivered and installed at CRT.

Fall 2016

Cori Phase 2 will be available to all users.