NERSCPowering Scientific Discovery Since 1974

Outage Log

This table lists events reported, ongoing, and resolved on NERSC systems. It is a historical record and may not be updated while a system event is in progress.

Event Date/TimeUp Date/TimeSystemComment
04/29/16 9:55 PDT04/29/16 10:40 PDTCoriSystem in degraded mode. Engineers are investigating an issue with the Cori scratch filesystem. Logins are available but may hang, jobs may be submitted, however new jobs will not run.
04/28/16 6:00 PDT04/28/16 14:00 PDTCoriDedicated runs. During system dedicated time, users can submit jobs but no user jobs will run.
04/27/16 8:00 PDT04/27/16 17:35 PDTEdisonScheduled maintenance. Login nodes will be unavailable.
04/25/16 14:53 PDT04/25/16 17:00 PDTCoriSystem in degraded mode. There is a software issue on Cori. Queues have been paused. Engineers were investigating.
04/25/16 10:05 PDT04/27/16 8:00 PDTEdisonSystem in degraded mode. There are hardware issues on Edison. Job scheduling is currently running and new jobs can start. Engineers are investigating.
04/24/16 8:40 PDT04/24/16 13:15 PDTESnet (WAN network)System in degraded mode. There were intermittent network issues connecting to NERSC from the Chicago area. ESnet engineers have re-routed traffic while they work to resolve this issue fully.
04/23/16 13:32 PDT04/23/16 15:25 PDTCoriSystem in degraded mode. /global/cscratch1 hardware issue.
04/20/16 10:00 PDT04/20/16 11:00 PDTNERSC WebsiteScheduled maintenance.
04/20/16 0:25 PDT04/20/16 11:10 PDTEdisonSystem in degraded mode.
04/19/16 7:30 PDT04/19/16 8:05 PDTNIMScheduled maintenance. The NIM database will be going through an upgrade. The NIM interface will be affected.
04/15/16 10:40 PDT04/15/16 13:05 PDTNX ServicesUnavailable. Engineers are actively investigating.
04/13/16 11:00 PDT04/13/16 12:00 PDTMATLABSystem in degraded mode. MATLAB is currently in degraded mode for license maintenance.
04/13/16 6:00 PDT04/13/16 9:00 PDTEdisonSystem in degraded mode.
04/10/16 0:10 PDT04/10/16 18:10 PDTEdisonDedicated runs.
04/07/16 7:10 PDT04/07/16 8:55 PDTEdisonSystem in degraded mode. Edison's "scratch2" filesystem was unavailable. Jobs using "scratch2" may have been affected.
04/06/16 10:25 PDT04/06/16 13:40 PDTProjectBUnavailable.
04/06/16 9:15 PDT04/06/16 19:30 PDTCoriDedicated runs. Full system reservation for dedicated science runs. Login nodes are available, however no other user jobs will start during the reservation.
04/06/16 9:00 PDT04/06/16 13:00 PDTHPSS UserScheduled maintenance.
04/05/16 12:00 PDT04/05/16 12:30 PDTScience GatewaysScheduled maintenance. newt.nersc.gov service will be switched to a new host. No downtime is expected but users may have to log back into the science gateways after the switch .
04/01/16 10:00 PDT04/01/16 16:00 PDTEdisonDedicated runs.
04/01/16 9:55 PDT04/01/16 14:30 PDTCoriDedicated runs. Full system reservation for dedicated testing and batch system upgrades. Login nodes are available, however no other user jobs will start during the reservation.
03/30/16 13:12 PDT03/30/16 14:45 PDTCoriUnavailable. Cori's local scratch filesystem is currently unavailable. This may impact or hang new user logins. Jobs have been disabled.
03/30/16 9:00 PDT03/30/16 14:15 PDTHPSS BackupScheduled maintenance.
03/29/16 10:50 PDT03/29/16 11:15 PDT GPFS filesystemsUnavailable.
03/29/16 5:00 PDT03/29/16 10:15 PDTNX ServicesUnavailable.
03/28/16 21:19 PDT03/29/16 10:00 PDTScience Gateway ServicesUnavailable.
03/28/16 16:25 PDT-EdisonMATLAB: Scheduled Maintenance 04/01/2016 from 10:00 - 12:00 PDT.
03/28/16 16:25 PDT-CoriMatlab: Scheduled maintenance 04/01/2016 from 10:00 - 12:00 PDT.
03/28/16 13:30 PDT03/28/16 16:19 PDTProjectSystem in degraded mode.
03/28/16 13:30 PDT03/28/16 16:19 PDTProjectBSystem in degraded mode.
03/28/16 13:30 PDT03/28/16 16:19 PDTGPFS Filesystem /project and /projectbSystem in degraded mode.
03/28/16 13:30 PDT03/28/16 16:19 PDTScience Gateway ServersUnavailable.
03/28/16 12:00 PDT03/28/16 13:30 PDTCoriScheduled maintenance.
03/28/16 8:00 PDT03/28/16 17:00 PDTNX ServicesScheduled maintenance.
03/28/16 6:00 PDT03/28/16 7:00 PDTEdisonSystem in degraded mode. /scratch3 was unavailable on Edison.
03/24/16 8:30 PDT03/24/16 14:24 PDTNERSC WebsiteScheduled maintenance.
03/24/16 8:00 PDT03/24/16 17:07 PDTScience Gateways ServicesScheduled maintenance. Science Gateways were not accessible while they were being relocated from OSF to CRT. Ipython and R services was down as part of this outage.
03/24/16 4:07 PDT03/25/16 16:20 PDTCoriSystem in degraded mode.
03/23/16 9:05 PDT03/23/16 11:43 PDTHPSS UserScheduled maintenance.
03/22/16 23:13 PDT03/23/16 1:10 PDTEdisonSystem in degraded mode. Edison had a scratch3 filesystem problem.
03/22/16 7:00 PDT03/22/16 18:20 PDTCoriScheduled maintenance. Cori Maintenance was extended while Cray worked on a few more issues.
03/21/16 13:55 PDT03/21/16 17:14 PDTCoriSystem in degraded mode.
03/21/16 9:52 PDT03/21/16 12:15 PDTCoriSystem in degraded mode.
03/19/16 21:08 PDT03/19/16 22:25 PDTCoriSystem in degraded mode.
03/17/16 11:50 PDT03/17/16 18:15 PDTCoriSystem in degraded mode.
03/17/16 7:00 PDT03/17/16 17:31 PDTOracle DatabasesScheduled maintenance. Oracle Databases was successfully relocated from OSF to CRT. The systems affected are back in production: gpodb07.nersc.gov, gpodb08.nersc.gov, gpodb11.nersc.gov, gpodb13.nersc.gov.
03/16/16 23:58 PDT03/17/16 20:50 PDTEdisonSystem in degraded mode. Logins are available, jobs can be submitted, and jobs are running.
03/16/16 8:00 PDT03/16/16 23:58 PDTEdisonScheduled maintenance. Maintenance has been completed
03/15/16 9:00 PDT03/15/16 9:15 PDTNIMScheduled maintenance. Users may experience brief disruption at this time.
03/13/16 11:00 PDT03/13/16 12:39 PDTProjectASystem in degraded mode. File operations to the "projecta" filesystem are currently hanging. Engineers are actively working to remedy this issue. Computing jobs which access "projecta" may also be hanging at this time.
03/07/16 22:28 PST03/07/16 23:00 PSTCoriUnavailable. Current jobs are running, logins are accessible and jobs can be submitted.
03/03/16 23:30 PST03/04/16 0:51 PSTCoriLogins available, batch jobs not running. Cori is now available.
02/29/16 8:59 PST-EdisonGlobus endpoint and collaboration accounts are unavailable on Edison. Engineers are investigating and will update by 12noon.
02/29/16 8:59 PST-CoriGlobus endpoint and collaboration accounts are unavailable on Cori. Engineers are investigating and will update by 10am.
02/27/16 4:00 PST02/28/16 0:00 PSTCoriScheduled maintenance. Site power maintenance has been complete. Cori is available.
02/27/16 4:00 PST02/27/16 23:53 PSTEdisonScheduled maintenance. Site power maintenance has been completed. Edison is available.
02/27/16 4:00 PST02/27/16 22:55 PSTNX ServicesScheduled maintenance. Site power maintenance has been completed. NX servers are available.
02/27/16 4:00 PST02/27/16 23:50 PSTGenepoolScheduled maintenance. Site power maintenance has been completed. Genepool is back to degraded mode.
02/27/16 4:00 PST02/27/16 22:00 PSTPDSFScheduled maintenance. Site power maintenance has been completed. PDSF is back to degraded mode.
02/27/16 4:00 PST02/27/16 23:43 PSTCoriScheduled maintenance. Site power maintenance has been completed. Cori is available.
02/25/16 11:38 PST-PDSFBatch system problems on PDSF. Engineers are investigating and will update at 2PM
02/24/16 10:00 PST02/24/16 19:00 PSTHPSS UserScheduled maintenance. Estimated uptime 19:00 PST.
02/18/16 20:28 PST02/18/16 21:31 PSTTelephone ServicesUnavailable. We were experiencing internal telephone issues. All phone lines are back to normal.
02/18/16 10:00 PST02/18/16 10:30 PSTNERSC WebsiteScheduled maintenance. Maintenance on www.nersc.gov crd.lbl.gov cs.lbl.gov my.nersc.gov Sites may be unavailable for short periods of time.
02/17/16 9:18 PST02/17/16 11:25 PSTHPSS BackupScheduled maintenance.

Show Older Outages: