NERSCPowering Scientific Discovery Since 1974

Outage Log

This table lists events reported, ongoing, and resolved on NERSC systems. It is a historical record and may not be updated while a system event is in progress.

Down Date/TimeUp Date/TimeSystemComment
01/30/13 15:20 PST01/30/13 16:30 PSTCarverSystem in degraded mode. System was in degraded mode due to NGF problem.
01/30/13 15:20 PST01/30/13 16:30 PSTDiracSystem in degraded mode. System was in degraded mode due to NGF problem.
01/30/13 15:20 PST01/30/13 16:30 PSTGenepoolSystem in degraded mode. System was in degraded mode due to NGF problem.
01/30/13 15:20 PST01/30/13 16:30 PSTHopperSystem in degraded mode. System was in degraded mode due to NGF problem.
01/30/13 15:20 PST01/30/13 16:30 PSTPDSFSystem in degraded mode. System was in degraded mode due to NGF problem.
01/30/13 15:20 PST01/30/13 16:30 PSTEuclidSystem in degraded mode. System was in degraded mode due to NGF problem.
01/27/13 16:31 PST01/27/13 22:31 PSTNGF/project unavailable.
01/23/13 16:30 PST01/23/13 17:10 PSTHopperSystem in degraded mode. Batch queues have been stopped while NGF issues are being resolved.
01/23/13 16:30 PST01/23/13 16:57 PSTCarverSystem in degraded mode. Batch queues have been stopped while NGF issues are being resolved.
01/23/13 16:30 PST01/23/13 16:57 PSTDiracSystem in degraded mode. Batch queues have been stopped while NGF issues are being resolved.
01/23/13 15:45 PST01/23/13 16:42 PSTScience Gateway NodesSystem in degraded mode. NGF degraded, affecting the science gateway nodes. Engineers are aware of the issue and working on it. ** This issue is resolved. **
01/23/13 15:54 PST01/23/13 16:00 PSTEuclidUnavailable. Euclid unavailable while being rebooted.
01/23/13 8:00 PST01/23/13 10:00 PSTPDSFScheduled maintenance. Rolling maintenance of PDSF interactive nodes for filesystem work - one system at a time starting with PDSF1.
01/21/13 23:07 PST01/21/13 23:40 PSTEuclidUnavailable.
01/16/13 9:00 PST01/16/13 12:27 PSTHPSS UserScheduled maintenance.
01/14/13 10:30 PST01/14/13 11:30 PSThelp.nersc.govScheduled maintenance. The trouble ticketing system may be affected during the outage. Users may not be able to submit a ticket or check on the existing status of a ticket during the maintenance window.
01/13/13 17:05 PST01/14/13 0:10 PSTHopperUnavailable. System was unavailable.
01/10/13 18:07 PST01/10/13 19:50 PSTCarverSystem in degraded mode. Logins available, batch system unavailable, jobs are still running.
01/10/13 18:07 PST01/10/13 19:50 PSTDiracSystem in degraded mode. Logins available, batch system unavailable, jobs are still running.
01/09/13 9:00 PST01/09/13 12:30 PSTHPSS BackupScheduled maintenance.
01/09/13 10:43 PST01/09/13 11:15 PSTNGF/projectb unavailable. /projectb was unavailable. Problem resolved.
01/09/13 1:47 PST01/09/13 3:04 PSTCarverLogins unavailable, batch jobs running. Intermittent login issue on multiple systems caused by cvrsvc04 node.
01/07/13 12:20 PST01/07/13 13:13 PSTEuclidUnavailable. System unavailable.
01/02/13 13:59 PST01/02/13 14:48 PSTCarverSystem in degraded mode. Logins available, batch system unavailable, jobs are still running.
01/02/13 13:59 PST01/02/13 14:28 PSTDiracSystem in degraded mode. Logins available, batch system unavailable, jobs are still running.
12/28/12 15:00 PST12/28/12 15:15 PSTCarverSystem in degraded mode. Logins available, batch system unavailable, jobs are still running
12/19/12 9:00 PST12/19/12 12:20 PSTHPSS UserScheduled maintenance.
12/17/12 14:00 PST12/17/12 14:32 PSTEuclidUnavailable.
12/16/12 2:40 PST12/16/12 11:35 PSTHPSS BackupUnavailable. Due to a hardware problem, HPSS system backup is not available.
12/15/12 22:16 PST12/16/12 0:05 PSTHopperSystem in degraded mode. Hopper was in degraded mode due to a cabinet power failure which affected connectivity to the /scratch2 file system.
12/13/12 9:50 PST12/13/12 10:33 PSTPDSFSystem in degraded mode. Logins access was slow.
12/12/12 16:12 PST12/12/12 16:20 PSTNGF/projectb unavailable. Configuration error.
12/10/12 16:16 PST12/10/12 17:20 PSTGenepoolSystem in degraded mode. Unmount of legacy JGI filesystem /ifs caused loss of sshd, gpfs services on gpintXX.
12/05/12 9:00 PST12/05/12 9:40 PSTHPSS BackupScheduled maintenance.
12/04/12 9:00 PST12/04/12 11:07 PSTEuclidScheduled maintenance.
12/04/12 4:04 PST12/04/12 4:25 PSTCarverSystem in degraded mode. Logins unavailable, batch commands unavailable.
11/30/12 14:20 PST11/30/12 19:02 PSTPDSFSystem in degraded mode. Due to a power issue, 48 PDSF compute nodes lost power.
11/30/12 8:00 PST11/30/12 10:15 PSTNERSC WebsiteUnavailable. Any required authentication to the website was affected. Problem resolved.
11/27/12 10:00 PST11/27/12 11:00 PSTGenepoolScheduled maintenance. UGE maintenance.
11/26/12 3:20 PST11/26/12 6:47 PSTGenepoolSystem in degraded mode. No new jobs can be submitted; existing jobs can't be queried.
11/22/12 13:30 PST11/22/12 13:50 PSTCarverSystem in degraded mode.
11/21/12 19:00 PST11/21/12 19:28 PSTCarverSystem in degraded mode. Batch system unavailable. Logins still available.
11/21/12 19:00 PST11/21/12 19:28 PSTDiracSystem in degraded mode. Batch system unavailable. Logins still available.
11/21/12 12:13 PST11/21/12 14:00 PSTGenepoolSystem in degraded mode.
11/21/12 9:00 PST11/21/12 13:27 PSTHPSS UserScheduled maintenance.
11/19/12 13:51 PST11/19/12 16:35 PSTEuclidUnavailable. Problem undetermined. Reason for outage pending.
11/18/12 10:43 PST11/18/12 14:26 PSTCarverSystem in degraded mode. Batch system unavailable , interactive access is available.
11/16/12 7:00 PST11/16/12 9:00 PSTPDSFScheduled maintenance. Rolling reboot (one node at a time) of interactive nodes for GPFS upgrade. Batch will be unaffected.
11/15/12 11:29 PST11/15/12 11:38 PSTCarverSystem in degraded mode. Batch not available. Jobs are running.
11/14/12 22:49 PST11/15/12 7:35 PSTCarverSystem in degraded mode. Batch system unavailable and jobs are not starting. Logins available.
11/14/12 22:49 PST11/15/12 7:35 PSTDiracSystem in degraded mode. Batch system unavailable and jobs are not starting. Logins available.
11/14/12 9:00 PST11/14/12 17:00 PSTEuclidScheduled maintenance.
11/14/12 9:00 PST11/14/12 14:45 PSTHPSS UserScheduled maintenance.
11/14/12 11:13 PST11/14/12 13:25 PSTCarverSystem in degraded mode. Batch system unavailable , Jobs are still running.
11/14/12 11:13 PST11/14/12 13:25 PSTDiracSystem in degraded mode. Batch system unavailable , Jobs are still running.
11/13/12 7:00 PST11/13/12 19:15 PSTGenepoolScheduled maintenance.
11/13/12 7:00 PST11/13/12 19:00 PSTNGF/projectb scheduled maintenance. /projectb will be unavailable.
11/13/12 7:00 PST11/13/12 16:40 PSTHopperScheduled maintenance.
11/13/12 8:54 PST11/13/12 9:38 PSTCarverSystem in degraded mode. Batch unavailable. Jobs still running.
11/09/12 18:02 PST11/10/12 0:25 PSTCarverScheduled maintenance. Carver back in full production at 00:25PST 11/10/12.

Show Older Outages: