NERSCPowering Scientific Discovery Since 1974

Outage Log

This table lists events reported, ongoing, and resolved on NERSC systems. It is a historical record and may not be updated while a system event is in progress.

Event Date/TimeUp Date/TimeSystemComment
02/12/19 10:37 PST02/12/19 11:06 PSTMyProxyScheduled maintenance. Service will be down briefly (10-15 min) within the window for kernel and system software updates and reboots.
02/12/19 10:37 PST02/12/19 12:07 PSTScience DatabasesScheduled maintenance. Services will be down briefly (10-15 min) within the window for kernel and system software updates and reboots.
02/12/19 9:00 PST02/12/19 9:10 PSTMyNERSC, NIMScheduled maintenance. NIM and MyNERSC services will be down briefly (10-15 min) within the window for kernel and system software updates and reboots.
02/12/19 8:30 PST02/12/19 12:25 PSTSpinScheduled maintenance. Services will be down briefly (1-2 min) within the window for urgent kernel and security updates and system reboots.
02/12/19 8:30 PST02/12/19 12:35 PSTJGI Web and Database serversScheduled maintenance. Servers will be down briefly (10-15 min) within the window for kernel and system software updates and reboots. The webfs storage system will be unavailable from approximately 9:30-11:30 PST for a firmware and software upgrade.
02/09/19 18:42 PST02/10/19 0:46 PSTCoriSystem in degraded mode. Access to all network filesystems may be slower than usual. Engineers are on site, we hope to have the issue resolved by 03:00 PST.
02/07/19 16:00 PST02/07/19 20:00 PSTMulti-Factor AuthenticationScheduled maintenance. Scheduled maintenance. Replacement of hardware components. No outage is expected.
02/07/19 15:18 PST-CoriNotes: (02/07/19 15:18 PST) - Burst Buffer: Staging operation to/from Persistent Reservations has limited functionality
02/07/19 8:00 PST02/07/19 17:00 PSTHPSS Archive (User)Scheduled maintenance.
02/06/19 7:00 PST02/06/19 19:15 PSTCoriScheduled maintenance. Maintenance Completed at 19:15 PST.
02/06/19 7:00 PST02/06/19 11:10 PSTData Transfer NodesScheduled maintenance.
02/05/19 6:00 PST02/05/19 7:05 PSTNetworkSystem in degraded mode. Users may notice intermittent sluggish performance accessing NERSC assets. Engineers are aware and investigating a DNS issue.
02/02/19 16:05 PST02/03/19 23:20 PSTCoriSystem in degraded mode.
02/01/19 12:51 PST02/01/19 13:37 PSTCoriUnavailable.
01/28/19 7:04 PST-HPSS Archive (User)Users may see delays or transient I/O errors while the HPSS archive transitions to CRT. If you get an I/O error, please retry your retrieval after several (5 - 6) hours. This work should be completed by 02/01.
01/26/19 11:30 PST01/26/19 13:00 PSTMulti-Factor AuthenticationScheduled maintenance. Scheduled maintenance. Configuration changes related to system redundancy will be made. No outage is expected.
01/25/19 7:21 PST01/25/19 9:57 PSTHPSS Archive (User)Unavailable. Engineers are actively investigating.
01/25/19 7:04 PST-HPSS Archive (User)Users may see delays or transient I/O errors while the HPSS archive transitions to CRT. If you get an I/O error, please retry your retrieval after several (5 - 6) hours. This work should be completed by 02/01.
01/24/19 13:10 PST01/24/19 14:14 PSTCoriUnavailable. Cori is currently unavailable, our engineers are troubleshooting the issue.
01/24/19 6:56 PST-HPSS Archive (User)Users may see delays or transient I/O errors while the HPSS archive transitions to CRT. If you get an I/O error, please retry your retrieval after several (5 - 6) hours. This work should be completed by 02/01.
01/22/19 7:22 PST-HPSS Archive (User)Users may see delays or transient I/O errors while the HPSS archive transitions to CRT. If you get an I/O error, please retry your retrieval after several (5 - 6) hours. This work should be completed by 02/01.
01/21/19 10:39 PST01/21/19 15:05 PSTHPSS Archive (User)Unavailable. The system is unavailable due to site power issue
01/21/19 10:39 PST01/21/19 14:09 PSTHPSS Regent (Backup)Unavailable. The system is unavailable due to site power issue
01/21/19 9:35 PST01/21/19 18:13 PSTCoriUnavailable. The system is unavailable due to a center wide power outage. Power supply is stable now, NERSC engineers are working to bring the system back up.
01/21/19 9:35 PST01/21/19 17:59 PSTEdisonUnavailable. The system is unavailable due to a center wide power outage. Power supply is stable now, NERSC engineers are working to bring the system back up.
01/21/19 9:35 PST01/21/19 19:11 PSTGenepoolUnavailable. The system is unavailable due to a center wide power outage. Power supply is stable now, NERSC engineers are working to bring the system back up.
01/21/19 9:35 PST01/21/19 19:11 PSTPDSFUnavailable. The system is unavailable due to a center wide power outage. Power supply is stable now, NERSC engineers are working to bring the system back up.
01/21/19 9:35 PST01/22/19 8:50 PSTNX ServicesUnavailable. SSH to other hosts currently requires password + OTP, engineers are investigating the issue.
01/21/19 9:35 PST01/21/19 18:46 PSTMongoDBUnavailable. The system is unavailable due to a center wide power outage. Power supply is stable now, NERSC engineers are working to bring the system back up.
01/21/19 9:34 PST01/21/19 12:03 PSTData Transfer NodesUnavailable. The system is unavailable due to a center wide power outage.
01/21/19 9:34 PST01/21/19 11:24 PSTProjectUnavailable. The system is unavailable due to a center wide power outage.
01/21/19 9:34 PST01/21/19 11:24 PSTProjectBUnavailable. The system is unavailable due to a center wide power outage.
01/21/19 9:34 PST01/21/19 11:24 PSTGlobal CommonUnavailable. The system is unavailable due to a center wide power outage.
01/21/19 9:34 PST01/21/19 11:24 PSTGlobal HomesUnavailable. The system is unavailable due to a center wide power outage.
01/18/19 15:16 PST-Cori
01/16/19 9:00 PST01/16/19 16:04 PSTHPSS Archive (User)Scheduled maintenance.
01/14/19 23:30 PST01/15/19 2:50 PSTCori/project unavailable. /global/project and /global/projecta are currently inaccessible from computing platforms. Engineers are investigating.
01/14/19 23:30 PST01/15/19 2:50 PSTEdison/project unavailable. /global/project and /global/projecta are currently inaccessible from computing platforms. Engineers are investigating.
01/14/19 23:30 PST01/15/19 2:50 PSTGenepool/project unavailable. /global/project and /global/projecta are currently inaccessible from computing platforms. Engineers are investigating.
01/14/19 23:30 PST01/15/19 2:50 PSTPDSF/project unavailable. /global/project and /global/projecta are currently inaccessible from computing platforms. Engineers are investigating.
01/11/19 16:44 PST01/11/19 18:06 PSTCoriUnavailable. Cori and Edison are currently unavailable. Logins are not available and no new jobs can be submitted at this time.
01/11/19 16:44 PST01/11/19 18:06 PSTEdisonUnavailable. Cori and Edison are currently unavailable. Logins are not available and no new jobs can be submitted at this time.
01/11/19 16:09 PST01/11/19 16:26 PSTCoriUnavailable. Cori and Edison access is available following a brief disruption. Engineers are investigating the issue.
01/11/19 16:09 PST01/11/19 16:26 PSTEdisonUnavailable. Cori and Edison access is available following a brief disruption. Engineers are investigating the issue.
01/11/19 16:09 PST01/11/19 18:06 PSTShibboleth loginSystem in degraded mode. NERSC web site and Shibboleth login are intermittently unavailable due to problems with web load balancer equipment. Work is ongoing.
01/11/19 16:09 PST01/11/19 18:06 PSTNERSC WebsiteSystem in degraded mode. NERSC web site and Shibboleth login are intermittently unavailable due to problems with web load balancer equipment. Work is ongoing.
01/10/19 8:45 PST01/10/19 12:29 PSTGenepoolUnavailable. Genepool is unavailable due to power work
01/10/19 8:45 PST01/10/19 11:32 PSTNX ServicesUnavailable. nxcloud is unavailable due to power work
01/10/19 8:43 PST01/10/19 12:29 PSTPDSFUnavailable. PDSF is unavailable due to power work
01/10/19 8:21 PST01/10/19 11:54 PSTmongodbUnavailable.
01/08/19 16:04 PST01/08/19 17:00 PSTHPSS Archive (User)Unavailable.
01/08/19 7:00 PST01/08/19 20:27 PSTCoriScheduled maintenance. Maintenance was completed at 20:27 PST.
01/08/19 7:00 PST01/08/19 20:00 PSTEdisonScheduled maintenance. Maintenance completed at 8:54
01/03/19 11:00 PST01/03/19 14:04 PSTNIMUnavailable. Access to the NIM interface is unavailable due to critical maintenance. Logins to NERSC systems remain available.
01/02/19 11:22 PST-CoriDataWarp users will have fewer resources available until we are able to bring all of the Burst Buffer nodes back in service. Existing Persistent Reservations may not be available.
01/01/19 9:01 PST-CoriDataWarp users will have fewer resources available until we are able to bring all of the Burst Buffer nodes back in service.
12/31/18 7:50 PST12/31/18 13:56 PSTEdisonUnavailable.
12/31/18 7:07 PST12/31/18 8:16 PSTSeqFSSystem in degraded mode. Filesystem is being affected by site power issues. Engineers are working on the issue.
12/31/18 5:25 PST12/31/18 20:40 PSTCoriUnavailable. DataWarp users will have fewer resources available until we are able to bring all of the Burst Buffer nodes back in service.
12/27/18 9:00 PST12/27/18 15:00 PSTHPSS Archive (User)Scheduled maintenance.
12/23/18 19:39 PST12/24/18 1:17 PSTCoriSystem in degraded mode.
12/23/18 15:14 PST12/23/18 15:46 PSTHPSS Archive (User)Unavailable.
12/22/18 13:51 PST12/22/18 15:02 PSTCoriLogins unavailable, batch jobs running. Logins were temporarily unavailable due to issues with the multi-factor authentication mechanism.
12/22/18 13:51 PST12/22/18 15:02 PSTEdisonLogins unavailable, batch jobs running. Logins were temporarily unavailable due to issues with the multi-factor authentication mechanism.
12/22/18 13:51 PST12/22/18 15:02 PSTGenepoolLogins unavailable, batch jobs running. Logins were temporarily unavailable due to issues with the multi-factor authentication mechanism.
12/22/18 13:51 PST12/22/18 15:02 PSTPDSFLogins unavailable, batch jobs running. Logins were temporarily unavailable due to issues with the multi-factor authentication mechanism.
12/22/18 13:51 PST12/22/18 15:12 PSTNERSC MFA LoginsUnavailable. NERSC MFA-enabled logins were unavailable due to LBLnet maintenance. Engineers have corrected the issue and logins across NERSC are available.
12/19/18 7:00 PST12/19/18 17:53 PSTCoriScheduled maintenance.

Show Older Outages: