NERSCPowering Scientific Discovery Since 1974

Outage Log

This table lists events reported, ongoing, and resolved on NERSC systems. It is a historical record and may not be updated while a system event is in progress.

Event Date/TimeUp Date/TimeSystemComment
10/18/17 1:55 PDT10/18/17 8:00 PDTHPSS UserSystem in degraded mode. Archive is in a degraded state. NERSC engineers are working with the vendor to resolve the issue.
10/17/17 9:00 PDT10/17/17 17:00 PDTPostgres Databases on scidb1/2Unavailable. Postgres databases on scidb1/2 aka nerscdb01/02 will be migrating to a new host and unavailable during this period. After this date these dbs should be accessed only from the new nerscdb03.nersc.gov host. Users of these services should have received an email with this info, please contact us with any issues. MySQL services are unaffected but will be migrated at a later date
10/15/17 13:35 PDT10/15/17 14:46 PDTCoriSystem in degraded mode. System filesystem issue, no new jobs will start.
10/06/17 15:00 PDT10/09/17 22:43 PDTNERSC CenterScheduled maintenance. NERSC quarterly maintenance. This will include power and cooling across the Center.
10/06/17 15:00 PDT10/10/17 20:23 PDTCoriScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTDNAScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 11:43 PDTData Transfer NodesScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 22:43 PDTEdisonScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/10/17 16:19 PDTGenepoolScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTGlobal CommonScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTGlobal HomesScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 15:30 PDTHPSS BackupScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 15:30 PDTHPSS UserScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/10/17 14:25 PDTPDSFScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTProjectScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTProjectAScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:38 PDTProjectBScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/10/17 11:23 PDTNIM; MyNERSC; NERSC, CS, & CRD web sitesScheduled maintenance. NERSC center-wide shutdown and electrical upgrades. A basic informational website will be available.
10/06/17 15:00 PDT10/09/17 11:10 PDTMongoDBScheduled maintenance. Scheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 13:20 PDTMatgenScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 12:50 PDT Science Gateways and DatabasesScheduled maintenance. Scheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 10:57 PDTNX ServicesScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 11:03 PDTJGI Oracle DatabaseScheduled maintenance.
10/06/17 15:00 PDT10/09/17 12:05 PDTGenepool Websites and DatabasesScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/08/17 18:30 PDTMyProxyScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/06/17 15:00 PDT10/09/17 12:58 PDTSpinScheduled maintenance. NERSC center-wide shutdown and electrical upgrades.
10/05/17 7:30 PDT10/05/17 8:50 PDTNX ServicesUnavailable. NX is unavailable. Engineers are investigating the problem
10/04/17 13:00 PDT10/04/17 15:00 PDTSpinScheduled maintenance. No impact on services is expected, but staff will be unable to create or reconfigure services during the maintenance.
09/22/17 14:29 PDT-CoriA large portion of KNL nodes will be reserved for science runs on Monday 9/25/17 at 11:00 to Tuesday 9/26/17 at 11:00. Login nodes, job submissions, and Haswell nodes will remain available. Knl_interactive, knl, and knl_regularx partitions, however, will have limited availability.
09/19/17 15:15 PDT09/19/17 18:30 PDTHPSS BackupUnavailable. HPSS is currently unavailable due to electrical issues on site. Updates will be provided as they become available.
09/19/17 15:15 PDT09/19/17 18:30 PDTHPSS UserUnavailable. HPSS is currently unavailable due to electrical issues on site. Updates will be provided as they become available.
09/19/17 11:00 PDT09/20/17 11:00 PDTCoriDedicated runs. Large scale reservations for science runs. All KNL nodes are reserved. Login nodes, job submission, and Haswell nodes will remain available.
09/13/17 9:10 PDT-HPSS BackupHPSS will remain available during the NERSC scheduled maintenance 10/06/17 13:00 - 10/10/17 17:00 PDT
09/13/17 9:10 PDT-HPSS UserHPSS will remain available during the NERSC scheduled maintenance 10/06/17 13:00 - 10/10/17 17:00 PDT
09/13/17 8:00 PDT09/13/17 13:37 PDTHPSS UserScheduled maintenance.
09/12/17 14:21 PDT09/12/17 14:32 PDTCoriSystem in degraded mode. Logins are available, batch jobs are running. New jobs will not start due to hardware work.
09/06/17 18:42 PDT09/07/17 6:00 PDTCoriDedicated runs. All KNL nodes are reserved for large scale science runs. KNL job submission is available, and Haswell nodes are open.
09/06/17 8:00 PDT09/06/17 18:42 PDTCoriScheduled maintenance.
09/05/17 15:40 PDT09/05/17 18:09 PDTCoriSystem in degraded mode. Due to extreme weather conditions in Berkeley the KNL partitions are down. New KNL jobs won't start.
09/05/17 13:31 PDT09/05/17 15:40 PDTCoriLogins available, batch jobs not running. Due to extreme weather conditions in Berkeley the batch scheduling partitions are down.
09/05/17 13:31 PDT09/05/17 15:40 PDTEdisonLogins available, batch jobs not running. Due to extreme weather conditions in Berkeley the batch scheduling partitions are down.
09/04/17 17:01 PDT09/04/17 20:40 PDTEdisonUnavailable.
09/04/17 13:42 PDT09/04/17 14:05 PDTCoriSystem in degraded mode. Slurm commands and job submission was unavailable during this time due to software issues, which have been corrected.
09/02/17 14:25 PDT09/02/17 18:35 PDTEdisonUnavailable. All queues have been paused due to high temperature conditions in Berkeley.
09/02/17 11:30 PDT09/02/17 19:00 PDTCoriSystem in degraded mode. All queues have been paused due to high temperature conditions in Berkeley.
09/02/17 11:30 PDT09/02/17 14:25 PDTEdisonSystem in degraded mode. All queues have been paused due to high temperature conditions in Berkeley.
09/01/17 18:58 PDT-EdisonDue to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
09/01/17 18:57 PDT-CoriDue to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
09/01/17 15:30 PDT09/01/17 19:30 PDTGenepoolLogins available, batch jobs not running. Due to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
09/01/17 15:30 PDT09/01/17 19:30 PDTPDSFLogins available, batch jobs not running. Due to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
09/01/17 14:43 PDT09/01/17 18:29 PDTCoriLogins available, batch jobs not running. Due to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
09/01/17 14:43 PDT09/01/17 18:34 PDTEdisonLogins available, batch jobs not running. Due to extreme temperatures in the Bay Area, the scheduler will be paused during the hot parts of the day this weekend. Queues will resume as the temperature permits.
08/31/17 8:23 PDT08/31/17 8:30 PDTNetworkUnavailable. Brief network interruption on circuits between ESnet, NERSC and LBL.
08/30/17 19:36 PDT08/30/17 22:00 PDTCoriSystem in degraded mode. Cori is in a degraded state. Engineers are investigating the issue
08/30/17 9:00 PDT08/30/17 13:11 PDTNX ServicesScheduled maintenance. NX will be unavailable for maintenance
08/25/17 7:00 PDT08/25/17 8:22 PDTNIMUnavailable. NIM will be unavailable while engineers update to the new ERCAP interface
08/24/17 8:52 PDT-EdisonLogins to edisongrid is currently unavailable.
08/23/17 18:24 PDT08/23/17 22:24 PDTEdisonLogins unavailable, batch jobs running. Logins are temporarily unavailable. Engineers are investigating the issue.
08/23/17 7:00 PDT08/23/17 18:24 PDTEdisonScheduled maintenance. Edison maintenance. System and logins will not be available.
08/22/17 1:53 PDT08/22/17 4:44 PDTHPSS UserUnavailable. Archive is unavailable. Engineers are investigating
08/17/17 4:48 PDT08/17/17 7:00 PDTEdisonSystem in degraded mode. No new jobs are running on Edison while engineers investigate a possible filesystem issue. Logins are available and jobs can be submitted.
08/15/17 14:00 PDT08/15/17 17:23 PDTCoriSystem in degraded mode. cscratch1 is unavailable while engineers investigate filesystem issues.
08/15/17 14:00 PDT08/15/17 17:23 PDTEdisonSystem in degraded mode. cscratch1 is unavailable while engineers investigate filesystem issues.
08/15/17 11:45 PDT08/15/17 13:30 PDTEdisonSystem in degraded mode.
08/14/17 10:35 PDT08/14/17 11:13 PDTEdisonSystem in degraded mode. Edison is experiencing filesystem issues. Engineering are actively working on the situation.
08/11/17 20:55 PDT-CoriCreation of new Datawarp Persistent Reservations (PR) does not work. A workaround is available if needed.
08/08/17 12:00 PDT08/08/17 12:15 PDTData Transfer NodesScheduled maintenance.

Show Older Outages: