NERSCPowering Scientific Discovery Since 1974

Outage Log

This table lists events reported, ongoing, and resolved on NERSC systems. It is a historical record and may not be updated while a system event is in progress.

Event Date/TimeUp Date/TimeSystemComment
06/26/15 7:30 PDT06/26/15 8:35 PDTNERSC WebsiteUnavailable.
06/24/15 21:52 PDT06/24/15 23:08 PDTHopperSystem in degraded mode.
06/22/15 17:00 PDT07/22/15 21:30 PDTEdisonScheduled maintenance. Dedicated Testing. Login nodes will be available for users, but no user jobs will be running during the reservation.
06/17/15 9:00 PDT06/17/15 11:27 PDTHPSS BackupScheduled maintenance.
06/16/15 10:00 PDT06/16/15 12:00 PDTNERSC WebsiteScheduled maintenance. One of the web server will be removed from the load balancer which willl affect the redundancy to NERSC website (http://nersc.gov); however, the site is expected to remain up and functional during the time
06/12/15 2:35 PDT06/12/15 3:00 PDTCarverUnavailable. Unscheduled Maintenance.
06/05/15 13:45 PDT06/05/15 14:20 PDTHopperSystem in degraded mode. Logins and jobs are hanging due to problems with the Lustre filesystem.
06/04/15 8:30 PDT06/04/15 12:10 PDTHopperScheduled maintenance. Logins are available.
06/03/15 9:00 PDT06/03/15 9:26 PDTNX ServicesScheduled maintenance. The NX services will be unavailable.
06/03/15 5:00 PDT06/03/15 13:15 PDTGrid Services, MyNERSC and Science GatewSystem in degraded mode. Engineers are investigating reports of failed logins to multiple grid based services. Globus, MyNERSC and Science Gateways are affected.
05/28/15 17:05 PDT05/28/15 20:50 PDTHopperSystem in degraded mode. Users can still log on and jobs are still running, Intermittent slow response.
05/27/15 9:00 PDT05/27/15 10:18 PDTHPSS UserScheduled maintenance.
05/27/15 9:00 PDT05/27/15 10:18 PDTHPSS BackupScheduled maintenance.
05/27/15 7:00 PDT05/27/15 18:00 PDTEdisonScheduled maintenance. The Login nodes will be unavailable.
05/21/15 23:11 PDT05/22/15 10:30 PDTportal-auth web servicesUnavailable. portal-auth.nersc.gov (sgn02) is currently unavailable.
05/20/15 8:00 PDT05/20/15 23:10 PDTEdisonScheduled maintenance.
05/19/15 9:25 PDT05/19/15 15:30 PDTLDAPUnavailable. Logins to most NERSC services are unavailable or unreliable. Engineers are actively investigating this issue. UPDATE: logins are currently working, but may be unreliable until the issue is fully resolved.
05/17/15 21:22 PDT05/18/15 12:30 PDTNIMUnavailable. Our engineers are currently working to resolve the issue.
05/14/15 11:56 PDT05/14/15 12:06 PDTProjectSystem in degraded mode. /project hang for writing.
05/14/15 11:00 PDT05/14/15 13:17 PDThpweb1Scheduled maintenance. We don’t expect any web site outage as a result of this since the web sites for nersc.gov, crd.lbl.gov and cs.lbl.gov will still be served by hpweb2.nersc.gov .
05/13/15 10:15 PDT05/13/15 14:09 PDTEdisonUnavailable. Engineers are working in resolving HSN (High Speed Network) issue.
05/13/15 10:00 PDT05/13/15 11:00 PDTProjectSystem in degraded mode. File creation and deletion was hanging during that time. There was no issue for reads
05/13/15 9:00 PDT05/13/15 11:30 PDTHPSS BackupScheduled maintenance.
05/13/15 9:00 PDT05/13/15 11:30 PDTHPSS UserScheduled maintenance.
05/12/15 12:00 PDT05/12/15 18:45 PDTNERSC MailmanScheduled maintenance. The maintenance on website https://mailman.nersc.gov/ has been completed.
05/12/15 8:00 PDT05/12/15 15:00 PDTGenepoolScheduled maintenance. All maintenance work on genepool, gpweb and gpdb hosts has been completed.
05/12/15 8:00 PDT05/12/15 8:52 PDTNIMScheduled maintenance. NIM interface was down.
05/12/15 7:00 PDT05/12/15 15:00 PDTScience Gateway ServicesScheduled maintenance. All maintenance work on the following hosts and services has been completed: 07:00-08:00 - Some Science Gateways that require authentication may be unavailable during this period. 13:00-15:00 - NERSC Certificate Services will be down for system upgrades. Globus Online/NEWT and Grid Services may be unavailable during this period. 07:00-15:00 - MySQL and Postgres DBs will be unavailable due OS upgrades on host machines scidb1/scidb2.
05/11/15 8:00 PDT05/11/15 17:00 PDTMATLAB License ServerScheduled maintenance.
05/11/15 6:40 PDT05/11/15 8:00 PDTHopperUnavailable. /scratch2 was not responding. Engineers resolved the issue.
05/07/15 6:00 PDT05/08/15 18:00 PDTEdisonSystem in degraded mode. /scratch1 file operations were slower than usual. Engineers resolved the issue.
05/05/15 12:06 PDT05/05/15 12:50 PDTNERSC WebsiteUnavailable. Intermittent access problems. Engineers are working to resolve this issue.
04/30/15 13:00 PDT04/30/15 14:36 PDTScience Gateway ServicesScheduled maintenance. Some science gateway services (openmsi, ptf) will be unavailable due to an urgent GPFS upgrade.
04/29/15 10:00 PDT04/29/15 10:52 PDTNIMScheduled maintenance.
04/24/15 15:30 PDT04/24/15 16:07 PDTScience Gateway ServicesUnavailable. Science Gateway Service (portal.nersc.gov) was unavailable.
04/22/15 9:03 PDT04/25/15 7:52 PDTProjectSystem in degraded mode.
04/22/15 9:03 PDT04/24/15 9:45 PDTScience Gateway ServicesUnavailable.
04/21/15 17:03 PDT04/21/15 20:30 PDTProjectSystem in degraded mode. Due to a hardware problem, the /global/project filesystem was in degraded mode.
04/21/15 8:00 PDT04/21/15 9:10 PDTNIMScheduled maintenance. NIM interface was down.
04/17/15 15:56 PDT-EdisonDedicated Science Run on Wednesday, 04/22/2015, 16:00-20:00 PDT. Logins will be available and the debug queue will be open, so users can run debug jobs during the reservation.
04/14/15 10:05 PDT04/14/15 11:00 PDTHopperUnavailable. The system Queues were stopped at 10:05 and were turned back on at 11:15. /scratch2 unavailable.
04/14/15 8:00 PDT04/14/15 9:00 PDTNIMScheduled maintenance.
04/11/15 8:00 PDT04/11/15 12:00 PDTEdisonLogins available, batch jobs not running. Dedicated science runs.
04/09/15 22:39 PDT04/09/15 22:56 PDTHPSS UserUnavailable. hpss_user (Archive) was down for unscheduled maintenance.
04/01/15 13:35 PDT-EdisonEdison /scratch1 filesystem appears to be slow. The issue is being looked into. Updates will be provided when available.
03/30/15 8:00 PDT03/30/15 9:47 PDTScience Gateway ServicesUnavailable.
03/26/15 9:02 PDT03/26/15 9:24 PDTData Transfer NodesSystem in degraded mode. The DTN02 had login issues.
03/25/15 14:45 PDT03/25/15 16:20 PDTEdisonUnavailable. Dedicated testing.
03/25/15 9:00 PDT03/25/15 12:30 PDTHPSS UserScheduled maintenance.
03/25/15 8:00 PDT03/25/15 14:45 PDTEdisonScheduled maintenance.
03/18/15 9:00 PDT03/18/15 12:50 PDTHPSS UserScheduled maintenance.
03/16/15 8:30 PDT03/16/15 9:15 PDTHopperSystem in degraded mode. Lustre/scratch2 unavailable
03/15/15 9:00 PDT03/15/15 10:20 PDTProjectBSystem in degraded mode. PROJECTB was in degraded mode with some lost disks.
03/12/15 17:00 PDT03/12/15 20:40 PDTProjectBSystem in degraded mode. /projectb was degraded for Genepool nodes.
03/11/15 10:00 PDT03/11/15 17:25 PDTData Transfer NodesScheduled maintenance. DTN03 and DTN04 will be unavailable; however, DTN01 and DTN02 will remain available.
03/11/15 10:00 PDT03/11/15 12:22 PDTMongoDB01Scheduled maintenance. MongoDB01 read/write operations will be locked during this maintenance.
03/11/15 9:00 PDT03/11/15 13:00 PDTHPSS BackupScheduled maintenance.
03/11/15 8:00 PDT03/11/15 22:45 PDTHopperScheduled maintenance. Hopper OS Upgrade to CLE 5.2 UP02.
03/06/15 17:56 PST03/06/15 20:06 PSTHPSS UserUnavailable. Unscheduled Maintenance.
03/05/15 17:28 PST-EdisonDedicated Large Scale Science Run on 3/6/15 08:00-16:30PT. User jobs will not be running but logins will be available.
03/04/15 15:00 PST03/04/15 17:00 PSTNIMScheduled maintenance.
03/03/15 13:36 PST-PDSF3/4/15 10am - 12pm pdsfdtn2 (PDSF newt and globus endpoint) will be unavailable.
03/01/15 9:39 PST03/01/15 10:55 PSTEdisonSystem in degraded mode. Hardware disk issues with /scratch1
02/28/15 8:39 PST02/28/15 9:22 PSTCarverSystem in degraded mode. Batch system degraded, logins are available.

Show Older Outages: