NERSCPowering Scientific Discovery Since 1974

Superfacility

September 25, 2019

Mission Statement

The Superfacility concept is a framework for integrating experimental and observational instruments with computational and data facilities. Data produced by light sources, microscopes, telescopes and other devices can stream in real-time to large computing facilities where it can be analyzed, archived, curated, combined with simulation data and served to the science user community via powerful computing, storage and networking systems. Connected with high-speed programmable networking, this superfacility model is more than the sum of its parts, allowing for discoveries across data sets, institutions and domains and making data from one-of-a-kind facilities and experiments broadly accessible. 

The NERSC Superfacility project is designed to identify the technical and policy challenges in this concept for an HPC center. It coordinates and manages the work to address these challenges, in partnership with target science teams. It is designed to ensure that the solutions developed are widely useful (rather than one-off engagements) and that will scale to multiple user groups, and will be scalable for NERSC staff to support. 

Services in Development

Data Management and Sharing

We are working to develop and deploy tools that can be used to handle the large volumes of data generated by superfacility partners.

Data Transfer

  • Globus is our tool of choice for large data transfers. We have several optimized data transfer nodes that can access every file system at NERSC.
  • We are working to offer a new interface into HPSS that eliminates much of the difficulty of bundling and uploading files.
  • A command line tool to do parallel transfers between file systems at NERSC (including HPSS) is under development.
  • Batch system integration of data movement is being explored

Data Discovery

  • The NERSC Data Dashboard lets you see where your data is on the Project file system.
  • A PI Dashboard is under development to allow PIs to address common issues (like permission drift) for the data they control

Data Sharing

  • Spin, a service platform for deploying science gateways, has been successfully deployed
  • Globus Sharing has been enabled for data on the Project file system

 

Recent papers related to the Superfacility Model

Recent articles about the Superfacility Model

Recent talks about the Superfacility model