NERSCPowering Scientific Discovery Since 1974

Multi-Factor Authentication (MFA)

Introduction

NERSC provides users with the ability to use Multi-Factor Authentication (MFA) for logging into NERSC resources. MFA provides greater protection than regular passwords against phishing and other modern threats to your digital security. With NERSC's MFA, you authenticate using your NIM password plus a "one-time password" (OTP). As the name implies, you can use an OTP only once. You can find instructions for configuring and using MFA below.

Currently MFA logins are supported for ssh to NERSC systems, and on a subset of NERSC websites. Through the next couple of months, we will make available some new technologies to enhance the MFA experience, and provide MFA support for most NERSC user resources.

We have implemented MFA in ways that only minimally impact how you work at NERSC. Read below for details on new and upcoming features, and on how to install and use MFA.

NEW: SSH Keys with MFA

A key new feature for MFA at NERSC is a service called "sshproxy." This service allows you to use MFA to get an ssh key that is valid for 24 hours, using a simple command-line shell script. The ssh key can be used to login to any NERSC systems for which you are authorized. With sshproxy, you only have to use MFA once per day. See below for details on how to obtain and use the client script.

NEW: MFA for (Some) Websites

A subset of NERSC websites now support MFA; users who have enabled MFA will be prompted to enter their OTP, and for most sites will have to only do so once per day across all of those sites. See below for details on how the login process changes.

Coming Soon: Backup Passwords

NERSC MFA uses an app that runs on your mobile device. If you lose your device, or don't have it with you, you cannot login. Backup passwords are a small set of one-time passwords that you can print or store in a document, to use if you don't have your mobile device on-hand. You simply enter the first backup password on the list and then scratch it out; the second time you need a backup password, you enter the next password on the list. You will be able to get a set of backup passwords through NIM. We expect to have this service available mid-late September, 2018.

Coming Soon: MFA for NIM logins

Since you use NIM to provision your MFA, it's only appropriate that we support MFA for NIM logins. Users who enable MFA will have to subsequently use MFA to login to NIM. NIM will provide a reset mechanism to use if you have lost your MFA device or otherwise are having problems with MFA. We expect to have this service available mid-late September.

Coming Soon: MFA for MyProxy and Other Websites

MFA support for MyProxy and most other NERSC websites will be available in early October, 2018. See below for more information about MFA for MyProxy and websites.

Coming Soon: MFA for HPSS Tokens

HPSS will support MFA for generating HPSS tokens. We expect to have this support available early November, 2018.

How NERSC MFA Works

MFA at NERSC makes use of an app that you install on your mobile device, which you configure through NIM (If you do not have an iOS or Android mobile device, see below for alternatives). The authenticator presents a 6-digit password that changes every 30 seconds. Each password can only be used for one login once, thus the name "one time password" or "OTP." When you login to an MFA-enabled system, you will enter your NIM password followed by the OTP displayed on your authenticator app.

Screen Shot 2017 05 05 at 12.03.01 PM

 

Configuring and Using an MFA Token

The one-time password entry in the authenticator app is sometimes called a "token," or more specifically, a "soft token." To use MFA, you create a token for NERSC and install it on the authenticator app.

The steps for configuring your NERSC token are:

  • Install the authenticator app
  • Enable MFA in your NIM account
  • Generate a NERSC OTP token via NIM
  • Install the token on the authenticator app

Installing the Authenticator App

NERSC primarily supports the Google Authenticator app, which runs on an Android or iOS device. If you do not have such a device, NERSC does support Authy, a desktop app for Windows and Mac computers. For those who would like a hardware token, NERSC can support Yubikey, but you have to purchase it yourself.

These instructions focus on using Google Authenticator. Instructions for Authy are further below. For information on using a Yubikey, contact us at https://help.nersc.gov.

Install Google Authenticator. For an Android device, you can download it from here; for an Apple device, download from here.

Please note that you do not need to have a cell phone signal or WiFi to use Google Authenticator. They generate OTPs using the internal clock in your device. Once configured, you can use the app without any phone or internet service.

Enabling MFA in NIM

Before creating a token, you need to select 'MFA Opt In' in your NIM account. Login to your NIM account using your login ID and NIM password at

https://nim.nersc.gov

Note that, if you have already opted in but have not created any tokens, you can login to your NIM account with just the NIM password (that is, without a one-time password).

To enable MFA, select 'MFA Opt In/Out' under the Actions pull-down menu:

nim mfa opt in 2

 

Select 'Enabled' and click the 'Save All Rows' button.

nim mfa enabling 5

 

Clicking the 'Contact Information' link above, you will see that the Contact Info page correctly indicates that MFA has been enabled:

mfa enabled nim contact info

 

Creating and Installing a Token

In this step you will "connect" the app on your mobile device to your NIM account. You first create a token in NIM, and then install it on your app.

To do so, go back to the previous page (MFA Opt In/Out). Click the 'Enter and manage MFA Tokens' button. It will display the content of the 'MFA Tokens' tab:

nim mfa enabling05

To add a new token, click on the 'Add Token' button. Then, it will show your NERSC account name in the 'Select User' field. You can add a descriptive text in the 'Enter Token Description'.

nim mfa enabling06


In the above example ("my phone"), the user is trying to generate a token that will be used with the Google Authenticator on a phone. If you have more than one token from different devices, you can add a proper description for each token for your reference.

Click the 'Submit' button. Then, the webpage will display a token and a QR code that is to be scanned into your device. Note the token ID (TOTP22856A68 in the following example).

nim mfa enabling07

Using Google Authenticator

If you use Google Authenticator, start up the app on your device and click the red '+' button on the bottom right (in case of an Android device), then 'Scan a barcode'.

googleauthenticator1bc

On Apple devices, look for the '+' sign at the top right of the app:

googleauthenticator1bi


If necessary, allow the app to access your camera, and point the camera at the QR code. It should register automatically and then show the new token in your list. Your mobile device is now connected to your NIM account for MFA logins.

When you add the token on your device, the token name, in the form of NERSC-nersc_login_id-token_id (NERSC-wyang-TOTP22856A68 in the above example), appears in the far left side under the token list on your device.

Using Authy

Authy is a desktop app with very similar functionality to Google Authenticator. If you choose to use Authy, start the app and click the '+' sign to add an account for which the app will generate OTPs for you. When you are prompted to enter a code "given by the website" as shown below, enter the 'secret' field value shown in the MFA token generated in NIM (that is, the code that is blacked out, beneath the QR code in the screenshot shown above). Select a name for this token (e.g., NERSC-TOTP38776DC3), color to be used for displaying the token in the app (e.g., 'Generic Black') and the token length (you must select '6-digit'), and then click 'Save'.

authy4 5

 

Testing Your New Token

To test if the new token is set correctly, click on the token in NIM. This will show detailed information about the token along with the 'Test' and 'Delete' buttons. Click on the 'Test' button, and you will see the following:

nim mfa testing01

Enter the one time password generated on your authenticator app in the 'Enter One-Time-Password' field (do not enter your NIM password). Then, click on the 'Test Now' button. If everything is successfully configured, the NIM page will show 'Success':

nim mfa testing02

 

Multiple Tokens

If you have more than one mobile device, you should create a token for each one. For example, you can have one for each phone and also for a tablet, as shown below:

nim mfa tokens

When you login to NERSC resources, you can use any token for authentication, and do not have to specify which one you will use – the server will match your one-time password against all of the tokens that you have created.

Checking and Managing Tokens

You can view all the tokens you have created from the 'MFA Tokens' tab in NIM (see the picture above).

Deleting a Token

If you don't need a token any more, you can delete it. Select the token in NIM that you want to remove and press the 'Delete' button.

If you want to delete all tokens, then click the 'Delete All' button under the MFA Tokens tab.

Using MFA with SSH

The siimplest way to use MFA with ssh is to ssh to a NERSC system. When you login with ssh to a NERSC machine, you will be prompted to enter 'Password + OTP':

$ ssh cori.nersc.gov
 *****************************************************************
 *                                                               *
 *                      NOTICE TO USERS                          *
 *                      ---------------                          *
...
Password + OTP:

Open your authenticator app and read the OTP code corresponding to the token generated for your device (for example, 'NERSC-wyang-TOTP22856A68' in the previous snapshot):

googleauthenticator4b

 

Enter your NIM password immediately followed by the OTP, all in one line at the "Password + OTP: " prompt. For example, if your NIM password is “iL0ve_Burrit0$” and your app shows "015 691", as displayed in the screenshot, you must type “iL0ve_Burrit0$015691”. (Don't type the space shown in the app).

NOTE: When you enable MFA on your account, any ssh key that you have registered in NIM will no longer work; you have to authenticate with MFA every time you ssh to a NERSC host. Read on to the next section about "sshproxy" to see how you can use MFA just once per day, and obtain keys that you can use with automated workflows.

sshproxy

NERSC has developed a service, called sshproxy, that allows you to use MFA to get an ssh key that is valid for a limited time (24 hours by default). sshproxy provides a type of single-sign-on capability for ssh to NERSC systems. Once you have obtained a key, you can use it to ssh to NERSC systems (e.g., Cori, Edison, Denovo, Genepool, PDSF, DTN, ...) without further authentication until the key expires.

The sshproxy service uses a RESTful API for requesting keys. NERSC provides a bash client script that you can use from the command line on a Unix-like computer. A python script is also available, and a Windows client that supports PuTTY will be available soon.

Installing the Client

You can download the client from a NERSC GitHub repository. On Unix-like machines (macOS included), run the following commands on the machine to download the bash client sshproxy.sh, and set the execute permission bit:

$ curl -O https://raw.githubusercontent.com/NERSC/NERSC-MFA/master/sshproxy.sh
$ chmod u+x sshproxy.sh

Alternatively, you can clone the Git repository on your local computer:

$ git clone https://github.com/NERSC/NERSC-MFA.git

The above Git command creates the directory named 'NERSC-MFA' in the current working directory where you can find the script. You can keep this Git repository and update it (with 'git pull') from time to time, so that you have the latest version of the utility if NERSC makes a change to the utility later.

Using sshproxy

The sshproxy client, without any arguments, will use your local username, and obtain an ssh key with the default lifetime (24 hours). The private and public key will have the names 'nersc' and 'nersc-cert.pub," and will be stored in your ~/.ssh directory.

Run the sshproxy.sh script from where you installed it. For example, if the script is in your current directory, type:

$ ./sshproxy.sh

The script will prompt you to enter your password and OTP, in the same manner as you would do to ssh to a NERSC system with MFA:

Enter your password+OTP: 

Enter you NIM password followed by OTP, as before. Upon successfully authenticating, the client will install an ssh key and display a message showing the path to the key pair installed on your local computer and the expiration date and time for the keys. By default, the name of the files will be ~/.ssh/nersc and ~/.ssh/nersc-cert.pub (you can change the name with a command-line argument).

$ ./sshproxy.sh
Enter your password+OTP: 
Successfully obtained ssh key /Users/wyang/.ssh/nersc
Key is valid: from 2018-08-30T12:24:00 to 2018-08-31T12:25:52

You will see two ssh key files (a private key and a certificate containing the corresponding public key) installed in the ~/.ssh directory on your computer:

$ ls -l ~/.ssh/nersc*
-rw-------  1 wyang  wyang  3179 Aug 30 12:25 /Users/wyang/.ssh/nersc
-rw-------  1 wyang  wyang  1501 Aug 30 12:25 /Users/wyang/.ssh/nersc-cert.pub

The above example shows that an ssh key pair has been created on your local machine. With these keys, you can ssh to NERSC machines without further authentication until these keys expire.

Checking Certificate Expiration

You can check the expiration date and time of an existing ssh key pair. If the ssh key certificate file is ~/.ssh/nersc-cert.pub, run the following command on your local computer:

$ ssh-keygen -L -f ~/.ssh/nersc-cert.pub | grep Valid
        Valid: from 2018-08-30T12:24:00 to 2018-08-31T12:25:52

Please note that the times printed are local time (your time), not NERSC time (Pacific Time).

Using sshproxy Keys

You can use the keys you get from the sshproxy to login to NERSC systems by specifying the key file on the command line. For example, to login to cori.nersc.gov with a key named 'nersc':

$ ssh -i ~/.ssh/nersc cori.nersc.gov

This will allow you to login without having to authenticate again.

sshproxy Command-line Options

sshporxy.sh has several command-line options to override its default behavior. You can run 'sshproxy.sh -h' to get a help message.

$ ./sshproxy.sh -h
Usage: sshproxy.sh [-u <user>] [-s <scope>] [-o <filename>] [-U <server URL>]
         -u <user>	Specify remote username (default: <your_login_name>)
         -o <filename>  Specify pathname for private key (default: /Users/<your_login_name>/.ssh/nersc)
         -s <scope>     Specify scope (default: 'default')
         -U <URL>       Specify alternate URL for sshproxy server (generally only used for testing purposes)

If your NERSC username is not the same as your local username, you can specify your NERSC username with the '-u' option:

$ ./sshproxy.sh -u myusername

If you would like to have a different name for the ssh key file, you can use the -o option to specify the output filename:

$ ./sshproxy.sh -o mynersc

The scope option (-s flag) is to accommodate special needs for your work. If automatic workflow needs keys for a long term, you can make a request in a ticket. Click here to submit a request. We will grant such a request only after a review. When your request is approved, we will provide information on how to set the scope.

SSH Configuration File Options

We recommend some options to put in your ssh config file. These options help avoid some potential problems with expiring ssh keys, and provide default key filenames to ssh so that you don't have to specify the key on the command line every time you use ssh. These options can all be overridden on the command-line at any time.

If you typically use only the default 'nersc' key from sshproxy, you can modify your ssh config file to automatically use that key, instead of having to specify it on the command line every time. To do so, edit the file ~/.ssh/config on your local computer to include the following lines:

Host cori.nersc.gov edison.nersc.gov denovo.nersc.gov genepool.nersc.gov pdsf.nersc.gov dtn*.nersc.gov
    IdentityFile ~/.ssh/nersc

With that entry, whenever you ssh to one of those NERSC systems, your ssh client will automatically use your proxy key.

We also recommend adding the following options to your config, to prevent sshproxy keys from being added to ssh-agent, and to prevent ssh from attempting to use ssh-agent when connecting to NERSC systems:

    IdentitiesOnly yes
    AddKeysToAgent no

We recommend these options because ssh-agent does not ignore expired ssh certificates (or "certs"). The agent will try to authenticate using those certs, which causes the ssh server to fail because of too many authentication failures. ssh-agent also does not have a simple means of removing expired ssh certs.

If your ssh client does not present a valid ssh key to the ssh server, the server will prompt you to authenticate with NIM password + OTP. Neither the server nor the client will tell you that your key has expired.

Login to NERSC Machines

After you set up ssh keys as above, you login with ssh to a NERSC computational machine without further authentication, as long as the keys hasn't expired:

$ ssh cori.nersc.gov
 *****************************************************************
 *                                                               *
 *                      NOTICE TO USERS                          *
 *                      ---------------                          *
...
$ # You're on cori

After you log in, you can build your code, submit batch jobs, debug your code, etc. as you would normally do on any login node.

You can transfer a file to or from a NERSC machine with scp, in the same manner as you use ssh:

$ scp myfile edison.nersc.gov:~
 *****************************************************************
 *                                                               *
 *                      NOTICE TO USERS                          *
 *                      ---------------                          *
...
myfile 100% 13 0.5KB/s 00:00

You will not be prompted to authenticate, either.

Login Attempts with an Expired Key

If you try to login with an expired key, the server will not tell you that the key has expired. It will just prompt you to login with MFA, as if you did not have an ssh key:

$ ssh cori.nersc.gov
 *****************************************************************
 *                                                               *
 *                      NOTICE TO USERS                          *
 *                      ---------------                          *
...
Password + OTP: 

You can generate new ssh keys by running the sshproxy.sh script at any time, as shown in the 'Using sshproxy' section above.

Host Based Authentication

NERSC HPC hosts are configured to use ssh "hostbased" authentication for logins between Cori, Edison, and Denovo. This means that, once you log in to a host in the list from a remote host, you can ssh from that NERSC host to another without having to authenticate or ssh-agent.

$ ssh cori.nersc.gov
 *****************************************************************
 *                                                               *
 *                      NOTICE TO USERS                          *
 *                      ---------------                          *
...
$ nersc_host # You're on Cori cori $ ssh edison # Go to Edison from Cori ***************************************************************** * * * NOTICE TO USERS * * --------------- * ... $ nersc_host # You're now on Edison edison

MFA for MyProxy

The NERSC MyProxy service will require MFA-enabled users to authenticate using their password and OTP.

MFA for Web Services

Most NERSC web sites authenticate users using one of two authentication services, Shibboleth or NEWT, which each provide single sign-on capability across participating sites. For each of those services, once you have authenticated to one NERSC site, you will be able to access all other sites using that service without having to authenticate again for 24 hours. Both Shibboleth and NEWT will require MFA-enabled users to enter their OTP in addition to their password.

Sites that use Shibboleth will present a login page displays NERSC login banner as shown below. Login with your NIM user name and password.

nersclogin

Then, you will be prompted to enter an OTP:

shib login02b

Sites that use NEWT will have login pages that look different than the Shibboleth login banner. Below is from My NERSC login page:

mynersc login

A few NERSC sites use neither Shibboleth nor NEWT for various technical reasons. For those sites, single sign-on is unavailable and you will have to individually authenticate to them using MFA. Login using NIM password and an OTP:

jupyter login01b

Some NERSC sites do not use Shibboleth for various technical reasons, e.g., those that use NEWT. For those sites, single sign-on is unavailable and you will have to individually authenticate to them using MFA.

The NIM User Portal will also require MFA-enabled users to login using MFA.

MFA for HPSS Tokens

The HPSS authentication token generation service will support MFA for creating tokens. This service will be available in November.

Status of MFA on User Systems

Currently, MFA is supported on most of the systems that users access via ssh, such as Cori, Edison, etc. Web and other services will start supporting MFA in September 2018. The table below shows the status of MFA on NERSC systems and services.

MFA Available Now

Authentication Host
SSH Cori
Edison
Denovo
PDSF
Genepool
Data Transfer Nodes
gpweb
gpdb
Shibboleth Online Help Desk (https://help.nersc.gov)
Science gateways with NERSC (Shibboleth) login banner
Others Jupyter-dev
RStudio

MFA Coming Soon

Authentication Host
NEWT My NERSC
Science gateways accepting NIM passwords not displaying the NERSC (Shibboleth) login banner
Others NIM
Jupyter
HPSS
NX and NX-cloud
TBD NERSC Spin Registry

MFA Not Applicable

Host
Other science gateways hosted on portal.nersc.gov
GRDC
WeFold
CRCNS
The Materials Project
QCD

Questions, Comments, ...

If you have any questions, problems or comments, please contact us at https://help.nersc.gov.