Multi-Factor Authentication (MFA)
NERSC users will be required to use Multi-Factor Authentication (MFA) beginning in Allocation Year 2019. MFA provides greater protection than regular passwords against phishing and other modern threats to your digital security. With NERSC's MFA, you authenticate using your NIM password plus a "one-time password" (OTP). As the name implies, you can use an OTP only once.
Currently MFA logins are supported for ssh to NERSC systems, and on a subset of NERSC websites. Through the next couple of months, we will make available some new technologies to enhance the MFA experience, and provide MFA support for most NERSC user resources.
We have implemented MFA in ways that only minimally impact how you work at NERSC. Read below for details on new and upcoming features, and on how to install and use MFA.
NEW: MFA for NoMachine (NX)
MFA support for NX is available. See below for more information about MFA for NoMachine (NX).
How NERSC MFA Works
MFA at NERSC makes use of an app that you install on your mobile device, which you configure through NIM (If you do not have an iOS or Android mobile device, see below for alternatives). The authenticator presents a 6-digit password that changes every 30 seconds. Each password can only be used for one login once, thus the name "one time password" or "OTP." When you login to an MFA-enabled system, you will enter your NIM password followed by the OTP displayed on your authenticator app.
The one-time password entry in the authenticator app is sometimes called a "token," or more specifically, a "soft token." To use MFA, you create a token for NERSC and install it on the authenticator app.
The steps for configuring your NERSC token are:
- Install the authenticator app
- Enable MFA in your NIM account
- Generate a NERSC OTP token via NIM
- Install the token on the authenticator app
NERSC primarily supports the Google Authenticator app, which runs on an Android or iOS device. Other TOTP (Time-based OTP) authenticator apps can work, too - you can search for "TOTP" to find other options. If you do not have such a device, NERSC does support Authy, a desktop app for Windows and Mac computers. These instructions focus on using Google Authenticator. Instructions for Authy are further below.
Please note that you do not need to have a cell phone signal or WiFi to use Google Authenticator. They generate OTPs using the internal clock in your device. Once configured, you can use the app without any phone or internet service.
Enabling MFA in NIM
Before creating a token, you need to select 'MFA Opt In' in your NIM account. Login to your NIM account using your login ID and NIM password at
Note that, if you have already opted in but have not created any tokens, you can login to your NIM account with just the NIM password (that is, without a one-time password).
To enable MFA, click the 'MFA Token' tab from a group of yellow horizontal tabs; alternatively, select 'MFA Enable/Disable' under the Actions pull-down menu:
Select 'Enabled' and click the 'Save All Rows' button.
Clicking the 'Contact Information' link above, you will see that the Contact Info page correctly indicates that MFA has been enabled:
Note: After you enable MFA, you will not be able to login to NERSC resources and services without using a MFA token. If you don't set up a MFA token (see below) or delete all the MFA tokens that you have created, you will see a warning message on the NIM webpage:
Creating and Installing a Token
In this step you will "connect" the app on your mobile device to your NIM account. You first create a token in NIM, and then install it on your app.
To do so, go back to the previous page (MFA Opt In/Out). Click the 'Enter and manage MFA Tokens' button. It will display the content of the 'MFA Tokens' tab:
To add a new token, click on the 'Add Token' button. Then, it will show your NERSC account name in the 'Select User' field. First, make sure that it matches your username -- some users with an old login name may find the old username displayed there. You can add a descriptive text in the 'Enter Token Description'.
In the above example ("my phone"), the user is trying to generate a token that will be used with the Google Authenticator on a phone. If you have more than one token from different devices, you can add a proper description for each token for your reference.
Click the 'Submit' button. Then, the webpage will display a token and a QR code that is to be scanned into your device. Note the token ID (TOTP22856A68 in the following example).
Using Google Authenticator
If you use Google Authenticator, start up the app on your device and click the red '+' button on the bottom right (in case of an Android device), then 'Scan a barcode'.
On Apple devices, look for the '+' sign at the top right of the app:
If necessary, allow the app to access your camera, and point the camera at the QR code.
If you prefer not to scan the QR code with your device's camera, select 'Enter a provided key' and enter the 'secret' field value shown in the MFA token. For the 'Account name' field, you can use the TOTP number shown in the MFA token (e.g., NERSC-TOTP22856A68). Then, select 'Time based'.
After either scanning the QR code or entering the secret code manually, it should register automatically and then show the new token in your list. Your mobile device is now connected to your NIM account for MFA logins.
When you add the token on your device, the token name, in the form of NERSC-nersc_login_id-token_id (NERSC-wyang-TOTP22856A68 in the above example), appears in the far left side under the token list on your device.
Authy is a desktop app with very similar functionality to Google Authenticator. If it is the first time to use the app, you will have to create an Authy account and you will be asked to provide a phone number and an email address for the registration purpose. Then, the app will display a code which you have to enter for verification when you get a text message or a phone call from Authy.
Click the '+' sign to add an account for which the app will generate OTPs for you. When you are prompted to enter a code "given by the website" as shown below, enter the 'Authy Web Code' field value shown in the MFA token generated in NIM (that is, the code that is blacked out, beneath the QR code in the screenshot shown above). Select a name for this token (e.g., NERSC-TOTP38776DC3), color to be used for displaying the token in the app (e.g., 'Generic Black') and the token length (you must select '6-digit'), and then click 'Save'.
Testing Your New Token
To test if the new token is set correctly, click on the token in NIM. This will show detailed information about the token along with the 'Test' and 'Delete' buttons. Click on the 'Test' button, and you will see the following:
Enter the one time password generated on your authenticator app in the 'Enter One-Time-Password' field (do not enter your NIM password). Then, click on the 'Test Now' button. If everything is successfully configured, the NIM page will show 'Success':
If you have more than one mobile device, you can create a token for each one. For example, you can have one for each phone and also for a tablet, as shown below:
You can have up to 4 tokens.
When you login to NERSC resources, you can use any token for authentication, and do not have to specify which one you will use – the server will match your one-time password against all of the tokens that you have created.
Checking and Managing Tokens
You can view all the tokens you have created from the 'MFA Tokens' tab in NIM (see the picture above).
Deleting a Token
If you don't need a token any more, you can delete it. Select the token in NIM that you want to remove and press the 'Delete' button.
If you lose your device or don't have it with you, you cannot login. Backup passwords are a small set of one-time passwords that you can use if you don't have your mobile device on-hand.
Click the 'Generate!' button near the bottom of the webpage to generate backup OTPs:
Please print or store these passwords in a document and keep in a safe place. When you need one, you simply enter the first unused password on the list and then scratch it out. Next time you need a backup password, you enter the next password on the list, and so on. You must use the passwords in the order given to you by NIM.
Note that if you generate a new set of backup OTPs, any unused ones generated previously become invalid.
MFA for NIM logins
Users who enable MFA will have to subsequently use MFA to login to NIM. The login page looks as shown below:
Lost Tokens (resetting MFA)
If your MFA tokens are lost permanently (for example, you replaced your mobile device), you can delete your existing tokens, and then log into NIM to create new ones. To do so, click the 'Lost your tokens?' link (shown above). You will receive a link from us via email which will take you to a page where you will confirm that you want to delete your tokens. After that, you will be able to login to NIM (but nothing else) with just your password, and create new tokens.
The siimplest way to use MFA with ssh is to ssh to a NERSC system. When you login with ssh to a NERSC machine, you will be prompted to enter 'Password + OTP':
$ ssh cori.nersc.gov ***************************************************************** * * * NOTICE TO USERS * * --------------- * ... Password + OTP:
Open your authenticator app and read the OTP code corresponding to the token generated for your device (for example, 'NERSC-wyang-TOTP22856A68' in the previous snapshot):
Enter your NIM password immediately followed by the OTP, all in one line at the "Password + OTP: " prompt. For example, if your NIM password is “iL0ve_Burrit0$” and your app shows "015 691", as displayed in the screenshot, you must type “iL0ve_Burrit0$015691”. (Don't type the space shown in the app).
Note for Authy users: The first OTP that Authy displays when you click a token is often a bad one, and using it can lead to a login failure. To get a correct OTP, click the back arrow in the top left corner in the app window and re-click the token. We have reported this problem to Authy.
Note: When you enable MFA on your account, any ssh key that you have registered in NIM will no longer work; you have to authenticate with MFA every time you ssh to a NERSC host. Read on to the next section about "sshproxy" to see how you can use MFA just once per day, and obtain keys that you can use with automated workflows.
NERSC has developed a service, called sshproxy, that allows you to use MFA to get an ssh key that is valid for a limited time (24 hours by default). sshproxy provides a type of single-sign-on capability for ssh to NERSC systems. Once you have obtained a key, you can use it to ssh to NERSC systems (e.g., Cori, Edison, Denovo, Genepool, PDSF, DTN, ...) without further authentication until the key expires.
The sshproxy service uses a RESTful API for requesting keys. NERSC provides a bash client script that you can use from the command line on a Unix-like computer. A python script will be also available. A Windows client that supports PuTTY will be available soon.
Installing the Client
On Unix-like machines (macOS included), you can download the bash client sshproxy.sh from a project directory:
$ scp firstname.lastname@example.org:/project/projectdirs/mfa/NERSC-MFA/sshproxy.sh .
where myusername is your NERSC login ID.
Or you can run the following commands on your machine to download the bash client from a NERSC GitHub repository, and set the execute permission bit:
$ curl -O https://raw.githubusercontent.com/NERSC/NERSC-MFA/master/sshproxy.sh $ chmod u+x sshproxy.sh
Alternatively, you can clone the Git repository on your local computer:
$ git clone https://github.com/NERSC/NERSC-MFA.git
The above Git command creates the directory named 'NERSC-MFA' in the current working directory where you can find the script. You can keep this Git repository and update it (with 'git pull') from time to time, so that you have the latest version of the utility if NERSC makes a change to the utility later.
The sshproxy client, without any arguments, will use your local username, and obtain an ssh key with the default lifetime (24 hours). The private and public key will have the names 'nersc' and 'nersc-cert.pub," and will be stored in your ~/.ssh directory.
Run the sshproxy.sh script from where you installed it. For example, if the script is in your current directory, type:
The script will prompt you to enter your password and OTP, in the same manner as you would do to ssh to a NERSC system with MFA:
Enter your password+OTP:
Enter you NIM password immediately followed by OTP as a single string, as before. Upon successfully authenticating, the client will install an ssh key and display a message showing the path to the key pair installed on your local computer and the expiration date and time for the keys. By default, the name of the files will be ~/.ssh/nersc and ~/.ssh/nersc-cert.pub (you can change the name with a command-line argument).
$ ./sshproxy.sh Enter your password+OTP: Successfully obtained ssh key /Users/wyang/.ssh/nersc
Key /Users/wyang/.ssh/nersc is valid: from 2018-08-30T12:24:00 to 2018-08-31T12:25:52
You will see three ssh key files (private and public keys, and a certificate containing the corresponding public key) installed in the ~/.ssh directory on your computer:
$ ls -l ~/.ssh/nersc* -rw------- 1 wyang wyang 3179 Aug 30 12:25 /Users/wyang/.ssh/nersc -rw------- 1 wyang wyang 1501 Aug 30 12:25 /Users/wyang/.ssh/nersc-cert.pub -rw------- 1 wyang wyang 1501 Aug 30 12:25 /Users/wyang/.ssh/nersc.pub
The above example shows that an ssh key pair has been created on your local machine. With these keys, you can ssh to NERSC machines without further authentication until these keys expire.
Checking Certificate Expiration
You can check the expiration date and time of an existing ssh key pair. If the ssh key certificate file is ~/.ssh/nersc-cert.pub, run the following command on your local computer:
$ ssh-keygen -L -f ~/.ssh/nersc-cert.pub | grep Valid Valid: from 2018-08-30T12:24:00 to 2018-08-31T12:25:52
Please note that the times printed are local time (your time), not NERSC time (Pacific Time).
Using sshproxy Keys
You can use the keys you get from the sshproxy to login to NERSC systems by specifying the key file on the command line. For example, to login to cori.nersc.gov with a key named 'nersc':
$ ssh -i ~/.ssh/nersc cori.nersc.gov
This will allow you to login without having to authenticate again.
sshproxy Command-line Options
sshporxy.sh has several command-line options to override its default behavior. You can run 'sshproxy.sh -h' to get a help message.
$ ./sshproxy.sh -h Usage: sshproxy.sh [-u <user>] [-s <scope>] [-o <filename>] [-U <server URL>] -u <user> Specify remote username (default: <your_login_name>) -o <filename> Specify pathname for private key (default: /Users/<your_login_name>/.ssh/nersc) -s <scope> Specify scope (default: 'default')
-a Add key to ssh-agent (with expiration) -U <URL> Specify alternate URL for sshproxy server (generally only used for testing purposes)
If your NERSC username is not the same as your local username, you can specify your NERSC username with the '-u' option:
$ ./sshproxy.sh -u myusername
If you would like to have a different name for the ssh key file, you can use the -o option to specify the output filename:
$ ./sshproxy.sh -o mynersc
Note the -a option can be used to automatically add the new key to your ssh-agent. It will also be set with an expiration that matches the keys expiration so that ssh does not try to use the key after it has expired.
If your computer has an old version of ssh (e.g., OpenSSH_7.2), you may have to use the -a flag. Otherwise, ssh and scp commands will require additional flags to work as in the example cases shown below. To see the version info, run the command, 'ssh -V'.
Long-term SSH Keys
The scope option (-s flag) is to accommodate special needs for your work. If automatic workflow needs keys for a long term, you can make a request in a ticket. Click here to submit a request. We will grant such a request only after a review. When your request is approved, we will provide information on how to set the scope.
SSH Configuration File Options
We recommend some options to put in your ssh config file. These options help avoid some potential problems with expiring ssh keys, and provide default key filenames to ssh so that you don't have to specify the key on the command line every time you use ssh. These options can all be overridden on the command-line at any time.
If you typically use only the default 'nersc' key from sshproxy, you can modify your ssh config file to automatically use that key, instead of having to specify it on the command line every time. To do so, edit the file ~/.ssh/config on your local computer to include the following lines:
Host cori*.nersc.gov edison*.nersc.gov denovo.nersc.gov gpint*.nersc.gov gpweb*.nersc.gov genepool.nersc.gov pdsf.nersc.gov dtn*.nersc.gov IdentityFile ~/.ssh/nersc
With that entry, whenever you ssh to one of those NERSC systems, your ssh client will automatically use your proxy key.
If your ssh client does not present a valid ssh key to the ssh server, the server will prompt you to authenticate with NIM password + OTP. Neither the server nor the client will tell you that your key has expired.
Login to NERSC Machines
After you set up ssh keys as above, you login with ssh to a NERSC computational machine without further authentication, as long as the keys hasn't expired:
$ ssh cori.nersc.gov ***************************************************************** * * * NOTICE TO USERS * * --------------- * ...
$ # You're on cori
After you log in, you can build your code, submit batch jobs, debug your code, etc. as you would normally do on any login node.
You can transfer a file to or from a NERSC machine with scp, in the same manner as you use ssh:
$ scp myfile edison.nersc.gov:~ ***************************************************************** * * * NOTICE TO USERS * * --------------- * ...
myfile 100% 13 0.5KB/s 00:00
You will not be prompted to authenticate, either.
Login Attempts with an Expired Key
If you try to login with an expired key, the server will not tell you that the key has expired. It will just prompt you to login with MFA, as if you did not have an ssh key:
$ ssh cori.nersc.gov ***************************************************************** * * * NOTICE TO USERS * * --------------- * ... Password + OTP:
You can generate new ssh keys by running the sshproxy.sh script at any time, as shown in the 'Using sshproxy' section above.
Host Based Authentication
NERSC HPC hosts are configured to use ssh "host based" authentication for logins between Cori, Edison, and NX. This means that, once you log in to a host in the list from a remote host, you can ssh from that NERSC host to another without having to authenticate or ssh-agent.
$ ssh cori.nersc.gov ***************************************************************** * * * NOTICE TO USERS * * --------------- * ...
$ nersc_host # You're on Cori cori $ ssh edison # Go to Edison from Cori ***************************************************************** * * * NOTICE TO USERS * * --------------- * ... $ nersc_host # You're now on Edison edison
Since host based authentication is enabled with NX, you can go to Cori or Edison from NX without any authentication.
When logging into NX, please enter your password followed by your six-digit OTP, as a single string, in the password field. Once logged into NX, no further authentication is required to connect to Cori, Edison.
The NERSC MyProxy service will require MFA-enabled users to authenticate using their password and OTP.
Most NERSC web sites authenticate users using one of two authentication services, Shibboleth or NEWT, which each provide single sign-on capability across participating sites. For each of those services, once you have authenticated to one NERSC site, you will be able to access all other sites using that service without having to authenticate again for 24 hours. Both Shibboleth and NEWT will require MFA-enabled users to enter their OTP in addition to their password.
Sites that use Shibboleth will present a login page displays NERSC login banner as shown below. Login with your NIM user name and password.
Then, you will be prompted to enter an OTP:
Sites that use NEWT will have login pages that look different than the Shibboleth login banner. Below is from My NERSC login page:
A few NERSC sites use neither Shibboleth nor NEWT, for various technical reasons. For those sites, single sign-on is unavailable and you will have to individually authenticate to them using MFA. Login using NIM password and an OTP:
The NIM User Portal will also require MFA-enabled users to login using MFA.
Status of MFA on User Systems
Currently, MFA is supported on most of the systems that users access via ssh, such as Cori, Edison, etc. Web and other services will start supporting MFA in September 2018. The table below shows the status of MFA on NERSC systems and services.
MFA Available Now
|Data Transfer Nodes|
|Shibboleth||Online Help Desk (https://help.nersc.gov)|
|Science gateways with NERSC (Shibboleth) login banner|
|Science gateways accepting NIM passwords not displaying the NERSC (Shibboleth) login banner|
|NX and NX-cloud|
|HPSS token generation|
MFA Coming Soon
MFA Not Applicable
|Other science gateways hosted on portal.nersc.gov|
|The Materials Project|
Frequently Asked Questions (FAQ)
(Q) I don't have a smartphone or tablet. What do I do?
A desktop authenticator app called Authy will work for Windows and Mac computers.
You can use Authy's Chrome browser extension, too. FoxAuth Authenticator extension for Firefox web browser (https://addons.mozilla.org/en-US/firefox/addon/foxauth/) also works. Other TOTP authenticator web browser plugins may work.
We encourage you to install the app or a web extension on a different machine from the one you use to connect to NERSC for a security reasons.
We are working on an option of using a hardware token (users must purchase their own tokens). We will provide details when the option becomes available.
(Q) The clock on my device often drifts from the correct time. This happens especially to my cellphone when traveling overseas with no cell phone connectivity. Can I still use OTPs generated there? What is the time synchronization requirement between the NERSC OTP server and a device?
The NERSC server adjusts for your clock skew and will drift with it, as long as each drift is less than 180 seconds. Where you will have a problem is if your clock drifts more than 180 seconds between successive MFA authentications. Most often that happens when your phone clock has been drifting for a while, then you get it online again, it syncs to the cell tower, and the clock suddenly changes by a large amount. In that case, the solution we have at the moment is to delete the token and have you create a new one.
(Q) What if I lose my device?
If you have another device where you have configured a NERSC MFA token with its app, you should login to your NIM account as soon as possible and delete all the MFA tokens associated with the lost device.
If you have no other device, then go to the NIM login page, https://nim.nersc.gov. Click on the 'Lost your tokens?' link to request for a validation code that you will use to login into your NIM account. Logging in with the validation code will delete all the MFA tokens in NIM. Once you login, create a new token for a new device. If you don't have a new device yet, you can generate backup OTPs and use them for a time being.
(Q) I have two devices. How do I copy or transfer my token from one to the other?
You cannot copy or transfer a NERSC OTP token from one device to another. However, when NIM generates a QR code along with the "secret" code (see the 'Creating and Installing a Token' section), you can create a token on each device using the same QR or secret code, if you want. Then, if multiple devices' internal clocks are running at the same rate and the time on the devices is the same, the authenticator apps on the multiple devices will show the identical OTP.
(Q) MFA is enabled for my account but I didn't set a MFA token on an authenticator app. How can I login to NIM to set up?
Click the 'Lost your tokens?' link in the NIM login page (https://nim.nersc.gov) to request for a validation code that you can use to login to your NIM account. You will receive a validation code by email. Once you login, create a MFA token and configure it on an authenticator app.
(Q) I have enabled MFA. My logins fail repeatedly. What should I do?
If this is with a particular host (Cori, Edison, etc.) only, then login to your NIM account at https://nim.nersc.gov. That will clear login failures that may have accumulated for the host. Then, try to login to the host again.
If you enter incorrect OTPs too many times, the NERSC MFA server locks you out. In that case, you have to wait for 15 minutes before you try again.
If you are using ssh keys generated via sshproxy.sh for authentication, check if the keys have expired.
A popular way of using ssh key authentication is via ssh-agent, the "authentication agent." You add an ssh private key to ssh-agent and it uses the key to authenticate to a remote host that has the matching public key. You may be knowingly or unknowingly using this method (especially, when you use the -a option with sshproxy.sh). Ssh-agent goes through the saved keys one by one to see if the correct key is found. If it cannot find the matching key within 6 tries, ssh authentication fails. When you have many keys stored in ssh-agent, including the correct one, login can fail if the correct key is not selected within the first 6 tries. To see how many keys are stored in ssh-agent, run the command 'ssh-add -l' on your laptop/desktop. If you see many keys there, you can delete all of them with the command 'ssh-add -D' and run the sshproxy.sh command again. You can also selectively remove an individual key with the '-d' flag (for info, see the ssh-add man page).
If you don't remember your password, then follow the steps in the 'Forgotten Passwords' section in https://www.nersc.gov/users/accounts/user-accounts/passwords.
If all your MFA tokens don't seem to work, click the 'Lost your tokens?' link in the NIM login page (https://nim.nersc.gov) to request for a validation code that you can use to login to your NIM account. You will receive a validation code by email. Please note that logging into NIM with the code will delete all the MFA tokens in NIM. Once you login, create a new one.
(Q) I login to a NERSC resource and type my OTP displayed on an authenticator app. I need to login to a different resource. Can I type the same OTP if it is still valid as the 30-second time window hasn't expired?
No, it will not work. It's because an OTP (One Time Password) can be used only once for authentication. You have to wait until the next 30-second time window starts to get a new OTP.
(Q) We are running automated jobs on NERSC machines. How can we continue to do this with MFA?
Please try the sshproxy service. It currently serves keys that are good for 24 hours, but longer ones are possible, depending on what your needs are. To request for long-term keys, fill out the request form.
(Q) What services are enabled with MFA?
Please check the MFA availability status tables in the 'Status of MFA on User Systems' section above.
(Q) A NERC resource that I want to use doesn't support MFA, yet. Does it mean that I cannot access the resource if I enable MFA for my account?
No, it simply means that you can login to that resource with your NIM password only.
(Q) Do we have SSH host based authentication among all NERSC hosts?
Currently, SSH host based authentication works between Cori, Edison, and NX only. That is, if you are on Cori, for instance, you can ssh to Edison without being prompted to enter password + OTP. With NX, you can go to Cori and Edison without further authentication. However, when you ssh to a different host (e.g., dtn01) from Cori or Edison, you will be prompted to authenticate, and vice versa. We are working on extending host based authentication to the other systems. For now, you can get a long-lived key (via sshproxy) and that could be forwarded to jump between hosts.
(Q) Is there an sshproxy client for Windows?
We are currently developing and testing a tool, and we hope to release one soon.
In the meantime, if you have Cygwin, you can use the Linux version of the sshproxy client in a Cygwin terminal. Make sure that your Cygwin contains curl and openssh packages.
(Q) How can I use a tool like BBEdit, FileZilla, WinSCP, etc. that requires authentication to a NERSC host?
If the tool supports ssh key authentication, you can use the SSH keys generated by the sshproxy client. In that case, the settings in the 'SSH configuration File Options' section above can be the only thing required for authenticating properly. Please check the tool's user manual or documentation for specific info. BBEdit and FileZilla support ssh key authentication although FileZilla appears to require some manual setup.
If you use WinSCP, select 'SCP' in the 'File protocol' field and enter your username in the 'User name' field in the 'Login' window. Leave the 'Password' field blank. Then, click the 'Login' button. Click 'Continue' in the 'Authentication Banner' window. Then, you will see the 'Server prompt' window where you enter your password immediately followed by an OTP.
Questions, Comments, ...
If you have any questions, problems or comments, please contact us at https://help.nersc.gov.