Franklin User Environment

Operating System

Franklin's compute nodes run a light weight Linux-based OS, Compute Node Linux (CNL), which is optimized for application performance. CNL interacts with with an application process in a limited way, including managing virtual memory addressing, providing memory protection and performing basic scheduling. CNL reduces system overhead, ensures reproducible run-times for MPP jobs, and high performance, low latency MPI and SHMEM communications, which are critical for the system to scale to thousands of processors.

User Environment

Shell Initialization Files

Environment initialization files ("dot-files") in home directories have been configured for maximum flexibility and ease of maintenance, both for the user and for NERSC staff. A long listing of a newly-created home directory (for user aaa) should look similar to this:

% ls -al
total 168
drwx------  4 aaa  aaa  4096 Mar 22 21:40 .
drwxr-xr-x  3 bin  bin  4096 Mar 15 13:19 ..
lrwxrwxrwx  1 aaa  aaa     8 Aug 25  2009 .bash_profile -> .profile
-rwxrwxr-x  1 aaa  aaa   610 Feb 19 16:04 .bash_profile.ext
lrwxrwxrwx  1 aaa  aaa    36 Aug 31  2009 .bashrc -> /global/homes/skel/read-only/.bashrc
-rwxrwxr-x  1 aaa  aaa   580 Feb 19 16:04 .bashrc.ext
lrwxrwxrwx  1 aaa  aaa    35 Aug 31  2009 .cshrc -> /global/homes/skel/read-only/.cshrc
-rwxrwxr-x  1 aaa  aaa   599 Oct 28 13:15 .cshrc.ext
lrwxrwxrwx  1 aaa  aaa     7 Aug 25  2009 .kshrc -> .bashrc
-rwxrwxr-x  1 aaa  aaa   575 Feb 19 16:04 .kshrc.ext
lrwxrwxrwx  1 aaa  aaa    35 Aug 31  2009 .login -> /global/homes/skel/read-only/.login
-rwxrwxr-x  1 aaa  aaa   599 Oct 28 13:16 .login.ext
-rw-rw-r--  1 aaa  aaa    17 Sep 14  2009 .modules
lrwxrwxrwx  1 aaa  aaa    37 Aug 31  2009 .profile -> /global/homes/skel/read-only/.profile
-rwxrwxr-x  1 aaa  aaa   585 Feb 19 16:05 .profile.ext
lrwxrwxrwx  1 aaa  aaa     6 Aug 25  2009 .tcshrc -> .cshrc
-rwxrwxr-x  1 aaa  aaa   604 Oct 28 11:22 .tcshrc.ext
lrwxrwxrwx  1 aaa  aaa    38 Aug 31  2009 .zprofile -> /global/homes/skel/read-only/.zprofile
-rwxrwxr-x  1 aaa  aaa   588 Feb 19 16:05 .zprofile.ext
lrwxrwxrwx  1 aaa  aaa    36 Aug 31  2009 .zshenv -> /global/homes/skel/read-only/.zshenv
-rwxrwxr-x  1 aaa  aaa   578 Feb 19 16:05 .zshenv.ext
lrwxrwxrwx  1 aaa  aaa    35 Aug 31  2009 .zshrc -> /global/homes/skel/read-only/.zshrc
-rwxrwxr-x  1 aaa  aaa   573 Feb 19 16:05 .zshrc.ext

The files .bash_profile, .bashrc, .cshrc, .kshenv, .login, .profile, and .tcshrc are links to read-only files, and should not be deleted. All individual customizations (aliases, environment variables, etc.) should be made in the files named .bashrc.ext, .cshrc.ext, .kshenv.ext, .login.ext, .profile.ext, and .tcshrc.ext. These .ext files are sourced by the corresponding dot-files.

If these links and .ext files are missing, please run the fixdots, as shown below, to re-create them; then log out and log in again. All current dot files will be saved in a directory named $HOME/KeepDots.$Current_Time.

% /usr/common/usg/bin/fixdots

After running the fixdots tool, modify the appropriate .ext files to customize the login shell, using the settings saved in the KeepDots directory.

Shells

The default login shell for all new accounts on Franklin is C-shell (csh). Other available login shells are bash, csh, ksh, and sh. Users may not change their login shell directly on Franklin; NIM is used to accomplish this function. See Change Default Login Shell.

Default Programming Environment

Upon login, the example list of the default modules being loaded are:
(Note: Please do not unload these modules unless being swapped to suitable equivalents!)

% module list 
Currently Loaded Modulefiles:
  1) modules/3.1.6.5                       12) cray/csa/3.0.0-1_2.0202.19602.75.1
  2) moab/5.2.5                            13) cray/account/1.0.0-2.0202.19482.49.3
  3) torque/2.4.1b1-snap.200905131530      14) cray/projdb/1.0.0-1.0202.19483.52.1
  4) xt-asyncpe/3.3                        15) Base-opts/2.2.48B
  5) xtpe-barcelona                        16) pgi/9.0.4
  6) xtpe-target-cnl                       17) xt-libsci/10.4.0
  7) xt-service/2.2.48B                    18) xt-mpt/3.5.0
  8) xt-os/2.2.48B                         19) xt-pe/2.2.48B
  9) xt-boot/2.2.48B                       20) PrgEnv-pgi/2.2.48B
 10) xt-lustre-ss/2.2.48B_1.6.5            21) cray/MySQL/5.0.64-1.0202.2899.21.1
 11) cray/job/1.5.5-0.1_2.0202.19481.53.6 

The default programming environment is PGI (via module PrgEnv-pgi). The default PGI compiler version (in this example, pgi/9.0.4) is also loaded.

Other loaded modules include Torque/Moab for batch system management (via modules torque and moab), Message Passing Tool Kit (via module xt-mpt) for MPI, Shmem, and OpenMP libraries, Cray LibSci math and scientific libraries (via module libsci), and Lustre support (via module xt-lustre-ss).

There are a few options for the command "module avail":

• module avail or module avail -A lists all available modules.
• module avail -U lists all user modules.
• module avail -L lists all library modules.
• module avail -T lists all tool modules.
• module avail -P lists all programming environment modules.
• module avail -X lists all target modules.

Process Limits

The following process limits are enforced on the Franklin login nodes:

   cputime	60 min
   datasize	soft limit 1 GB, hard limit 2 GB
   stacksize	soft limit 128 MB, hard limit 256 MB
   memoryuse	soft limit 1 GB, hard limit 2 GB
   vmemoryuse	soft limit 2 GB, hard limit 2 GB
   descriptors  soft limit 2048, hard limit 2048
   maxproc      soft limit 150, hard limit 200