Systems
Franklin Home
About Franklin
Access to Franklin
User Environment
Accounts/Charges
File Storage
Programming
Running Jobs
Software
Performance Tools
Performance Monitoring
IO Strategies
Franklin Timeline
Vendor Manuals
Status & Stats
NERSC MOTD
Announcements
Known Problems
Current Queue Look
Completed Jobs List
Job Stats
Franklin File Storage
Online File Systems
Quotas and Usage
Archival Storage
Online File Systems
All user-accessible local file storage is provided by the Lustre File System, which provides scalable, high performance data storage and is available from all nodes. In particular, home directories and scratch directories are both provided by Lustre.
Home Directories
Upon logging in, users are placed in your global home directory by default. Home directories are limited in size and are meant for permanent storage of source code and other relatively small files. While home directories are available from the compute nodes, this file system is not configured for high performance.
Home directories should always be referred to by the environment variable $HOME. The absolute path to a home directory (e.g., /u4/joe/) may change, but the value of $HOME will always be correct.
Home directories are backed up as insurance against catastrophic file failures. Individual files or directories can not be restored, so users must back up important files to HPSS, to local storage, or elsewhere. Please back up all important files on a regular basis.
For security reasons, never allow "world write" access to the directories $HOME or $HOME/.ssh. NERSC scans for such security weakness, and, if detected, will change the permissions on the directories to disable "world write" access. Furthermore, SSH will not allow you to login if $HOME/.ssh is writeable by anyone other than you.
Home directories contain various shell initialization files ("dot-files", e.g. .login, .cshrc, .profile). These are symbolic links to NERSC-defined and controlled files that contain definitions needed by all users. Please do not not remove or modify these links. Customizations of login behavior can be specified in "extension" files with names such as .login.ext, .cshrc.ext, and .profile.ext.
Scratch Directories
Franklin is configured with two distinct scratch file systems named /scratch and /scratch2. Each user therefore has access to two scratch directories, which should always be referenced using the environment variables $SCRATCH and $SCRATCH2. Both file systems are available from all nodes, and are tuned for high performance. Users may run out of either scratch file system, but are encouraged to choose one or the other for their primary work.
The contents of $SCRATCH and $SCRATCH2 will be deleted ("purged") as needed as the system's disks approach capacity. In general, files in $SCRATCH and $SCRATCH2 will persist for at least seven days; users risk losing files in $SCRATCH or $SCRATCH2 that are older than that. NERSC purges files under $SCRATCH and $SCRATCH2 regularly; the interval may be adjusted as needed. As of early 2009, files last accessed more than 12 weeks prior were being purged. Please back up your important files frequently to HPSS.
There is a single (large) quota for each user that applies to the combined contents of $SCRATCH and $SCRATCH2. If your combined usage of $SCRATCH and $SCRATCH2 exceeds your quota, you will not be able to submit batch jobs until you reduce your combined usage. See Quotas for more information about monitoring your usage and quotas.
WARNING: Do not attempt to explicitly use a file system named /tmp. Your job may fail or be deleted if it writes to /tmp.Some software tools (editors, compilers, etc.) use the location specified by the $TMPDIR environment variable to store temporary files. Additionally, Fortran codes which open files with status="scratch" will write those files into $TMPDIR. On many Unix systems, $TMPDIR is set to /tmp. NERSC has set $TMPDIR to be $SCRATCH. Please do not redefine $TMPDIR!
Project Directories
The NERSC Global Filesystem (NGF) provides a large-capacity file storage resource that is shared among the major compute platforms. Storage in NGF is allocated to projects by request of project administrators.
Quotas
Note: this section is specifically about the scratch file systems. Quotas in NGF are handled differently than in these local file systems.
Users are alloted a quota in each file system. Quotas are in force for space and inode usage. Each file or directory counts as one inode.
| Machine | File system | Space quota | Inode quota |
|---|---|---|---|
| Franklin | global $HOME | 40GB | 500,000 |
| $SCRATCH and $SCRATCH2 combined | 750GB | 1,000,000 | |
| Bassi | $HOME | 5GB | 7,500 |
| $SCRATCH | 250GB | 50,000 | |
| Jacquard | $HOME | 5GB | 15,000 |
| $SCRATCH | 50GB | 50,000 |
NERSC sometimes grants temporary quota increases for legitimate purposes. To apply for such an increase, please see Disk Quota Change Request Form.
The myquota command (with no options) will display a user's current usage and quota. For example:
The output shows current usage and quotas for space and inodes. In this example, user "elvis" has a non-default SCRATCHTOT space quota of 1024 GB (1 terabyte), but has the default SCRATCHTOT inode quota. His combined usage in /scratch and /scratch2 exceeds his space quota there. This user will have to reduce his usage in $SCRATCH and/or $SCRATCH2 before he will be allowed to submit batch jobs on Franklin.
Franklin /scratch Usage
Disk Space Usage
April 1, 2009 to Date
March 2008 to March 2009
Inode Usage
April 1, 2009 to Date
March 2008 to March 2009
Franklin /u0 ($HOME) Usage
