NERSCPowering Scientific Discovery Since 1974

File storage and I/O

Disk Quota Change Request Form

Franklin File Systems

The Franklin system has 4 different file systems mounted which provide different levels of disk storage, I/O performance and file permanence.  The table below describes the various Franklin file systems

File SystemHomeLocal ScratchProject
Environment
Variable
Definition
$HOME $SCRATCH
$SCRATCH2
No environment variable
/project/projectdirs/
Description
  • Global homes file system shared with other NERSC systems.
  • All NERSC machines mount the same home directory.
  • GPFS filesystem.
  • Where users land when they log into the system.
  • Two 200TB Lustre local file systems. 
  • Local means the files can not be viewed on other NERSC systems.
GPFS global file system mounted on all NERSC systems.
Default Quota
  • 40 GB
  • 1 million inodes
  • 750 GB
  • 1 million inodes
  • combined over $SCRATCH and
    $SCRATCH2
  • 1 TB
  • 1 million inodes
Intended Purpose
  • Shell initializations
  • Storing source code
  • Compiling codes
  • Not intended for IO intensive applications
  • Running production applications
  • I/O intensive jobs
  • Temorary storage of large files.
  • Running production applications
  • Groups needing shared data access
  • Projects running on multiple NERSC machines
Peak Performance Low, ~100 MB/sec 17 GB/sec for each 40 GB/sec
Purged? No Yes, files older than 12 weeks are purged. No

Scratch Directories

Franklin is configured with two distinct scratch file systems named /scratch and /scratch2. Each user therefore has access to two scratch directories, which should always be referenced using the environment variables $SCRATCH and $SCRATCH2. Both file systems are available from all nodes, and are tuned for high performance. See Optinizing I/O Performance on Lustre File Systems. Users may run out of either scratch file system, but are encouraged to choose one or the other for their primary work.

The contents of $SCRATCH and $SCRATCH2 are deleted ("purged") if the files are older (defined by last access time) than 12 weeks.  Please back up your important files frequently to HPSS.

There is a single (large) quota (space and inode) for each user that applies to the combined contents of $SCRATCH and $SCRATCH2. If your combined usage of $SCRATCH and $SCRATCH2 exceeds your quota, you will not be able to submit batch jobs until you reduce your combined usage.

The "myquota" command (with no options) will display a user's current usage and quota.  NERSC sometimes grants temporary quota increases for legitimate purposes. To apply for such an increase, please see Disk Quota Increase Form.

Do Not Use /tmp Explicitly

WARNING: Do not attempt to explicitly use a file system named /tmp. Your job may fail or be deleted if it writes to /tmp.  Some software tools (editors, compilers, etc.) use the location specified by the $TMPDIR environment variable to store temporary files. Additionally, Fortran codes which open files with status="scratch" will write those files into $TMPDIR. On many Unix systems, $TMPDIR is set to /tmp. NERSC has set $TMPDIR to be $SCRATCH. Please do not redefine $TMPDIR!