NERSCPowering Scientific Discovery Since 1974

CNL Programming Considerations on Franklin

Shared Libraries (not supported)

The Cray XT series currently do not support dynamic loading of executable code or shared libraries. Also, the related LD_PRELOAD environment variable is not supported. It is recommened to run Shared Library applications on Hopper.

GNU C Runtime Library glibc Functions

The light weight OS on the compute nodes, Compute Node Linux (CNL), is designed to optimize application performance by reducing system overhead. In order to achieve this goal, only a subset of the GNU C runtime library, glibc, is implemented:

  • CNL provides limited support of the process control functions such as popen(), fork() and exec(); the resulting processes execute in the limited RAM disk environment on each compute node.
  • CNL supports cpuinfo and meminfo /proc files that contain information about your login node.
  • CNL does not support getgrgid(), getgrnam(), getpwnam(), and getpwuid() functions.
  • CNL does not support custom functions that require a daemon.
  • Please see Programming Environment User's Guide, Appendix A, for a complete list of the glibc functions that CNL supports.

Compute Node Page Size

Franklin runs CNL OS on compute nodes and only supports a single (small) page size of 4 KB.

I/O Support in the CNL Environment

The I/O functions allowed in CNL applications are Fortran, C/C++ I/O calls; Cray MPICH2 and Cray Shmem I/O functions; and the underlying Linux Lustre client I/O functions.

stdin, stdout, and stderr are handled by the aprun utility. Only processing element 0 (PE 0) can read stdin; all PEs can wri
te to stdout and stderr.

Timing Support in the CNL Environment

dclock() and etime() are not supported on the compute nodes. It is recommended to use MPI_Wtime(), or a Fortran intrinsic timing routine such as cpu_time() for time. The elapsed time (in seconds) could be obtained by the difference between two calls of the same function calls.

Turning Off Fortran Stop Message

The "stop" statement at the end of a Fortran program would cause every process that participates the job (every MPI tasks in a parallel job) to write to standard out a "FORTRAN STOP" message. This is not very scalable, would affect application performance, and could cause potential file system with a large concurrency job. To turn off this FORTRAN STOP message, you could add in your batch job script:

   setenv NO_STOP_MESSAGE   (for csh or tcsh)
   export NO_STOP_MESSAGE   (for bash or ksh)

Little-endian Support

Franklin is a little-endian machine. Being little-endian means the lower-order byte of a number is stored at the lower add
ress in memory while the higher-order byte stored in higher address. You could use the PGI fortran compiler option "-Mbyte
swapio" on Franklin to read big endian binary files from machines that have big-endian support. There is no C/C++ option f
or converting between big endian and little endian binary files.