NERSCPowering Scientific Discovery Since 1974

Core Specialization

Core Specialization (CS) is a feature of the Cray operating system that allows the user to reserve one or more cores per node for handling system services, and thus reduce the effects of timing jitter due to interruptions from the operating system at the expense of (possibly) requiring more nodes to run an application. The specialized cores may also be used in conjunction with Cray's MPI asynchronous progress engine [1] to improve the overlap of communication and computation for applications that use non-blocking MPI functions. In the absence of CS, the compute cores must service their own non-blocking calls.  The extent to which this can provide benefit is very application dependent.

Hyper-Threading complicates questions about the most effective use of processor resources. HT doubles the number of compute stream (i.e. processes or threads) that can schedule computation, but does not increase other resources such as floating-point units or cache. A key question for users is whether the additional 'logical cores' are best used by allocating additional MPI processes to the nodes, by using multiple threads per MPI task, or by devoting some to the operating system or MPI progress engine via CS. 

Use of the CS and HT features is determined at runtime by options provided to the aprun command. 

References:

  1. H. Pritchard, D. Roweth, D. Henseler, and P. Cassella, "Leveraging the Cray Linux Environment Core Specialization Feature to Realize MPI Asynchronous Progress on Cray XE Systems" Proc. Cray User Group, 2012.