PAPI at NERSC

Introduction

The Performance API (PAPI) specifies a standard application programming interface (API) for accessing hardware performance counters available on most modern microprocessors. PAPI was developed at the Innovative Computing Laboratory at the University of Tennessee.

PAPI provides two interfaces to the underlying counter hardware; a simple, high level interface for the acquisition of simple measurements and a fully programmable, low level interface directed towards users with more sophisticated needs.

PAPI provides portability across different platforms. It uses the same routines with similar argument lists to control and access the counters for every architecture. As part of PAPI, there is a predefined set of events that represents the lowest common denominator of most counter implementations. The intent is that the same source code will count similar and possibly comparable events when run on different platforms.

Using PAPI

To use PAPI to examine the performance of your program, you must insert calls to one or more PAPI routines into your code and compile with the PAPI library. The full functionality of PAPI is only available to C programs, although many routines are callable from Fortran starting with version 1.2.

The PAPI library is made available through the module command. Use the following to compile and link a program.

Franklin and Hopper

% module load xt-papi

% cc -c a.c $PAPI_INCLUDE_OPTS
% cc -o a.out a.o b.o ... $PAPI_POST_LINK_OPTS

% ftn -c a.f $PAPI_INCLUDE_OPTS
% ftn -o a.out a.o b.o ... $PAPI_POST_LINK_OPTS

Carver

% module load papi

% mpicc -c a.c -I$PAPI_INCLUDE
% mpicc -o a.out a.o b.o ... -L$PAPI_LIB -lpapi

% mpif90 -c a.f -I$PAPI_INCLUDE
% mpif90 -o a.out a.o b.o ... -L$PAPI_LIB -lpapi

Getting Started

One of the high-level PAPI routines is PAPI_flops. This routine counts flips (floating point instructions), real and processor time, and shows Mflips/s. The first call to PAPI_flops will initialize PAPI, setup the counters to monitor the PAPI_FP_INS and PAPI_TOT_CYC events and start the counters. Subsequent calls will read the counters and return total real time, total process time, total floating point instructions since the start of the measurement and Mflips/s rate since last call to PAPI_flops().

The calling sequence for PAPI_flops is:

#include <papi.h> 

int PAPI_flops (float *rtime, float *ptime, 
			long long *flpins, float *mflips);

for C programs, and:

include "f90papi.h"

integer iret
real (kind=4) rtime, ptime, mflips
integer (kind=8) flpins

call PAPIF_flops(rtime, ptime, flpins, mflips, irc)

for Fortran programs.

There are two other PAPI header files for Fortran:

• fpapi.h - this interface relies on a preprocessing step that IBM's xlf Fortran compiler will perform using filenames with suffix .F or .F90. With this file, you must use the '#include' syntax.
• f77papi.h - this file is formatted for Fortran 77 fixed format source code.

To obtain performance data for your program, insert a call to PAPI_flops at the beginning and end of the main program, and print the values produced by the second call.

PAPI Documentation

The PAPI Home Page has links to a number of presentations on PAPI, and a set of PAPI Manual Reference Pages.

In addition, man pages are available. See man PAPI or man PAPIF for an introduction.