IPM with I/O Tracing

• 1. Overview
• 2. Usage

1. Overview

The 'ipmMEGA' library is an in-development modification of the IPM library that adds the capability of tracing low-level POSIX I/O calls. In addition, a Perl script 'ipmio' parses the trace output to generate statistics and plots of I/O activity, including:

• a color-coded timeline that breaks down the types and durations of the POSIX I/O functions called by each task;
• histograms of read/write buffer sizes; and
• histograms of read/write bandwidths.

Sample output can be seen here.

2. Usage

On Franklin, load the module with:

franklin> module load ipmio

To link against the ipmMEGA library, add the environment variable $IPMIO to the *end* of your link call:

franklin> cc -o ... $IPMIO

When you run your executable, a directory called 'libctrace' will be created with a text file 'ioIPMXXXXX.txt' for each MPI task. These traces need to be tarred into a single file before they can be parsed by the Perl script:

franklin> tar -cf traces.tar libctrace/

The usage for the Perl script is:

franklin> ipmio -i input.tar -o outputDir [-e fileExt -x widthX]

where

• input.tar contains the ioIPMXXXXX.txt trace files
• outputDir destination for the HTML and image output
• fileExt file extensions to keep (other file handles, such as stdout and stderr, will be ignored)
• widthX the width in pixels of the x-axis of the timeline plot (the y-axis has pixels equal to the number of tasks