Application Interconnect Requirements

As leading supercomputing systems move towards designs employing tens of thousands of processors, there has been renewed interest in finding a network topology with cost that grows linearly with system scale, such as torus and mesh networks. However, torus and mesh networks create significant restrictions on application communication performance and job scheduling in comparison to fully-connected interconnection networks, such as fat-tree and crossbar topologies. We investigate alternative interconnect topologies and technology that meets the communication requirements of DOE Office of Science HPC applications in a cost-effective and scalable manner.

In order to profile the communication characteristics of scientific applications for our study, we employ the Integrated Performance Monitoring (IPM) tool -- an application profiling layer that allows us to non-invasively gather the communication characteristics of these codes as they are run in a production environment. IPM brings together multiple sources of performance metrics into a single profile that characterizes the overall performance and resource usage of the application. It maintains low overhead by using a unique hashing approach which allows a fixed memory footprint and minimal CPU usage. IPM is open source, relies on portable software technologies and is scalable to thousands of tasks.

Related Information

• Hybrid Flexibly Assignable Switch Topology (HFAST)(pdf)
• Raw IPM Profiles of Key NERSC Applications(dir)