The disparity among programming and execution models across HPC system architectures has inhibited the development and migration of third party applications and required substantial effort for ports as well as learning curves as users move between HPC systems. HPC systems must include a basic set of capabilities and tools with similar, if not identical, user interfaces. Systems of diverse components can be used as an ensemble for computing but critical infrastructure and resource management systems must be developed to make this practical.
To a large extent, new tools and software subsystems for HPC environments are developed as stand-alone products. Interoperability, especially with the diverse and rapidly changing HPC system software components, is rarely a near term requirement. Yet, the inability of a user's tool-sets to work together or the lack of means to integrate a new tool into an already established software context constrains the effectiveness, usability, and progress of advanced HPC software tools. Tools must, in general, be able to share common data, exchange products, and work together.
It is end-user applications of economic importance that will drive the demand and market for HPC systems, and provide a sustaining economic base for their future development. But the uncertainties involved in developing major codes for HPC systems impose too much risk to warrant investment by third party-software vendors. A base of economically important applications must emerge for HPC systems and, initially, these must be identified and sponsored through government support in cooperation with industry that would provide the initial sustaining customer base. The degree and rate of success will depend on the extent to which the base of applications can be broadened beyond the conventional science and engineering applications.
Because a major aspect of HPC system software is to simplify the programming interface to and manage the myriad resources of complex parallel systems, the difficulty in realizing effective system software is in large part due to the structure and low level logical interface which make up the architecture of such systems. Although processor architecture is driven by the workstation market, within that constraint processor/system architecture needs to evolve to be more supportive of scalable parallel processing applications and tools.
Symmetric Multi-Processors (SMP) are emerging as a major class of computing system, derived from and extending the range of capabilities of high end workstations. These are likely to be the first high volume parallel computers and the target for commercial third party parallel software. While not truly scalable beyond a small number of processors, these systems will be the dominant form of parallel architecture with sufficient market presence to justify industrial investment. The degree and likelihood of the future success of HPC systems will be significantly enhanced through their ability to leverage software investment for SMPs. This requires an almost seamless relationship between SMPs and HPC where end-user applications may migrate up and down the spectrum of parallel computer organizations from single user workstations to HPCs comprising thousands of processors.