Building toward Blue Waters

04.15.09 -

NCSA has several systems that are being used as stepping stones for the sustained petaflop system that will come online in 2011.

The Blue Waters project faces an interesting challenge: The IBM POWER7 hardware on which that sustained-petaflop supercomputer will be based doesn't exist yet. Because there is much application development work and other research to be done in preparation for Blue Waters and other future systems, NCSA has installed four systems to help with these critical tasks. These systems are available to project team members and collaborators who are working on application enhancement, system software development, and education projects for Blue Waters.

An IBM POWER575+ cluster nicknamed BluePrint serves as a testbed where NCSA can validate the performance of compilers, the development environment, monitoring and management tools, and other elements of the software environment that IBM will provide for Blue Waters. BluePrint is composed of 120 POWER5+ nodes, each having 16 cores and 64 GB of memory. The AIX operating system is installed, with Linux to be available for specific projects.

Two IBM POWER6 systems are an important part of the development of the Blue Waters archival storage environment and are also being used as a platform for application development. In order for scientists and engineers to be able to productively use Blue Waters from day one, those researchers are working closely with computing experts to port, optimize, and scale applications to effectively use the system's more than 200,000 processors. These Petascale Application Collaboration Teams (PACTs) are using the POWER6 platform for development and testing.

Another important tool for the PACTs is the IBM system simulator environment (called Mambo). This software, which runs on an x86-compatible system, allows the Blue Waters team to simulate the performance of the POWER7 processor. This sneak peak at the Blue Waters' processor is essential to understanding how codes must be adjusted to use the architecture most effectively.

"You can see clock tick by clock tick what will happen inside the processor," says NCSA's Mike Showerman.

Details of the POWER7 architecture and the simulated performance are still confidential at this time.

NCSA's Innovative Systems Laboratory (ISL) also has an accelerator cluster with 16 nodes, each featuring two dual-core 2.4 GHz AMD Opterons, four NVIDIA Quadro 5600 GPUs, and one Nallatech H101-PCIX FPGA accelerator. This cluster is used for collaborative projects in which ISL staff work with scientists and engineers to adapt their applications to run more quickly on these specialized architectures, which may be a key feature of future petascale systems.

The cluster is also used for classes offered at the University of Illinois, for workshops, and for summer schools offered by the Virtual School of Computational Science and Engineering, which helps prepare the current and next generation of scientists and engineers to use leading-edge computer systems.

Work is under way to double the number of compute nodes on this development cluster.

National Science Foundation

Blue Waters is supported by the National Science Foundation through awards ACI-0725070 and ACI-1238993.