A petascale sized effort

Over the past year, NCSA and IBM have made incredible progress toward the realization of the Blue Waters system. You have read about many of those achievements in the last four issues of Access as well as other Blue Waters project information, such as the completion of the National Petascale Computing Facility (NPCF), the progress of the 18 science teams, our workshops, virtual school sessions, and webinars. So, in this issue, I want to point out some new information that has only recently become available.

First, there is an increasing amount of information on the POWER7 processors and systems. IBM is now shipping many versions of the POWER7 systems. NCSA actually acquired a p780 as part of our interim systems environment. Since May, science team members and NCSA staff have been using the POWER7 “Bluedrop” system to port and re-engineer their applications in preparation for the arrival of the production Blue Waters equipment. The results are very positive, and early access is giving the science teams and IBM a firm basis for performance estimations and optimizations.

To expand the insight gained using Bluedrop, NCSA staffers are testing early hardware that has both the POWER7 processors and the hub chips in operation. The equipment is similar to the POWER7 IH drawer IBM demonstrated as SC09, with 256 cores, eight operating system images, and the full interconnect. While still in engineering prototype form, we have been able to run all the key benchmarks and tests on this early equipment as well as some of the science applications from the teams with Petascale Computing Resource Allocations (PRAC) from the National Science Foundation.

The early access and testing is mutually beneficial to NCSA and IBM. Both staff get to see the actual performance on the first integrated hardware. We have been able to explore the interconnect performance to a modest scale and to start to use some of the advanced hardware features that will be in Blue Waters. The other advantage of the earliest possible access is it provides ways to uncover problems early enough to allow them to be corrected by the time production hardware is built. IBM staff worked with NCSA application and system specialists to uncover subtle interactions that could inhibit performance and to start low-level tuning of the hardware parameters based on real science applications. The software environment is also improved based on this real experience. This work will continue up through the Blue Waters deployment in the NPCF, with more science codes being run. We are particularly focusing on the problems, challenges, and solutions for petascale storage and I/O by expanding the detailed design, quality expectations, and performance expectations for the storage subsystem.

One of the most exciting developments for people interested in Blue Waters occurred in late August when IBM released many of the details of the Blue Waters interconnect at the Hot Chips 22 conference and the Hot Interconnects conference. With these presentations and papers, one coauthored by NCSA’s Torsten Hoefler, we now can discuss publicly most of the interconnect characteristics that will be important to moving science applications to the system in a highly effective manner. The details of this powerful, new interconnect are now summarized on our public Blue Waters website.

For the near future, we continue working with the science teams, including some newly announced teams this fall. The teams, and others who will take advantage of Blue Waters, are more than what many have traditionally called “users.” They are equal partners in this enterprise. The Blue Waters team builds the tool and helps computational scientists and engineers use it effectively so scientific partners drive discovery through the research they conduct on Blue Waters. The approach is to create an expert-to-expert, mutually beneficial relationship between the Blue Waters project team and the science, engineering, and educational teams with Blue Waters allocations. This results in active partnerships with all the science teams and ultimately enhances their ability to use Blue Waters to further their goals.

This fall we hold two major events for our science team partners, a four-day workshop that helps partners tune and re-engineer their codes, and another at SC10 that will be our first Blue Waters partner meeting. Of course, we will continue to engage with IBM on bringing to realization the full promise that is the Blue Waters system in 2011, a petascale-sized effort in its own right.

William Kramer
Deputy Project Director, Blue Waters
NCSA

Disclaimer: Due to changes in website systems, we've adjusted archived content to fit the present-day site and the articles will not appear in their original published format. Formatting, header information, photographs and other illustrations are not available in archived articles.