Scientific impacts from just three years of Blue Waters

Highlights from an investigation and evaluation of the scientific results from the NCSA Blue Waters supercomputer system

“Build it and they will come” is one way to approach building a supercomputer, but it’s not what the National Center for Supercomputing Applications (NCSA) did with Blue Waters, the largest leadership-class National Science Foundation supercomputer. Prior to the system going online in April 2013, Blue Waters staff worked with more than 20 science teams to determine a unique, balanced hardware configuration—a process now known as co-design.

Three years later, a sample of 31 science teams that have used Blue Waters were surveyed and interviewed as part of a report meant to judge the effectiveness and productivity of this unique system—housed at NCSA’s home institution, the University of Illinois at Urbana-Champaign.

Using information gathered in the surveys, the report’s authors at International Data Corporation’s HPC division (now known as Hyperion Research) ranked the impact of each team’s findings into an “innovation index”—using a methodology they developed to analyze the effectiveness of 700-plus scientific projects, including international HPC projects. The Hyperion Research analysts noted in the report that “NCSA did an unusually thorough job of preparing [science teams] for Blue Waters.”

“These findings confirm that Blue Waters has proven to be an exceptionally—and in some cases uniquely—competent platform for accelerating scientific innovation,” the Hyperion authors wrote in the report. “The Blue Waters-enabled innovations described and ranked in this study will produce strong benefits for the scientific disciplines they belong to.”

The report’s release coincides with a comprehensive workload study of all the scientific applications run on the supercomputer since its start in 2013. That study, written about extensively here, found most of the applications run on the system have been extreme-scale. It also found there were a number of problems that couldn’t be run anywhere else, and confirms the balance of hardware the Blue Waters team chose—from high network bandwidth, to a high node count with large memory—is being used effectively.

The Hyperion report reinforces those points from the workload study by showing how Blue Waters projects rank with “Innovation Impact.” Hyperion has a database of similar surveys run for the Department of Energy and several governments in Europe and Asia, which gives them an approximate average to compare to. Notably, Hyperion found far fewer Blue Waters science innovations fell into the category of “least important” than average, and more Blue Waters projects were evaluated in the top three categories for impact.

“The innovations produced so far by Blue Waters users are crucially important as a group because they constitute substantial steps forward in major disciplines,” the authors wrote in the report.

“Not once during the 31 interviews [Hyperion] conducted with Blue Waters users for this study did any of them point to shortcomings of the supercomputer,” the authors added. “On the contrary, all of the researchers had praise for the system and enthusiastically reported on the progress it has enabled for their work.”

Outside of the science Blue Waters has enabled, it has also re-established the importance of a balanced system—a supercomputer that can effectively do a complete range of simulation problems, as well as data intensive problems at scale. Doing so required that the Blue Waters team not focus entirely on increasing the “peak calculations per second” metric used in the high-profile TOP500 list (which Blue Waters refused to participate in), instead disbursing project funds to invest in other vital components of the machine, like network bandwidth and memory.

“[Hyperion] applauds NCSA’s bold decision not to optimize Blue Waters for superior performance on the narrow benchmark test used to determine rankings on the semi-annual TOP500 supercomputers list. If more leadership-class HPC sites resisted this political temptation, vendors would be more motivated to design HPC systems that are applicable to a broader range of user needs,” the authors wrote.

Follow this link to download the report and learn more about its findings and methodologies. This includes the Scientific Breakthroughs section, which has direct quotes from researchers. Other sections focus on cost savings, preparing to support better future projects, as well as sections that call out specific projects that have helped society, or discovered something new.

Disclaimer: Due to changes in website systems, we've adjusted archived content to fit the present-day site and the articles will not appear in their original published format. Formatting, header information, photographs and other illustrations are not available in archived articles.