NCSA releases 2017 Blue Waters Project Annual Report detailing innovative research and scientific breakthroughs

11.09.17 -

The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign today released the 2017 Blue Waters Project Annual Report. For the project's fourth annual report, research teams were invited to present highlights from their research that leveraged Blue Waters, the National Science Foundation's (NSF) most powerful system for sustained computation and data analysis. Spanning economics to engineering, geoscience to space science, Blue Waters has accelerated research and impact across an enormous range of science and engineering disciplines throughout its more than 4-year history covered by the report series. This year is no different.

"To date, the NSF Blue Waters Project has provided over 20 billion core-hour equivalents to science, engineering and research projects, supported not just by the NSF, but also by NIH, NASA, DOE, NOAA, and other funders. Without Blue Waters, these funded investigations might not even be possible," said Blue Waters Director and Principal Investigator, Dr. William "Bill" Kramer. "In this year's report, we are using a 'badge' to show the projects that are Data-Intensive (39 projects), GPU-Accelerated (34), Large Scale Greater than 1,000 nodes (65), Memory-Intensive (18), Only on Blue Waters (27), Multi-Physics/Multi-Scale (47), Machine Learning (9), Communication-Intensive (32) and Industry (5). This shows the breadth and depth of the uses world-class science is making on Blue Waters."

"I continue to be amazed by the vast range of creative, limit-pushing research that scientists submit to this publication year after year. With the support of the National Science Foundation and the University of Illinois, NCSA's Blue Waters Project continues to empower scientists to make discoveries that have immense impact in a diverse range of fields, spark new understanding of our world, and open new avenues for future research," said Dr. William "Bill" Gropp, Director of NCSA.

The annual report also highlights the Blue Waters Project's strong focus on education and outreach. Blue Waters provides the equivalent of 60 million core-hours of the system's computational capacity each year for educational projects, including seminars, courseware development, courses, workshops, institutes, internships, and fellowships. To date, there have been more than 200 approved education, outreach, and training projects from organizations across the country. These allocations have directly benefitted over 3,700 individuals in learning about different aspects of computational and data-enabled science and engineering at more than 160 institutions, including 41 institutions in EPSCoR jurisdictions and at 14 Minority Serving Institutions.

The Blue Waters Annual Report highlights how the project is helping other domain specialists reach petascale sustained performance, specifically through their recently-expanded Petascale Application Improvement Discovery (PAID) program, where the Project provided millions of dollars to science teams and computational and data experts to improve the performance of applications.

Gropp continued, "Even more remarkable breakthroughs will be forthcoming as NCSA continues to partner with scientists around the nation to change the world as we know it."

Distribution of annual actual usage by discipline, June 2016-May 2017.

This year's annual report features 130 research abstracts from various allocation types, categorized by space science, computer science, geoscience, physics and engineering, biology, chemistry, and health, and social science, economics, and humanities. One project from each section of the Blue Waters report is featured below to show the span of research possibilities enabled by Blue Waters. Click on the category headlines to visit that area of the report or click here to download the full report.

Biology, Chemistry, and Health

Gregory A. Voth — "Large-Scale Coarse-Grained Molecular Simulations of the Viral Lifecycle of HIV-1"
Allocation: NSF PRAC/1,200,000 node hours (38,400,000 core hour equivalents)

A key step in the lifecycle of human immunodeficiency virus type-1 (HIV-1) is the production of enveloped particles containing viral proteins and genomes from infected cells. However, the molecular details of this process have remained elusive.

Using large-scale coarse-grained molecular simulations, enabled by Blue Waters, Voth and his team were able to investigate a network of critical interactions that regulate the early stages of HIV-1 assembly, packaging, and budding. Thanks to Blue Waters' large amounts of computational power and superior network performance, this team revealed several new insights about the interactions that regulate HIV-1 viral assembly dynamics, which could be used in the future to help design a new therapeutic approach to viral infection.

Space Science

Eric J. Lentz — "Exploring the Nature of Exploding Massive Stars with High Resolution"
Allocation: NSF PRAC/6,000,000 node hours (192,000,000 core hour equivalents)

Core-collapse supernovae (CCSNe) are violent explosions of massive stars at the end of their lives. CSSNe play an important role in the dynamics of galaxies by injecting energy, producing and expelling heavy elements (which subsequently contributes to the elemental composition of planets, such as Earth), and triggering formation of new star systems.

With the help of Blue Waters, Lentz's team is using 3D simulations to explore the variation in explosions and ejecta that result from the known variations in properties of massive stars (initial mass, composition, rotation, etc.) to better understand the impact of resource limitations on those outcomes. Because CCSNe simulations are large, lengthy, and expensive to conduct, Blue Waters was the only practical choice to conduct these. Even a single 3D simulation can overwhelm the available allocations for a single project at other large sites, but with Blue Waters, Lentz's team can perform about three simulations per year.

Physics and Engineering

Paolo Gardoni — "3D Probabilistic Physics-Based Seismic Hazard Maps for Regional Risk Analysis"
Allocation: Illinois Exploratory/50,000 node hours (1,600,000 core hour equivalents)

Earthquakes have the potential to wreak enormous amounts of damage and economic loss, especially in highly-populated areas. Understanding the possible risk in advance of a crisis can help communities prepare for and manage the earthquake crisis better, which is exactly what Gardoni and his team have set out to accomplish on Blue Waters. With this year's exploratory allocation, Gardoni and his team were able to set up SPEED software (an existing, open-source high-performance software package) on Blue Waters and evaluate/tune the SPEED code, priming them to develop 3D probabilistic physics-based hazard maps to address seismic risk in the future on Blue Waters. This will contribute to a comprehensive understanding of earthquake physics and effects. Blue Waters was the best choice for this research, as the ability to run hundreds of simulations is essential in future work to generate the predictions needed for 3D probabilistic physics-based hazard maps.

Computer Science and Engineering

Tandy Warnow — "Parallel Algorithms for Big Data Phylogenomics, Proteomics, and Metagenomics"
Allocation: Illinois/125,000 node hours (4,000,000 core hour equivalents)

For computational molecular biology, massive data sets are necessary to accurately investigate the complexities of life. In three areas in particular, these data sets face particularly strenuous computational challenges: phylogeny estimation, protein structure and function prediction, and environmental sample analysis. Tandy Warnow's work, in turn, seeks to ease these constraints, and did so by using Blue Waters to establish four new biological methods and evaluations: SDVquest (an efficient method for species tree estimation), HIPPI (a protein classification method), an evaluation of screening genes in phylogenomic analyses, and an evaluation of protein benchmark data sets. Just this year, five papers based on this work performed on Blue Waters have been published, with another two in the submission process.


Jamesina J. Simpson — "Location-Specific Space Weather Hazards to Electric Power Grids Calculated on a Global Scale"
Allocation: NSF PRAC/5,000,000 node hours (160,00,000 core hour equivalents)

In 1859, a geomagnetic solar storm collided into Earth's magnetosphere, causing an electric shock that left telegraph systems across the globe in shambles and telegraph operators injured. Due to the sheer unpredictability of this event and the historical record surrounding it, it is reasonable to expect another cosmic weather event will affect Earth in the future.

Jamesina Simpson, a researcher at the University of Utah, utilized NCSA's Blue Waters supercomputer to study the effect of a cosmic weather event on a modern Earth, which depends much more on electrotechnology than ever before. To determine this potential effect, Simpson used Maxwell's equations models of the Earth-ionosphere waveguide to calculate the hazard that a cosmic weather event could have on specific local power grids and provide a reasonable expectation of how well these systems will work in the event of disturbed conditions. This, in turn, will allow power grids to combat large-scale blackouts in the event of a space weather event.

Blue Waters, which provided more than 100,000 processors for this study, was used primarily to increase modeling resolution from 40 x 40 x 5 km to 1 x 1 x 1 km, which allowed researchers to study electric field behavior on ocean-continent boundaries and to develop models of electromagnetic waves as they travel through uncertain regions of Earth's ionosphere.

Given Simpson's research, certain localities will now be able to combat massive cosmic weather events that could potentially have a catastrophic effect on Earth's entire electronic infrastructure.

Social Science, Economics and Humanities

Yongyang Cai — "Policy Responses to Climate Change in a Dynamic Stochastic Economy"
Allocation: Great Lakes Consortium for Petascale Computation (GLCPC)/250,000 node hours (8,000,000 core hour equivalents)

To analyze the impact and effectiveness of climate change response policy, researchers previously used simple Integrated Assessment Models (IAMs) of climate and the economy.

Yongyang Cai and his team developed an improved computational framework, called the Dynamic Stochastic Integration of Climate and the Economy (DSICE), which takes into account more complex factors than ever before, such as the heterogeneity of the economy in order to better analyze the complex relationship between climate change and policy responses. This year, the team further developed the DSICE framework to incorporate several new tipping points, including spatial structure of temperature and economy, sea level rise, thawing of permafrost, partial competition and collaboration, and adaptation.

However, this improved framework also requires much larger amounts of computational resources than previous IAMs. The parallel algorithms used to solve these intensive economic problems require optimal communication between nodes and the ability to solve large quantities of problems in a short time, making Blue Waters the optimal choice to conduct this research.

Graduate Fellows

Larissa Reames — "Simulated Effects of Urban Environments on the Dynamics of a Supercell Thunderstorm"
Allocation: Blue Waters Fellow/200,000 node hours (64,000,000 core hour equivalents)

According to the United Nations, the Earth's population is increasingly concentrated in urban areas, with nearly two-thirds of the world's population expected to live in urban areas by 2050. As the number of people within cities grows, it is becoming more important to understand and to correctly predict the interactions between urban environments and the atmosphere, and the types of weather events that urban areas will be more prone to.

Blue Waters Fellow Larissa Reames did just that with her time on Blue Waters, identifying surface roughness as one of the most important factors when predicting how an urban area (in this case, a Great Plains urban area) will interact with supercell storms, as opposed to thermodynamics, which has been the subject of previous study. Reames' allocation has since been extended, allowing for continued use of Blue Waters to study the effects of urban environments and their surface roughness on supercell storms.

About the National Center for Supercomputing Applications

The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign provides supercomputing and advanced digital resources for the nation's science enterprise. At NCSA, University of Illinois faculty, staff, students, and collaborators from around the globe use advanced digital resources to address research grand challenges for the benefit of science and society. NCSA has been advancing one third of the Fortune 50® for more than 30 years by bringing industry, researchers, and students together to solve grand challenges at rapid speed and scale.

About NCSA's Blue Waters Project

The Blue Waters petascale supercomputer is one of the most powerful supercomputers in the world, and is the fastest supercomputer on a university campus. Blue Waters uses hundreds of thousands of computational cores to achieve peak performance of more than 13 quadrillion calculations per second. Blue Waters has more memory and faster data storage than any other open system in the world. Scientists and engineers across the country use the computing and data power of Blue Waters to tackle a wide range of challenges. Recent advances that were not possible without these resources include computationally designing the first set of antibody prototypes to detect the Ebola virus, simulating the HIV capsid, visualizing the formation of the first galaxies and exploding stars, and understanding how the layout of a city can impact supercell thunderstorms.

Media contact

Kristin Williamson
Senior Assistant Director, NCSA
University of Illinois at Urbana-Champaign
Public Affairs | Marketing | Facilities
Office: 217-300-2933
Cell: 217-343-1594

National Science Foundation

Blue Waters is supported by the National Science Foundation through awards ACI-0725070 and ACI-1238993.