CalTech Researcher Uses Blue Waters to Model Galactic Atmospheres

02.11.19 -

Boswell Hutson (NCSA)

Since the beginning of astronomy, scientists have historically spent countless hours, and more recently countless compute cycles, to understand the formation of our galaxy or galaxies in general. This research has advanced to the point where scientists can reasonably estimate how galaxies begin, but some mysteries remain.

Before present-day, researchers were limited in the scope of what they were able to observe. Simulations and models could realistically decipher how the center of a galaxy formed, but simulations could simply not account for the interactions that happened outside of a galaxy’s core, and as such, were missing vital data and insight about how gasses on the outskirts of galaxies behave during formation.

Enter Cameron Hummels, an astrophysicist at the California Institute of Technology (CalTech) and an NSF Astronomy and Astrophysics Postdoctoral Fellow, who is part of a team working to address these simulation challenges. Also included in the team is Blue Waters researcher Brian O’Shea and past Blue Waters Graduate Fellow Iryna Butsky.

“What we’re looking at is a projection of what is termed neutral Hydrogen, a cool, electrically neutral gas that exists all around the universe,” said Hummels. “For the purposes of this technique, we’re looking at a common observational constraint that we have: how much of this cool HI (pronounced H one) gas is prevalent in the region around the galaxy?”

“When you under resolve certain structures, as you can see [in the left-side simulation below], it does some un-physical things and tends to wipe out these cold gas structures that are being probed by the neutral Hydrogen.”

By employing their Enhanced Halo Resolution (EHR) modeling technique on Blue Waters, one of the most powerful supercomputers in the world, Hummels and his research team are able to, more accurately than ever before, account for cool hydrogen gas (HI) that is spewed into galactic outskirts in vast quantities following galaxy formation.

In order to visualize this phenomenon and the leaps in advances made by EHR, Hummels was tasked with creating video clips of simulations that illustrate cool HI gas in higher resolutions, allowing it to be viewed by the naked eye. Take a look at the videos below, the EHR technique is displayed on the right:

This technique is quite novel; it is the first time simulations have been carried out to accurately depict what happens to cool hydrogen gas during galaxy formation. In turn, the results provide a more accurate overall picture to corroborate observations. This past November, the work was submitted to Astrophysical Journal, with a publication date expected in the near future.

These simulations and the subsequent paper would not have been possible, however, without the use of a massively-parallel leadership supercomputer, which is where Blue Waters system at NCSA comes into play.

“I’ve been in computational astrophysics for about ten years now, and have used a number of machines including NCSA’s Abe and the NSF’s XSEDE program,” said Hummels. But when it came to this new form of EHR modeling, Blue Waters was the perfect fit for the job.

“Blue Waters has been great,” Hummels said. “Right now, things just work. Blue Waters has taken out the difficulty of software wrangling, and that’s been extremely beneficial for both me as an individual and our entire team.”

“Many other allocated computational resources are so over-subscribed, trying to get a job submitted and through the queue is extremely challenging, especially during deadline times,” Hummels continued.

“Due to the way the Blue Waters system is set up, with the number of people that can apply being limited and relatively large allocations being doled out, we haven’t had nearly as many issues as we see on other systems, meaning research is run on a much faster time-scale. Relative to other national resources, we can get simulations done on an order of two to three times sooner on Blue Waters.”

These movies and this publication are merely the tip of the iceberg for this research, however. Many members of Hummels’ team are working to publish their own insights gained from these EHR simulations, which will hopefully pave the way for an even deeper understanding of galactic formation for both researchers and the general public.

“Our allocation on Blue Waters is now officially complete, but we are continuing to work on analyses on these data sets,” said Hummels. “This is the first of several different papers, including from my colleagues who share this PRAC allocation. This was all enabled by the presence of Blue Waters and I, for one, am very appreciative.”


The Blue Waters petascale supercomputer is one of the most powerful supercomputers in the world, and is the fastest sustained supercomputer on a university campus. Blue Waters uses hundreds of thousands of computational cores to achieve peak performance of more than 13 quadrillion calculations per second. Blue Waters has more memory and faster data storage than any other open system in the world. Scientists and engineers across the country use the computing and data power of Blue Waters to tackle a wide range of challenges. Recent advances that were not possible without these resources include computationally designing the first set of antibody prototypes to detect the Ebola virus, simulating the HIV capsid, visualizing the formation of the first galaxies and exploding stars, and understanding how the layout of a city can impact supercell thunderstorms.


The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign provides supercomputing and advanced digital resources for the nation's science enterprise. At NCSA, University of Illinois faculty, staff, students, and collaborators from around the globe use advanced digital resources to address research grand challenges for the benefit of science and society. NCSA has been advancing one third of the Fortune 50® for more than 30 years by bringing industry, researchers, and students together to solve grand challenges at rapid speed and scale.

National Science Foundation

Blue Waters is supported by the National Science Foundation through awards ACI-0725070 and ACI-1238993.