Tropical cyclones are known by many names – hurricanes in the Atlantic and Northeast Pacific, and typhoons in the Northwest Pacific. They are some of the most powerful weather events on the planet. When a cyclone spins toward land, millions upon millions of people can be impacted. Predicting them takes an enormous amount of math when all the variables are taken into account. Fluid dynamics calculations play an essential role in predicting what cyclones will do, and these are exactly the types of simulations that GPUs excel at producing. Researchers from the Naval Postgraduate School have proven that their model, which uses GPUs, is five times faster than the CPU model by using NCSA’s GPU-based Delta supercomputer.
Soonpil Kang, currently a postdoctoral researcher at Lawrence Livermore National Laboratory (LLNL), was a postdoctoral researcher at the Naval Postgraduate School (NPS) working with Frank Giraldo, a distinguished professor in the Department of Applied Mathematics at NPS, when this research was conducted. Kang’s team wanted to port a CPU-based simulation, a non-hydrostatic atmospheric model called xNUMA, to a GPU-based simulation using the OpenACC directive-based programming model.
“We focused on how to efficiently simulate a large-scale, realistic tropical cyclone by leveraging GPU capabilities,” said Kang. “Efficiency here means that the algorithm is constructed on GPUs and that the solution is faster than for a CPU-only simulation.”
Sometimes, creating an accurate test requires comparing the simulation results to a real-world event. In this case, Kang’s team used a hurricane that had a lot of data they could test their simulation against. “To replicate a real-world hurricane, we collaborated with Dr. Stephen Guimond, an atmospheric scientist at Hampton University, who provided observational data for Hurricane Guillermo,” said Kang. “His collaboration and data support were instrumental in setting up the simulation.”
Kang’s team used the data from Guimond’s work to recreate Hurricane Guillermo, the ninth-most intense Pacific hurricane on record. In doing this, they could see if their virtual hurricane evolved in the same way the real-world counterpart did in 1997.
“For virtually creating this atmospheric phenomenon,” explained Kang, “we mathematically model a tropical cyclone using partial differential equations (Navier-Stokes). We integrate a real-world data set (airborne data for Hurricane Guillermo) as input to this mathematical system to numerically recreate the hurricane.”
Creating a fast, efficient simulation is just one part of the challenge. The team also had to ensure that the model could capture all the necessary details to accurately predict when a cyclone would intensify. Rapid intensification is one of the deadliest and most unpredictable stages of a cyclone. Kang’s team was able to accurately simulate a rapid intensification, and they achieved this at a lower computational cost.
“The simulations that we produced are similar to how weather is predicted using supercomputers,” said Kang. “For weather prediction, a key is efficient simulation to provide a quick prediction, especially for hurricane simulations that may severely affect coastal communities. Our work shows that prediction can be made more efficient using GPUs and showcases a programming approach. Our simulations are efficient not only in terms of computing time but also in terms of the required energy consumed to complete the computations. Our results show that using GPUs can save both time and energy compared to pure CPU-based simulations. We hope that this work can contribute to the acceleration of numerical weather prediction models.”
Computational fluid dynamics has greatly benefited from the use of GPUs. Here, we are merely leveraging this capability for problems of interest to the general population in the atmospheric sciences. We expect this trend to continue in the atmospheric and ocean sciences.
postdoctoral researcher, Lawrence Livermore National Laboratory
This work couldn’t have been completed without the resources allocated through the U.S. National Science Foundation ACCESS program. Kang’s team initially worked with a contact at NVIDIA to help them get the simulation running on a GPU. “This research was a great exercise for collaboration between academia and industry. We built our mathematical model in a computer program, and a software engineer at Nvidia helped us make it run on their GPUs,” he said.
After they had the model tested, they needed a supercomputer to run it. Kang’s team chose NCSA’s Delta supercomputer for their ACCESS allocation. Kang hopes that their success will give other research centers working on modeling atmospheric events the confidence and proof of concept that simulating these events on a GPU-based system is worthwhile.
“We used Nvidia A100 GPUs that are available on NCSA’s Delta machine,” said Kang. “When comparing four GPUs to 128 CPUs, the GPU simulation time is about five times faster and five times more energy efficient than the CPU. This presents a compelling case for using GPUs in weather prediction models; however, not all weather centers have fully embraced this technology for a variety of reasons, and we would like to help change that.”
Research related to this breakthrough has been published in the Journal of Advances in Modeling Earth Systems, and the team has just published a paper about their new GPU-based work in the International Journal of High-Performance Computing Applications.
ABOUT DELTA AND DELTAAI
NCSA’s Delta and DeltaAI are part of the national cyberinfrastructure ecosystem through the U.S. National Science Foundation ACCESS program. Delta (OAC 2005572) is a powerful computing and data-analysis resource combining next-generation processor architectures and NVIDIA graphics processors with forward-looking user interfaces and file systems. The Delta project partners with the Science Gateways Community Institute to empower broad communities of researchers to easily access Delta and with the University of Illinois Division of Disability Resources & Educational Services and the School of Information Sciences to explore and reduce barriers to access. DeltaAI (OAC 2320345) maximizes the output of artificial intelligence and machine learning (AI/ML) research. Tripling NCSA’s AI-focused computing capacity and greatly expanding the capacity available within ACCESS, DeltaAI enables researchers to address the world’s most challenging problems by accelerating complex AI/ML and high-performance computing applications running terabytes of data. Additional funding for DeltaAI comes from the State of Illinois.