Understanding Earth’s Geomagnetism with Blue Waters

Deep in the center of the Earth is a fluid outer core that generates Earth’s magnetic field like a magnet with its two magnetic poles aligning closely to the geographic north and south poles. This alignment has been long used by mankind for navigation. But the magnetic field of the Earth plays a far more critical role in protecting the Earth’s habitats, by providing a strong magnetic shield to deflect solar wind, coronal mass ejection and solar energetic particles.

But the Earth’s core is not a conventional magnet, its magnetic field, called the geomagnetic field, changes substantially in both space and time, due to a turbulent dynamo action within the core. Thus it is very challenging to accurately predict geomagnetic variations in even several years in future. Dr. Nikolaos Pavlis, a scientist with the National Geospatial Intelligence Agency, and Dr. Weijia Kuang, a geophysicist in the Geodesy & Geophysics Lab at the NASA Goddard Space Flight Center in Greenbelt, Maryland, have been using the Blue Waters supercomputer at the National Center for Supercomputing Applications (NCSA) at the University of Illinois to learn more about geomagnetic variations and their underlying mechanisms, so that better forecasting models can be developed. Dr. Kuang recently answered some questions about this research via email.

Q: What can you tell about your research?

A: The collaboration that we are working on is in the area of geomagnetism, an important discipline of Earth science. The research goal of this collaboration is, in one sentence, utilizing geomagnetic observations and geodynamo models to make accurate forecasts of geomagnetic temporal variation on five-year to 20-year time scales. To reach this goal, we focus first on numerically understanding the forecast accuracy convergence with the ensemble size used for the ensemble Kalman filter type algorithm employed in our system.

There are a lot of technical details embedded in the above short description. Therefore I am writing a few more details and hope they are helpful. It is well known that Earth possesses a strong magnetic field (called the geomagnetic field) in much of its history (~ 4.5 billion years). This field is dominantly dipolar at the Earth’s surface and aligns approximately with the spin axis of the Earth, making the two poles pointing approximately to north and south, respectively. The fields are similar to the magnetic fields of a simple bar magnet. This north-south alignment has been used by mankind for navigation for several thousands of years.

Like many other geophysical quantities, the geomagnetic field changes in time and in space. Its changes in time are called “secular variation” (SV). Such changes are due to vigorous fluid motion, called convection, in the Earth’s fluid outer core which is approximately 3,000 km below the surface.

The fundamental geodynamical process governing the core convection and the geomagnetic field is called “geodynamo.” At present, numerical modeling is the main tool to understand this dynamical process, its consequence on geomagnetic variation that is observable at the Earth’s surface, and its relevance to Earth’s evolution on geological time scales. Effort on accurate forecast of SV serves both the fundamental science and societal application needs.

Q. How are you using Blue Waters for this research?

A:We use Blue Waters for two main research tasks: (1) obtaining large ensemble of high-resolution geodynamo simulation solutions; and for (2) testing of forecast accuracy convergence with the ensemble size. These two can provide the knowledge on optimal ensemble sizes for geomagnetic forecast with given numerical resolutions and forecast accuracies. As you will find in the answers to the next two questions, the optimal ensemble size ensures cost-effective means for our research.

Q. How many cores are you using on Blue Waters? How long do your runs take?

A:Our project is computationally expensive. If we use 128 cores as the nominal usage for a single geodynamo simulation run, then 512 simultaneous runs will use 65,536 cores (or 2,048 nodes). However, due to research and technical reasons, we have tested so far only 1,024 cores (32 nodes).

Q. Would this research be possible without Blue Waters?

A. One main bottleneck of our research is the computing resource, in particular the CPU time. A typical geodynamo simulation requires ~10¹³ floating-point operations or “flops” with our current numerical resolution (100x100x100 in the three-dimensional space) and will require 10¹⁷ flops (100 petaflops) if higher resolutions are used for “Earth-like” parameters. Geomagnetic data assimilation can require three orders of magnitude more CPU time with ~1,000 ensemble members. If we look at it from the wall-clock time perspective, a single geodynamo simulation run can take up to two weeks (depending on numerical resolution and number of nodes used) on Blue Waters. Therefore, an ensemble of 512 simulation runs (which is expected to be typical) could last 10 years if they were executed sequentially. Blue Waters will enable us to have the entire ensemble runs executed simultaneously (parallel computation), thus allowing assimilation runs completed in the time frame comparable to that of a single run. Without Blue Waters (or any comparable computing facilities), we would have to scale back our ensemble size in order to complete all simulations within a reasonable time frame. This will certainly limit our ability to achieve meaningful research and application goals.

Disclaimer: Due to changes in website systems, we've adjusted archived content to fit the present-day site and the articles will not appear in their original published format. Formatting, header information, photographs and other illustrations are not available in archived articles.