Skip to main content

Dark Energy Survey Data Management group lays groundwork for next survey at NCSA


Understanding the 14-billion-year history of cosmic expansion requires a deep and wide map of the universe. The fainter and more distant the source of light is, the farther back in time astronomers can look to interrogate their theories on how the universe came to be.

Using larger telescopes and more sensitive cameras, astronomers have begun systematically mapping the Earth’s night sky to greater depths. Right now, about a quarter of the southern sky is being mapped by the Dark Energy Survey (DES), using a new, large camera on the 4-meter Blanco Telescope in Chile, which is capturing the light of hundreds of millions of galaxies over a 5,000 square degree region.

The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana Champaign is one of the three founding members of DES, chosen to lead the data management component of the project. NCSA Research Scientist Felipe Menanteau and the DES Data Management team are using the Blue Waters supercomputer—as well as Illinois’ Campus Cluster and Fermilab’s Fermigrid—to process the raw telescope images into high-quality scientific data products.

DES began searching the southern skies on August 2013 and will continue observing for around 100 nights a year until the end of 2018. During a normal night of observations, DES produces about 1 TB of raw data, including science and calibration images, which are transported automatically from Chile to NCSA to be archived and reduced. The DES data management system is in charge of processing, calibrating and archiving these images.

Menanteau says Blue Waters staff spent almost two years working with his team to get the “perfectly parallel” scientific pipelines the survey uses to work on the system. These pipelines are responsible for automatically turning an image taken at multiple exposures into accessible and organized data. He explains this process more in the following video.

A next generation survey, the Large Synoptic Survey Telescope (LSST), is set to start operations in early 2020, and NCSA will host its Data Facility for operations. LSST is building off knowledge developed over the course of DES that allows for astronomical pipelines to be run on systems like Blue Waters. This knowledge is helping LSST prepare innovative pipelines for a more ambitious and expansive look at the universe..

“We’re using galaxies to properly map the distribution of matter in the universe,” says Menanteau, adding that gaining such knowledge will allow scientists to learn more about dark energy, a mysterious force that’s pushing galaxies and clusters of galaxies away from each other at an accelerating rate.

Every year DES releases all observations made to date to the project’s collaborators, who use the data for rapid science analysis and to enhance their understanding of the evolution of the universe. The survey has discovered previously unknown dwarf planets and dwarf galaxies in and around our Milky Way as well as millions of more distant, previously undetected galaxies. The survey will publicly release half of their data in December 2017, with a follow-up public release at the end of the project.

Felipe Menanteau is also a Research Professor of Astronomy at the University of Illinois. The Dark Energy Survey is supported by the National Science Foundation grant #1536171, the Department of Energy, foreign funding agencies, and the DES collaborating institutions, which are listed at www.darkenergysurvey.org.

Much of the footage in the video came from NCSA’s Advanced Visualization Lab’s movie “Seeing the Beginning of Time,” now available on Amazon Prime. Footage and timelapses also originally used in “Wonders of the Chilean Sky” was provided by Bjoern Soergel of the Institute of Astronomy, Cambridge.

Disclaimer: Due to changes in website systems, we've adjusted archived content to fit the present-day site and the articles will not appear in their original published format. Formatting, header information, photographs and other illustrations are not available in archived articles.

Back to top