NCSA awarded $7.7 million for Dark Energy Survey data management

10.05.11 -

The National Science Foundation has awarded a grant of $7.7 million over five years to the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign to operate a sophisticated data management pipeline for the Dark Energy Survey, a collaborative astronomy project focusing on uncovering the nature of dark energy.

Beginning in 2012, the DES will use a 500-megapixel camera (DECam) to deeply image 10 percent of the sky for 500 nights, producing a precise picture of the largest-scale structures in the universe and a detailed measurement of how those structures have evolved over time. DES will gather an enormous amount of data, capturing terabytes every night.

"This type of data-intensive, data-driven science requires networking, computing, and archiving capabilities and sophisticated tools to efficiently process the data," says Don Petravick, who leads the DES data management project. "It's our role at NCSA to provide those capabilities and tools as an integral part of the DES collaboration. This allows astronomers to focus on analysis of science-ready data, rather than spending their time on preliminary processing or technical issues."

Working closely with the DES collaboration and the Illinois Department of Astronomy, NCSA has developed a system for processing, calibrating, and archiving the wealth of data that will be gathered by the DES. This system will use high-performance computing resources provided by the NSF's XSEDE (Extreme Science and Engineering Discovery Environment) project.

NCSA and the University of Illinois have been involved in the DES collaboration since 2005. The data management tools have been tested through periodic Data Challenges, working with simulated data that has progressively become closer and closer in volume and complexity to what will be gathered when DES comes online next year.

NCSA is also involved in the data management system for the upcoming Large Synoptic Survey Telescope (LSST), which will use an 8.4 meter telescope and 3 gigapixel camera to produce a wide-field astronomical survey of the universe that tracks its changes over time. Like DES, LSST will collect terabytes of data every night.

"We'll process the images of 30 million stars and galaxies for each full night of observing," Petravick says.

For more information, go to: http://cosmology.illinois.edu/DES/.