Data and Information Science and Technologies

NCSA is at the forefront of data-intensive research.

NCSA and its collaborators provide a data management framework for the Dark Energy Survey, which is using the world's largest digital camera to undertake the largest galaxy survey ever attempted. This framework processes, calibrates, and archives the massive amounts of data—quadrillions of bytes over the lifetime of the survey—that will be collected for the DES.

NCSA is embarking on a $10 million effort (referred to as "Brown Dog") to develop software to manage and make sense of vast amounts of digital data. While technology has made is easy for everyone to create and share vast amounts of digital data—including images, video, and audio—searching, sorting, accessing and understanding that data is very challenging. Among the major issues: lack of metadata (the data about the data that describe when and how and by whom is was produced) and difficulty of access data in outdated formats. The NCSA team will develop services to make the contents of uncurated data collections accessible.

The Large Synoptic Survey Telescope will use an 8.4-meter telescope and 3-gigapixel camera to produce a wide-field astronomical survey of the universe that tracks its changes over time; in addition to probing the mysteries of dark energy and dark matter, LSST is designed to detect exploding supernovae, potentially hazardous near-Earth asteroids, and distant Kuiper Belt Objects. LSST will collect tens of terabytes of data every night, which will be processed, calibrated, and archived by NCSA.

NCSA and the University of Illinois also are at the forefront of efforts to develop a National Data Service, which would provide infrastructure to enable researchers across all disciplines to easily find, reuse, and publish data. More than 70 representatives from organizations across the United States and around the world gathered in Boulder, Colorado, in June to discuss this NDS and organize a consortium to turn their vision into reality. The NDS consortium is an international federation of data providers, data aggregators, community-specific federations, publishers, and cyberinfrastructure providers.