Created in 1987, NCSA HDF is a data file format that assists users in storing, manipulating, and sharing scientific data. HDF works across diverse operating systems, including UNIX, Windows, and Linux. It features a library of callable routines, utility programs, and tools for creating HDF files, converting files into HDF, and analyzing and visualizing data. When HDF was created, scientists were using a wide assortment of computer systems and software. They needed to share data in a world where standardization was rare.

"It began with support for eight-bit raster images," says Folk. "Scientific visualization was becoming more popular, and people wanted to view visual data on their desktop PCs. With HDF they were able to move that data from a supercomputer to the desktop and among different kinds of desktop computers." HDF has since expanded dramatically, supporting images and data compiled in multiple dimensions as well as tabular datasets that include a variety of data types.


















HDF allows users to group datasets in a single file, much as PC users group data into folders on their desktops. Among file management programs, this feature is unique to HDF, says Folk, and it means researchers can easily organize data by any characteristic. Another unique feature of HDF is its ability to include information about the dataset as annotations inside the data file. A climate researcher, for example, might receive a file from a colleague that includes annotations telling him he is looking at temperature data taken on a specific day and at specific intervals. Other annotations might tell the researcher the latitude and longitude of the temperature data.

"As a researcher, what I like about HDF is the ability to use one set of software to pull in all sorts of data," says Barkstrom. "And the fact that HDF allows you to include all sorts of peripheral information about the data in your files definitely makes data analysis easier, quicker, and more accurate."