A bridge between GridFTP and HPSS

By Nicole Gaynor

Big data requires new tools for transferring data between machines, especially when the source and target machines use different data transfer protocols. The GridFTP High Performance Storage System data storage interface (GridFTP HPSS DSI), developed at the National Center for Supercomputing Applications, is exactly that.

HPSS, a highly scalable, hierarchical storage system, and GridFTP, a newer transfer protocol, use different types of commands. The HPSS DSI translates communications between the GridFTP protocol and the HPSS API (application programming interface, the set of commands HPSS understands).

“With this enabling piece of software, existing and new HPSS installations can join with grid infrastructure as first-class citizens and leverage emerging transfer solutions such as Globus Online,” says Jason Alt, developer of the HPSS DSI and programmer at NCSA.

The new Blue Waters Data Sharing Service is only one of myriad possible implementations of the HPSS DSI. The service will allow current and former supercomputer users to share their research data with colleagues who do not have access to the supercomputer.

More information on the HPSS DSI and the code base is available at https://github.com/JasonAlt/GridFTP-DSI-for-HPSS.

Disclaimer: Due to changes in website systems, we've adjusted archived content to fit the present-day site and the articles will not appear in their original published format. Formatting, header information, photographs and other illustrations are not available in archived articles.