Radiant Takes on the Cloud

Unless you’re completely off the grid, you probably use the cloud to access and manage data. From bank accounts to virtual photo albums, the world relies now, more than ever, on phantom servers housed somewhere beyond the rainbow. And, according to Doug Fein, assistant director for cyberinfrastructure at the National Center for Supercomputing Applications, those services come at a cost. Eventually.

For example, even if you aren’t paying for premium service to host a website or store other data, it could cost you if you want to migrate that data. And depending on how much data, it could cost you a lot. Another consideration is control over your data. What happens when the existing hosting server becomes obsolete? What happens to your precious family photos? There are inherent inefficiencies, Fein says, when data is managed far from its original source.

When you’re talking about massive amounts of data, like that being modeled on supercomputers and analyzed in the cloud, the stakes are even higher. “Science revolves around data,” Fein says.

“When using containers,” adds Rob Kooper, NCSA lead research programmer, “it has become easier to move the compiled applications to where the data is and leverage the compute next to the data, unlike in the past where we moved the data to where applications are installed. With the large amount of data (up to PetaBytes), moving the data to the compute is no longer financially and time feasible.”

These huge data sets may require networking multiple resources to obtain the speed expected from high-performance supercomputers. Because networking can also be a costly endeavor in the commercial cloud environment, a team of University of Illinois at Urbana-Champaign data scientists have been working to create a solution allowing for more flexible data management at a fraction of the commercial cost.

“In 2015, we built Nebula, running on open-stack/open-source software, as an experiment,” Fein says. They started with 10 nodes and grew to 50 nodes in two years. While Nebula, with no long-term funding, was never meant to be a permanent solution, the experiment worked as a proof of concept, showing a clear need for this type of data management.

In 2019, the team started imagining how to administer what Nebula offered to a larger campus community. And thus, Radiant was born.

Launched in January 2021, Radiant allows users serving data through the system total control and significant cost savings over other options. While the team is working on NIST 800.171 compliance, Radiant adheres to NCSA’s stringent security requirements. And with 100-gigabit nodes, Radiant is incredibly fast.

Radiant is a long-term solution. It’s really about being able to store as much as you want. You could start with one [virtual machine] and expand for 500, for example. And you don’t have to worry about data loss when a grant ends. Radiant is about one-third the cost of Amazon Web Services and similar commercial solutions. We’re not looking to compete with them; we just want to offer another option.

Doug Fein, NCSA Cyberinfrastructure Assistant Director

Time on Radiant is charged by reservation, with three charges: storage (charged per terabyte), CPUs (charged by number of cores and memory) and VMs (charged by the number of instances). “It can be as low as $100 a year per instance,” Fein says. Not only is Radiant a flexible, affordable data management solution, it also offers the support of a team housed at NCSA.

A Radiant webpage is in the works, but you don’t have to wait; visit the Radiant allocations wiki for more information about getting a project on Radiant.

---

ABOUT NCSA

The National Center for Supercomputing Applications at the University of Illinois Urbana-Champaign provides supercomputing and advanced digital resources for the nation’s science enterprise. At NCSA, University of Illinois faculty, staff, students and collaborators from around the globe use these resources to address research challenges for the benefit of science and society. NCSA has been advancing many of the world’s industry giants for over 35 years by bringing industry, researchers and students together to solve grand challenges at rapid speed and scale.