Innovators Bridging Research and Software Engineering

Research in nearly every discipline has benefitted from the development of software tailor-made to address its needs – whether that means modeling complex geophysical phenomena, building data management systems that enable meaningful research insights, or something more. As countless domains have begun to draw on software developers’ expertise, a new career path has emerged.

Over the past 10 years, developers working in research have self-organized and named their field Research Software Engineering. The Society of Research Software Engineers, which emerged from these efforts, defines a Research Software Engineer as “combining professional software engineering expertise with an intimate understanding of research.” This blend of strong technical prowess and research experience affords unique insights into solving complex problems and has made RSEs invaluable members of their teams.

While this field might seem new, NCSA has been supporting groundbreaking research with software for the past 35 years, from developing the Mosaic web browser to the diverse range of research software developed here today. A number of developers at the Center are members of the United States Research Software Engineer Association (US-RSE), a professional community co-founded by NCSA Chief Scientist Daniel S. Katz. I spoke with a few research software engineers at NCSA to learn more about their experience in this field and the advice they have for those interested in it.

The RSE Difference

Dipannita Dey, a research programmer at NCSA’s Software Directorate, was brand new to research when she joined the Center. “Industry is focused on getting product to market,” she says, reflecting on her prior work. “I wanted opportunities to explore, read papers, get hands-on with the research.”

At NCSA, her work on Clowder gives her the opportunity to not only build a data management platform for academics but to do research of her own on federated search applications, satisfying her own research interests and creating opportunities for publication.

“You’re bridging two very different communities,” notes Chen Wang, a fellow research programmer. “You have to understand the research to an extent in order to communicate with scientists about what they need and be able to translate that into a software application.”

Chen’s work on the IN-CORE platform allows researchers to model the impact of natural disasters and inform community resilience planning – a field that encompasses many different domains itself. “People definitely speak different languages!” she laughs. “I might be given just an equation and need to work with the researchers to translate it into code and then fit that into our larger platform, and be able to visualize the results.”

“There’s no book that tells you how to do it,” adds T. Andrew Manning, a senior research programmer. “You are doing R&D work on your own, innovating creatively, figuring out what solutions are needed for the problem at hand. The researchers you’re working with might not know what they need exactly and you have to help them get there. It’s about being a thought leader and innovator but also a maintainer, a bug fixer and so on. You’re giving scientists the resources they need to do their science.”

His work at NCSA is with the Center for Astrophysical Surveys – “getting tons of data to answer lots of cosmological questions,” as he puts it. “We’ve had to be very innovative in how to capture and store and make that data accessible so that scientists can use it effectively to answer their research questions.”

Despite the challenges, all three are very enthusiastic about the benefits of the role. “I really like writing code,” Wang says. “The SMILE project started as three people in a room together and now has 3,000 users from around the world. I feel like I’m helping others.”

“I like getting stuff to work. I like being able to say, I built this thing and it runs,” Manning adds. 

“If a community benefits from my products, I feel great,” Dey says. “I love what I do every day.”

The Path to this Role

RSEs come from many different backgrounds. Some, like Dey, began as developers working in industry before migrating into research based on their interests. Others, like Wang and Manning, began as researchers but were drawn to programming. “I always wanted to get better at it,” Wang says, who studied climate change and bioenergy at Illinois. “My research was always computational and that process intrigued me.” Manning completed a Ph.D. in physics at the University of Maryland and worked in other fields before realizing his true interest was in the “whirligigs and doodads” of software development. “Everyone knows software and cyberinfrastructure are critical to doing science,” he says.

All three had helpful advice for those who might be interested in becoming a Research Software Engineer.

“There are so many different programming languages out there. Pick a commonly used one to learn in-depth and get a basic understanding of how programming works. Once you know that much, you can pick up others more easily. And stay calm!”

Dipannita Dey

“If you’re coming from a research background, harden your programming skills because that’s the core of the work. If you’re coming from a computer science background, don’t get too stressed if you don’t understand the research at first. Be passionate and open and flexible.”

Chen Wang

“Just go for it! But you have to find an institution to work at that understands this concept. Outside of NCSA, I’m not sure how well other institutions understand the value of this combination of skills. You don’t want to be fighting an uphill battle to get your role defined.”

Andrew Manning

The Bigger Picture

That value is very clear at NCSA, per Kenton McHenry, associate director of the Software Directorate. “We’re software developers, but we’re also academics. We’re part of the team – collaborators, partners, equals,” he says. At NCSA, RSEs are completely integrated into the research process: writing code, but also publishing, presenting, writing proposals, mentoring students and more. “This isn’t a disposable role that’s hired on contract – it’s a crucial part of the whole organization,” he says. “That allows us to see common needs and build a knowledge base.”

At the end of the day, it’s that mindset that makes this work possible. “You have to have a mentality of always learning,” McHenry says. “You have to want to change something. If you want to make a difference, be a part of cutting-edge projects, then come on in.”

Contributor Acknowledgement
NCSA’s KJ Naum, project manager, contributed to this story.