NCSA NEWS

LEAD weathers first big trial

Story posted November 9, 2006

A multi-disciplinary effort involving nine institutions -- including NCSA -- and more than 100 scientists, students, and technical staff, LEAD comprises a complex array of services, applications, interfaces, and local and remote computing, networking, and storage resources that can be used in a stand-alone fashion or linked together in workflows to study mesoscale weather (meteorological phenomena approximately 2-200 kilometers in horizontal extent, such as thunderstorms and squall lines).

The LEAD (Linked Environments for Atmospheric Discovery) framework provides users with an almost endless set of capabilities ranging from simply accessing data and perhaps visualizing it, to running highly complex and linked data ingest, assimilation and forecast processes in real time and in a manner that adjusts dynamically to inputs as well as outputs. The methodologies and infrastructures being developed are extensible to other fields, including medicine, ecology, hydrology, geology, oceanography, and biology.

"We had dedicated time on both NCSA's Mercury and Tungsten clusters, using 256 processors of each machine" says Jay Alameda, NCSA project manager, of the LEAD trial that took place July 13 as part of a Unidata workshop at the National Center for Atmospheric Research in Boulder.

Following a tutorial, users configured their accounts, developed their forecasts and launched the associated workflows on the TeraGrid. According to Alameda, two runs were made, with 25-30 users simultaneously in each run. Once both runs were complete, users came together in a plenary session to view and discuss their results.

"Some aspects of the project worked very well, other parts, we discovered, did not," says Alameda. Teams are identifying the modifications that will be needed for those areas.

Participants overall impression was that the capabilities provided by LEAD (Linked Environments for Atmospheric Discovery) were useful. The University of Michigan's Katherine Lawrence and her community engagement team were present to observe the use of the orchestration systems and identify further usability issues related to functional navigation and the use of documentation. The team also recorded explicit suggestions for improving the consistency of software with existing conventions and expectations of meteorologists and students (for example, preferred notation for specifying date and time).

The workshop afforded a unique opportunity to unveil the rapidly maturing capabilities of LEAD to a select group of university researchers and faculty who are particularly interested in running forecast models and applying related tools in the classroom. In addition, the team obtained both quantitative and qualitative feedback and conducted the first scalability and stability tests of the entire LEAD system, including in particular the ability of the TeraGrid to handle the simultaneous submission of dozens of weather research and forecasting jobs.