Predicting Tornados with Data Driven Workflows: Building a Service Oriented Grid Architecture for Mesoscale Meteorology Research

Each year the loss of life and property due to mesoscale storms such as tornados and hurricanes is substantial. The current state of the art in predicting severe storms like tornados is based on static, fixed resolution forecast simulations and storm tracking with advanced radar. It is not good enough to accurately predict these storms with the accuracy needed to save lives. What is needed is the ability to do on-the-fly data mining of instrument data and to use the acquired information to launch adaptive workflows that can dynamically marshal resources to run ensembles simulations on-demand. These workflows need to be able to monitor the simulations and, where possible, retarget radars to gather more data to initialize higher resolution models that can focus the predictions. This scenario is not possible now, but it is the goal of the NSF LEAD project. To address these problems we have built a service oriented architecture that allows us to dynamically schedule remote data analysis and computational experiments. The Grid of resources used include machines at Indiana, Alabama, NCSA, Oklahoma, UNC and Ucar/Unidata, and soon Teragrid. The users gateway to the system is a web portal and a set of desktop client tools. Five primary persistent services are used to manage the workflows: a metadata repository called MyLEAD that keeps track of each users work, a WS-Eventing based pub-sub notification system, a BPEL based workflow engine, a web service registry for soft-state management of services and the portal server. An application factory service is used by the portal to create transient instances of the data mining and simulation applications that are orchestrated with the BPEL workflows. As the workflows execute they publish status metadata via the notification system to the users MyLEAD space. The talk will present several open research challenges that are common to many e-Science efforts.