Data Assimilation Group

Group Leader: W. Bourke

Team Leaders: J. Kepert, P.Steinle

Objectives: To conduct research on advanced data assimilation systems for use in numerical models.

Research activities

The generalised multivariate statistical interpolation scheme (GenSI) which is a major extension of the operational Optimal Interpolation (OI) based system (as used currently in GASP and LAPS) has been the primary focus of the work of the group over the past several years. The GenSI assimilation system supports through a single binary, a range of resolutions for both the Australian region and the global domain. It is based on an iterative solver algorithm and supports 3D radiance assimilation, large volume data selection, extensive data types and improved quality control. The software has been implemented to execute both in shared memory and distributed memory computing architectures. The software has been ported to, and highly optimised for, the new multi-node SX-6 facilities.

The use of polar-orbiting satellite infra-red and microwave soundings is of critical and continuing importance to data assimilation in the southern hemisphere. Extensive efforts are being made to use this data as effectively as possible in both the GASP and LAPS assimilation systems. The use of the expanded cloud drift wind (CDW) coverage has required a significant focus on quality control and data processing in the GenSI scheme, and the use of Scatterometer QuikScat winds has been demonstrated as robust within both assimilation systems.

The Meteorological Archive and Retrieval System (MARS) data handling has expanded and now supports a wide range of operational and research data. A major upgrade of the MARS software has been prepared for the new platforms ( IBM p690) to be available at the new headquarters at 700 Collins St and will also be used in migrating the current archive to the new environment. An important upgrade to the MARS capability is the integration of browsing/graphics/retrieval/ via WEB-MARS. Metview continues to provide a valuable application layer for processing and displaying data for a number of operational and experimental systems.

Implementation of GenSI /3D-Var assimilation for GASP and LAPS

The GenSI assimilation software has been substantially upgraded in functionality and robustness, and the major parallel trials of GASP and LAPS at current operational resolutions have been conducted, yielding significant positive impact over current operations. Additionally increased vertical resolution at 33 and 50 levels in GASP and at 50 levels in LAPS have also yielded a significant positive impact.

A major streamlining of the scripts controlling the GenSI has been developed to support unified assimilation of the GASP and LAPS systems, assimilation on the thinned Gaussian grid, more efficient grid-point format interface between model and assimilation systems, and to facilitate a soil-moisture nudging algorithm. Much of this work has been based on design developed in collaboration between the Data Assimilation and Model Development groups.

Enhanced satellite data assimilation for both GASP and LAPS

The important use of Advanced TIROS Operational Vertical Sounder (ATOVS)/ Advanced Microwave Sounding Unit (AMSU)-A and High Resolution Infrared Sensor (HIRS) in both LAPS and GASP has been maintained in both research and operations. Extending the GASP and LAPS systems to 50 levels (0.1 hPa uppermost level) has been successfully validated in extensive assimilation and prediction experiments. An important logistical aspect of the ATOVS processing has been to arrange receipt of both real-time global radiances (via the UK Met Office) and local read-out of ATOVS radiances. The UKMO data stream provides ATOVS/AMSU-B moisture channel radiances and the experimental AQUA/AMSU-A radiances. Evolving to the use of UKMO and local AAPP 1D radiances, has introduced a further need to perform HIRS cloud clearing. A straightforward paired window channel discrimination has been developed and successfully tested to identify clear/cloudy IR radiances.

Cloud drift winds in BUFR format and with agreed international quality control indicators (QI) are now available from the major CDW producers: NESDIS, EUMETSAT and JMA (GOES-9). The ability to set and use appropriate thresholds based on these QI has been an essential aspect of the successful trials of the GenSI system in 2003/04. Assimilation of QuikScat scatterometer data has been successfully validated in GenSI within a 33 level GASP system and within a 29 level LAPS system nested within the GASP system. The assessment of GPS based retrieval of Total Precipitable Water has been restricted to documenting and completion of the original year 2000 intercomparison with Sondes/Numerical analyses. The GenSI data handling has been extended to support more extensive diagnostics on data usage. Prototype diagnostics for some data types have been developed within Metview but remain to be extensively used.

Three-dimensional error covariance modelling

Ensemble based covariance generation

The GASP T239/L29 ensemble prediction error statistics, based upon ensembles of differences between 6-hour forecast background fields, generated by parallel analysis-forecast cycles with perturbed observation, have been prepared and tested in three GenSI assimilation experiments, with further evaluations in more extensive trials on the new SX-6 facilities.

Ensemble Kalman Filter (EKF) assimilation

An important workshop on EKF assimilation was organised by the Group and held in BMRC in November 2003 with a number of leading international experts in the field attending under Bureau sponsorship. This meeting provided an excellent update on current international efforts in the field as well as stimulating our own efforts in developing a prototype EnKF system.

Co-ordinate transformations

The use of potential temperature based covariance in the vertical has been implemented within GenSI but awaits testing and evaluation.

High resolution Australian-region surface analysis.

A robust routine Mesoscale Surface Analysis System (MSAS), and accompanying web page and archive to MARS have been put in place under the control of the SMS scheduler. The system of hourly 4 km resolution Australian Region surface analyses has been used to support the ‘National alerting system for severe tropical storms using NWP guidance’ system, operated from the Western Australian Regional Forecast Centre.

GASP Ensemble Prediction System (with Model Development Group)

The GASP Ensemble Prediction System (EPS) has been operated successfully now for three years by NMOC. The web page has been extended with some useful diagnostics. The ECMWF based verification software has been refined and now supports verification of EPS predictions available within the MARS archive. A detailed quantitative evaluation of the GASP and ECMWF EPS systems has been conducted. The evaluation suggests that the relatively poorer performance of the GASP EPS is very much constrained by the basic GASP system performance. This highlights the importance of the imminent upgrades of GenSI and the BAM model, which are expected to deliver an improved basic deterministic control forecast. An ongoing dynamics issue within the GASP EPS is the occasional generation of spurious inertial modes in the singular vector decomposition.

Optimisation of the assimilation and prediction system codes.

The GenSI code has been further optimised including improvements arising from the Benchmark NEC optimisation. The code was successfully ported to the SX-6 facilities and validated in a range of extensive GASP assimilation and prediction experiments.

The implementation of synchronous observation – first-guess deviations has been completed and tested in experimental trials. Comprehensive testing of this procedure is expected to deliver useful impact on system performance. An algorithm for moisture nudging within the assimilation/model cycle has been implemented and used successfully by the Model Development group in tests of the ECMWF land-surface parametrisation scheme.

Validation of the sensitivity of the GASP prediction system to a range of initial conditions has been examined in an attempt to clarify the relative role of initial conditions versus model. Initial conditions from ECMWF and from National Centers for Environmental Prediction (NCEP) have both been used, with the studies confirming that the ECMWF initial condition has a significant positive impact relative to the GASP operational initial condition in the southern hemisphere and the tropics, while maintaining a smaller but positive impact on northern hemisphere prediction relative to GASP. The NCEP initial conditions show a similar impact although less marked in the southern hemisphere. A major component of the discrepancy in current performance of the GASP system in the southern hemisphere is attributable to the capability of the currently operational assimilation system.