NMOC Quarterly Summary April-June 2002

Summary of System Performance
Model Skill Scores

Analyses & Numerical Prediction | About Products | Map/Image/Chart Archives

These graphs show a time filtered graph with the plot centred on the 6th month of a 12 month period. The x-axis label refers to January for the year of the label.
 

The above graph shows the historical improvemant of the Australian Region LAPS model predistion versus persistence. The Bureau target of 30 points difference has been exceeded continuously since early 2000.

Anomaly Correlation Scores

Definition of Terms

Anomaly Correlation is the area-averaged correlation between the anomaly of the forecast versus climatology and the anomaly of the verifying analysis versus climatology. It measures the skill in forecasting the position of systems, and is insensitive to overall bias or error in scale. It is usually expressed as a percentage, with 100% being perfect. Evaluation of forecast charts by staff at ECMWF showed that 60% was about the lower limit for a forecast to provide useful guidance.

Mean Error (or bias) is the spatial average of the differences between the forecast and the analysis valid for the same time.

Root Mean Square Error (rmse) is the spatial average of the squared differences between the forecast and the analysis valid for the same time. It is a measure of how well a model captures both the pattern and the intensity of systems. It also includes the effect of any biases.

S1 Skill Score is the sum of the absolute horizontal gradients of the differences between the forecast and analysis values, normalised by the sum of the maximum absolute gradients of a forecast or analysis value. It measures the model ability to forecast correctly the horizontal gradients of scalar variables such as MSLP. The full range of the S1 score is from 0 to 200, with a low score being better than a high score. A perfect score of 0 occurs when the forecast and analysis gradients are the same, even though the values may be different. Typically a score of about 60 represents the limit of usefulness. Using persistence for a 24-hour forecast gives a score of about 55 over the Australian Region.

Australian Region refers to an irregular verification grid that covers the area 15° - 45°S and 100°-170° E but with additional points to 55° S between 130°-170° E. The grid spacing used is 10° but grid points are staggered in longitude and in latitude. It uses all 58 points at MSLP but 48 points (excluding northern and southern most points) at upper levels. TLAPS verification uses the grid that covers the area 15°-45° S and 100°-170° E.

Australian Tropics refers to a verification grid covering the domain of 25° S-25° N and 100° E-160° E with the grid spacing of 2.5°.

Metanal refers to the successive correction method (SCM) numerical analysis that was run at NMOC between 1969 and 1994 for the traditional Australian Region domain.

Persistence is the forecast that is obtained by using the initial analysis as the forecast. It is often used as a basis of comparison in the assessment of the success attained in conventional forecasting methods such as NWP.

Self refers to verifying the forecast from each system against its own analysis.

12-month Moving Average is the average of a parameter over a 12-month period, centred on the 6th month. It was also called 12-month running mean.