Australian Government - Bureau of Meteorology Home | About Us | Contacts | Help | Feedback |

Global | Australia | NSW | Vic. | Qld | WA | SA | Tas. | ACT | NT | Ant. |
Weather & Warnings | Hydrology | Climate | Numerical Prediction | About Services | Learn About Meteorology | Registered User Services |

Analyses & Numerical Prediction

Analysis and Prediction Operations Bulletin No. 60
31 March 2005

Operational Consensus Forecasts (OCF)


Introduction

An initial configuration of the Operational Consensus Forecasts (OCF) system was operationally implemented in NMOC beginning with the 00 UTC run on 8 March 2005. The OCF system, formally known in the non-operational state as BOG (Best Objective Guidance) was developed by the Weather Forecasting Group in BMRC.

OCF optimally combines the direct model output from a number of numerical atmospheric models and produces forecasts of various weather elements for specified sites throughout Australia. More specifically the operational configuration, as implemented on 8 March, produces forecasts of maximum, minimum and ground temperatures, rainfall amounts and associated probabilities, sunshine hours and evaporation for more than 600 sites throughout Australia. The available forecast period can be up to 7 days depending on weather elements. Currently the NWP models used in OCF include the local systems: GASP, LAPS_PT375, MESO_LAPS_PT125, MESO_LAPS_PT050 (SEQLD, SYDNEY and VICTAS domains) and those from the overseas' centres: ECMWF, NCEP, UKMO and JMA.


Bias Correction and Compositing

Direct model output (DMO) usually has a systematic bias due to the model resolution and physical parameterisation used in the model. Additional sources of bias comes from spatial and temporal interpolation errors when the model output is used to produce a forecast for a specific site and time. Model Output Forecasts (MOF) suffers from biases in different ways. A model upgrade can result in an outdated predictor-predictand relationship and therefore coefficients in the relationship need recalculation. Since this recalculation requires 2 to 3 years of development data it is very time-consuming and hence impractical to do the recalculation for every model update. OCF overcomes these bias problems for both DMO and MOF by using a bias-correction scheme using a 30-day running window. In this scheme an estimate using the Best Easy Systematic estimator (BES) is computed for the sample of forecast errors from the previous 30 days:

Formula
where q0.25 and q0.75 are the lower and upper quartiles respectively for the sample of all the forecast errors in the last 30 days, and q0.5 is the median for the same sample.

This bias estimate is calculated for all the component models. Then for each model the corresponding BES is removed from the current forecast. The result is a set of bias-corrected component DMO and MOF forecasts.

After the bias-correction is applied to each component model the current model forecasts are combined to produce an optimal consensus forecast. The compositing method is based on giving different weights to individual components. The Mean Absolute Error (MAE) of each bias-corrected component forecast over the previous 30 days is computed. A weighting is then given to each individual component forecast according to:

Formula
where MAEi is the mean absolute error for the bias-corrected forecast of the ith component model for the past 30 days, and N is the total number of available component models.

This means the greatest weighting is given to the component model which after bias-correction showed the least MAE over the last 30 days. The weighting calculated using the above formula is then used to combine all the bias-corrected current component forecasts to produce a consensus:

Formula
where wi is the weighting given to the bias corrected ith component forecast of the weather element of interest, fi.

The optimal weighting relies on the assumption that the model biases present in the last 30 days continue to apply to the current forecast situation.


Implementation of OCF in NMOC

In the current setup in NMOC there are 8 component models whose forecasts are used in DMO form: GASP, LAPS_PT375, MESO_LAPS_PT125, MESO_LAPS_PT050 (MESOQLD, MESOSYD and MESOVICTAS), ECSP, UKGC, USAVM and JMA. Additionally there are 2 MOFs, one derived from LAPS_PT375 (MOF-LAPS375) and another from GASP (MOF-GASP).

How a model is used in OCF compositing is dependent on a number of factors: the model's forecast time interval (foreast projection interval), the maximum forecast period (maximum forecast projection), availability of fields, etc. For example, for UKGC and USAVM models 6-hourly forecasts are available up to 48 hours. Thereafter the forecast interval increases to 12-hourly. Since the interpolation scheme used for calculating maximum and minimum temperatures require at least a 6-hourly forecast interval, max/min temperature forecasts are available for at most up to day 2 for these two models. Availability of temperature forecasts for OCF component models and their descriptions are shown in Table.1.
Component

Component

Contribution (+day)

Resolution of grid in Database

Name

Basetime

Max T

Min T

Spatial (deg)

Temporal

ECSP

-

-

-

1.5

6 hourly reducing to 12 after forc hr=120

12Z

1 2 3 4 5 6 7 8 9

1 2 3 4 5 6 7 8 9

GASP

00Z

1 2 3 4 5

1 2 3 4 5 6

1.0

3 hourly

12Z

1 2 3 4 5 6

1 2 3 4 5 6

MOF-GASP

00Z

1 2 3 4 5 6

1 2 3 4 5 6

-

-

12Z

1 2 3 4 5 6 7

1 2 3 4 5 6 7 8

UKGCM

00Z

1

1 2

1.25

6 hourly reducing to 12 after forc hr=48

12Z

1 2

1 2

USAVN

00Z

1

1 2

1.25

6 hourly reducing to 12 after forc hr=48

12Z

1 2

1 2

JMA

00Z

1 2

1 2 3

1.25

6 hourly reducing to 12 after forc hr=84

12Z

1 2 3

1 2 3

LAPS_PT375

00Z

1 2

1 2 3

0.375

3 hourly

12Z

1 2 3

1 2 3

MOF-LAPS375

00Z

1 2

1 2

-

-

12Z

1 2

1 2

MESO_LAPS_PT125

00Z

1

1 2

0.125

3 hourly

12Z

1 2

1 2

MESO_LAPS_PT050

00Z

1

1

0.05

Hourly

12Z

1

1 2

 

Table.1 This table shows configurations of OCF component models. The contribution is the number of days that a model's actual forecast goes out to operationally.

All DMO and MOF components are updated twice a day except ECSP component model which is updated only once a day.

One critical factor that has a large impact on operational utilitsation of OCF is the arrival times of the component models. The Bureau's limited area model, MESO_LAPS_PT125 is one of the first models to arrive and is always included in OCF. However because GASP and most of the global models from overseas centres arrive later than the time OCF compositing process runs the global models used in the OCF lag the limited area models by 12 hours. ECSP data can be older still: as the model is updated only once a day the available ECSP data can be older than other components for 12Z OCF run. In some cases one or more models are not available by the time OCF suite runs, in which case warning flags are displayed on the OCF webpage to indicate which component models are missing.

The fields that are forecast by OCF vary from station to station. At most there are 7 weather elements that are forecast for each site. These are:

  • Maximum air temperatures
  • Minimum air temperatures
  • Rainfall
  • Probability of rain
  • Ground minimum temperature
  • Sunshine hours
  • Evaporation

Not all elements are forecast for a particular site. It depends on the availability of the fields in the model grid, temporal resolution of model grids and the availability of observations for bias-correction.


Verification

Figure 1 shows comparison between temperature forecasts produced by OCF and those issued by RFC's for a period from 1 June, 2003 and 31 May, 2004. The aggregated MAEs were calculated for all available OCF sites. The statistics show that the OCF has outperformed official RFC maximum temperature forecasts for day1 and day2 but its performance was worse at day3 and day4 relative to RFC. However for a minimum temperature forecast OCF has outperformed RFC at all projection hours.

Comparison of temperature forecasts

Figure 1. Mean absolute errors of OCF maximum (X) and minimum (N) temperature forecasts and the official RFC forecasts for all the OCF sites. The numbers following X and N refer to forecast periods in days. The comparison covers the period from 1 June, 2003 and 31 May, 2004. The number of component models used are only the largest possible. Most of the days less than optimal number of component forecasts were used as model grids were not available at the time of commencement of OCF compositing process. Data were extracted from the real-time Forecast Database (FDB) and consequently the acculmulation used here simulates the real-time operational environment as closely as possible. Note that in some RFCs the time period for recording maximum/minimum temperatures was defined to be the 24-hour period from midnight to midnight. OCF adopts the usual convention of the 24-hour period from 9am to 9am. Note also that the OCF forecasts were not rounded to the nearest degree prior to verification which is not the practice used in the RFC's.

A special interest to forecasters is the verification results for the capital cities. Table 2 shows maximum/minmum temperature verification statistics accumulated from June 2003 to May 2004. For capital cities the table shows that the official RFC maximum temperature forecasts were slightly better than those produced by OCF. For the minimum temperature forecasts the difference between RFC's and OCF is minimal.

Site Element Day Events MAE
Official OCF
PERTH METRO MAX +1 340 1.20 1.40
DARWIN AIRPORT 338 0.94 0.97
ADELAIDE 328 1.21 1.24
BRISBANE 339 1.26 1.55
SYDNEY OBS HILL 335 1.43 1.46
CANBERRA AP 332 1.26 1.27
MELBOURNE 340 1.47 1.48
HOBART 339 1.50 1.61
PERTH METRO MAX +2 339 1.44 1.69
DARWIN AIRPORT 337 0.99 1.11
ADELAIDE 327 1.60 1.58
BRISBANE 338 1.42 1.63
SYDNEY OBS HILL 334 1.71 1.66
CANBERRA AP 331 1.60 1.56
MELBOURNE 338 1.76 1.75
HOBART 338 1.78 1.72

PERTH METRO MAX +3 339 1.66 2.04
DARWIN AIRPORT 337 1.05 1.25
ADELAIDE 327 1.80 1.96
BRISBANE 338 1.45 1.79
SYDNEY OBS HILL 334 1.97 2.49
CANBERRA AP 331 1.74 1.89
MELBOURNE 338 1.94 2.08
HOBART 338 1.87 2.17
PERTH METRO MAX +4 339 1.84 2.23
DARWIN AIRPORT 337 1.10 1.27
ADELAIDE 327 1.99 2.48
BRISBANE 338 1.64 2.00
SYDNEY OBS HILL 334 2.15 2.56
CANBERRA AP 331 1.93 2.17
MELBOURNE 338 2.05 2.31
HOBART 338 1.95 2.48
PERTH METRO MIN +1 337 1.50 1.51
DARWIN AIRPORT 338 0.98 0.94
ADELAIDE 328 1.38 1.21
BRISBANE 339 1.28 1.24
SYDNEY OBS HILL 335 0.98 0.94
CANBERRA AP 332 1.65 1.67
MELBOURNE 340 1.23 1.25
HOBART 339 1.24 1.21
PERTH METRO MIN +2 339 1.87 1.79
DARWIN AIRPORT 337 1.08 1.05
ADELAIDE 327 1.73 1.44
BRISBANE 174 1.51 1.48
SYDNEY OBS HILL 334 1.17 1.05
CANBERRA AP 331 2.08 2.00
MELBOURNE 338 1.41 1.32
HOBART 338 1.42 1.32

PERTH METRO MIN +3 340 2.08 2.03
DARWIN AIRPORT 338 1.13 1.20
ADELAIDE 328 1.81 1.74
BRISBANE 175 1.83 1.80
SYDNEY OBS HILL 335 1.30 1.32
CANBERRA AP 332 2.31 2.32
MELBOURNE 339 1.47 1.54
HOBART 339 1.47 1.51

PERTH METRO MIN +4 339 2.20 2.30
DARWIN AIRPORT 337 1.18 1.38
ADELAIDE 327 1.96 2.01
BRISBANE 175 1.97 2.04
SYDNEY OBS HILL 334 1.48 1.70
CANBERRA AP 331 2.42 2.50
MELBOURNE 338 1.50 1.57
HOBART 338 1.59 1.72


Table 2. Mean absolute errors of forecast maximum (MAX) and minimum (MIN) temperatures for the capital cities. The accumulation period for the statistics was from June 2003 till May 2004. The forecasts were extracted from the real-time FDB so not all component models were used to produce OCF consensus. The 174 events for Brisbane minimum temperature forecasts for day2, day3 and day4 is due to the fact that QLD RFC did not issue minimum temperature forecasts beyond day1 during the first half of the verification period.

OCF relies on high quality observational data for its bias-removal step. If however bias develops in observations OCF will drift and its performance will deteriorate. A good quality control procedure for the observational network is therefore very important and OEB should be notified promptly if a site fault is suspected.



Product Availability

Web

The webpage is updated twice a day after MESO_LAPS_PT125 model run becomes available.
External URL


Acknowledgement

The work of Frank Woodcock and Chermelle Engel in the Weather Forecast Group of BMRC is fully acknowledged.


Future Developments

Enhancements to the current operational configuration are expected in the near future. They inlcude:

  • Updates of station dictionary due to observation site closures or changes
  • Hourly OCF
  • Improved forecast preparation forms which directly ingest OCF data
  • Inclusion of MESO_LAPS_PT050 with newer domains - Adelaide, Perth and Darwin
  • Dissemination of OCF to external users


References

  1. Upgrade to NMOC's Weather Forecast Guidance Best Objective Guidance (BOG)
  2. Woodcock, F. and Engel, C., 2005: Operational Consensus Forecasts.Weather and Forecasting, 20, 101-111.
 

Home | About Us | Learn about Meteorology | Contacts | Search | Help | Feedback
Weather and Warnings | Climate | Hydrology | Numerical Prediction | About Services | Registered Users | SILO
© Copyright Commonwealth of Australia 2008, Bureau of Meteorology (ABN 92 637 533 532)
Please note the Copyright Notice and Disclaimer statements relating to the use of the information on this site and our site Privacy and Accessibility statements. Users of these web pages are deemed to have read and accepted the conditions described in the Copyright, Disclaimer, and Privacy statements. Please also note the Acknowledgement notice relating to the use of information on this site. No unsolicited commercial email.