Evidence Targeted Automation (ETA)

The Bureau of Meteorology aims to produce forecasts that provide the most value and impact for the Australian community.The ETA project is developing an automated forecast system, with evidence to inform its use, ensuring that quality weather forecasts are provided by the Bureau in an efficient manner with an optimal mix of human and automated processes.

ETA is developing a suite of tools to evaluate gridded weather forecasts including comparison of various forecasts from various sources. The assessment techniques are being designed in consultation with meteorologists to ensure they are relevant to the definition, design and use of the forecasts.

ETA is providing results as they become available, enabling incremental improvement of Bureau forecasts and the efficiency of their production. We have developed techniques to quantify the improvement, and confidence of any improvement, provided by the human contribution to the official forecasts and work on streamlining the human effort. We published a research report documenting some of our early work and the general framework we use to support decisions around automation at the Bureau of Meteorology.

Our work has assessed the accuracy and value of forecasts for the following parameters:

• 3-hour and 24-hour chance of rainfall
• Minimum and maximum temperatures
• Wind direction and magnitude
• Dewpoint
• Fire Danger Indices for Grass and Forest.

The plot above shows a scaled version of the Huber loss (an error metric) of the wind speed at weather stations located on exposed peaks across Australia. Smaller mean errors (lower down on the plot) are better. The graph shows how the forecasts improve from a 7-day lead time to a 1-day lead time. The Official forecasts (green) are compared to two automated alternatives. The improvement from the initial automated alternative (blue) to the better one (purple) was our automated implementation of a manual technique previously used by forecasters.