Australian Bureau Of Meteorology

Public User Surveys

Introduction
Collection of Vital User information
Service Changes
Accountability
Background
Research Objectives
Methodology and Sample
Other Surveys
The Future

Introduction

The Australian Government Bureau of Meteorology (hereafter the 'Bureau') provides a rich and varied information service to the Australian public. The service is designed to cater for the wide spectrum of users in the community. The service has evolved over many years into its present state and continues to change to meet the ever-changing information requirements of an increasingly weather savvy public.

In order to best meet community needs, the Bureau endeavours to understand 'the user' and, particularly, aspects such as the range of weather dependent activities and what weather information is used. The Bureau is also interested in the public's perception of the accuracy of weather information and whether the information is available in sufficient time to allow it to be used in an optimal way.

Collection of Vital User information

The Bureau utilises a range of approaches to determine user behaviour with regard to weather information. These include attending agricultural field days and boating shows, community talks (for example schools and Probus clubs), attendance at regular consultative meetings with representatives in a number of community sectors, liaising directly with specific customers and groups of customers, as well as one-on-one interactions between weather forecasters and the general public every day of the year. The latter are conducted from the Bureau's main forecasting centres in each state and territory capital city, and the many more manned observation stations distributed across Australia.

The Bureau also conducts 'Public User Surveys' via a third party organisation, using the Computer Aided Telephone Interviewing (CATI) technique. These surveys provide a snapshot of a variety of information related to the public's use of weather services.

Service Changes

The Bureau is constantly reviewing its services and products to best provide a variety of weather information available in the most appropriate formats, at the most useful times and readily accessible through the many media and technology platforms that exist.

Improvements in the Bureau's services may result from a combination of any or all of the triggers below:

• Improvements in the science of meteorology may manifest either through increased accuracy of existing weather forecast information, or through the introduction of new products and services

• New technologies can allow the Bureau and/or its partners in the private meteorological sector to deliver innovative services. For example, the variety of weather and other information provided using mobile phone technology, such as SMS, MMS and, in the past, WAP

• Innovation occurs inside the Bureau. Bureau staff have a wide range of knowledge and experience and are themselves users of the Bureau's services. Hence they are often a source of new ideas for service improvements

• Direct feedback from users can help to refine existing products and services or lead to the creation of new services

Accountability

The Bureau exists as an Executive Agency within the Commonwealth Department of Environment and Heritage, and is largely funded by the Australian taxpayer. As such, the Bureau is accountable, as any Public Service Department or Agency is, to carry out its responsibilities in an effective and efficient manner.

The Bureau is required to report on the effectiveness and efficiency of its activities, both within government through a variety of mechanisms, and externally through its Annual Report, as well as via a number of informal relationships with specific user groups.

The Bureau also believes it important to share some of the information that is gleaned from the public via the public user surveys. The associated pages presented here aim to do this. Readers of these pages are encouraged to contribute their thoughts and responses to the information presented through the feedback page.

Background

The Bureau has conducted regular seasonal telephone surveys of the Australian public since 1997 to assess a variety of aspects of the Bureau's performance in meeting the public's weather information requirements. Samples are drawn from households in metropolitan, rural and regional Australia.

Beginning with the Summer 2001/2002 survey, a new phase of research commenced. The metropolitan surveys were continued on a seasonal basis until 2003 when they were cut back to just two: Summer (conducted in February) and Winter (conducted in August). Separate Regional and Rural surveys were conducted on a once per year basis.

Since Summer 2001/2002 a greater emphasis has been placed on what weather information is used and how this information is accessed to make weather-dependent decisions. The research also encompasses the public's perception of accuracy of forecasts for different weather elements, and for the accuracy of short and medium-range (2-4 day) forecasts, as well as for overall satisfaction with the Bureau.

For the most part, the same questions are asked in consecutive surveys. This allows the Bureau to determine if there is any noticeable trend in the measures being assessed. This will alert the Bureau in a timely fashion to aspects of the service that require attention and possibly some adjustment in the short term.

On occasion, additional questions will be inserted as part of the survey to focus on a particular aspect of the Bureau's service. The questions may refer, for example, to a specific product, the public's interpretation of certain 'weather-speak', or be canvassing for ideas on the direction of, or reaction to, new products or services.

The reduction in the frequency of the surveys in 2003 in no way implies the Bureau is less willing to determine the public's impressions, more that the Bureau has recognised that some of the measures have remained unchanged. The information provided by these metrics would not be adversely affected by limiting the frequency of the surveys.

The questions asked in the latest Metropolitan, Regional and Farmer (Rural) surveys are shown at:

• Metropolitan

• Regional

• Rural

Research Objectives

In determining the various research objectives, the Bureau considered the broad range of users of weather information. Regular measurement of the following aspects of the Bureau service allow appropriate and focussed use of resources, and evolution of the Bureau's service.

The specific objectives of recent public user surveys that are common across Metropolitan, Regional and Rural samples include:

• Measure overall satisfaction with the Bureau's service

• Measure perceptions of forecast accuracy, including short (24 hour) and medium range (2-4 day) forecasts (including Seasonal Climate Outlooks in the case of the Farming survey), and forecasts for different weather elements

• Determine if there is a public perception that the weather forecasts are improving

• Determine the types of weather parameters of particular interest, and the particular products that contain the information required

• Measure perceptions of the timeliness of weather information delivery

• Understand how weather information is used to make weather-dependent decisions and whether it meets user requirements

• Determine how weather information is accessed, including media used, frequency of access and how far ahead.

In addition, the surveys provide an opportunity to garner attitudes and knowledge via one-off questions that may relate to specific aspects of the service or a specific products. Examples in the recent past have included:

• Public understanding of commonly used terminology (including 'fine', 'dry', 'rain' and 'showers')

• Determining if the public was aware of the existence of particular weather information (in this case soil temperature and evapo-transpiration - both of which are useful for the farming community)

• User experience of severe thunderstorm warnings

Methodology and Sample

The Bureau uses an external company proficient in Computer Aided Telephone Interviewing techniques to conduct the surveys. Research interviews are conducted over evenings and weekends. Surveys are conducted for samples comprising householders from Metropolitan (capital city) centres, Regional (provincial/country) centres and Rural (farming) areas.

The samples for the latest Metropolitan, Regional and Farmer (Rural) surveys are shown at:

• Metropolitan

• Regional

• Rural

Other Surveys

A number of other sections in the Bureau conduct either regular or ad hoc surveys of their client/user base. While the majority of the information remains within the Bureau and supports research and product innovation, some has found its way into a variety of reports available externally - particularly the Bureau's Annual Report (see http://www.bom.gov.au/inside/eiab/reports/ar03-04/index.shtml ). Regional forecast offices (located in the capital cities) have also conducted one-off surveys, mostly at the various agricultural field days and boat and other shows.

At various times results of these surveys will be reported in a range of contexts including changes to service, public education and required reporting.

The Future

The Bureau intends to continue to survey its customers to aid with the continuous improvement of services, and also to satisfy internal and external reporting requirements. The nature and content of future surveys will depend on the amount of resources available and the changing external environment. The organisation will also move to use new technologies and techniques (including the Internet) to gain this information, while being conscious of any possible biases in using these means.

The Bureau's feedback page on its external web site (see http://www.bom.gov.au/other/feedback ) already attracts a large number of queries and suggestions. The Bureau values any information users may wish to provide us and is particularly interested in the way people use weather information in their daily lives. As well, you may like to provide us with examples of how the use of weather information has saved (or cost) you money. You may also like to comment on what you would like to see measured in determining the successes or failures of the Bureau's service.