About Water Data Transfer Format (WDTF)

What is WDTF?

WDTF is an XML data transfer format designed to support the water industry to share and deliver water data to the Bureau of Meteorology as required under the Water Regulations 2008.

The current version of the WDTF supports data for Regulations categories 1, 2, 3, 4, 5, 7, 9, 10a and 10b.

Why was a data transfer file format required?

In July 2008, the Bureau of Meteorology was scheduled to receive data from more than 250 organisations listed in the Regulations. At the time there were no applicable national or international format standards. Data providers were using various formats to supply data ranging from mature in-house or commercial data management systems to spread sheets configured for a combination of data storage and reporting purposes. Segments of the water industry with common data storage products were able to exchange data with common formats, but data meaning was inconsistent between organisations.

Variability in data formats and potential format instability presented significant challenges for the Bureau when it came to using the data. Data interchange between organisations and researchers was also likely to be problematic. The Bureau identified the need for a data transfer format to support the transfer of regulations data. The water data transfer format (WDTF) has now been developed and adopted as the preferred method for data delivery.

Existing data transfer standards and formats.

The Bureau's existing format for Hydrological data transfer is called 'Hydrological comma separated' (HCS). This format is used in the established and mature flood warning role but was unable to support the full range of data required by the regulations..

WDTF initial development

The regulations requirements were reviewed with industry representatives to establish data transfer requirements. Input was sought from CSIRO, Greenspan, Kisters, Rubicon and SRA Information Technology with several content workshops held to define WDTF data. The data content was then encoded into XML through the WIRADA project.

XML encoding is based on a GML Simple Features (GML- SF) implementation of the Observations and Measurements (O&M) standard. GML- SF is a profile that only uses a subset of GML, reflecting the most common WFS (Web Feature Service) use of GML3.

Walker, G. , P. Taylor, S. Cox and P. Sheahan (2009). Water Data Transfer Format (WDTF): Guiding principles, technical challenges and the future. In Anderssen, R.S., R.D. Braddock and L.T.H. Newham (eds) 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand and International Association for Mathematics and Computers in Simulation, July 2009, pp. 2377-2383. ISBN: 978-0-9758400-7-8.

WDTF support

Many IT/data management companies have undertaken work to assist organisations to implement WDTF data transfer to the Bureau.
See WDTF Providers for a list of companies.

Future directions

Components of WDTF address similar requirements to existing standards. Work is being completed to harmonise these efforts using a standards-based, model-driven approach. The design of WDTF is compatible with this model-driven approach. WDTF data requirements are a superset of those being considered in the harmonisation activity. Once completed, the harmonised components will be integrated into WDTF. It is anticipated that this work will be the basis for release version 2.0.

The harmonisation work is being completed through the OGC's hydrology domain working group (Hydrology DWG). The Hydrology DWG is a forum for collaboration and development of standards for hydrological data. The purpose of the Hydrology DWG is to provide a venue and mechanism for seeking technical and institutional solutions to the challenge of describing and exchanging data describing the state and location of water resources, both above and below the ground surface. The path to adoption will be through OGC papers and standards, advanced to ISO where appropriate, and also through the World Meteorological Organization's (WMO) and its Commission for Hydrology (CHy) and Water Information Systems (WIS) activities.

WDTF release dates

WDTF package v0.1 released — 26 August 2008
WDTF package v0.2 released — 22 September 2008
WDTF package v0.3 released — 24 March 2009
WDTF package v1.0 released — 30 October 2009