|
||||||||||||||||||||||||||||||||||||||||||
The Bureau of Meteorology's new water information functions are described in the Commonwealth Water Act 2007 under Part 7 - Water Information. These new functions are supported by the Water Regulations 2008.
CD, DVD or on an external storage device. The Regulations also provide for historical information to be given by File Transfer Protocol (FTP) or via Web Services. However, given the potentially large file sizes, we advise against FTP. Each zip file sent via FTP should not be greater than 30MB. The Bureau does not currently have the operational capacity to poll Web Services, please alert us if you are considering giving water information via Web Services so we can discuss the arrangements for this process.
Address to send Historical Information on CD or DVD:
Manager Water Data
Water Data Management Branch
GPO Box 2334
ACT 2601
For a definition of Historical Information, see Historical Information and New Information.
For a definition of Ongoing/New Information, see Historical Information and New Information.
Time Series Data and Meta-data should be provided by FTP or via Web Services. The Bureau does not currently have the operational capacity to poll Web Services, please alert us if you are considering giving water information via Web Services so we can discuss the arrangements for this process.
This information should be provided by FTP. However, where information is required at discrete and widely spaced time intervals, such as at the end of each financial year it can be given by CD, DVD or on an external storage device.
The Bureau of Meteorology has registered all persons named in the Regulations on its FTP service, to support the giving of regular water information (hourly, daily, weekly, monthly, annually), as required in accordance with the Regulations under the Water Act 2007. This service will allow you to copy and transfer the water information files you are required to give to the Bureau of Meteorology on an ongoing basis.
Each agency has the following Folder structure:

All incoming data files whether time series, metadata or other must be contained in zip files for FTP transfer. Zip files should be uploaded directly to the data, metadata, or timeseries folders, as above. Please do NOT attempt to create subfolders within these folders. The file transfer mode of the FTP client must be set to binary. The total size of the zip files transferred in any one hour period should not exceed 30MB, or the upload may fail.
The zip files should be named using only alphanumeric, hyphens, and underscores in filenames, i.e. no spaces or metacharacters. (See Data file content and naming conventions for information on how to name the data files within a zip file.) Zip file names should be chosen to be unique for each transfer, e.g. by encoding date and time or sequence number in the name, unless the data is intended to replace previously supplied data, in which case the previous name should be reused.
To access the service you will need to connect to the internet. Your organisation will receive by email the User Id and password which you will require to access the service.
The FTP service is available 24 hours per day, every day.
More information about the Bureau of Meteorology's FTP service can be found at http://www.bom.gov.au/reguser/ under FTP Access procedures.
Windows Explorer can also be used as a simple FTP client to connect to the Bureau of Meteorology's FTP service.
Download FTP instructions.
Some organisations named in the Water Regulations 2008 employ service providers such as contractors or other agencies to collect and store data on their behalf. With few exceptions, this information is defined under the regulations as being in the named organisation's control, although it may not be stored in their database. The named organisations are therefore required to ensure its provision to the Bureau. In the case where information is being sourced from a third party service provider, the Bureau can, on instruction from the named organisation, provide a separate FTP login to the service provider. The service provider can then supply data into their own FTP account on behalf of a named person. Please contact us if this situation applies to your organisation.
After you have uploaded one or more zip files to the Bureau's FTP server, the data files will be processed by Bureau systems. The process will assess data files and will either accept (import) or reject the files. The actions completed are recorded in log files that are on the FTP site. The log files are the method of confirming acceptance or rejection of data files with data providers.
The Water Transfer Data Transfer Format (WDTF) is our preferred mode of data delivery and we are promoting this as the standard format for the transfer of water data. This format will facilitate the transfer of Time Series data, and Metadata.
There are a number of commercial companies that have produced export tools to facilitate the forwarding of data to the Bureau. Organisations wishing to update their data manangement systems to support WDTF can apply for funds from the Bureau under the Modernisation and Extension of Hydrologic Monitoring Systems Program.
There is no specified format for water information, other than time series data and metadata.
When you fill in the Data Survey please describe the systems in which you currently store that information. An example of water information that is not time series data is Information Sub-Category 3f.
To assist with the ingestion of data into Bureau systems, there are some basic guidelines for data file content file name structure.
It is theoretically possible to deliver all data relating to many monitoring stations in a single file. However the Bureau has set a constraint that a data file should only contain data for a single site and the data for that site be split across two transfer file types. If required, data of each file type can be provided independently of the other. The two file types and their contents are described below.
This file type is used to transfer data that is continuously monitored. It is typically of high temporal density and is usually recorded electronically.
This file type can contain all site data other than time series. It can contain discreet water quality sample data.
The required file name format is:
FormatName.OrganisationId.DateTime.SiteId-FileType.xml
Element |
Mandatory ? |
Description |
||||
|---|---|---|---|---|---|---|
FormatName |
Yes |
Declares the name of the data file format.
|
||||
OrganisationId |
Yes |
The identifier used to describe a person named under the Water regulations. These identifiers are assigned by the Bureau of Meteorology, it is the login name you use to connect to the FTP server. |
||||
DateTime |
Yes |
The date and time (standard time) that the file was created, in the form YYYYMMDDHHMMSS. |
||||
SiteId |
Yes |
The site identifier used by the data custodian. This identifier should be unique within the organisation. The list of valid identifiers is maintained by the data custodian. |
||||
FileType |
No |
This element is used to indicate that a file contains continuous time series data (ctsd). If used, the only valid content is 'ctsd' |
Organisation 'w00XXX' - 'Data provider XXX' who creates an export at 8am on 1/July/2008. The data is for monitoring site G8210009. The file format is not one of the Bureau declared formats, the organisation calls it a 'HHRR' format.
Hhrr-meta-v1.w00010.20080701080000.G8210009.txt
Home | About Us | Learn about Meteorology | Contacts | Search | Help | Feedback Weather and Warnings | Climate | Hydrology | Numerical Prediction | About Services | Registered Users |
|
© Copyright Commonwealth of Australia 2009, Bureau of Meteorology (ABN 92 637 533 532) Please note the Copyright Notice and Disclaimer statements relating to the use of the information on this site and our site Privacy and Accessibility statements. Users of these web pages are deemed to have read and accepted the conditions described in the Copyright, Disclaimer, and Privacy statements. Please also note the Acknowledgement notice relating to the use of information on this site. No unsolicited commercial email. |