Using the Water Data FTP Sites

File Transfer Protocol (FTP)

The Bureau of Meteorology has registered all named persons or organisations in the Regulations on its FTP service. This supports the giving of regular water information (hourly, daily, weekly, monthly, annually), as required in accordance with the Regulations under the Water Act 2007. This service will allow you to copy and transfer the water information files you are required to give to the Bureau.

Each agency has the following Folder structure:

All incoming data files, whether time series, metadata or other, must be contained in zip files for FTP transfer. Zip files should be uploaded directly to the data, metadata, or timeseries folders, as shown above. Please do NOT attempt to create subfolders within these folders. The file transfer mode of the FTP client must be set to binary. The total size of the zip files transferred in any one hour period should not exceed 30MB, or the upload may fail.

Zip files should be named using only alphanumeric text, hyphens and underscores in the filename. The names should not contain spaces or metacharacters. See Data file content and naming conventions for information on how to name the data files within a zip file. Zip file names should be unique for each transfer by encoding date and time or sequence number in the name for example, unless the data is intended to replace previously supplied data. In that case the previous name should be reused.

To access the service you will need to connect to the internet. Your organisation will be sent an email containing a User Id and password that is required for accessing the service.

The FTP service is available 24 hours every day.

More information about the Bureau's FTP service can be found at http://www.bom.gov.au/reguser/ under FTP Access procedures.

Windows Explorer can also be used as a simple FTP client to connect to the Bureau FTP service. Download FTP instructions.

After you have uploaded a zip file

After you have uploaded one or more zip files to the Bureau's FTP server, the data files will be processed by Bureau systems. The process will assess data files and will either accept (import) or reject the files. Completed actions are recorded in log files that are on the FTP site. The log files confirm acceptance or rejection of data files with data providers.

The Bureau's FTP data retrieval and receipt process.

• FTP folders are scanned and uploaded files are examined once every hour. Any files currently being transferred are ignored until the next hour.
• If a file is in the correct format and is a valid zip archive it is accepted for further processing and moved from the upload folder.
• If a file is not in the zip format or is not a valid zip archive, it will be rejected and deleted from the upload folder.
• For each accepted file, a line will be added to an 'acceptedYYMM.txt' logfile that is created within the upload directory.
• For each rejected file, a line will be added to a 'rejectedYYMM.txt' logfile that is created within the upload directory.

About FTP data delivery log files

Logfiles are a cumulative record of all files accepted and rejected for a given year (YY) and month (MM). The current month's log files may be deleted without loss as they will be recreated the next time any files are processed, however, a previous month's log files will no longer be recreated after a new month has started. Data suppliers should delete log files from the FTP site once they are no longer required. Where the accumulation of log files becomes excessive the Bureau may delete older log files. The Bureau will not delete the current month and previous month's files from the FTP site.