Satellite Image Home Page

Description of ASDA Format

Draft version 5, 18 April 1997 (version history)

This document describes the Australian Satellite Data Archive (ASDA) format for storing satellite pass meta-data in the Parameter Value Language (PVL). It Fields are classified as mandatory, recommended or optional.

The contents of the PVL header is recommended to be:

1. Preamble, mandatory
2. Format, mandatory
3. Per-Data Description Block, some mandatory, some recommended
4. Full Example Archive Header.

Preamble

The preamble contains the ASDA Version number for this archive file.

ASDA_Version, mandatory

This supplies the version number and the date that the version was settled upon.

Example: "V1.0 March 1997"

ASDA_Information, recommended

This supplies information about the header and the desginers of it.

Header_Contents, mandatory

This list gives the sequence of entries in this archive file.

Example: (Format, HRPT_Data_Description)

Format Group

The Format Group defines the data storage format within the archive file. This is a mandatory field, as it tells a user or user program how to extract the data from the file bistream. See below for an example.

tape_physical_record, mandatory on tape systems

PVL integer of blocksize used on tape. It is the first actual field (after the version field) which contains data, and defines the tape blocking size. A user who is unsure of the block size will be able to read in the first hundred bytes or so and simply read off (in plain 7-bit ASCII) the actual block size to use.

Content Groups, mandatory

This group gives the size of each content block of the archive. It also gives an indication of the record size and record type of the data. The record type given should directly correspond to an entry in a data description group of the same name as the content group. For example, the HRPT data stream:

HRPT_Data group in the Format group indicates that a data block of HRPT data exists in the archive file and has records of length 13864 bytes which are formatted as HRPT_Line which is described in the HRPT_Data_Description group.

Example of Format Group

begin_group = Format
 File_Contents = (PVL_Header, HRPT_Data);
 begin_group = PVL_Header;
  length = 65536 <bytes>;
 end_group = PVL_Header;
 begin_group = HRPT_Data;
  length = 72383944 <bytes>;
  record_size = 13864 <bytes>;
  record_type = HRPT_Line;
 end_group = HRPT_Data;
end_group = Format;

Per-Data Description Group

Scene_Description Group, recommended

AVHRR_scene, recommended if polar orbiter scene

this is a PVL set containing the latitude and longitude coordinates of the top left corner and bottom right corner of the AVHRR scene

Example: AVHRR_scene = {(-10.3,140.1),(-45.3,150.3),(-9.6,142.1),(-45.2,154.3)}

unique_identifier, recommended

this is the composite foreign key that is used to look up the archive catalogue to determine archive file location.

Example: unique_identifier = "NOAA-14,RAW,1996-04-30T10:03:45Z,Hobart"

CEOS_inventory

160 characters, as per the Committee on Earth Observations Satellites (CEOS) Inventory Exchange Format (IEF), version 2.1, February 1992

Example:

N14AVHHRP 950502 034500 ?????? HBTHBT01731A ????????????? 0hrpt   16? Xc01mar961
-50.82 184.00-58.05 137.24-10.32 133.29-06.11 160.43A? ????????????????????????2

Satellite Group, mandatory

This group defines the identity and state of the satellite during the time of image capture.

name, mandatory

The unique code for the satellite.

Example: name = "NOAA-12"

identity, mandatory

the identity (as given by ...) of the satellite

orbit, mandatory if polar orbiter

Orbit number of satellite when data was sensed defined at the acquisition time.

pass_direction, mandatory if polar orbiter

direction in orbit, "ascending" or "descending"

acquisition_start, mandatory

satellite clock at start of data acquisition

acquisition_end, mandatory

satellite clock at end of data acquisition

Navigation Group, mandatory

This group gives all available navigation information for the satellite.

Instruments Group, recommended

This group gives all available instrument information for the satellite.

Station Group, recommended

This group defines the identity and state of the station that captured the image.

name, recommended

the name of the station

identity, recommended

the identity (as given by ...) of the station

location, recommended

latitude/longitude coordinates of the reciever

station_clock_accuracy, recommended

description of the source of the station clock

Example: "XNTP daemon under UNIX"

acquisition_start, recommended

station clock at start of data acquisition

acquisition_end, recommended

station clock at end of data acquisition

Data Description Group, mandatory

The data description group has no formal structure. It should describe the data that is stored from the satellite. The description will mostly be straight ASCII (PVL Character Set) with a few tables. It is intended that this be used as a programmer or scientist reference. This description should include:

1. A full data set name and/or reference to the document used to compile this description.
2. The organisation of the data, be it by lines, by fields, by grid or whatever.
3. The method by which the data was generated. References to the instruments as described in the "Satellite Group", "Instruments Group" above where appropriate.
4. The storage format of the data. This should include the bitfield descriptions of data values where appropriate.

Example

The HRPT data contains information described in the HRPT_Line data description. A partial HRPT_Line data description group could be:

 begin_group = Data_Description;
  begin_group = HRPT_Line;
   size = 13864 <bytes>;
   elements = (pre_sync, identity, time, telemetry, back_scan, space_data, sync, TI
P, spare, AVHRR, post_sync, fill, error_codes);
   begin_group = pre_sync;
    elements = 10 <bits>;
    number_elements = 6;
    description = " first 60 bits from a 63-bit pseudo noise generator,
                      generator polynomial x6+x5+x2+x+1, start all 1's,
                      bit 1, element, 1 first";
    format = '1010000100 0101101111 1101011100 0110011101 1000001111
                0010010101';
   end_group = pre_sync;
   begin_group = identity;
    elements = 10 <bits>;
    number_elements = 2;
    description = 'minor frame number, spacecraft address and frame status';
    format = 'element 1(7)
                bit 1    0 = internal_sync   1 = AVHRR sync
                bit 2,3  00 = not used       01 = minor frame 1
                         10 = minor frame 2  11 = minor frame 3
                bit 4-7  spacecraft address  (4 = MSB, 7 = LSB)
                bit 8    0 = frame stable    1 = frame resync occurred
                bit 9,10 spare, bit 9 = 0    bit 10 = 1
                element 2(8)  spare';
   end_group = identity;
   begin_group = time;
    elements = 10 <bits>;
    number_elements = 4;
    description = 'time code day to milliseconds from spacecraft clock';
 . 
 .
 .

Data_Quality Group, recommended

The Data Quality Group, at a minimum, has to define how the user of this data is to extract the data quality information from either the data stream or from this meta-data file.

description, recommended

Description of how to determine the quality of the data. For HRPT, this would describe what the contents of the Error Field in each line mean.

bad_lines, option

Number of lines that are determined by the ingestor to not be good. This entry is used typically when the data quality information is embedded within the data stream.

line_quality_table, option

The line quality table is used to store the data quality information externally to the actual data stream. Typical entries are for flywheeled (repeated by reception hardware - typically not recieved) lines and lines which are known to have corrupted data.

Version History

Draft version 2, 27 May 1996

Initial public release.

Draft version 3, 13 November 1996

Clarification of the Base Elements of the elements field of the Data Format, particularly with respect to binary data.

Draft version 4, 12 December 1996

Group discussion with P.J. Turner, B.A. Berzins, R.T. Jones and P.C. Tildesley found that the following needed to be rectified in the current format used:

1. The boundaries between data storage and data format had been made less clear by the BoM extensions to the Format group.
2. The overall structure of the ASDA format should be rearranged to allow multiple data formats within one archive file.

The following changes have been made:

1. The Ancillary group has been replaced by the HRPT_Data_Description group (this corresponds directly to the name given to the raw data in the Format group plus the _Description tag). The Data_Description group has been moved into the HRPT_Data_Description group. Thus, there are two high-level groups now - the Format group and the HRPT_Data_Description group. If a SST product was included in the archive, the appropriate Format description would be added along with a top-level SST_Data_Description group describing the data.
2. The BoM Format group extensions have been incorporated into the Data_Description group, allowing programmatical access to that information.
3. A size parameter has been added to data format groups that consist of complex elements which is the total size of those elements.

Draft version 5, 18 April 1997

Version 1.0 of the archive header has been finalised with no major changes from 12 December. Some reformatting of the HRPT description entries was made.