Time Series Foundation Models - current state and future directions
The HDF-EOS5 Tutorial
1. The HDF-EOS5 Tutorial
Ray Milburn
L3 Communciations, EER Systems Inc.
1801 McCormick Drive, 170
Largo, MD 20774 USA
Ray.Milburn@L-3com.com
2. What is HDF-EOS ?
• HDF (Hierarchical Data Format) is a disk-based format
chosen by NASA as a standard for EOS program
• HDF-EOS is an extension to HDF which standardizes
common Earth Sciences data structures
• HDF-EOS2 based on HDF4 is used operationally by EOS
TERRA and AQUA instrument teams
• HDF-EOS5, based on HDF5 is newer and will be used by
EOS AURA instruments
• Contains Geolocation Information:
– Structural metadata links Geolocation and science data
2
3. HDF-EOS 5 Library
• Provides Grid/Point/Swath/ZA interfaces
• Allows to create/access Grid/Point/Swath/ZA and
manipulate their objects (data sets, attributes, etc.)
• Written in C with FORTRAN-77 wrappers
• C-function names prefixed with “HE5_”
• FORTRAN-77 calls have similar naming
convention
3
4. Grid Structure
• Data organized by regular geographic
spacing, specified by projection parameters.
• Structure
– Multidimensional data arrays
– Geolocation information is given implicitly by
projection parameters contained in structural
metadata
4
6. Swath Structure
• Data is organized by time or track parameter
• Data spacing can be irregular
• Structure
– Geolocation information stored explicitly in
Geolocation Field (2-D array)
– Data stored in multidimensional arrays
– Time stored in 1-D or 2-D array
– SM links Geolocation and science data
6
8. Point Structure
• Data is specified temporally and/or spatially,
but with no particular organization
• Structure
– Tables used to store science data at a particular
Lat/Long/Height
– Up to eight levels of data allowed. Structural
metadata specifies relationship between levels.
8
10. Zonal Average (ZA) Structure
• Generalized array structure with no
geolocation linkage
• Data can be organized by time or track
parameter
• Data spacing can be irregular
• Structure
– Data stored in multidimensional arrays
– Time stored in 1-D or 2-D array
– SM links science data
10
11. HDF-EOS Grid Structure
• Usage - Data which is organized by regular
geographic spacing, specified by projection
parameters.
• Structure
– Any number of 2-D to 8-D data arrays per
structure, one per data type (e.g. temperature)
– Geolocation information contained in projection
formula, coupled by structural metadata.
– Any number of Grid structures per file allowed.
11
12. HDF-EOS Swath Structure
• Usage - Data which is organized by time,
monotonic geolocation, track parameter.
Spacing can be irregular.
• Structure
– Geolocation information stored explicitly in
Geolocation Field (2-D array)
– Data stored in 2-D or 3-D arrays
– Time stored in 1-D or 2-D array, TAI is standard.
– Geolocation/science data connected by structural
12
metadata
13. HDF-EOS Point Structure
• Usage - Data which is specified temporally
and/or spatially, but with no particular
organization.
• Structure
– Tables (Vdatas) used to store science data at a
particular Lat/Long/Height.
– Up to eight levels of data allowed. Structural
metadata specifies relationship between levels.
13
14. HDF-EOS ZA Structure
• Usage - Data which is organized by time,
track parameter. Spacing can be irregular.
• Structure
– Data stored in 2-D or 3-D arrays
– Time stored in 1-D or 2-D array, TAI is standard.
– Science data connected by structural metadata
14
15. Basic HDF-EOS5 Functions
• Access - Open, create, close objects/files
• Definition - Set key parameters of data sets
• Input/Output - Read/Write data
• Inquiry - Return information about data
• Subset - Read data from specified region
15
32. Programming Model
• Open the HDF-EOS5 file
• Create or Attach to a specified
Grid/Swath/Point/ZA object
• Define key object features and/or
perform necessary operations (with
the data sets, attributes, etc)
• Detach from the Grid/Swath/Point/ZA
object
• Close the HDF-EOS5 file
32
33. HDF-EOS Tools and Utilities
• heconvert - command-line utility to convert
HDF-EOS2 objects to HDF-EOS5 objects
• HE5View - X-Motif based tool which
displays all parts of the HDF-EOS5 objects
• JEB - Java EOS Browser - Java based tool
to display HDF4, HDF-EOS2 and HDFEOS5 objects
33