The HDF Group provides support for NPP/NPOESS in a number of ways, including development and maintenance of software capabilities in HDF5 libraries and tools that help NPP/NPOESS data producers and users, software testing on platforms of importance to NPP/NPOESS, high quality rapid response user support for NPP/NPOESS, and performance of special projects. The purposes of this presentation are to apprise attendees of the areas of emphasis for FY 2010, and to solicit ideas and opinions that will help the project understand how best to use its resources in order to best serve the needs of NPP/NPOESS.
1. The HDF Group
Support for NPP/NPOESS
by The HDF Group
Mike Folk, Elena Pourmal, Peter Cao
The HDF Group
November 5, 2009
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
1
www.hdfgroup.org
2. Goal
To develop and adapt HDF5
software and provide HDF5
support for NPP/NPOESS.
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
2
www.hdfgroup.org
3. Areas of immediate need
⢠Ensure easy and intuitive data access and
use by diverse communities
⢠Enable producers and consumers to view
content, manage metadata, and convert data
to other formats.
⢠Provide high-quality, rapid-response support
of HDF5 for NPOESS users at all levels.
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
3
www.hdfgroup.org
4. 2009-2009 Priorities
⢠Data accessibility and usability
⢠Developed library of high level APIs to support
NPP/NPOESS data management
⢠Tool development
⢠Modify h5dump to enable viewing of region
references and quality flags
⢠Modify HDFView to enhance viewing of
references and quality flags
⢠System maintenance
⢠User support
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
4
www.hdfgroup.org
5. NPP/NPOESS Project Information
⢠Project Web site
⢠http://www.hdfgroup.org/projects/npoess/
⢠Project description
⢠Released software
⢠Documentation
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
5
www.hdfgroup.org
7. NPOESS Software FTP Site
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
7
www.hdfgroup.org
8. NPOESS Project Information
⢠Project Wiki site
⢠http://wiki.hdfgroup.org/NPOESS-Project
⢠Monthly progress reports, links to software
under development, presentations, etc.
⢠Requires a password
⢠Contact epourmal@hdfgroup.org
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
8
www.hdfgroup.org
10. The HDF Group
Library of High-Level APIs
to support NPOESS data
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
10
www.hdfgroup.org
11. NPOESS High-Level Library
⢠Library to support HDF5 region references and
packed bits in support of NPOESS quality flags
â˘
â˘
â˘
â˘
C APIs
Documentation
Tests
Examples
⢠Source available from
⢠ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/s
ource/
⢠Documentation is available from
⢠http://www.hdfgroup.org/projects/npoess/
⢠Still a prototype!
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
11
www.hdfgroup.org
13. The HDF Group
h5dump
New Features to Support Region
References and Quality Flags
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
13
www.hdfgroup.org
14. h5dump and Region Reference Data
⢠New flag âR to display data pointed to by
region references
⢠Improved display of subsets
⢠An element of a dataset is specified by its
indices
⢠Available in HDF5 1.8.4, November 2009
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
14
www.hdfgroup.org
15. h5dump and Quality Flags
⢠New flag âM to display packed bit-field data
⢠Available only in the HDF5 NPOESS branch
⢠Will NOT be in HDF5 1.8.4 release
⢠Source and pre-built binaries can be found at
⢠ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/NPOESS/
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
15
www.hdfgroup.org
16. The HDF Group
HDFView
New Features to Support
References and Quality Flags
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
16
www.hdfgroup.org
17. HDFView and references
⢠Aggregated granules: show the whole
dataset pointed to by an object reference
⢠Granules: show data pointed to by a dataset
region reference
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
17
www.hdfgroup.org
18. Objects Pointed to by Object References
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
18
www.hdfgroup.org
19. Data Pointed to by Region References
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
19
www.hdfgroup.org
20. HDFView and quality flags
⢠By default, integers with quality flags are
displayed as 8-bit decimal values
⢠Needed: a way to show values of specific
quality flag bit fields
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
20
www.hdfgroup.org
21. Default display: 8-bit decimal integers
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
21
www.hdfgroup.org
22. Displaying quality flags in 8-bit integers
⢠Use âOpen Asâ on dataset with quality flags.
⢠Apply bitmask to choose individual quality
flags
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
22
www.hdfgroup.org
23. Examples of bit masks
⢠Applied mask in red at top-right, e.g., âBy bitmask {0, 1}â
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
23
www.hdfgroup.org
24. Quality Flags
⢠Data can be displayed in decimal, binary, or hexadecimal
al
cim
de
ry
i na
b
d
xa
he
November 3-5, 2009
ci
e
al
m
HDF/HDF-EOS Workshop XIII
24
www.hdfgroup.org
25. HDFView plug-in to Show Quality Flags
⢠Implemented using âTreeViewâ plug-in
â˘
â˘
â˘
â˘
Open HDFView
Select âUser Optionsâ
Select NPOESS plug-in
Restart HDFView
⢠Available with
HDFView 2.5 patch3
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
25
www.hdfgroup.org
26. The HDF Group
Possible Future Work
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
26
www.hdfgroup.org
27. Possible future work
â˘
â˘
â˘
â˘
Add new features
Enhance current features
System support
User support
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
27
www.hdfgroup.org
28. Maintenance
⢠Maintenance NPOESS/NPP features
â˘
â˘
â˘
â˘
High-Level Library
H5dump, other tools?
HDFView
Testing, documentation, releases
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
28
www.hdfgroup.org
29. New features
⢠Support Variable-Length datasets
⢠Ease access to information about quality flags
⢠Use definition of quality flags, which are in:
⢠The Common Data Format Control Book
⢠XML files as part of the CDFCB-X delivery
⢠Apply the structure of the XML to determine
whether a field is a packed quality flag byte
⢠Tell us what you need!
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
29
www.hdfgroup.org
30. Enhance current features
⢠Extend reference display options
⢠E.g., show properties of the referenced data
⢠Revise default TreeView and TableView to
improve NPOESS plug-in support
⢠Improve quality of testing, documentation, and
binary and source distribution
⢠Tell us what you need!
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
30
www.hdfgroup.org
31. NPOESS/NPP System Support
⢠Daily testing on systems critical to NPOESS/NPP
⢠Linux 32 and 64-bit
⢠Windows
⢠AIX 5.3
⢠Solaris 32 and 64-bit
⢠Regular testing of NPOESS high-Level library
⢠Linux 32 and 64-bit
⢠Windows
⢠Solaris 32 and 64-bit
⢠Porting to requested compilers
⢠Performance tuning for applications
⢠Improved support for C++
⢠Support for netCDF-4 access to NPOESS SDR and EDR
products
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
31
www.hdfgroup.org
32. NPOESS/NPP User Support
⢠Priority support to NPOESS/NPP users
⢠Send email to help@hdfgroup.org
⢠Add NPOESS to the subject field
⢠Example:
Subject: [NPOESS] Where is the plug-in?
⢠Response in less than 4 hours
⢠Continue gather requirements and enhance
HDF5 software as needed
⢠We want to hear from you!
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
32
www.hdfgroup.org
33. Special Research Projects
⢠Improvements to HDF5 or HDF5 NPOESS
products with the data management portion
of the Raytheon Algorithm Development
Library (ADL)
⢠Implement OpeNDAP server for HDF5
NPOESS SDR and EDR products
⢠Develop NPOESS specialized version of the
HDF5 tool h5check
⢠Performance tuning of HDF-NPOESS
product files
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
33
www.hdfgroup.org
36. Acknowledgements
This work was supported by cooperative agreement
number NA08AANEG0128from the National
Aeronautics and Space Administration (NASA).
Any opinions, findings, conclusions, or
recommendations expressed in this material are
those of the author[s] and do not necessarily reflect
the views of the National Aeronautics and Space
Administration.
November 3-5, 2009
HDF/HDF-EOS Workshop XIII
36
www.hdfgroup.org