SlideShare ist ein Scribd-Unternehmen logo
1 von 18
Downloaden Sie, um offline zu lesen
Generalized Conversion of HDFEOS Products to GIS Compatible
Formats
Larry Klein, Ray Milburn, Cid
Praderas, and Abe Taaheri
Emergent Information Technologies,
Inc.
February 28, 2002
larry@eos.hitc.com

1
EOS Data Product Access
• Standard products are stored and distributed in
HDF-EOS format, based on HDF4.
• The format is self-describing and portable.
• The format was developed to provide a
convention for geo-locating data from
disparate instruments.
• This allowed product developers to use the
same data structures, limiting need to develop
access software.
• For example, EOS-AURA instruments will use the
same file structure.
• However…..
2
EOS Data Access Issues
• The format did not provide standards in detail.
For example:
– Products contain HDF as well as HDF-EOS
objects.
– There are different standards for fill data.
– MODIS products have little uniformity.
– Geolocation not uniform - ASTER uses
geocentric coordinates and geodetic
coordinates
– Geolocation data in swaths not always
available pixel by pixel.
3
Challenges for Data Access
• Demand for EOS data products is extending
beyond instrument development teams, eg.
GIS users. This implies less storage format
expertise.
• GIS applications don’t always like HDF-EOS.
• It is desirable to compare ASTER/MODIS/MISR
products with the same analysis tools.
• Users may desire area spanning more than a
single file. No generalized mosaicing software
is available.
• Common commercial tools (e.g. IDL, ENVI)
4
don’t accept all EOS products. They accept
‘generic’ HDF-EOS based products.
EOS Data Converter
Requirements
• Convert EOS ASTER, MODIS, MISR to GeoTIFF.
Initially, ~50 EOS land products selected.
• Make details of internal file structures
transparent to users.
• Convert HDF-EOS Swath to HDF-EOS Grid.
• Allow file selection from users local storage.
• Make functionality available through graphical
and command line interfaces.
• Integrate with ECS Data Pool servers

5
Data Converter Requirements
• Functionality
– Re-projection: USGS, MODIS Integerized
Sinusoidal, EASEGrid.
– Stitching (Mosaicing).
– Subsetting by band/parameter.
– Subsetting by geolocation.
– Metadata preservation/creation.
– Resampling.

6
Data Converter Requirements
• User Interface
– Portable, written in C and Java.
– Initially operable on Sun, SGI, Linux.
– Not dependent on COTS (eg. IDL).
• Development in Near Future
– Output Multiple-band GeoTIFFs
– 4-D data sets (e.g. MOD043)
– Additional geometric corrections
– MODIS Ocean/Atmosphere products
7
Re-Use of Tools
• Some requirements can be met by modifying and
integrating existing tools. For example:
– HDF-EOS swath/grid format conversion tool
prototype from GSFC. This was not adapted to
particular products and needed heavy
modification.
– Tools are being developed at EOS archive
centers, which are focused on classes of
products. For example, the MODIS Reprojection
Tool from USGS.
– Specialized tools, eg. removal of edge effects
(MODIS)
8
• Our approach was to develop what is missing and
Converter Tool Schematic

Users Local Storage
EOS data product (HDF-EOS

Java-Based GUI
Select input file:
• Single, multiple file
• Bounding box
Select output format
Select output
projection
Resampling algorithm
Underlying Functionality (C Code)
Generalized HDF-EOS conversion
Mosaic/Subset
Rasterize
Output GeoTiff, Binary, HDF-EOS
Re-Projection
Metadata preservation/creation

Users Local Storage
Output file

Application
ArcView
ERDAS
ENVI

9
User Interface: Conversion

10
User Interface: Stitch/Subset

11
MODIS L2 Surface Reflectance: Safari 2000

12
MODIS L2/ASTER L1B Surface Reflectances
Safari 2000 Campaign - NW Botswana

Lake Ngami

Town of Maun

Boteti River

13
One km. Digital Terrain Data
HDF-EOS
Grid Files

San Fransisco Area

14
One km. Digital Terrain Data

Subsetted

Four Tiles
Stitched
GeoTIFF File

15
ASTER Swath Stitching/Subsetting
Swath
Files

Grid
File

16
MISR L1B Mt. Etna Eruption

17
ASTER L1B Mt. Etna - Stitched

18

Weitere ähnliche Inhalte

Ähnlich wie Generalized Conversion of HDF-EOS Products to GIS-Compatible Formats

GIS Standards and Interoperability
GIS Standards and InteroperabilityGIS Standards and Interoperability
GIS Standards and Interoperability
Nasr Khashoggi
 

Ähnlich wie Generalized Conversion of HDF-EOS Products to GIS-Compatible Formats (20)

HDF-EOS APIs, tools, etc.
HDF-EOS APIs, tools, etc.HDF-EOS APIs, tools, etc.
HDF-EOS APIs, tools, etc.
 
HDF-EOS Tools
HDF-EOS ToolsHDF-EOS Tools
HDF-EOS Tools
 
SEEDS Standards Process
SEEDS Standards ProcessSEEDS Standards Process
SEEDS Standards Process
 
HDF-EOS Development: Current Status and Tools
HDF-EOS Development: Current Status and ToolsHDF-EOS Development: Current Status and Tools
HDF-EOS Development: Current Status and Tools
 
ESDIS Status (2002)
ESDIS Status (2002)ESDIS Status (2002)
ESDIS Status (2002)
 
MODIS Land and HDF-EOS
MODIS Land and HDF-EOSMODIS Land and HDF-EOS
MODIS Land and HDF-EOS
 
Generalized EOS Data Converter: Making Data Products Accessible to GIS Tools
Generalized EOS Data Converter: Making Data Products Accessible to GIS ToolsGeneralized EOS Data Converter: Making Data Products Accessible to GIS Tools
Generalized EOS Data Converter: Making Data Products Accessible to GIS Tools
 
HDF-EOS Maintenance, Current Development and Tools
HDF-EOS Maintenance, Current Development and ToolsHDF-EOS Maintenance, Current Development and Tools
HDF-EOS Maintenance, Current Development and Tools
 
Metadata in EOSDIS
Metadata in EOSDISMetadata in EOSDIS
Metadata in EOSDIS
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
Hdg geo discussion
Hdg geo discussionHdg geo discussion
Hdg geo discussion
 
GIS Standards and Interoperability
GIS Standards and InteroperabilityGIS Standards and Interoperability
GIS Standards and Interoperability
 
ICESat-2 H5-ES Product Development Strategy
ICESat-2 H5-ES Product Development StrategyICESat-2 H5-ES Product Development Strategy
ICESat-2 H5-ES Product Development Strategy
 
HDF and netCDF Data Support in ArcGIS
HDF and netCDF Data Support in ArcGISHDF and netCDF Data Support in ArcGIS
HDF and netCDF Data Support in ArcGIS
 
HDF-EOS Development - Current Status and Schedule
HDF-EOS Development - Current Status and ScheduleHDF-EOS Development - Current Status and Schedule
HDF-EOS Development - Current Status and Schedule
 
HDF-EOS Development Current Status
HDF-EOS Development Current StatusHDF-EOS Development Current Status
HDF-EOS Development Current Status
 
HDF Lessons from NPOESS & Future Opportunities
HDF Lessons from NPOESS  & Future OpportunitiesHDF Lessons from NPOESS  & Future Opportunities
HDF Lessons from NPOESS & Future Opportunities
 
HDF-EOS Status and Developments
HDF-EOS Status and DevelopmentsHDF-EOS Status and Developments
HDF-EOS Status and Developments
 
Development of a Prototype Web GIS Server for HDF-EOS Data based on OGC Web M...
Development of a Prototype Web GIS Server for HDF-EOS Data based on OGC Web M...Development of a Prototype Web GIS Server for HDF-EOS Data based on OGC Web M...
Development of a Prototype Web GIS Server for HDF-EOS Data based on OGC Web M...
 

Mehr von The HDF-EOS Tools and Information Center

Mehr von The HDF-EOS Tools and Information Center (20)

Cloud-Optimized HDF5 Files
Cloud-Optimized HDF5 FilesCloud-Optimized HDF5 Files
Cloud-Optimized HDF5 Files
 
Accessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDSAccessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDS
 
The State of HDF
The State of HDFThe State of HDF
The State of HDF
 
Highly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance FeaturesHighly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance Features
 
Creating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 FilesCreating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 Files
 
HDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance DiscussionHDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance Discussion
 
Hyrax: Serving Data from S3
Hyrax: Serving Data from S3Hyrax: Serving Data from S3
Hyrax: Serving Data from S3
 
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLABAccessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
 
HDF - Current status and Future Directions
HDF - Current status and Future DirectionsHDF - Current status and Future Directions
HDF - Current status and Future Directions
 
HDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and FutureHDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and Future
 
HDF - Current status and Future Directions
HDF - Current status and Future Directions HDF - Current status and Future Directions
HDF - Current status and Future Directions
 
H5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only LibraryH5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only Library
 
MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10
 
HDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDFHDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDF
 
HDF5 <-> Zarr
HDF5 <-> ZarrHDF5 <-> Zarr
HDF5 <-> Zarr
 
HDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server FeaturesHDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server Features
 
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
 
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
 
HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?
 
HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020
 

Kürzlich hochgeladen

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 

Generalized Conversion of HDF-EOS Products to GIS-Compatible Formats

  • 1. Generalized Conversion of HDFEOS Products to GIS Compatible Formats Larry Klein, Ray Milburn, Cid Praderas, and Abe Taaheri Emergent Information Technologies, Inc. February 28, 2002 larry@eos.hitc.com 1
  • 2. EOS Data Product Access • Standard products are stored and distributed in HDF-EOS format, based on HDF4. • The format is self-describing and portable. • The format was developed to provide a convention for geo-locating data from disparate instruments. • This allowed product developers to use the same data structures, limiting need to develop access software. • For example, EOS-AURA instruments will use the same file structure. • However….. 2
  • 3. EOS Data Access Issues • The format did not provide standards in detail. For example: – Products contain HDF as well as HDF-EOS objects. – There are different standards for fill data. – MODIS products have little uniformity. – Geolocation not uniform - ASTER uses geocentric coordinates and geodetic coordinates – Geolocation data in swaths not always available pixel by pixel. 3
  • 4. Challenges for Data Access • Demand for EOS data products is extending beyond instrument development teams, eg. GIS users. This implies less storage format expertise. • GIS applications don’t always like HDF-EOS. • It is desirable to compare ASTER/MODIS/MISR products with the same analysis tools. • Users may desire area spanning more than a single file. No generalized mosaicing software is available. • Common commercial tools (e.g. IDL, ENVI) 4 don’t accept all EOS products. They accept ‘generic’ HDF-EOS based products.
  • 5. EOS Data Converter Requirements • Convert EOS ASTER, MODIS, MISR to GeoTIFF. Initially, ~50 EOS land products selected. • Make details of internal file structures transparent to users. • Convert HDF-EOS Swath to HDF-EOS Grid. • Allow file selection from users local storage. • Make functionality available through graphical and command line interfaces. • Integrate with ECS Data Pool servers 5
  • 6. Data Converter Requirements • Functionality – Re-projection: USGS, MODIS Integerized Sinusoidal, EASEGrid. – Stitching (Mosaicing). – Subsetting by band/parameter. – Subsetting by geolocation. – Metadata preservation/creation. – Resampling. 6
  • 7. Data Converter Requirements • User Interface – Portable, written in C and Java. – Initially operable on Sun, SGI, Linux. – Not dependent on COTS (eg. IDL). • Development in Near Future – Output Multiple-band GeoTIFFs – 4-D data sets (e.g. MOD043) – Additional geometric corrections – MODIS Ocean/Atmosphere products 7
  • 8. Re-Use of Tools • Some requirements can be met by modifying and integrating existing tools. For example: – HDF-EOS swath/grid format conversion tool prototype from GSFC. This was not adapted to particular products and needed heavy modification. – Tools are being developed at EOS archive centers, which are focused on classes of products. For example, the MODIS Reprojection Tool from USGS. – Specialized tools, eg. removal of edge effects (MODIS) 8 • Our approach was to develop what is missing and
  • 9. Converter Tool Schematic Users Local Storage EOS data product (HDF-EOS Java-Based GUI Select input file: • Single, multiple file • Bounding box Select output format Select output projection Resampling algorithm Underlying Functionality (C Code) Generalized HDF-EOS conversion Mosaic/Subset Rasterize Output GeoTiff, Binary, HDF-EOS Re-Projection Metadata preservation/creation Users Local Storage Output file Application ArcView ERDAS ENVI 9
  • 12. MODIS L2 Surface Reflectance: Safari 2000 12
  • 13. MODIS L2/ASTER L1B Surface Reflectances Safari 2000 Campaign - NW Botswana Lake Ngami Town of Maun Boteti River 13
  • 14. One km. Digital Terrain Data HDF-EOS Grid Files San Fransisco Area 14
  • 15. One km. Digital Terrain Data Subsetted Four Tiles Stitched GeoTIFF File 15
  • 17. MISR L1B Mt. Etna Eruption 17
  • 18. ASTER L1B Mt. Etna - Stitched 18