SlideShare a Scribd company logo
1 of 21
An Overview of
HDF-EOS
(Part I)

Doug Ilg
Raytheon STX
Doug.Ilg@gsfc.nasa.gov
(301) 441-4089
1
Outline
What is HDF-EOS?
The Grid Interface
The Point Interface

2
What is HDF-EOS?
An HDF “Profile”
An extension to HDF
A library built “on top” of HDF
Three new data objects
Three new programming interfaces

3
Why HDF-EOS?
Standard HDF lacks well defined ways of
handling some key needs of EOSDIS
Data structures for Earth remote
sensing data and in-situ measurements
with:
– tightly coupled geolocation information
– subsetting services based on geolocation

ECS metadata model

4
HDF-EOS Platforms
HDF-EOS Version 2.3 is available for:
Sun SPARC - Solaris
SGI - IRIX
DEC Alpha - Digital UNIX
HP 9000 - HP-UX
IBM RS/6000 - AIX
PC - Windows 95/NT
5
HDF-EOS Interfaces
C and FORTRAN Interfaces for:
Grid Data (GD)
Point Data (PT)
Swath Data (SW)

6
HDF-EOS Programming
Model
Writing
–
–
–
–
–
–
–
–

open file
create object
define structure
detach object*
attach object*
write data
detach object
close file

Reading
–
–
–
–
–
–

open file
attach object
inquire object
read data
detach object
close file

7
A Grid Data Set

8
A Grid Structure
Xdim
Size: 2000

Projinfo
Ydim
Size: 800

9
Projections Supported
Geographic
Transverse Mercator
Universal
Transverse Mercator
Hotine Oblique
Mercator
Space Oblique
Mercator
Polar Stereographic

Lambert Azimuthal
Equal Area
Lambert Conformal
Conic
Polyconic
Interrupted Goode’s
Homolosine
Integerized
Sinusoidal
10
Components of the Grid
Interface
Access
Definition
Basic I/O
Inquiry
Subset
Tiling
11
Tips on Writing a Grid
Order of calls is significant:
– Setting a compression method affects all
subsequently defined fields
– Setting a tiling scheme affects all
subsequently defined fields

12
Grid Subsetting Features
By Geolocation
– GDdefboxregion/Gdextractboxregion

By “Vertical” Field
– GDdefvrtregion/GDextractvrtregion

By Time (special case of vertical)
Tip: use Geolocation, then Vertical/
Temporal
13
Compression Methods for
Grids
Run-Length Encoding
Adaptive Huffman
Gzip

14
A Point Data Set
Lat
61.12
45.31
38.50
38.39
30.00
37.45
18.00
43.40
34.03
32.45
33.30
42.15
35.05
34.12
46.32
47.36
39.44
21.25
44.58
41.49
25.45

Lon Temp(C) Dewpt(C)
-149.48 15.00 5.00
-122.41 17.00 5.00
-77.00 24.00 7.00
-90.15 27.00 11.00
-90.05 22.00 7.00
-122.26 25.00 10.00
-76.45 27.00 4.00
-79.23 30.00 14.00
-118.14 25.00 4.00
-96.48 32.00 8.00
-112.00 30.00 10.00
-71.07 28.00 7.00
-106.40 30.00 9.00
-77.56 28.00 9.00
-87.25 30.00 8.00
-122.20 32.00 15.00
-104.59 31.00 16.00
-78.00 28.00 7.00
-93.15 32.00 13.00
-87.37 28.00 9.00
-80.11 19.00 3.00

15
A Point Structure
Lat
Long
Buoy ID
25.2645 091.2564
0126
22.3549 -93.4657
3564
23.2564 -89.2546
1256

Buoy ID
0126
0126
3564
1256
1256
0126
3564

Time Wave Height(ft) Temp(C)
01:26
2.54
18.4
05:56
3.58
18.2
06:28
12.64
16.4
08:12
7.58
17.1
09:58
7.76
17.2
09:59
4.23
20.1
10:16
10.23
17.5

16
The Point Interface
Access
Definition
Basic I/O
Inquiry
Subset

17
Tips on Writing a Point
Every level in a Point data set must be
linked into the hierarchy.
Before two levels can be linked, a link
field must exist.

18
Point Subsetting Features
By Time
– PTdeftimeperiod/PTextractperiod

By Geolocation
– PTdefboxregion/PTextractregion

Tip: use one or the other, not both

19
Compression Methods for
Points
NONE

20
Tips for HDF-EOS Coding
Most operations (read, write, subset)
work on a single field at a time.
Region IDs and Period IDs are interchangeable and can be reused to
further reduce a subset.
Partial writes (appending) on
compressed fields are only supported
through tiling.
21

More Related Content

What's hot

Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA SystemFast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Yuichiro Yasui
 
WSE 6A-Octo-X Terrain Mapping UAV
WSE 6A-Octo-X Terrain Mapping UAVWSE 6A-Octo-X Terrain Mapping UAV
WSE 6A-Octo-X Terrain Mapping UAV
Manuel De La Cruz
 
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
Johan Andersson
 

What's hot (20)

Real-Time Visual Simulation of Smoke
Real-Time Visual Simulation of SmokeReal-Time Visual Simulation of Smoke
Real-Time Visual Simulation of Smoke
 
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA SystemFast & Energy-Efficient Breadth-First Search on a Single NUMA System
Fast & Energy-Efficient Breadth-First Search on a Single NUMA System
 
Neighbourhood Preserving Quantisation for LSH SIGIR Poster
Neighbourhood Preserving Quantisation for LSH SIGIR PosterNeighbourhood Preserving Quantisation for LSH SIGIR Poster
Neighbourhood Preserving Quantisation for LSH SIGIR Poster
 
Advancements in-tiled-rendering
Advancements in-tiled-renderingAdvancements in-tiled-rendering
Advancements in-tiled-rendering
 
GDC16: Improving geometry culling for Deus Ex: Mankind Divided by Nicolas Trudel
GDC16: Improving geometry culling for Deus Ex: Mankind Divided by Nicolas TrudelGDC16: Improving geometry culling for Deus Ex: Mankind Divided by Nicolas Trudel
GDC16: Improving geometry culling for Deus Ex: Mankind Divided by Nicolas Trudel
 
NUMA-aware thread-parallel breadth-first search for Graph500 and Green Graph5...
NUMA-aware thread-parallel breadth-first search for Graph500 and Green Graph5...NUMA-aware thread-parallel breadth-first search for Graph500 and Green Graph5...
NUMA-aware thread-parallel breadth-first search for Graph500 and Green Graph5...
 
LHCb Computing Workshop 2018: PV finding with CNNs
LHCb Computing Workshop 2018: PV finding with CNNsLHCb Computing Workshop 2018: PV finding with CNNs
LHCb Computing Workshop 2018: PV finding with CNNs
 
ARPS Architecture 1
ARPS Architecture 1ARPS Architecture 1
ARPS Architecture 1
 
Performance Analysis with Scalasca, part II
Performance Analysis with Scalasca, part IIPerformance Analysis with Scalasca, part II
Performance Analysis with Scalasca, part II
 
30th コンピュータビジョン勉強会@関東 DynamicFusion
30th コンピュータビジョン勉強会@関東 DynamicFusion30th コンピュータビジョン勉強会@関東 DynamicFusion
30th コンピュータビジョン勉強会@関東 DynamicFusion
 
BEFLIX
BEFLIXBEFLIX
BEFLIX
 
Logistic Regression in R-An Exmple.
Logistic Regression in R-An Exmple. Logistic Regression in R-An Exmple.
Logistic Regression in R-An Exmple.
 
ARPS Architecture
ARPS ArchitectureARPS Architecture
ARPS Architecture
 
WSE 6A-Octo-X Terrain Mapping UAV
WSE 6A-Octo-X Terrain Mapping UAVWSE 6A-Octo-X Terrain Mapping UAV
WSE 6A-Octo-X Terrain Mapping UAV
 
論文紹介"DynamicFusion: Reconstruction and Tracking of Non-­‐rigid Scenes in Real...
論文紹介"DynamicFusion: Reconstruction and Tracking of Non-­‐rigid Scenes in Real...論文紹介"DynamicFusion: Reconstruction and Tracking of Non-­‐rigid Scenes in Real...
論文紹介"DynamicFusion: Reconstruction and Tracking of Non-­‐rigid Scenes in Real...
 
P1341cle
P1341cleP1341cle
P1341cle
 
Archaeological Surveying in the Middle East
Archaeological Surveying in the Middle EastArchaeological Surveying in the Middle East
Archaeological Surveying in the Middle East
 
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
The Intersection of Game Engines & GPUs: Current & Future (Graphics Hardware ...
 
Graph Regularised Hashing
Graph Regularised HashingGraph Regularised Hashing
Graph Regularised Hashing
 
Coq for ML users
Coq for ML usersCoq for ML users
Coq for ML users
 

Viewers also liked

Viewers also liked (20)

Metadata Requirements for EOSDIS Data Providers
Metadata Requirements for EOSDIS Data ProvidersMetadata Requirements for EOSDIS Data Providers
Metadata Requirements for EOSDIS Data Providers
 
HDF Project Update
HDF Project UpdateHDF Project Update
HDF Project Update
 
IBM Visualization Data Explorer
IBM Visualization Data ExplorerIBM Visualization Data Explorer
IBM Visualization Data Explorer
 
HDF Explorer
HDF ExplorerHDF Explorer
HDF Explorer
 
HDF Server
HDF ServerHDF Server
HDF Server
 
HDF-EOS Development Status and Maintenance Support
HDF-EOS Development Status and Maintenance SupportHDF-EOS Development Status and Maintenance Support
HDF-EOS Development Status and Maintenance Support
 
MrSID
MrSIDMrSID
MrSID
 
HDF And HDF-EOS Tools
HDF And HDF-EOS ToolsHDF And HDF-EOS Tools
HDF And HDF-EOS Tools
 
An Introduction to HDF (1997)
An Introduction to HDF (1997)An Introduction to HDF (1997)
An Introduction to HDF (1997)
 
MATLAB and HDF-EOS
MATLAB and HDF-EOSMATLAB and HDF-EOS
MATLAB and HDF-EOS
 
Dataset Independent Subsetting
Dataset Independent SubsettingDataset Independent Subsetting
Dataset Independent Subsetting
 
Current HDF Tools (1997)
Current HDF Tools (1997)Current HDF Tools (1997)
Current HDF Tools (1997)
 
EOS Overview
EOS OverviewEOS Overview
EOS Overview
 
Incorporating ISO Metadata Using HDF Product Designer
Incorporating ISO Metadata Using HDF Product DesignerIncorporating ISO Metadata Using HDF Product Designer
Incorporating ISO Metadata Using HDF Product Designer
 
PCMDI Software System
PCMDI Software SystemPCMDI Software System
PCMDI Software System
 
Desktop Support for HDF and HDF-EOS
Desktop Support for HDF and HDF-EOSDesktop Support for HDF and HDF-EOS
Desktop Support for HDF and HDF-EOS
 
Indexing HDF5: A Survey
Indexing HDF5: A SurveyIndexing HDF5: A Survey
Indexing HDF5: A Survey
 
America Runs on Excel and HDF5 - Glued together by Python
America Runs on Excel and HDF5 - Glued together by PythonAmerica Runs on Excel and HDF5 - Glued together by Python
America Runs on Excel and HDF5 - Glued together by Python
 
His Expert's Voice
His Expert's VoiceHis Expert's Voice
His Expert's Voice
 
HDF
HDFHDF
HDF
 

Similar to An Overview of HDF-EOS (Part 1)

Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
Naoki (Neo) SATO
 
BWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 PresentationBWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 Presentation
lilyco
 
D3 D10 Unleashed New Features And Effects
D3 D10 Unleashed   New Features And EffectsD3 D10 Unleashed   New Features And Effects
D3 D10 Unleashed New Features And Effects
Thomas Goddard
 
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
Rudolf Husar
 

Similar to An Overview of HDF-EOS (Part 1) (20)

The HDF-EOS5 Tutorial
The HDF-EOS5 TutorialThe HDF-EOS5 Tutorial
The HDF-EOS5 Tutorial
 
HDF-EOS Overview and Status
HDF-EOS Overview and StatusHDF-EOS Overview and Status
HDF-EOS Overview and Status
 
Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
Deep Learning, Microsoft Cognitive Toolkit (CNTK) and Azure Machine Learning ...
 
EECSCon Poster
EECSCon PosterEECSCon Poster
EECSCon Poster
 
Dpdk applications
Dpdk applicationsDpdk applications
Dpdk applications
 
BWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 PresentationBWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 Presentation
 
Your Game Needs Direct3D 11, So Get Started Now!
Your Game Needs Direct3D 11, So Get Started Now!Your Game Needs Direct3D 11, So Get Started Now!
Your Game Needs Direct3D 11, So Get Started Now!
 
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdfS51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
 
D3 D10 Unleashed New Features And Effects
D3 D10 Unleashed   New Features And EffectsD3 D10 Unleashed   New Features And Effects
D3 D10 Unleashed New Features And Effects
 
Status of HDF-EOS, Related Software and Tools
Status of HDF-EOS, Related Software and ToolsStatus of HDF-EOS, Related Software and Tools
Status of HDF-EOS, Related Software and Tools
 
060128 Galeon Rept
060128 Galeon Rept060128 Galeon Rept
060128 Galeon Rept
 
Achitecture Aware Algorithms and Software for Peta and Exascale
Achitecture Aware Algorithms and Software for Peta and ExascaleAchitecture Aware Algorithms and Software for Peta and Exascale
Achitecture Aware Algorithms and Software for Peta and Exascale
 
RAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data ScienceRAPIDS – Open GPU-accelerated Data Science
RAPIDS – Open GPU-accelerated Data Science
 
DAW: Duplicate-AWare Federated Query Processing over the Web of Data
DAW: Duplicate-AWare Federated Query Processing over the Web of DataDAW: Duplicate-AWare Federated Query Processing over the Web of Data
DAW: Duplicate-AWare Federated Query Processing over the Web of Data
 
Location based services for Nokia X and Nokia Asha using Geo2tag
Location based services for Nokia X and Nokia Asha using Geo2tagLocation based services for Nokia X and Nokia Asha using Geo2tag
Location based services for Nokia X and Nokia Asha using Geo2tag
 
State of the Art Web Mapping with Open Source
State of the Art Web Mapping with Open SourceState of the Art Web Mapping with Open Source
State of the Art Web Mapping with Open Source
 
Klessydra t - designing vector coprocessors for multi-threaded edge-computing...
Klessydra t - designing vector coprocessors for multi-threaded edge-computing...Klessydra t - designing vector coprocessors for multi-threaded edge-computing...
Klessydra t - designing vector coprocessors for multi-threaded edge-computing...
 
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
2006-01-11 Data Flow & Interoperability in DataFed Service-based AQ Analysis ...
 
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
0603 Esip Fed Wash Dc Tech Pres 060103 Esip Aq Tech Track
 
Presentation NBMP and PCC
Presentation NBMP and PCCPresentation NBMP and PCC
Presentation NBMP and PCC
 

More from The HDF-EOS Tools and Information Center

More from The HDF-EOS Tools and Information Center (20)

Cloud-Optimized HDF5 Files
Cloud-Optimized HDF5 FilesCloud-Optimized HDF5 Files
Cloud-Optimized HDF5 Files
 
Accessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDSAccessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDS
 
The State of HDF
The State of HDFThe State of HDF
The State of HDF
 
Highly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance FeaturesHighly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance Features
 
Creating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 FilesCreating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 Files
 
HDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance DiscussionHDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance Discussion
 
Hyrax: Serving Data from S3
Hyrax: Serving Data from S3Hyrax: Serving Data from S3
Hyrax: Serving Data from S3
 
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLABAccessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
 
HDF - Current status and Future Directions
HDF - Current status and Future DirectionsHDF - Current status and Future Directions
HDF - Current status and Future Directions
 
HDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and FutureHDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and Future
 
HDF - Current status and Future Directions
HDF - Current status and Future Directions HDF - Current status and Future Directions
HDF - Current status and Future Directions
 
H5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only LibraryH5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only Library
 
MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10
 
HDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDFHDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDF
 
HDF5 <-> Zarr
HDF5 <-> ZarrHDF5 <-> Zarr
HDF5 <-> Zarr
 
HDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server FeaturesHDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server Features
 
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
 
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
 
HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?
 
HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Your enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4jYour enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4j
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 

An Overview of HDF-EOS (Part 1)

  • 1. An Overview of HDF-EOS (Part I) Doug Ilg Raytheon STX Doug.Ilg@gsfc.nasa.gov (301) 441-4089 1
  • 2. Outline What is HDF-EOS? The Grid Interface The Point Interface 2
  • 3. What is HDF-EOS? An HDF “Profile” An extension to HDF A library built “on top” of HDF Three new data objects Three new programming interfaces 3
  • 4. Why HDF-EOS? Standard HDF lacks well defined ways of handling some key needs of EOSDIS Data structures for Earth remote sensing data and in-situ measurements with: – tightly coupled geolocation information – subsetting services based on geolocation ECS metadata model 4
  • 5. HDF-EOS Platforms HDF-EOS Version 2.3 is available for: Sun SPARC - Solaris SGI - IRIX DEC Alpha - Digital UNIX HP 9000 - HP-UX IBM RS/6000 - AIX PC - Windows 95/NT 5
  • 6. HDF-EOS Interfaces C and FORTRAN Interfaces for: Grid Data (GD) Point Data (PT) Swath Data (SW) 6
  • 7. HDF-EOS Programming Model Writing – – – – – – – – open file create object define structure detach object* attach object* write data detach object close file Reading – – – – – – open file attach object inquire object read data detach object close file 7
  • 8. A Grid Data Set 8
  • 9. A Grid Structure Xdim Size: 2000 Projinfo Ydim Size: 800 9
  • 10. Projections Supported Geographic Transverse Mercator Universal Transverse Mercator Hotine Oblique Mercator Space Oblique Mercator Polar Stereographic Lambert Azimuthal Equal Area Lambert Conformal Conic Polyconic Interrupted Goode’s Homolosine Integerized Sinusoidal 10
  • 11. Components of the Grid Interface Access Definition Basic I/O Inquiry Subset Tiling 11
  • 12. Tips on Writing a Grid Order of calls is significant: – Setting a compression method affects all subsequently defined fields – Setting a tiling scheme affects all subsequently defined fields 12
  • 13. Grid Subsetting Features By Geolocation – GDdefboxregion/Gdextractboxregion By “Vertical” Field – GDdefvrtregion/GDextractvrtregion By Time (special case of vertical) Tip: use Geolocation, then Vertical/ Temporal 13
  • 14. Compression Methods for Grids Run-Length Encoding Adaptive Huffman Gzip 14
  • 15. A Point Data Set Lat 61.12 45.31 38.50 38.39 30.00 37.45 18.00 43.40 34.03 32.45 33.30 42.15 35.05 34.12 46.32 47.36 39.44 21.25 44.58 41.49 25.45 Lon Temp(C) Dewpt(C) -149.48 15.00 5.00 -122.41 17.00 5.00 -77.00 24.00 7.00 -90.15 27.00 11.00 -90.05 22.00 7.00 -122.26 25.00 10.00 -76.45 27.00 4.00 -79.23 30.00 14.00 -118.14 25.00 4.00 -96.48 32.00 8.00 -112.00 30.00 10.00 -71.07 28.00 7.00 -106.40 30.00 9.00 -77.56 28.00 9.00 -87.25 30.00 8.00 -122.20 32.00 15.00 -104.59 31.00 16.00 -78.00 28.00 7.00 -93.15 32.00 13.00 -87.37 28.00 9.00 -80.11 19.00 3.00 15
  • 16. A Point Structure Lat Long Buoy ID 25.2645 091.2564 0126 22.3549 -93.4657 3564 23.2564 -89.2546 1256 Buoy ID 0126 0126 3564 1256 1256 0126 3564 Time Wave Height(ft) Temp(C) 01:26 2.54 18.4 05:56 3.58 18.2 06:28 12.64 16.4 08:12 7.58 17.1 09:58 7.76 17.2 09:59 4.23 20.1 10:16 10.23 17.5 16
  • 18. Tips on Writing a Point Every level in a Point data set must be linked into the hierarchy. Before two levels can be linked, a link field must exist. 18
  • 19. Point Subsetting Features By Time – PTdeftimeperiod/PTextractperiod By Geolocation – PTdefboxregion/PTextractregion Tip: use one or the other, not both 19
  • 21. Tips for HDF-EOS Coding Most operations (read, write, subset) work on a single field at a time. Region IDs and Period IDs are interchangeable and can be reused to further reduce a subset. Partial writes (appending) on compressed fields are only supported through tiling. 21