SlideShare ist ein Scribd-Unternehmen logo
1 von 7
HDF-EOS Datablade:
Efficiently Serving Earth Science Data

http://heineken.gsfc.nasa.gov/hdfeos/
What is a Datablade?
Image
HDF-EOS

Spatial

DataBlade API

Text

Web

Extending the server by
adding DataBlades: new Types
and new functions

Extensible
OR Engine

Scaleable
Data
Manager

INFORMIX-Universal Server

http://heineken.gsfc.nasa.gov/hdfeos/
What does the HDF-EOS Datablade
provide?
• Inventory management of large volume data and
metadata
• Flexible access to the data as a file or BinaryLarge-Object inside the database.
• Cross-granule searching and subsetting
• Re-use other datablades such as spatial search
tools from Informix
• Implements Grid subsetting operations in the
Database
http://heineken.gsfc.nasa.gov/hdfeos/
Plans for the Second Year
•
•
•
•

Implement Swath, Point, HDF datatypes
General subsetting tool: stored queries
Metadata search and query tools
Implement MISR Quality-Assessment database

http://heineken.gsfc.nasa.gov/hdfeos/
HDF-EOS Datablade Software Architecture
Informix DB

SQL
SQL
InterInterface
face
and
API

HDF-EOS
C library
(Unchanged)

Valuable
Data
Exposed
in Searchable
Tables

File
I/O
Emulator

http://heineken.gsfc.nasa.gov/hdfeos/

Granules
in
File
System

Granules
in
Informix
Overlapping Granule Scenario

Latitude

90
80
70
60
50
40
30
20
10

Subset: Lat: 65 to 75
Lon: 15 to 25

0
1
2
3
4
10

30

50

70

90

Longitude

http://heineken.gsfc.nasa.gov/hdfeos/
Overlapping Granule Scenario

Latitude

90
80
70
60
50
40
30
20
10

Subset: Lat: 65 to 75
Lon: 15 to 25

0
1
2
3
4
10

30

50

70

90

Longitude

http://heineken.gsfc.nasa.gov/hdfeos/

Weitere ähnliche Inhalte

Was ist angesagt?

Media iQ fifth elephant teaser
Media iQ   fifth elephant teaser Media iQ   fifth elephant teaser
Media iQ fifth elephant teaser
prabhuprakash
 

Was ist angesagt? (20)

view_hdf
view_hdfview_hdf
view_hdf
 
PoolParty Search Server
PoolParty Search ServerPoolParty Search Server
PoolParty Search Server
 
MongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and VisualizationMongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and Visualization
 
HDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the CloudHDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the Cloud
 
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
 
Improved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the MassesImproved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the Masses
 
Scientific Computing and Visualization using HDF
Scientific Computing and Visualization using HDFScientific Computing and Visualization using HDF
Scientific Computing and Visualization using HDF
 
041018 Esds Poster
041018 Esds Poster041018 Esds Poster
041018 Esds Poster
 
Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!
 
Hadoop 2 cluster architecture
Hadoop 2 cluster architectureHadoop 2 cluster architecture
Hadoop 2 cluster architecture
 
Media iQ fifth elephant teaser
Media iQ   fifth elephant teaser Media iQ   fifth elephant teaser
Media iQ fifth elephant teaser
 
Mapreduce Tutorial
Mapreduce TutorialMapreduce Tutorial
Mapreduce Tutorial
 
NASA Terra Data Fusion
NASA Terra Data FusionNASA Terra Data Fusion
NASA Terra Data Fusion
 
Hadoop 1 vs hadoop2
Hadoop 1 vs hadoop2Hadoop 1 vs hadoop2
Hadoop 1 vs hadoop2
 
VINEYARD Overview - ARC 2016
VINEYARD Overview - ARC 2016VINEYARD Overview - ARC 2016
VINEYARD Overview - ARC 2016
 
Data Center Operating System
Data Center Operating SystemData Center Operating System
Data Center Operating System
 
SPD and KEA: HDF5 based file formats for Earth Observation
SPD and KEA: HDF5 based file formats for Earth ObservationSPD and KEA: HDF5 based file formats for Earth Observation
SPD and KEA: HDF5 based file formats for Earth Observation
 
Practical Use of a NoSQL Database
Practical Use of a NoSQL DatabasePractical Use of a NoSQL Database
Practical Use of a NoSQL Database
 
Cloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management SystemCloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management System
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineageObservability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineage
 

Andere mochten auch

Marketing Research - Heineken
Marketing Research - HeinekenMarketing Research - Heineken
Marketing Research - Heineken
Khoa Fanda
 
Heineken summary
Heineken summaryHeineken summary
Heineken summary
JCDecauxUK
 
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
Tom De Ruyck
 
Marketing 360 presentation by lawrence villegas - Connections Media examples...
Marketing 360 presentation by lawrence villegas -  Connections Media examples...Marketing 360 presentation by lawrence villegas -  Connections Media examples...
Marketing 360 presentation by lawrence villegas - Connections Media examples...
Lawrence Villegas
 
Heineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tvHeineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tv
Mitya Voskresensky
 
Heineken presentation bingül&eda
Heineken   presentation bingül&edaHeineken   presentation bingül&eda
Heineken presentation bingül&eda
Yalcin Bingul
 

Andere mochten auch (20)

Beer Facts
Beer FactsBeer Facts
Beer Facts
 
Marketing Research - Heineken
Marketing Research - HeinekenMarketing Research - Heineken
Marketing Research - Heineken
 
Heineken summary
Heineken summaryHeineken summary
Heineken summary
 
People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2
 
Carlsberg: Student Trends
Carlsberg: Student TrendsCarlsberg: Student Trends
Carlsberg: Student Trends
 
Best cases time
Best cases timeBest cases time
Best cases time
 
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
 
WFA Annual Report 2013
WFA Annual Report 2013WFA Annual Report 2013
WFA Annual Report 2013
 
Ethnic Marketing
Ethnic MarketingEthnic Marketing
Ethnic Marketing
 
The Best Intranets in Europe
The Best Intranets in EuropeThe Best Intranets in Europe
The Best Intranets in Europe
 
Heineken Experience
Heineken ExperienceHeineken Experience
Heineken Experience
 
Marketing 360 presentation by lawrence villegas - Connections Media examples...
Marketing 360 presentation by lawrence villegas -  Connections Media examples...Marketing 360 presentation by lawrence villegas -  Connections Media examples...
Marketing 360 presentation by lawrence villegas - Connections Media examples...
 
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...
 
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
 
Heineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tvHeineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tv
 
Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...
 
QR Codes: A Point of View
QR Codes: A Point of ViewQR Codes: A Point of View
QR Codes: A Point of View
 
Heineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers AnalysisHeineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers Analysis
 
Heineken presentation bingül&eda
Heineken   presentation bingül&edaHeineken   presentation bingül&eda
Heineken presentation bingül&eda
 
Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International
 

Ähnlich wie HDF-EOS Datablade: Efficiently Serving Earth Science Data

Hw09 Rethinking The Data Warehouse With Hadoop And Hive
Hw09   Rethinking The Data Warehouse With Hadoop And HiveHw09   Rethinking The Data Warehouse With Hadoop And Hive
Hw09 Rethinking The Data Warehouse With Hadoop And Hive
Cloudera, Inc.
 
Hadoop Administration pdf
Hadoop Administration pdfHadoop Administration pdf
Hadoop Administration pdf
Edureka!
 
Research on vector spatial data storage scheme based
Research on vector spatial data storage scheme basedResearch on vector spatial data storage scheme based
Research on vector spatial data storage scheme based
Anant Kumar
 

Ähnlich wie HDF-EOS Datablade: Efficiently Serving Earth Science Data (20)

Hw09 Rethinking The Data Warehouse With Hadoop And Hive
Hw09   Rethinking The Data Warehouse With Hadoop And HiveHw09   Rethinking The Data Warehouse With Hadoop And Hive
Hw09 Rethinking The Data Warehouse With Hadoop And Hive
 
Modernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APSModernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APS
 
What is hadoop
What is hadoopWhat is hadoop
What is hadoop
 
Hadoop.pptx
Hadoop.pptxHadoop.pptx
Hadoop.pptx
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
Testing Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopTesting Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of Hadoop
 
hadoop resume
hadoop resumehadoop resume
hadoop resume
 
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data PlatformLessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
 
Hadoop Administration pdf
Hadoop Administration pdfHadoop Administration pdf
Hadoop Administration pdf
 
Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)
 
hadoop exp
hadoop exphadoop exp
hadoop exp
 
M. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptxM. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptx
 
Research on vector spatial data storage scheme based
Research on vector spatial data storage scheme basedResearch on vector spatial data storage scheme based
Research on vector spatial data storage scheme based
 
Big data applications
Big data applicationsBig data applications
Big data applications
 
Hadoop jon
Hadoop jonHadoop jon
Hadoop jon
 
M017619194
M017619194M017619194
M017619194
 
Hadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & RetrievalHadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & Retrieval
 
Robin_Hadoop
Robin_HadoopRobin_Hadoop
Robin_Hadoop
 
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformModernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
 
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
 

Mehr von The HDF-EOS Tools and Information Center

Mehr von The HDF-EOS Tools and Information Center (20)

Cloud-Optimized HDF5 Files
Cloud-Optimized HDF5 FilesCloud-Optimized HDF5 Files
Cloud-Optimized HDF5 Files
 
Accessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDSAccessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDS
 
The State of HDF
The State of HDFThe State of HDF
The State of HDF
 
Highly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance FeaturesHighly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance Features
 
Creating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 FilesCreating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 Files
 
HDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance DiscussionHDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance Discussion
 
Hyrax: Serving Data from S3
Hyrax: Serving Data from S3Hyrax: Serving Data from S3
Hyrax: Serving Data from S3
 
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLABAccessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
 
HDF - Current status and Future Directions
HDF - Current status and Future DirectionsHDF - Current status and Future Directions
HDF - Current status and Future Directions
 
HDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and FutureHDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and Future
 
HDF - Current status and Future Directions
HDF - Current status and Future Directions HDF - Current status and Future Directions
HDF - Current status and Future Directions
 
H5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only LibraryH5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only Library
 
MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10
 
HDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDFHDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDF
 
HDF5 <-> Zarr
HDF5 <-> ZarrHDF5 <-> Zarr
HDF5 <-> Zarr
 
HDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server FeaturesHDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server Features
 
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
 
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
 
HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?
 
HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020
 

Kürzlich hochgeladen

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Kürzlich hochgeladen (20)

AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 

HDF-EOS Datablade: Efficiently Serving Earth Science Data

  • 1. HDF-EOS Datablade: Efficiently Serving Earth Science Data http://heineken.gsfc.nasa.gov/hdfeos/
  • 2. What is a Datablade? Image HDF-EOS Spatial DataBlade API Text Web Extending the server by adding DataBlades: new Types and new functions Extensible OR Engine Scaleable Data Manager INFORMIX-Universal Server http://heineken.gsfc.nasa.gov/hdfeos/
  • 3. What does the HDF-EOS Datablade provide? • Inventory management of large volume data and metadata • Flexible access to the data as a file or BinaryLarge-Object inside the database. • Cross-granule searching and subsetting • Re-use other datablades such as spatial search tools from Informix • Implements Grid subsetting operations in the Database http://heineken.gsfc.nasa.gov/hdfeos/
  • 4. Plans for the Second Year • • • • Implement Swath, Point, HDF datatypes General subsetting tool: stored queries Metadata search and query tools Implement MISR Quality-Assessment database http://heineken.gsfc.nasa.gov/hdfeos/
  • 5. HDF-EOS Datablade Software Architecture Informix DB SQL SQL InterInterface face and API HDF-EOS C library (Unchanged) Valuable Data Exposed in Searchable Tables File I/O Emulator http://heineken.gsfc.nasa.gov/hdfeos/ Granules in File System Granules in Informix
  • 6. Overlapping Granule Scenario Latitude 90 80 70 60 50 40 30 20 10 Subset: Lat: 65 to 75 Lon: 15 to 25 0 1 2 3 4 10 30 50 70 90 Longitude http://heineken.gsfc.nasa.gov/hdfeos/
  • 7. Overlapping Granule Scenario Latitude 90 80 70 60 50 40 30 20 10 Subset: Lat: 65 to 75 Lon: 15 to 25 0 1 2 3 4 10 30 50 70 90 Longitude http://heineken.gsfc.nasa.gov/hdfeos/

Hinweis der Redaktion

  1. &amp;lt;number&amp;gt;
  2. &amp;lt;number&amp;gt;
  3. Features of the HDFEOS Datablade Software Architecture The HDF-EOS library, although non-reentrant, still runs unchanged in the database, with no code modifications. The File-I/O-Emulator (FIOE) catches HDF-EOS I/O calls, and is optimized to handle subsetting I/O patterns. Potential to optimize for other patterns as well. The FIOE allows us to manage the data inside the database, or allow visibility to the data outside. User is not forced into one world or the other. Advantage of putting it inside is that you get extra safety features such as transaction management and recovery. Legacy science software can continue to run on the data if left outside. Can expose valuable indexable data inside the database as necessary. (Vdata example) So how does this architecture perform?….
  4. This picture goes with demo #2.