2. Outline
• Vision of the Complete Observing System
• Vision of Spectral Databases
• State of the Art of Spectral Databases
• Architecture and essential Functionality
• The Metadata Model
• Input and Data Storage
• Metadata Editing
• Retrieval, Processing and Output
• Data Exchange
• Administration Tools
• GIS Connectivity
• SPECCHIO Online
• Data Policy
• Software availability
2
3. Vision of the Complete Observing System
Understanding the Earth system
and the changes induced by
anthropogenic and natural causes
National Research Council, Earth Science and Applications GEO, Global earth observation system of systems
from Space: National Imperatives for the Next Decade and GEOSS: 10-Year implementation plan reference
Beyond. Washington, DC: The National Academies Press, document. 2200 AG Noordwijk, The Netherlands ESA
2007.
Publications Division 2005.
3
4. Vision Statement
Vision of the Complete Observing System
Enable a healthy public, economy,
and planet through an integrated,
comprehensive, and sustained
Earth observation system.
Acquisition of Earth
surface information in
support of Earth System
Science.
Image sources:
earthobservatory.nasa.gov
CENR/IWGEO. 2005. Strategic Plan for the U.S. Integrated Earth Observation System, National Science and www.boeing.com
Technology Council Committee on Environment and Natural Resources, Washington, DC.
jaxa.jp
4
5. The Complete Observing System
Archiving and
Sensors and Platforms
Data Management
Spacebased
Airborne
In situ
Processing
Processors
Image sources:
Google Earth
www.boeing.com
5
6. Implementing the System of Systems: Data Grids
Clearing House
Data clearing houses present
users with a uniform interface
to the heterogeneous
storage/service components
of the data grid.
Users
E. J. Christian, "GEOSS Architecture Principles and the
GEOSS Clearinghouse," IEEE Systems Journal, vol. 2,
pp. 333 - 337, 2008.
6
7. Data Assimilation & CAL/VAL in a Data Grid
Vision
Data grids encompass spacebased, airborne and in situ data,
offering the possibility of seamless data integration at different scales
of observation and thus facilitating data assimilation & calibration/
validation processes as standard operations.
Metadata
Metadata
Metadata
Assimilation
&
CAL/VAL
Processes
See also CEOS CAL/VAL activities:
ceoswgcv-ivos.org/
7
8. Spectral Databases: Components of a Complete
Observing System
Archiving and
Data Management
Archiving and
Data Management
Image sources:
• GEO, Global earth observation system of systems GEOSS: 10-Year implementation plan reference document. 2200 AG Noordwijk, The Netherlands ESA
Publications Division 2005.
• www.brest.ird.fr/us025/observatoire/station_meteo_sao_tome.jpg
8
9. Definition: Spectral Databases
Spectral databases are systems for the
organised storage of spectral signatures
accompanied by associated metadata.
Metadata
A. Hueni, et al. "The spectral database SPECCHIO for improved long term usability and data sharing," Computers & Geosciences, vol. 35(3), pp. 557-565, 2009.
9
11. Metadata Space
Metadata define a multi-dimensional space; a primary resource is
identified by its metadata parameters and thus has a defined position in
metadata space.
11
12. Vision of Spectral Databases
Spectral information is at the scientistʼs fingertips
and comprehensive metadata and automatically
derived quality indicators facilitate the assessment
of data quality and hence their suitability for new
applications.
Clearing House
Scientist
12
13. Roadmap to Data Sharing and Long-term Usage
Registration as
data grid
component
(e.g. GEOSS)
Homogenized
metadata for
eased inclusion
as data grid
service
Automated
quality indicator
generation
Spectral
database setup
for the organised
storage of
spectral data and
metadata
Data Acquisition
of high accuracy,
well-documented
spectral data
13
14. State of the Art of Spectral Databases
Logo
Institute
Online
Public
Active
# of # of Reference
www
spectra*
install.
RSL, University of ✔
✔
✔
3316
15
A. Hueni, et al. "The spectral
Zurich, database SPECCHIO for improved
Switzerland
long term usability and data
www.specchio.ch
sharing," Computers &
Geosciences, vol. 35(3), pp.
557-565, 2009.
DLR, ✔
✔
✔
2008
1
Becvar, M. et al, (2006 - 2008) DLR
Oberpfaffenhofen, Spectral Archive. Online at: http://
Germany
cocoon.caf.dlr.de/intro_en.html,
cocoon.caf.dlr.de
accessed on: 25.11.2009.
SSI, Australia
✔
✔
✘
A few 1
Ferwerda, J. et al, 2006. A free
www.hyperspectra dozen
online reference library for
l.info/
hyperspectral reflectance
signatures. SPIE Newsroom, 1-2,
http://newsroom.spie.org/
x5275.xml.
SSD, Darwin, ✘
✘
>> 1000
1
Pfitzner, K., et al, 2008, 29.09. -
Australia
✔
03.10.2008. SSD's Spectral Library
environment.gov.a Database. In: Proceedings 14th
u/ssd/research/ Australasian Remote Sensing and
protect/ Photogrammetry Conference,
rehabilitation.html
Darwin, AU.
* Available in the online system on 25.11.2009
14
15. State of the Art of Spectral Databases: Technology
Logo
Database
Interface
Local Import formats
Export formats
installation
possible
MySQL
Java and PHP
✔
ASD binary, GER,
CSV
ENVI SLB,
ENVI SLB
ASCII,
XML
XML
?
Web
✘
ASD binary, ASCII
?
MySQL
PHP
✘
ASD binary
ASCII
ASD text
ENVI SLB
GER, ASCII
JCAMP
ENVI SLB
SQL Server
?
✘
ASD binary
?
15
16. Critical Design Criteria
• Extendable, generic data model
• Defined value sets for categorical variables
• Native file reading routines for automatic metadata generation
• Intuitive interface for metadata editing with minimal user interaction
(multiple updates)
• Data retrieval via flexible metadata searches
Success Criteria
• Users can accept changes in their traditional workflow
• Users will only enter their data in a well-documented fashion if the
user interfaces are friendly and quick
• Spectral databases must become the ʻone stop shopʼ for spectral
signatures within and across institutions.
16
17. SPECCHIO: RSLʼs Spectral Database
A. Hueni, et al. "The spectral database SPECCHIO for improved long term usability and data sharing," Computers & Geosciences, vol. 35(3), pp. 557-565, 2009.
17
18. Main components of the spectral database SPECCHIO
Query Builder
Metadata Editor
Data Loader
Space Network Processor
Instrumentation Administration
File Export
Data Report
18
25. Selected Metadata of Spectral Data
Spatial position per spectrum.
Used for GIS, sun angle
calculations and spatial
searches.
Automatic calculation of
sensor and illumination
geometry (in case of
FIGOS and location/time
is known respectively).
General sampling information.
25
29. Processing: The Space Network
Components:
• Nodes: spaces (data sources/sinks) and processing modules
• Edges: directed connections between nodes
A. Hueni, M. Kneubuehler, J. Nieke, and K. Itten, "PROCESSING EXTENSION FOR THE SPECTRAL DATABASE SPECCHIO," in EARSeL SIG IS,
Tel Aviv, Israel, 2009.
29
32. Available Modules in the Space Network Processor
• Radiance to Reflectance Transformation
• Reference Panel Correction Factors
• Correct for Reference Panel Non-Idealness
• Delta (A-B as matrix operation)
• Waveband Filter
• Broadband and Narrowband Filters
• Visualisation:
Spectral Line Plot, Spectral Scatter Plot , Gonio Sampling Points
Plot, Gonio Hemisphere Explorer, Time Line Plot, Time Line
Explorer
• File Export
32
33. Output: Visualisation of spectrodirectional data
Interactive Gonio Hemisphere Explorer in SPECCHIO
A. Hueni, S. Rey, D. Schläpfer, J. Schopfer, and M. Kneubuehler, "VISUALISATION, PROCESSING AND STORAGE OF SPECTRODIRECTIONAL DATA
BASED ON THE SPECTRAL DATABASE SPECCHIO," in IGARSS 09, Cape Town, South Africa, 2009.
33
36. Data Exchange between SPECCHIO Instances
Central Server @ RSL
SPECCHIO Online Server
SPECCHIO
SPECCHIO
SPECCHIO
SPECCHIO
Edu
Prod
Prod
Test
Workstation @ RSL
Workstation @3rd
party
SPECCHIO
SPECCHIO
A. Hueni, T. Malthus, M. Kneubuehler, and M. Schaepman, "Data Exchange between distributed Spectral
Databases," Computers & Geosciences, submitted.
36
37. Data Exchange between SPECCHIO Instances
Automated export/import of spectral sampling
campaigns including the whole metadata
context using XML style files.
37
40. GIS Connectivity
- ODBC connection
- Access to all tables
- Spatial mapping
Mapping of sampling locations using ArcMap (ESRI)
40
41. SPECCHIO Online: www.specchio.ch
- Personal accounts via www.specchio.ch
- Upload your own spectra, browse and download all spectra in the online database
41
43. Data Policy
- Data on the online system is freely accessible (read-only for all, full access for owner)
- Import/Export function for SPECCHIO systems allows to easily include our own data
in the online system (exact copy including full metadata context)
43
44. Software availability
- Online user account creation
- Java software download
- Full installation package for local installation available: email admin@specchio.ch
44
