The document discusses Thematic Exploitation Platforms (TEPs) and the Geohazards Exploitation Platform (GEP) specifically. The GEP provides access to Earth observation data and processing tools for geohazards research. It allows users to access large datasets, run distributed processing, and collaborate. The GEP currently has over 1,100 registered users processing data for projects on volcanoes, earthquakes, and landslides. It also partners with other initiatives like CEOS, the International Charter, and EPOS to support their work.
2. â R&D activities to create an ecosystem of
interconnected Thematic Exploitation
Platforms
â Users access a work environment
containing the data and resources
required, as opposed to downloading
and replicating the data âat homeâ
Thematic Exploitation Platforms
The fundamental principle is to move
the User to the data and tools
3. Specific fields in the domain
â Geohazards domain
â Earthquakes, Volcanoes, Landslides âŠ
â Hydrology domain
â Hydrological modelling, Flood mapping, Water quality
monitoring âŠ
â âŠ
Data-intensive research
â Targeted data
â Targeted software
THEMATIC Exploitation Platforms
4. â Very fast access to large volume of data
â EO data
â In-situ data
â Distributed processing
â Hybrid Cloud infrastructure
â Commercial, Public and Private Clouds
Thematic EXPLOITATION Platforms
5. â Collaborative tools
â Referencing Publications
â Sharing processing results
â Different roles
â Community scientists
â Data providers
â Resource providers
Thematic Exploitation PLATFORMS
6. GEP designed in the context of:
âȘ Geohazards Supersite initiative (GSNL)
âȘ CEOS Disasters Working Group
User-driven model for partnership and
community building
âȘ Started from Int. Forum on Satellite
EO and Geohazards organised by ESA
and GEO in Santorini in 2012 (140+
participants)
Geohazards Exploitation Platform | GEP
7. Platform based on virtualization & federation of
satellite EO data
âȘ Provide services & support to the geohazards community
On-demand & systematic processing services
âȘ Cloud Compute power, managing multi-tenant resources
Access to Copernicus Sentinels repositories
âȘ Plus access to hundred TBs of EO data archives (ERS and
ENVISAT), and other EO missions (ALOS-2, Cosmo-Skymed
and TerraSAR-X) under CEOS WG Disaster and the GSNL
agreements
Geohazards Exploitation Platform | GEP
8. GEP | Initial Usage Scenarios
Users from geoscience centres hosting geohazard experts
âȘ with skills in EO (to process, analyze, validate, integrate data)
âȘ generating DRM products for decision-makers (End Users)
âȘ End Users arenât intended to be direct users of the platform
Initially targeting three main scenarios:
âȘ EO Data Exploitation - A user runs on-demand processing services
âȘ New EO Service Development - A user integrates a new processing chain or
modifies/enhances an on-line processing chain
âȘ New EO Product Development - A user runs a chain global/systematic to generate
a new derived information layer
9. GEP | Roadmap
GEP Early Adopters Programme
Want to apply as early adopter of the GEP (limited slots)? contact@geohazards-tep.eu
Open under invitations/approval
Consolidation and evolution of the service
Access to EPOS IP users (within available slots)
GEP v1 validation with early adopters
Limited to early adopters
(under invitation), evolution
of the service on-going
Open,
consolidated and
stable service
GEP v2 pre-operations GEP v2 pre-ops ext.
170+ platform early adopters
100+ new users in the period
Today
GEP v2 engineering
50 platform early adopters
28 new users in the period
22 platform early adopters
integrating application or
exploiting on demand processing
Nov 2015 Jul 2018
KO AR
Dec 2016
CDR
Dec 2018Feb 2015
As of Dec 2018:
1100+ registered users
from 86 countries
10. âȘ 20+ on-demand services using Optical & SAR
data grouped in Thematic Apps, according to
the defined goals of Community Managers
âȘ New basic services providing full resolution and
change detection imagery for rapid online
visualization
âȘ 8 systematic services delivering continuously
updated information layers on GEP, including
the large scale production of Sentinel-1 InSAR
browse images at both 100m and 50m
resolution over tectonic regions and volcanoes
GEP | Enriched Services Portfolio
13. GEP | Early Adopters (July 2018)
âȘ 172 users (from 77 organisations and 35 countries) are carrying out projects on the platform via the Early Adopters Programme
âȘ Mainly European users, but also from the rest of the world: Asia (Turkey, China, Thailand, South Korea, Indonesia, India, Malaysia & Iran),
Africa (Morocco, Nigeria, Ethiopia), Latin America (Mexico, Chile, Ecuador, Brazil and Argentina) and North America (USA and Canada).
14. User Scheduled Processing
The data are automatically selected on a
regular basis, according to different policies:
â Time driven
â Data driven
Scheduled User Processing
https://geohazards-tep.eu
Supported by
15. Itâs a data driven systematic processing. The
service has followed a ramp-up process
starting from Dec 2016 until Aug 2017:
â EU Tectonic area
â World tectonic area (25%)
â World tectonic area (40%)
It currently processes 150+ Sentinel-1 SLC
pairs per day.
DLR InSAR Browse Medium
Resolution Service
Supported by
BELNET-BEGRID (Belgium)
Data Driven Scheduled Processing
https://geohazards-tep.eu
16. Data Driven Scheduled Processing
Itâs a data driven systematic processing. This
service provides an amplitude and coherence
product at 50m systematically from
Sentinel-1 data for a number of volcanoes
(20+) globally.
DLR InSAR High Resolution
Change Monitoring Service
https://geohazards-tep.eu
Supported by
ReCaS Bari (Italy)
17. The User triggers the processing services from
the Geobrowser Web interface.
The User searches and filters the EO data
from an Opensearch Catalogue, then passes
the selected inputs to a processing service.
User-Driven processing
https://geohazards-tep.eu
User Driven Processing
Supported by
18. This service provides the coherence and
intensity changes for Sentinel-1 TOPSAR IW
data pairs performed through SNAP. SNAP is
a common architecture for all Sentinel
Toolboxes, which is ideal for Earth
Observation processing and analysis.
ESA SNAP Sentinel-1 COherence
and INtensity (COIN)
Supported by
User Driven Processing
https://geohazards-tep.eu
19. P-SBAS stands for Parallel Small BAseline
Subset and it is a DInSAR processing chain
for the generation of Earth deformation time
series and mean velocity maps. Input: SLC
(Level-1) Sentinel-1 data.
CNR-IREA P-SBAS Sentinel-1
processing on-demand
https://geohazards-tep.eu
Will be
supported by
BELNET-BEGRID (Belgium)
User Driven Processing
20. GEP | Monthly Statistics (Dec 2018)
EO Data collections accessible
(i.e. the user can run a processing against these
data on the GEP)
Global Coverage with open access:
Sentinel-1, Sentinel-2, Sentinel-3, Landsat-8, ERS (SAR IM Level-0), ENVISAT (ASAR
IM Level-0)
Limited coverage collections with restricted access:
ALOS-2, TerraSAR-X, COSMO SkyMed, RADARSAT-2, Pleiades
Several datasets from various EO missions for the Charter Platform Prototype:
ALOS-2, TerraSAR-X, SPOT6/7, Pleiades, UK-DMC-2, Kompsat-2/3/5, Resurs-P,
Kanopus-V, RAPIDEYE, GF2, PROBA-V, SUOMI
Number of processing requests launched
[by data source, month]
Sentinel-1 (4.534), Sentinel-2 (1.039), ENVISAT (30), Landsat-8 (138), ERS (30),
KOMPSAT-5 (61), Pleiades/SPOT (88), UK-DMC-2 (20), ALOS-2 (4), TerraSAR-X (2),
SUOMI (8)
Quantity of processing hours [month] 7.664 CPU/hours
Number of products generated [month] 18.072
Number of products published [month] 8.375
Number of products downloaded [month] 6.029
21. GEP | Users Uptake (Dec 2018)
The GEP has a total of 1131 registered users from 86 countries world wide
24. Enhanced Data Gateway
âȘ Automatic multi-sourcing to optimise data access requests
âȘ Programmable and systematic data caching
âȘ Data usage accounting
âȘ Personal cloud storage (repository)
GEP | Cloud Platform - Data
Supported Data Providers
Daily figures
25. Application Integration
â Applications developed in any programming language supported
â C/C++, Java, Python, Matlab and IDL
â Continuous Integration and Deployment Environment with
automatic packaging and deployment in production environments
26. â OGC Testbeds activity to identify a strategy for
packaging applications able to run in different
Cloud computing production centres
â OWS Context documents allow applications to
be specified and deployed using:
â Information about the Docker container
providing a consistent environment for
execution
â WPS as the execution interface information
about the input data and respective
â OpenSearch catalogue end-points.
GEP | Standardization
27. â Improved Production Center, with (auto)scalability allowing
cost-effective data processing on Cloud Computing clusters
â Deployment in multiple Cloud-based
processing environments
â EGI.eu, Amazon AWS, PSNC,
CloudFerro (EO IPT Poland)
â On going/in negotiation with
â Serco ONDA DIAS: integration of the GEP as Reference Front Office
â CloudFerro CREODIAS: integration of systematic chains of the GEP
â EODC in the context of the EOSC-Hub project
GEP | Cloud Platform - Computing
28. âȘ User Communities remain the owners of their
created assets, and decide how to share these
on the platform, for other users to find and
reuse these assets
âȘ They are also empowered to decide on the
target production environment of their
packaged applications
âȘ no lock-in with Cloud provider
âȘ Open APIs needed for emerging marketplaces
(e.g. EOSC-HUB)
GEP | Cloud APIs - SaaS
29. Uptake & Sustainability
CEOS Pilots
âȘ Support for CEOS Pilots for complex workflows
âȘ Multiple-pairwise image correlation for the
monitoring of surface deformation from
optical image time-series)
âȘ Processing service for measuring horizontal
displacements from time-series of optical
satellite images
âȘ ESA and CNES to setting up a mechanism
to bring Pleiades data on GEP for on-line
processing
Sentinel-2
surface velocities
over the
Harmaliere
landslide
in French Alps
MPIC-OPT service developed by
CNRS / EOST (Strasbourg)
S-2 image tile
HarmaliĂšre landslide
(French Alps)
30. Uptake & Sustainability
International Disasters Charter
âȘ International Charter Major and Space
Disasters Board selected GEP as Prototype
processing platform
âȘ Testing the support of Project Manager and
Value Adding partners for Charter activations.
âȘ Help rapidly screen data with data
processing (co-registering multi-sensor
collections), and image analysis (e.g.
change detection)
âȘ Automatic notifications about Charter
activations
âȘ Supporting 18 missions
31. Uptake & Sustainability
Scientific networks - EPOS
âȘ EPOS, European Plate Observing System, is a
long-term plan to facilitate integrated use of
data, data products, and facilities from
distributed research infrastructures for solid
Earth science in Europe.
âȘ GEP was selected as the gateway for the
Satellite Data Thematic Core Service in EPOS
âȘ EPOS Implementation: the GEP will provide the
interface for the Satellite Data
âȘ EPOS Operations: GEP users will have access to any
other data available within EPOS IP and
âȘ EPOS ERIC will purchase GEP services (estimative
~100k/year) with ESA as an associated
partner within the Satellite Data TCS
governance scheme
33. â Dedicated Geohazards TEP Zenodo community
â Users upload a products file (+ metadata)
â Reviewed by the Scientific Communicator
â After approval, automated process to create
Digital Object Identifiers (DOI) for each product
â The author citations are managed and
sharable from Zenodo
â The publication of approved products on the
GEP Community area is fully transparent for
users (GEP uses the Zenodo API)
Publishing GEP Results via Zenodo
34. â Exploring the Earth
Observation
Catalogues from a
Notebook
â Accessing
OpenSearch API
directly from Python
â Analyse results as
data structures
(Panda/Geopanda)
Data Exploitation: Target Reproducibility
36. â Notebook analysis
of a stack of
Sentinel-1 data
â Backscatter profiles
for reference image
used in flood
analysis
Data Exploitation: Time Series Analyses
37. â Invoking GEP
services from a
notebook
â DIAPASON
Stripmap service
over Fogo volcano
through a WPS
Service
Data Exploitation: WPS Services
38. GEP | Take-home Messages
âȘ Benefit from fast access to EO data, storage
capacity and processing resources offered by
platform-based solutions
âȘ A hosted processing platform is a partnership
that needs the support of scientist/developer
to offer operational services providing value
added information to the community
âȘ Platform e-collaboration and reproducible
knowledge promote innovation and response
capacity
39. Platforms: Take-home Messages
Cloud Platform Solutions built for Earth sciences
communities:
âȘ to share their findings, streamline their
creation of new data products and make
these accessible, interoperable
âȘ to document their developments as
reproducible experiments
âȘ with FAIR guiding principles to make data
findable, accessible, interoperable and reusable
40. âȘ Agriculture monitoring
âȘ Air quality
âȘ Biodiversity
âȘ Climate Change
âȘ Cold Regions Monitoring
âȘ Crop Monitoring
âȘ Disaster Risk Reduction
âȘ Earthquakes
âȘ Energy
âȘ Food Security
âȘ Land management
âȘ Oceanography
âȘ Volcanology
âȘ Water quality
âȘ Wetlands
User communities
41. Open Cloud Strategy
â Freedom of movement
Users are free to move data as needed.
By supporting distributed instances of its
EO Data management layer, deliver the
required level of data locality to ensure high
performance processing with optimized costs
â Developers community
Support and nurture Cloud communities that
collaborate on evolving open source
technologies, including at the level of the
Platform engineering team, when it comes to
deliver modular extensions.
â Self-service provisioning
The Platformâs end-users are able to
self-provision and manage their required
ICT resources transparently and perform their
tasks autonomously
â Open APIs
Embrace Cloud bursting APIs that can be
easily plugged into the Platformâs codebase
to expand the Platform with Providers offering
complementary strategic advantages for
different user communities