Copernicus academy and relays webinar 2021

1. May 26, 2021
Copernicus Academy and Relays Webinar 2021
Manolis Koubarakis
National and Kapodistrian University of Athens
Artificial Intelligence in the Earth Observation Domain:
Current European Research

2. 2
Motivation
• The Copernicus programme of the European Union
• Artificial Intelligence advances in Europe and internationally in the last few years
• Question: How can AI technologies impact Earth Observation? What are current European efforts in this
area?

3. 3
Four Projects
• “ExtremeEarth: From Copernicus Big Data to Extreme Earth Analytics”
o The project started in January 2019 and finishes in December 2021.
o See http://earthanalytics.eu/ .
• “AI4Copernicus: Reinforcing the AI4EU Platform by Advancing Earth Observation Intelligence,
Innovation and Adoption”
o The project started in January 2021 and finishes in December 2023.
o See https://ai4copernicus-project.eu/

4. 4
Four Projects (cont’d)
• “DeepCube: Explainable AI pipelines for big Copernicus data”
o The project started in January 2021 and finishes in December 2023.
o See https://deepcube-h2020.eu/
• “Callisto: Copernicus Artificial Intelligence services and data fusion with other distributed data
sources and processing at the edge to support DIAS and HPC infrastructures”
o The project started in January 2021 and finishes in December 2023.
o See https://callisto-h2020.eu/

5. 5
ExtremeEarth Main Objective
• The main objective of ExtremeEarth is to develop Artificial Intelligence and Big Data techniques and
technologies that scale to the PBs of big Copernicus data, information and knowledge, and apply these
technologies in two of the ESA Thematic Exploitation Platforms: Food Security and Polar.
• The technologies to be developed will extend the European Hopsworks data intensive AI platform of
partner Logical Clocks to offer unprecedented scalability to extreme data volumes and scale-out
distributed deep learning for Copernicus data.
• The extended Hopsworks platform will run on CREODIAS and will be available as open source to enable
its adoption by the strong European Earth Observation downstream services industry.
• The technologies to be developed will also extend the linked geospatial data systems GeoTriples,
JedAI, Strabon and SemaGrow pioneered by project partners UoA and NCSR in the past, so that they
scale to the extreme volumes of Copernicus data.

6. 6
ExtremeEarth Consortium
1. National and Kapodistrian University of Athens (UoA)
2. VISTA
3. The Arctic University of Norway (UiT)
4. University of Trento (UNITN)
5. The Royal Institute of Technology (KTH)
6. National Center for Scientific Research – Demokritos (NCSR-D)
7. German Aerospace Center (DLR)
8. Polar View
9. Norwegian Meteorological Institute (METNO)
10.LogicalClocks
11.British Antarctic Survey (UKRI-BAS)

7. 7
The Food Security Use Case
• The objective is to develop high resolution water availability maps for agricultural areas allowing a
new level of detail for wide-scale irrigation support for farmers. The Danube and Duero catchments will
be targeted.

8. 8
The Polar Use Case
• The objective is to produce high resolution ice maps for maritime users from massive volumes of
heterogeneous Copernicus data.

9. 9
Scalable Deep Learning Techniques for Copernicus
Big Data
• Developed an LSTM deep neural network architecture for crop type mapping from Sentinel 2 data.
This has been implemented on Hopsworks and it is being used in the Food Security use case.
• Developed various machine learning architectures (LDA, CNN, variational autoencoders and GANs)
for sea-ice classification from Sentinel 1 data. These have been implemented on Hopsworks and are
being used in the Polar use case.

10. 10
Very Large Training Datasets for Deep Learning
Architectures
• Developed a training dataset consisting of ~1M pixels of 16 Sentinel 2 images located in Austria
where each pixel is labelled with one of 13 crop types. This dataset was developed using existing crop
type maps and Sentinel 2 data and it was used to train the LSTM network for the Food Security use
case.
• Available publicly very soon (http://earthanalytics.eu/datasets.html).

11. 11
Very Large Training Datasets for Deep Learning
Architectures (cont’d)
• Developed a training dataset consisting of 63048 patches of 30 Sentinel 1 images located in the
European Arctic where each patch is labelled with one of 6 ice types. This dataset was developed by
expert photo-interpretation and it was used to train three of the CNN networks for the Polar use case.
• Developed a training dataset consisting of ~62M patches of 24 Sentinel 1 images located in the
Belgica bank of Greenland Sea where each patch is labelled with one of 11 ice types. This dataset was
developed using active learning and it was used to train the LDA and one of the CNN networks for the
Polar use case.
• See http://earthanalytics.eu/datasets.html.

12. 12
Big Linked Geospatial Data Systems
• Developed the system GeoTriples-Spark for transforming big geospatial data from their legacy formats
into RDF. GeoTriples-Spark can transform 2TBs of geospatial data into RDF in 50 minutes.
• Developed the system JedAI-spatial for interlinking big linked geospatial data. JedAI-spatial has been
tested with >100 GB of geospatial data and has been shown to scale almost linearly.
• Developed the system Strabo2 for querying big linked geospatial data using the OGC standard
GeoSPARQL. Strabo2 can process queries of the Geographica benchmark over 171GBs in 30 to
400 seconds.
• Developed the version 3 of system Semagrow for federating big linked geospatial data sources.
Semagrow can process queries used in the Food Security use case over <1GB of data in <20
seconds.

13. 13
Integration in Hopsworks, TEPs and CREODIAS
• We integrated the AI and Big Data technologies presented above in the Hopsworks data platform and
deployed them in CREODIAS and the two TEPs for developing the two use cases.

14. 14
New Deep Learning Functionalities in Hopsworks
• We extended the filesystem HopsFS and the resource scheduler HopsYARN of Hopsworks for managing
EO data.
• We extended the Hopsworks metadata and security model with APIs for EO metadata.
• We carried out the following extensions to the Hopsworks platform that enable large-scale distributed data
processing and building ML/DL pipelines: EO Data Management, Feature Store, Experiment API,
Maggy framework for asynchronous parallel execution of trials for machine learning experiments,
distribution oblivious training functions, Maggy support for hyperparameter tuning and parallel
ablation studies.

15. 15
AI4Copernicus
• Motivation
o The AI4EU AI-on-demand platform (https://www.ai4europe.eu/) is a one-stop-shop for anyone
looking for AI knowledge, technology, tools, services and experts.
o EO data and services have reached a significant level of maturity via the DIAS platforms and produce
value in various domains.
• Main Objective
o AI4Copernicus aims to bridge these two worlds and make the AI4EU platform, the platform of
choice for users of Copernicus data along the value chain (scientists, SMEs, non-tech sector).

16. 16
Scientific and Technical Contributions

17. 17
Open Calls
• AI4Copernicus will also offer incentives to the AI and Copernicus communities in Europe to solve real
problems of business and societal value using the AI4Copernicus software:
o 4 open calls
▪ 8 small-scale experiments (smaller, single-beneficiary experimental projects targeting technology-
advanced users)
▪ 9 use cases (larger-budget projects, involving at least one non-technology user)
• The project differentiates between pre-defined domain-specific themes (top-down) in four thematic areas
(energy, agriculture, security and health) and open themes/social causes (bottom-up).
• First open call out in June 2021.

18. 18
DeepCube (Explainable AI pipelines for big
Copernicus data)
• Motto: New AI architectures for new EO problems
• Objectives:
o Develop scalable AI architectures to solve high impact problems that involve quantitative estimation
of geophysical variables
o Use AI techniques to accommodate non EO data such as social network data, industry specific data,
sensor data etc.
o Make use of AI architectures on interferometric SAR data

19. 19
Use Cases
• Forecasting extreme drought and heat impacts in Africa
• Climate induced migration in Africa
• Fire hazard forecasting in the Mediterranean
• Global volcanic unrest detection and alerting
• Critical infrastructure monitoring in Qatar
• Sustainable tourism for Brazil

20. 20
Scientific and Technological Contributions

21. 21
Callisto (Copernicus Artificial Intelligence services and data
fusion with other distributed data sources and processing at the
edge to support DIAS and HPC infrastructures)
• Objectives
o Develop a very high precision digital model of our planet to monitor and simulate human and
environmental activity to enable sustainable development.
o Combine satellite with in-situ and socioeconomic data to derive digital twins, dynamic replicas of our
planet that mimic its behaviour, based on the integration of federated cloud platforms, big data
technologies, AI, Deep Learning, HPC and processing on the edge.

22. 22
Use Cases
• Virtual presence for CAP monitoring
• Virtual presence for water quality assessment
• Sensor journalism
• Land border change detection

23. 23
Scientific and Technological Contributions

24. 24
Prevailing Themes
• Artificial Intelligence
o Deep learning
▪ Distributed deep learning platforms
▪ Explainability, causality and hybrid modeling
o Semantic technologies and linked data
• DIASes
• Cloud computing
• High Performance computing
• Edge computing
• Interesting, high impact use cases

25. Thank you!
Follow us on Twitter:
@ExtremeEarth_EU
@AI4Copernicus
@DeepCube_H2020
@CALLISTO_H2020
@mkoubarakis