Artificial intelligence (AI) is experiencing steadily growing interest over the recent years. For good reason, since these innovative algorithms and methods, such as machine learning and deep neural networks, in which knowledge is acquired and applied based on data, enable the automation of a wide range of processes and quickly deliver precise results. AI is also getting more and more popular in the space sector. The Institute of Space Technology & Space Applications (ISTA) at the Universität der Bundeswehr in Munich is conducting research around AI for space operations, science, and technology. An overview of activities and current developments towards fault management, autonomous collision avoidance, autonomous landing, as well as radio science at ISTA will be presented.

1. AI4Space
-
Artificial Intelligence at ISTA
M. Hülsmann & B. Haser
Science Coffee
June 11th 2021

2. 6/15/2021 Conference/Workshop/Event 2
2
ESA Science Coffee 11 June 2021
Benjamin Haser, MSc.
a
 Research Associate for AI4Space since 2019 at UniBw ISTA
 MSc. and BSc. in Physics, with specialization in particle &
nuclear fusion physics from LMU Munich (2013 – 2019)
Maren Hülsmann, MSc.
 Research Associate for AI4Space since 2019 at UniBw ISTA
 Experience as Flight Dynamics Engineer at DLR GSOC
(2017 – 2019)
 MSc. and BSc. in Industrial Mathematics from
University of Bremen (2011 – 2017)
 Volunteer at Space Generation Advisory Council since 2019:
 Co-Lead of Space Safety and Sustainability PG
 Event Manager SG[Germany]
Introduction

3. 3
6/15/2021 Conference/Workshop/Event 3
ESA Science Coffee 11 June 2021
Outline
• The Institute of Space Technology & Space Applications
• Research Topics and Expertise
• AI4Space at ISTA
• Overview
• Examples
• AI-based Fault Management
• Autonomous Collision Avoidance
• Super Resolution and Autonomous Landing
• Radio Science – Parameter Estimation
• Summary & Outlook

4. 4
6/15/2021 Conference/Workshop/Event 4
ESA Science Coffee 11 June 2021
Institute of Space Technology & Space Applications
https://www.unibw.de/lrt9/lrt-9.1

5. 5
6/15/2021 Conference/Workshop/Event 5
ESA Science Coffee 11 June 2021
Institute of Space Technology & Space Applications
Research Topics:
• Highly Autonomous Spacecraft Operations
• System Safety and Reliability
• Fault Management
• Autonomous Landing on Moons, Asteroids, Comets
• System Design
• System simulation and systems engineering
• Mission and system design
• Radio Science
• Participation in missions: MarsExpress, VenusExpress, Rosetta, New Horizons, Juice, …
• Mission and system studies for DLR/BMWI, ESA and EU

6. 6
6/15/2021 Conference/Workshop/Event 6
ESA Science Coffee 11 June 2021
AI4Space at ISTA
AI4Space research group at ISTA serves as a link between AI engineer and space
technology developer as well as satellite integrator
Aiming at understanding both sides
Locate and develope suitable applications of AI for usage on board of spacecraft or within ground operations
• New space and digitalisation are rapidly changing the space ecosystem
• Operating thousands of satellites at once is challenging with available traditional strategies and technology
• Artificial Intelligence and Machine Learning are enablers of higher autonomous systems
• AI and Space are two very different domains with different „languages“

7. 7
6/15/2021 Conference/Workshop/Event 7
ESA Science Coffee 11 June 2021
• AI-based Fault Management
• Anomaly detection in satellite telemetry data
• Goal: Prediction of non-normal trends, anomalies and fault events
• Autonomous Collision Avoidance for Multi-Spacecraft Systems
• Safe Autonomy and optimal behaviour
• Detection of possible collisions
• Formation reconfiguration in case of a failure
• Munich Aerospace Research Group
• Partnering with German Aerospace Center (DLR)
• Data Analysis
• Radio Science Experiments (e.g. Structure Analysis of Phobos)
• Image Processing
• Super-Resolution and object detection for Landing on small planetary bodies (LiDAR data)
• Object Detection and pre-processing for Earth Observation missions
• Verification and Validation of AI-based systems
Research Focus – AI Applications

8. 8
6/15/2021 Conference/Workshop/Event 8
ESA Science Coffee 11 June 2021
• FDIR
• critical part of a Spacecraft OBSW
• guarantees safety and service availabilities of Spacecraft/ Mission
• Traditionally done with “hard-coded” limit checks
• Fault Detection
• detect presence of faults and time of occurrence
• Fault Isolation
• determine type and location
• Fault Identification
• determine size and time-varying behaviour
• estimate severity and effects on system
• System Reconfiguration
• select and execute optimal actions to solve or compensate failure
Fault Detection
Fault Isolation
Fault Identification
System
Reconfiguration
taken from MA Mehring/Oetjen, July 2020
AI-Focus I: Fault Management
taken from: M. Hülsmann, R. Förstner: Predictive
Maintenance as Enhancement of Fault Detection,
Isolation and Recovery Strategies of Spacecraft, IAC 2020.

9. 9
6/15/2021 Conference/Workshop/Event 9
ESA Science Coffee 11 June 2021
AI-Focus I: AI4FDIR onboard SeRANIS
AI4FDIR Experiment
• Implementation and in-orbit verification of AI-based fault management concepts
• Prediction of fault events and detection of non-nominal trends in the satellite telemetry onboard
• In-Orbit test on board SeRANIS
• SeRANIS
• UniBw M small satellite mission to be launched in 2025
• “Multifunctional satellite laboratory for digital services in and from orbit”
• Funded by dtec.bw
https://dtecbw.de/home/forschung/unibw-m/projekt-seranis
https://seranis.de/

10. 10
6/15/2021 Conference/Workshop/Event 10
ESA Science Coffee 11 June 2021
Proposed Concept:
• Fault Prediction
• Prediction of TM ahead by N time steps (e.g. LSTM)
• Classification of predicted TM (Anomaly or Nominal)
• Fault Explanation
• Isolation of anomaly/ fault
• Identification of fault and estimation of severity
• Fault Prevention
• Evaluation of fault severity and prevention actions available
• Decision about actions and selection of optimal prevention actions
Prediction Classification Isolation Identification
TM Data Evaluation
System Level
Actions
System Model
AI-Focus I: AI4FDIR onboard SeRANIS
Anomaly
developing
taken from: M. Hülsmann, R. Förstner: Predictive Maintenance as Enhancement of Fault Detection, Isolation and Recovery Strategies of Spacecraft, IAC 2020.

11. 11
6/15/2021 Conference/Workshop/Event 11
ESA Science Coffee 11 June 2021
AI-Focus II: AI4COLA
„Autonomous Collision Avoidance for Multi-Spacecraft Systems“
• Munich Aerospace Research Group (2021 – 2024)
• Lead topic: Safety in Orbit
• In collaboration with the German Space Operations Center (DLR GSOC)
• Focus on three scenarios
• Satellite Formation
• Satellite Swarm
• (Mega-) Constellation
https://www.munich-aerospace.de/en/research/research-groups

12. 12
6/15/2021 Conference/Workshop/Event 12
ESA Science Coffee 11 June 2021
AI-Focus II: AI4COLA
Research Focus
1. Mission Concepts
2. Collision Avoidance
• Ground-based
• Autonomy in orbit
3. Mission Safety
• Safety Guided Design
• Formation Control
• Fault Management
4. Verification, Validation & Test
• Verification and Validation of AI-based systems
• (Sub-)System Test
• Emulation Test with drones
Visualisation of the OneWeb constellation
Tandem-X and TerraSAR-X Formation ©DLR

13. 13
6/15/2021 Conference/Workshop/Event 13
ESA Science Coffee 11 June 2021
AI-Focus III: Super Resolution - Motivation
Improve GN&C process with AI:
• Increase resolution of the incoming shape map
• Detect hazard objects earlier  save fuel
Useable in many other scenarios
[1]

14. 14
6/15/2021 Conference/Workshop/Event 14
ESA Science Coffee 11 June 2021
AI-Focus III: Super Resolution - Motivation
Improve GN&C process with AI:
• Increase resolution of the incoming shape map
• Detect hazard objects earlier  save fuel
Useable in many other scenarios
[1]

15. 15
6/15/2021 Conference/Workshop/Event 15
ESA Science Coffee 11 June 2021
AI-Focus III: Super Resolution - Models
[3]
[2]

16. 16
6/15/2021 Conference/Workshop/Event 16
ESA Science Coffee 11 June 2021
AI-Focus III: Super Resolution - Models
[1]

17. 17
6/15/2021 Conference/Workshop/Event 17
ESA Science Coffee 11 June 2021
AI-Focus III: Radio Science – Theory
Motion of a Satellite:
̈
𝑟𝑟 =
𝐹𝐹 𝑟𝑟, 𝑡𝑡, 𝑣𝑣
𝑚𝑚
= 𝛻𝛻𝛻𝛻
r,t,v: position and kinematic information of the satellite in a non-rotating
geocentric coordinate system
Radial symmetric Force:
𝐹𝐹 = −𝐺𝐺𝐺𝐺
𝑀𝑀⊕
𝑟𝑟2
𝑒𝑒𝑟𝑟
Potential:
𝑈𝑈 = 𝐺𝐺 �
𝜌𝜌(⃗
𝑠𝑠)
𝑟𝑟 − 𝑠𝑠
𝑑𝑑𝑑𝑑
𝜌𝜌(⃗
𝑠𝑠) : density at point s
𝑟𝑟 > 𝑠𝑠: Expression in spherical harmonics
1
𝑟𝑟 − 𝑠𝑠
=
1
𝑟𝑟
�
𝑠𝑠
𝑟𝑟
𝑛𝑛
𝑃𝑃 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 , 𝑤𝑤𝑤𝑤𝑤𝑤𝑤 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 =
⃗
𝑟𝑟⃗
𝑠𝑠
𝑟𝑟𝑟𝑟
𝑈𝑈 =
𝐺𝐺𝑀𝑀⊕
𝑟𝑟
� �
𝑅𝑅⊕
𝑛𝑛
𝑟𝑟𝑛𝑛
𝑃𝑃𝑛𝑛,𝑚𝑚 𝑐𝑐𝑐𝑐𝑐𝑐𝜃𝜃 𝐶𝐶𝑛𝑛,𝑚𝑚𝑐𝑐𝑐𝑐𝑐𝑐 𝑚𝑚𝜑𝜑 + 𝑆𝑆𝑛𝑛,𝑚𝑚𝑠𝑠𝑖𝑖𝑖𝑖 𝑚𝑚𝜑𝜑

18. 18
6/15/2021 Conference/Workshop/Event 18
ESA Science Coffee 11 June 2021
AI-Focus III: Radio Science – Stoke Coefficients
Contain whole information about the mass distribution of a body:
𝐶𝐶𝑛𝑛,𝑚𝑚 =
2 − δ0,𝑛𝑛
𝑀𝑀⊕
𝑛𝑛 − 𝑚𝑚 !
𝑛𝑛 + 𝑚𝑚 !
�
𝑟𝑟𝑛𝑛
𝑅𝑅⊕
𝑛𝑛 𝑃𝑃𝑛𝑛,𝑚𝑚 𝑐𝑐𝑐𝑐𝑐𝑐𝜃𝜃 𝑐𝑐𝑐𝑐𝑐𝑐 𝑚𝑚𝜑𝜑 𝜌𝜌(⃗
𝑟𝑟) 𝑑𝑑𝑑𝑑𝑑𝑑𝜑𝜑𝑑𝑑𝜃𝜃
𝑆𝑆𝑛𝑛,𝑚𝑚 =
2 − δ0,𝑛𝑛
𝑀𝑀⊕
𝑛𝑛 − 𝑚𝑚 !
𝑛𝑛 + 𝑚𝑚 !
�
𝑟𝑟𝑛𝑛
𝑅𝑅⊕
𝑛𝑛 𝑃𝑃𝑛𝑛,𝑚𝑚 𝑐𝑐𝑐𝑐𝑐𝑐𝜃𝜃 𝑠𝑠𝑠𝑠𝑠𝑠 𝑚𝑚𝜑𝜑 𝜌𝜌(⃗
𝑟𝑟) 𝑑𝑑𝑑𝑑𝑑𝑑𝜑𝜑𝑑𝑑𝜃𝜃
For sufficient small enough volume elements:
𝐶𝐶𝑛𝑛,𝑚𝑚 =
2−δ0,𝑛𝑛
𝑀𝑀⊕
𝑛𝑛−𝑚𝑚 !
𝑛𝑛+𝑚𝑚 !
∑ ∑ ∑
𝑟𝑟𝑛𝑛
𝑅𝑅⊕
𝑛𝑛 𝑃𝑃𝑛𝑛,𝑚𝑚 𝑐𝑐𝑐𝑐𝑐𝑐𝜃𝜃 𝑐𝑐𝑐𝑐𝑐𝑐 𝑚𝑚𝜑𝜑 𝜌𝜌(⃗
𝑟𝑟) 𝑑𝑑𝑑𝑑𝑑𝑑𝜑𝜑𝑑𝑑𝜃𝜃
𝑆𝑆𝑛𝑛,𝑚𝑚 =
2−δ0,𝑛𝑛
𝑀𝑀⊕
𝑛𝑛−𝑚𝑚 !
𝑛𝑛+𝑚𝑚 !
∑ ∑ ∑
𝑟𝑟𝑛𝑛
𝑅𝑅⊕
𝑛𝑛 𝑃𝑃𝑛𝑛,𝑚𝑚 𝑐𝑐𝑐𝑐𝑐𝑐𝜃𝜃 𝑠𝑠𝑠𝑠𝑠𝑠 𝑚𝑚𝜑𝜑 𝜌𝜌(⃗
𝑟𝑟) 𝑑𝑑𝑑𝑑𝑑𝑑𝜑𝜑𝑑𝑑𝜃𝜃
Direct limitation for accuracy:
𝑅𝑅⊕ =
3 3
4𝜋𝜋
𝑉𝑉

19. 19
6/15/2021 Conference/Workshop/Event 19
ESA Science Coffee 11 June 2021
AI-Focus III: Radio Science – Shape Model
Characteristics:
Shape: Ellipsoid
𝐴𝐴 = 13400, 𝐵𝐵 = 11200, 𝐶𝐶 = 9200
Phobos Willner
Describtion of the surface up to 17th order by using 665
control point and spherical harmonics of latitude and
longitude
Model: Polygon (Icosahedron)
20 ∗ 4𝑛𝑛
n=6  81920 equilateral triangles
Epsilon: 1x10−3
∶ Used for uneven surface correction
Insert spherical density anomalies:
Waterice or porosities
 Measure influence on gravitational coefficients

20. 20
6/15/2021 Conference/Workshop/Event 20
ESA Science Coffee 11 June 2021
AI-Focus III: Radio Science – Results
𝑅𝑅2 = 1 −
∑ 𝑦𝑦𝑖𝑖 − 𝑦𝑦𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚
2
∑ 𝑦𝑦𝑖𝑖 − �
𝑦𝑦 2

21. 21
6/15/2021 Conference/Workshop/Event 21
ESA Science Coffee 11 June 2021
AI-Focus III: Radio Science – Problems
Random Samples Percentage deviation between model and real value Number of anomalies

22. 22
6/15/2021 Conference/Workshop/Event 22
ESA Science Coffee 11 June 2021
Summary and Outlook - AI4Space at ISTA
• Link between both worlds – AI and space
Focus in the Area of AI:
• Machine Learning
• Deep Learning
• Reinforcement Learning
• Hybrid Methods
• Explainable AI (XAI)
On-going Student Theses
• TM Forecasting
• Automated ML-Pipeline
• Simulation of space-related faults for DNNs
• Reinforcement Learning based navigation controller
Other Topics
• Verification & Validation of AI-based Systems
SeRANIS IR & optical Camera Experiment
• Object detection
• Denoising & Cloud removal
• Data transformation & encrypting

23. Thank you for your attention
AI4Space Team
Benjamin Haser
benjamin.haser@unibw.de
Maren Hülsmann
maren.huelsmann@unibw.de
Sanjay Swami
sanjay.swami@unibw.de
Woo Seok Park
woo.park@unibw.de

24. 24
6/15/2021 Conference/Workshop/Event 24
Sources
[1] Ledig, Christian & Theis, Lucas & Huszar, Ferenc & Caballero, Jose & Cunningham, Andrew & Acosta, Alejandro & Aitken, Andrew & Tejani, Alykhan & Totz, Johannes & Wang, Zehan & Shi, Wenzhe.
(2017). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 105-114. 10.1109/CVPR.2017.19.
[2] https://journals.plos.org/plosone/article/figure?id=10.1371/journal.pone.0235487.g002
[3] C. Dong, C. C. Loy, K. He and X. Tang, "Image Super-Resolution Using Deep Convolutional Networks," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 1
Feb. 2016, doi: 10.1109/TPAMI.2015.2439281.