P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017.
Sensitivity Analysis for Building Adaptive Robotic Software
1. SENSITIVITY ANALYSIS FOR
BUILDING ADAPTIVE ROBOTIC SOFTWARE
Pooyan Jamshidi, Miguel Velez and Christian Kästner
INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION
REDUCING COSTS WITH TRANSFER LEARNING
USE CASES
Systematic System Evolution
To automate or guide intelligent design choices.
Runtime Adaptation
To enable runtime adaptation of software configurations
to maintain quality of performance under dynamic
conditions (changing environment, goals, and tasks).
Performance Debugging
To guide robot software developers to identify potential
bugs causing low quality of performance.
RESULTS
Motivation:
Robotic software expose configurable parameters.
These tunable parameters affect performance of robots.
This can be leveraged to optimize performance.
Source Response Target Response
Transfer learning combines:
Lots of data gathered cheaply from
the simulator
With much less data gathered
expensively from the target robot
To make better predictions overall
PUBLICATIONS
P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer
learning for improving model predictions in highly configurable software. Int’l
Symp. Software Engineering for Adaptive and Self-Managing Systems
(SEAMS), 2017.
P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving &
adaptive robotic software, Workshop on Autonomous Mobile Service Robots
(WSF), 2016.
Predictive
Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Use for
analysis
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
Using only a few real data points to
predict yields poor results across
configuration space
Using transfer learning to combine the
few real data points with lots of
approximate data yields a good model
Machine
Learning
Configuration
Parameters
Design of
Experiment
Configuration
Space
Predictive
Model
Sensitivity
Analysis
DataMeasurem
ents
Configuration
Space
Data
Accuracy
Energy
CPU
0 5 10 15 20 25 30 35
mean CPU utilization
0
500
1000
1500
2000
2500
3000
3500
numberofconfigurations
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
6
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
5
10
15
20
25
30
35
40
45
CPU
Localisation Error