SlideShare ist ein Scribd-Unternehmen logo
1 von 1
Downloaden Sie, um offline zu lesen
SENSITIVITY ANALYSIS FOR
BUILDING ADAPTIVE ROBOTIC SOFTWARE
Pooyan	Jamshidi,	Miguel	Velez	and	Christian	Kästner
INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION
REDUCING COSTS WITH TRANSFER LEARNING
USE CASES
Systematic System Evolution
To automate or guide intelligent design choices.
Runtime Adaptation
To enable runtime adaptation of software configurations
to maintain quality of performance under dynamic
conditions (changing environment, goals, and tasks).
Performance Debugging
To guide robot software developers to identify potential
bugs causing low quality of performance.
RESULTS
Motivation:
Robotic software expose configurable parameters.
These tunable parameters affect performance of robots.
This can be leveraged to optimize performance.
Source Response Target Response
Transfer learning combines:
Lots of data gathered cheaply from
the simulator
With much less data gathered
expensively from the target robot
To make better predictions overall
PUBLICATIONS
P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer
learning for improving model predictions in highly configurable software. Int’l
Symp. Software Engineering for Adaptive and Self-Managing Systems
(SEAMS), 2017.
P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving &
adaptive robotic software, Workshop on Autonomous Mobile Service Robots
(WSF), 2016.
Predictive
Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Use for
analysis
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
Using only a few real data points to
predict yields poor results across
configuration space
Using transfer learning to combine the
few real data points with lots of
approximate data yields a good model
Machine
Learning
Configuration
Parameters
Design of
Experiment
Configuration
Space
Predictive
Model
Sensitivity
Analysis
DataMeasurem
ents
Configuration
Space
Data
Accuracy
Energy
CPU
0 5 10 15 20 25 30 35
mean CPU utilization
0
500
1000
1500
2000
2500
3000
3500
numberofconfigurations
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
6
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
5
10
15
20
25
30
35
40
45
CPU
Localisation Error

Weitere ähnliche Inhalte

Ähnlich wie Sensitivity Analysis for Building Adaptive Robotic Software

Saving resources with simulation webinar 092011
Saving resources with simulation webinar 092011Saving resources with simulation webinar 092011
Saving resources with simulation webinar 092011
Scott Althouse
 
Development of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural networkDevelopment of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural network
IJAAS Team
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD Editor
 
Software performance simulation strategies for high-level embedded system design
Software performance simulation strategies for high-level embedded system designSoftware performance simulation strategies for high-level embedded system design
Software performance simulation strategies for high-level embedded system design
Mr. Chanuwan
 

Ähnlich wie Sensitivity Analysis for Building Adaptive Robotic Software (20)

SOHOMA19 Conference slides
SOHOMA19 Conference slidesSOHOMA19 Conference slides
SOHOMA19 Conference slides
 
Measuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software MetricsMeasuring the Code Quality Using Software Metrics
Measuring the Code Quality Using Software Metrics
 
Software project management Software economics
Software project management Software economicsSoftware project management Software economics
Software project management Software economics
 
A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine...
A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine...A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine...
A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine...
 
Saving resources with simulation webinar 092011
Saving resources with simulation webinar 092011Saving resources with simulation webinar 092011
Saving resources with simulation webinar 092011
 
Effective Software Effort Estimation Leveraging Machine Learning for Digital ...
Effective Software Effort Estimation Leveraging Machine Learning for Digital ...Effective Software Effort Estimation Leveraging Machine Learning for Digital ...
Effective Software Effort Estimation Leveraging Machine Learning for Digital ...
 
Modelling Open-Source Software Reliability Incorporating Swarm Intelligence-B...
Modelling Open-Source Software Reliability Incorporating Swarm Intelligence-B...Modelling Open-Source Software Reliability Incorporating Swarm Intelligence-B...
Modelling Open-Source Software Reliability Incorporating Swarm Intelligence-B...
 
Software cost estimation
Software cost estimationSoftware cost estimation
Software cost estimation
 
implementing_ai_for_improved_performance_testing_the_key_to_success.pptx
implementing_ai_for_improved_performance_testing_the_key_to_success.pptximplementing_ai_for_improved_performance_testing_the_key_to_success.pptx
implementing_ai_for_improved_performance_testing_the_key_to_success.pptx
 
Srikanth QA Analyst
Srikanth QA AnalystSrikanth QA Analyst
Srikanth QA Analyst
 
A Defect Prediction Model for Software Product based on ANFIS
A Defect Prediction Model for Software Product based on ANFISA Defect Prediction Model for Software Product based on ANFIS
A Defect Prediction Model for Software Product based on ANFIS
 
A Defect Prediction Model for Software Product based on ANFIS
A Defect Prediction Model for Software Product based on ANFISA Defect Prediction Model for Software Product based on ANFIS
A Defect Prediction Model for Software Product based on ANFIS
 
Development of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural networkDevelopment of software defect prediction system using artificial neural network
Development of software defect prediction system using artificial neural network
 
2cee Master Cocomo20071
2cee Master Cocomo200712cee Master Cocomo20071
2cee Master Cocomo20071
 
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
IJERD (www.ijerd.com) International Journal of Engineering Research and Devel...
 
A Machine learning based framework for Verification and Validation of Massive...
A Machine learning based framework for Verification and Validation of Massive...A Machine learning based framework for Verification and Validation of Massive...
A Machine learning based framework for Verification and Validation of Massive...
 
Trends in Embedded Software Engineering
Trends in Embedded Software EngineeringTrends in Embedded Software Engineering
Trends in Embedded Software Engineering
 
Towards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java ApplicationsTowards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java Applications
 
Software performance simulation strategies for high-level embedded system design
Software performance simulation strategies for high-level embedded system designSoftware performance simulation strategies for high-level embedded system design
Software performance simulation strategies for high-level embedded system design
 
A Runtime Evaluation Methodology and Framework for Autonomic Systems
A Runtime Evaluation Methodology and Framework for Autonomic SystemsA Runtime Evaluation Methodology and Framework for Autonomic Systems
A Runtime Evaluation Methodology and Framework for Autonomic Systems
 

Mehr von Pooyan Jamshidi

Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
Learning LWF Chain Graphs: A Markov Blanket Discovery ApproachLearning LWF Chain Graphs: A Markov Blanket Discovery Approach
Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
Pooyan Jamshidi
 
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Pooyan Jamshidi
 
Transfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable SoftwareTransfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable Software
Pooyan Jamshidi
 
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
Transfer Learning for Software Performance Analysis: An Exploratory AnalysisTransfer Learning for Software Performance Analysis: An Exploratory Analysis
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
Pooyan Jamshidi
 
Learning Software Performance Models for Dynamic and Uncertain Environments
Learning Software Performance Models for Dynamic and Uncertain EnvironmentsLearning Software Performance Models for Dynamic and Uncertain Environments
Learning Software Performance Models for Dynamic and Uncertain Environments
Pooyan Jamshidi
 
Self learning cloud controllers
Self learning cloud controllersSelf learning cloud controllers
Self learning cloud controllers
Pooyan Jamshidi
 

Mehr von Pooyan Jamshidi (20)

Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
Learning LWF Chain Graphs: A Markov Blanket Discovery ApproachLearning LWF Chain Graphs: A Markov Blanket Discovery Approach
Learning LWF Chain Graphs: A Markov Blanket Discovery Approach
 
A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...
 A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn... A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...
A Framework for Robust Control of Uncertainty in Self-Adaptive Software Conn...
 
Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...
Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...
Machine Learning Meets Quantitative Planning: Enabling Self-Adaptation in Aut...
 
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural ...
 
Transfer Learning for Performance Analysis of Machine Learning Systems
Transfer Learning for Performance Analysis of Machine Learning SystemsTransfer Learning for Performance Analysis of Machine Learning Systems
Transfer Learning for Performance Analysis of Machine Learning Systems
 
Transfer Learning for Performance Analysis of Configurable Systems: A Causal ...
Transfer Learning for Performance Analysis of Configurable Systems:A Causal ...Transfer Learning for Performance Analysis of Configurable Systems:A Causal ...
Transfer Learning for Performance Analysis of Configurable Systems: A Causal ...
 
Machine Learning meets DevOps
Machine Learning meets DevOpsMachine Learning meets DevOps
Machine Learning meets DevOps
 
Learning to Sample
Learning to SampleLearning to Sample
Learning to Sample
 
Integrated Model Discovery and Self-Adaptation of Robots
Integrated Model Discovery and Self-Adaptation of RobotsIntegrated Model Discovery and Self-Adaptation of Robots
Integrated Model Discovery and Self-Adaptation of Robots
 
Transfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable SoftwareTransfer Learning for Performance Analysis of Highly-Configurable Software
Transfer Learning for Performance Analysis of Highly-Configurable Software
 
Architectural Tradeoff in Learning-Based Software
Architectural Tradeoff in Learning-Based SoftwareArchitectural Tradeoff in Learning-Based Software
Architectural Tradeoff in Learning-Based Software
 
Production-Ready Machine Learning for the Software Architect
Production-Ready Machine Learning for the Software ArchitectProduction-Ready Machine Learning for the Software Architect
Production-Ready Machine Learning for the Software Architect
 
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
Transfer Learning for Software Performance Analysis: An Exploratory AnalysisTransfer Learning for Software Performance Analysis: An Exploratory Analysis
Transfer Learning for Software Performance Analysis: An Exploratory Analysis
 
Architecting for Scale
Architecting for ScaleArchitecting for Scale
Architecting for Scale
 
Learning Software Performance Models for Dynamic and Uncertain Environments
Learning Software Performance Models for Dynamic and Uncertain EnvironmentsLearning Software Performance Models for Dynamic and Uncertain Environments
Learning Software Performance Models for Dynamic and Uncertain Environments
 
Machine Learning meets DevOps
Machine Learning meets DevOpsMachine Learning meets DevOps
Machine Learning meets DevOps
 
Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...
Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...
Fuzzy Self-Learning Controllers for Elasticity Management in Dynamic Cloud Ar...
 
Configuration Optimization for Big Data Software
Configuration Optimization for Big Data SoftwareConfiguration Optimization for Big Data Software
Configuration Optimization for Big Data Software
 
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
 
Self learning cloud controllers
Self learning cloud controllersSelf learning cloud controllers
Self learning cloud controllers
 

Kürzlich hochgeladen

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Kürzlich hochgeladen (20)

Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 

Sensitivity Analysis for Building Adaptive Robotic Software

  • 1. SENSITIVITY ANALYSIS FOR BUILDING ADAPTIVE ROBOTIC SOFTWARE Pooyan Jamshidi, Miguel Velez and Christian Kästner INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION REDUCING COSTS WITH TRANSFER LEARNING USE CASES Systematic System Evolution To automate or guide intelligent design choices. Runtime Adaptation To enable runtime adaptation of software configurations to maintain quality of performance under dynamic conditions (changing environment, goals, and tasks). Performance Debugging To guide robot software developers to identify potential bugs causing low quality of performance. RESULTS Motivation: Robotic software expose configurable parameters. These tunable parameters affect performance of robots. This can be leveraged to optimize performance. Source Response Target Response Transfer learning combines: Lots of data gathered cheaply from the simulator With much less data gathered expensively from the target robot To make better predictions overall PUBLICATIONS P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017. P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving & adaptive robotic software, Workshop on Autonomous Mobile Service Robots (WSF), 2016. Predictive Model Learn Model Measure Measure Data Source Target Simulator (Gazebo) Robot (TurtleBot) Predict Performance Predictions Adaptation Use for analysis 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 CPU usage [%] CPU usage [%] (a) (b) (c) (d) Prediction without transfer learning 5 10 15 20 25 5 10 15 20 25 10 15 20 25 Prediction with transfer learning Using only a few real data points to predict yields poor results across configuration space Using transfer learning to combine the few real data points with lots of approximate data yields a good model Machine Learning Configuration Parameters Design of Experiment Configuration Space Predictive Model Sensitivity Analysis DataMeasurem ents Configuration Space Data Accuracy Energy CPU 0 5 10 15 20 25 30 35 mean CPU utilization 0 500 1000 1500 2000 2500 3000 3500 numberofconfigurations 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 6 8 10 12 14 16 18 20 22 24 26 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 5 10 15 20 25 30 35 40 45 CPU Localisation Error