Automatically Improve Software Architecture Models for Performance, Reliability, and Costs using Evolutionary Algorithms

1. PerOpteryx Automatically Improve Software Architecture Modelsfor Performance, Reliability, and Costs using Evolutionary Algorithms Anne Martens Karlsruhe Institute of Technology (KIT), GermanyHeiko Koziolek, Steffen Becker, Ralf Reussner WOSP / SIPEW 2010

2. Software Performance Engineering Anne Martens C1 3 sec A Transform Solve Change component and deployment 2.5 sec C2 A Solve Transform Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

3. Not only Performance! Anne Martens C1 3 sec p(fail) 0.01%$ 5700 A Transform Solve Change component and deployment 2.5 sec p(fail) 0.02% $ 12000 C2 A Solve Transform Optimise multiple criteria at once Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

4. Multicriteria Optimisation Anne Martens ArchitecturalCandidate A Time 5s $40K Costs Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

5. Multicriteria Optimisation Anne Martens Generated & Evaluated A Time 5s $40K Costs Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

6. Multicriteria Optimisation Anne Martens A Time 5s B 3s C 2s Pareto-optimal $40K $33K $20K Costs Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

7. Related Work: Quality Optimisation Anne Martens Rule-based approaches: Single quality only Parsons2008, Cortellessa2009, PerformanceBooster (Xu&Woodside2008),ArchE (McGregor2007) Multicriteria evaluation: No improvement Bondarev2007, Grunske2007 Optimisation: Limited degrees of freedom ArcheOpteryx (Aleti2009), Canfora2005,Kavimandan2009, Sassy (Menascé2010) Missing: Flexible multicriteriaoptimisationatthe design level Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

8. PerOpteryx Approach Anne Martens Flexible degreesoffreedom Multiplequalities Multi-criteriaoptimization Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

9. Degrees of Freedom Anne Martens Design decision that can still be made C Degree of freedom Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

10. Types of Degrees of Freedom in CBSE Anne Martens Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

11. Instances of Degrees of Freedom Anne Martens Each component Allocation Componentselection for D Each server Processor speed Component selection Search alternatives Component selection for C Allocation of D C D Processor speed of server 1 Allocation of C Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

12. Search Problem Anne Martens evaluate Response in 2.5 s P(failure) 0.02% Cost $6000 transform evaluate initial model candidate model Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

13. SearchImplementation Anne Martens NSGA-II [Deb2002] Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

14. Quality evaluation Anne Martens PalladioComponentModel [Becker2007] PCM2LQN [Koziolek2008] PCM2Cost [Martens2010] PCM2DTMC [Brosch2009]  Cost Reliability Performance Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

15. CaseStudywith PerOpteryx (1/2) Anne Martens Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

16. CaseStudywith PerOpteryx (1/2) 1235 candidates 58 Pareto optimal 8h running time Anne Martens Componentallocation Processingrates Componentselection Response Time POFOD Costs Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

17. Case Study with PerOpteryx (2/2) Anne Martens RT: 1.34 s POFOD: 5.2E-4 Cost: 69.83 Onlyfour, but fasterserversDifferent Webserver RT: 2.2 s POFOD: 6E-4 Cost: 98 Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

18. Future Work Anne Martens ? Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

19. Conclusions Anne Martens AutomatedArchitectureImprovement Flexible degreesoffreedom Multiplequalities Multi-criteriaOptimization http://sdqweb.ipd.kit.edu/wiki/PerOpteryx Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

20. Conclusions Anne Martens Multi-criteriaOptimization PerOpteryx Benefits Performance, Reliability, Costs Implemented& Validated Automatedimprovement Performance istrade-off Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

21. PerOpteryx Implementation (1/3) Anne Martens Degrees of Freedom: ProcessingRate ComponentSelection ComponentAllocation ? ? e.g., 1.5 – 3 GHz Motivation – Related Work – Approach – Case Study – Future Work –Conclusion

22. SearchProcess Anne Martens 3 sec p(fail) 0.01%5700 $ Change alongdegrees 2.8 sec p(fail) 0.02% 6000 $ Change alongdegrees 2.5 sec p(fail) 0.02% 12000 $

23. Not only Performance! Anne Martens C1 3 sec A Transform Solve Change component and deployment 2.5 sec C2 A Solve Transform