This document introduces QTIP-WEB, a proof-of-concept for a benchmarking as a service platform. It allows users to import inventory, run benchmark playbooks from a GitHub repository, and view results without needing local installations. The platform is based on Ansible Semaphore and supports customizing playbooks, self-hosting, and integrating with other OPNFV services. Its goal is to provide a shared benchmarking service for the OPNFV community.

2. QTIP-WEB
A small step towards Benchmarking as a Service

3. Benchmarking as a Service
What is your understanding?

4. Content
Introduction
PoC Demo
Roadmap to Benchmarking as a Service
Q&A

5. Introduction
QTIP performance benchmarking utilize Ansible as test runner
• Inventory => SUT (system under test)
• Playbooks => benchmark plan
• Roles => test suite
• Tasks => test case, test step
• Modules => extension (such as installer)
• Plugins => post process, report formatter

6. qtip-web
• User interface of qtip benchmarking service
• Based on ansible-semaphore
• https://github.com/ansible-semaphore/semaphore

7. Benchmarking as a Service
PoC Demo

8. Quick start
• Install
• Sign in qtip benchmarking service
• Import key-pairs to access your inventory
• Add playbook repository - https://github.com/qtipbench/qtip-boilerplate
• Add inventory
• Run testing
• Check result

9. Benefits
• No local installation required
• Value-added services
• Report publishing
• Result pushing
• Data visualization
• Less pain for upgrading the testing toolkit
• Testing results is more trustable and consistent

10. Not just a UI
Enhance OPNFV support with qtip
• OPNFV installer modules for inventory discovery
• Verified performance testing suite on OPNFV
• Integration of OPNFV performance testing projects
• ...
• What else do you expect?

11. Customization
Fork boilerplate and hack the playbook

12. Self hosting supported
• Composed by open source projects
• Deployable on containers
• Less security concerns
• Less ssh connection problems

13. Even more ideas
Integration of OPNFV community services
• Result storage => pnda.io from Bamboo
• Reporting API => testapi from releng
• Dashboard => reporting from releng

14. Benchmarking as a Service
Deploy once, used by everyone in the community
https://wiki.opnfv.org/display/qtip
Join us to convert the idea to reality