1. How to automate your QA process?
Add-on Discovery Day
Sergio Freire
Xray’s Product Manager
Xray for JIRA empowers test teams to manage and deliver rock solid software solutions
2. Agenda
1. QA challenges
2. Test Automation
3. Automation on workflows
4. Automation on reporting
5. Live showcase
6. Wrap-up
4. 1. Releasing faster (speed vs quality)
2. Manual and automated testing
3. Fast feedback, where you need it
4. Integration with many different automation frameworks
5. Easy integration
6. Support for different methodologies (waterfall, Agile, “hybrid”)
7. Losing time doing all kinds of extra stuff
QA Challenges (well… some of them)
5. 1. Releasing faster
Speed vs Quality - are they real enemies?
Let these help you out:
• Agile
• DevOps, DevTestOps
• Continuous Integration
• Automated Testing
• Continuous Testing
• Continuous Delivery
Release often, with confidence. Motivate your team, make your customers happy.
Image by www.genalice.com.
7. Can you automate everything? Or should you?
• Tests can be automated but choose well what to automate
• Defect creation and analysis will be manual (AI will take time to achieve
this goal)
• Some collateral activities can be automated (e.g. environment setup,
reporting, etc)
• Try to focus on what gives you most value and automate the repetitive
tasks
Automation
8. The cost of bugs
Code Complete, Steve McConnell, 2004
• The cost of fixing bugs grows exponentially depending on the phase in which
they are detected
• The fix cost also depends on the stage where bugs are initially introduced
=> Automated testing to the rescue!
Phase in which a Defect is
introduced
Phase in which a Defect is detected
9. Supported formats
• Xray JSON
• Cucumber
• Behave
• JUnit
• Nunit
• Robot framework
• Calabash
• … more to come
Automation in mind…
Integrating with automated test frameworks
13. Possible scenarios
• Disallow executions with workflow statuses
• For Test Executions in specific
status
• In progress Test Execution workflow status
• After the first run
• Resolved Test Execution workflow status
• When all runs are in final status
• Resolution for Resolved Test Execution
Workflow automation example for Test Executions
Manage execution life-cyle
15. Real status of requirements with in-context information
With Xray:
• Real-time information about the
coverage status of the requirement
for a given version
• Real-time information about the
associated test results, for a given
version
• A ”requirement” may be any issue
type you may have; test coverage
can be made in any issue type
16. Provides a quick way of evaluating the current (coverage) status of your project's
requirements, so you will be able to see if they are ready to be or not.
JIRA Dasboards: Requirement Coverage Gadgets
17. Useful, yet lack quality related information.
Standard Agile Boards
18. Xray gives you visibility of your testing results and its progress.
Enhanced Agile Boards with Xray
19. Use Xporter to generate fully customizable Word, Excel or PDF files with your testing
reports to give to your management, colleagues or to your customers
(e.g. Test specification, Test Execution progress and respective results, Test Plan details).
Reports as deliverable assets
21. 1. Integrating with Junit automated tests
• Automatic report sent to email using Xporter
2. Automated testing using BDD with Cucumber
3. Consolidated results in Test Plan
Live showcase
22. • Cucumber supported natively
• BDD – Specify tests in a business-readable domain-specific language
• Test definition serves as documentation
• Automated test steps may be implemented in Ruby, Java, .Net, etc
• Export specification and execute it during Continuous Integration
Specifying “Automated Cucumber Tests”
24. 1. Use JIRA with Xray for managing the complete lifecycle of your software projects
2. Xray is the perfect tool for Test Management inside JIRA, enabling automation features
3. Xporter complements Xray, enabling advanced reports in Word, Excel, PDF with custom layouts
Deliver faster and better releases with JIRA and Xray
Xray for JIRA empowers test teams to manage and deliver rock solid software solutions
25. w w w. x p a n d - a d d o n s . c o m
Thank you.
sergio.freire@xpand-it.com
Hinweis der Redaktion
1. You your managers want to release faster while at the same time achieve quality (show next slide)
2. Dealing simultaneously with manual and automated tests in a seamless way
3. Getting feedback should not be something you have to ask. It should be something out-of-the-box, natural, by seeing in-context information where you need it
4. A software project is composed of multiple components, using different frameworks and languages, so automated tests may also be written in different frameworks and using all sorts of test runners
5.Implementing continuous integration should be easy to setup. You want your team to get feedback on the quality of their code, as fast as possible and you don’t want the process of setting this up be an excuse to not setting it up at all
6. Every organization, every team is different; you need to be able to adapt to how teams prefer to manage and optimize their workflow
7. You need many and clean testing environments, you need to set them up.. You also need to generate all sorts of reports for different stakeholders. You loose time in many extra tasks
Not really. You can achieve both if you adopt practices such as Agile, DevOps, Continuous Integration/Testing for example.
They can be used to ensure quality while allowing you release often.
All results related with a given test specification are always reported to the same Test entity
Runs history, comments, etc, all will be in the same entity
Tests may be added to Test Sets, Test Plans and may be part of Test Executions
Tests may be linked with requirements
test results will be reflected on requirements
Standard agile board lacks crucial information about the real progress of user stories and testing progress. So it seems you would need to do something in order to obtain this information right? Well, actually you don’t.
In-context information of the real coverage and testing status of your requirements, in the Agile boards.
You can also track the progress of your testing execution and plans in the board.
This is valuable information that is shown you, without effort.