1. Test improvement process as a
result of QA process audit
If the test processes at your organization or project are to improve, you
are probably the one who will make that happen...

2. About Speaker
Speaker: Ramella Basenko
Role: Lead QA Engineer at AgileEngine, ISTQB
certified Tester Full Advanced Level
7+ years of experience in QA area, 5 years of QA team
management experience.

3. Agenda
1. What is QA Process Audit and why do we need it
2. What do we gain as a result of the Software QA process audit
3. From what to start and steps to perform QA process audit
4. Continuous Test Improvement Process
5. Industry-standard Test Process Improvement models
6. My personal experience
7. QA session

4. What is QA process audit and why do we need it
Audits ensure your quality assurance system is sound. Audits are also necessary for ISO 9001 registration.
Types of Audits
● 3rd Party Audits
● 2nd Party Audits
● 1st Party Audits
3rd Party Audit are done by an outside independent audit organization. In case of an ISO 9001 audit the independent
organization is the registrar. They normally do not have an interest in the final outcome. The 3rd party could be a
registrar, government employee, or a firm hired by the company. The result of the audit could be a certification, license,
acceptance, or an award.
2nd Party Audit are audits from those who have a deep interest in the final outcome. This could be you auditing your
suppliers, or your customers auditing you. The audit could be called a survey or an assessment.
1st Party Audits are audits done internally. They are a management tool with the emphasis on continuous
improvement. The auditor(s) must be independent of the audit area. The audit should be aligned with the company
goals and metrics.

5. What is QA process audit and why do we need it
If you find yourself in one of the following situations, it is highly recommended to perform
a QA process audit:
● You have no clear visibility of software quality or
experience process bottlenecks.
● Your project workload increases, but your current QA
process doesn’t allow scaling.
● Your current QA process prevents you from
transitioning to a different development
methodology, taking a different approach to
infrastructure management or rolling other process
changes.
● You need to pass a product/process certification
and want to make sure your QA process complies with
the requirements set by a corresponding standard.

6. What do we gain as a result of the Software QA process audit
1. Get detailed analysis of current situation on a project.
2. Improve existing process according to recommendations and using chosen improvement
model
3. Improve testing Efficiency
4. Get ROI of process improvement and testing overall
5. Make our customers satisfied with project results

7. From what to start and steps to perform QA process audit
Steps to kick-off and perform QA process audit :
1. Initiate the process audit
2. Diagnose the present circumstances
3. Gather information by interviewing of people
involved to the current process (QA lead, team
members), eg Review the existing process
4. Compose Audit Summary document based on
replies analysis and selected test process
improvement models
5. Suggest improvement models or plans that are
best for the current project.

8. Continuous Test Improvement Process
The Deming Cycle, or PDCA Cycle (also known as
PDSA Cycle), is a continuous quality improvement
model consisting out of a logical sequence of four
repetitive steps for continuous improvement and
learning: Plan, Do, Check (Study) and Act.
It can be applicable to any kind of project, not limited to
software development.

9. Industry-standard Test Process Improvement models
Externally developed process assessment models can provide a starting point, a standard
framework, and a way of measuring your process. Process assessment can be performed
using a process model and opportunities to improve current process can be identified.
There is a generic process improvement model called IDEAL that can be used for any
process.
At the same time four industry-standard Test Process Improvement models are widely
used in the world: Testing Maturity Model integration(TMMi), Test Process Improvement
Next(TPI Next), Critical Testing Process (CTP) and Systematic Test and Evaluation Process
(STEP).

10. Industry-standard Test Process Improvement models
TMMi or Test Maturity Model Integration improves the software testing practices for the
organizations and also alleviates the IT standards for it. TMMi is gradually finding its way in
many IT organizations to streamline and ease their testing process. TMMi was curated by
Illinois Institute of Technology whereas TMMi was curated by TMMi foundation. The intent
behind it was to develop guidelines and reference framework for improving the testing process.
TPI Next – reflects the changes in today’s business dynamics and technology developments.
TPI Next offers a specific, balanced and correlated improvement path for test organization. The
TPI Next model also provides a better understanding of the correlation between current test
process and adjacent processes. It will help discussing and addressing issues for improving the
overall software process.

11. Industry-standard Test Process Improvement models
Critical Testing Process (CTP) is a non-prescriptive process model. It describes the
important software processes and what should happen in them, but it doesn’t put them in any
order of improvement. This makes CTP a very flexible model. It allows you to identify and deal
with specific challenges to your test processes. It identifies various attributes of good
processes, both quantitative and qualitative. It allows you to use business value and
organizational pain to select the order and importance of improvements. It is also adaptable to
all software development lifecycle models.
Systematic Test and Evaluation Process (STEP) is a structured testing methodology, also
used as a content-based model for improving the testing process. Systematic Test and
Evaluation Process (STEP) does not require that improvements occur in a specific order.

12. My Practical experience
1. TMMi model was chosen to perform
analysis
2. Questionnaire has been created to use
during interviews with team members.
3. Questionnaire contained different
sections like Test Planning, test
Execution, Test Monitoring and Control
etc.
4. Based on audit results, Audit Summary
document was created containing
recommendations for improving the
current process
Maturity goals and levels of the Testing Maturity Model

13. My Practical experience
Steps to apply TMMi model to certain process:
1) Identify existing testing maturity level:
Knowing the current maturity level of the organization is very important and the first step
towards TMMi. TMMi helps IT organizations evaluate their current maturity level. It also
evaluate if the current testing processes are reactive or proactive. It calculates the current
maturity level of the organization and the efforts required to move to the next maturity level.
2) Set a Target:
After all, evaluation is done next is the time to set the target and the milestones to achieve the
next maturity level. After setting the targets the organization have to work upon achieving them
and reaching to wanted maturity level.

14. My Practical experience
3) Steps by step structuring:
When the organization figures out the gap between the current and desired maturity level, it
prepares a step by step break of the structure to attain its end goal. Doing things step by step helps
in achieving the goals easily and hence organisations break their complete efforts into steps.
4) Review, feedback, solution, and repeat:
Organizations at any maturity level on TMMi require continuous review, feedback, solution, and
repeat. It helps in improving the testing process and hence helping the organization reach the top
maturity. It also helps in reviewing and refining industry standards and processes.
Implementation of TMMi without a doubt helps IT organizations to enhance their Testing practices
and brings many benefits to the organization. With the improving TMMi maturity levels, IT
organizations also benefit by improving maturity levels of the SDLC, particularly the planning and
development phases.

15. References
1. TMMi documents https://www.tmmi.org/tmmi-documents/
2. Brief decription of test process improvement models http://tryqa.com/software-
testing-process-improvement-models-tmmi-tpi-next-ctp-step/
3. TPI Next model whitepapers https://www.qualitestgroup.com/assets/test-process-
improvement.pdf
4. Guide to test process improvement from Guru99 https://www.guru99.com/a-
complete-guide-to-test-process-improvement.html
5. How to Use The Deming Cycle for Continuous Quality Improvement
https://www.process.st/deming-cycle/
6. Example of survey based on TMMi level 2 to 5 http://experimentus.tmmi-ind-
2019.sgizmo.com/s3/
7. Test process improvement - practical guide on TPI https://www.amazon.com/Test-
Process-Improvement-step-step/dp/0201596245?language=en_US

16. Q&A

17. Thanks for your
attention!