QA Fest 2019. Дмитрий Гуменюк. Test to code mapping, code coverage by automated and manual tests, actionable coverage – easy! We made it, open sourced and free!
Have you ever collected code coverage with tests? Most likely it was unit tests. But have you tried to collect them from automated tests, manual tests? What if you even had an incredible number of 100% coverage, does it give you any understanding? Can you make it actionable? In order to solve this issue - we wrote our own tooling, made it simple and pluggable, without app code affect, and accidentally did more than competitors, but open-sourced. With this talk, I will introduce the new tool, share our approach, describe benefits of test to code mapping, tell you how to make coverage numbers actionable, introduce first steps to try it at home, and will share our plans going forward and how we plan to shift black-box testing to white-box.
QA Fest 2019. Дмитрий Гуменюк. Test to code mapping, code coverage by automated and manual tests, actionable coverage – easy! We made it, open sourced and free!
1. TEST TO CODE MAPPING
CODE COVERAGE BY AUTOMATED
AND MANUAL TESTS – EASY!
We made it,
but open-sourced
and free
Dmitriy Gumeniuk
QA Fest
Kyiv, September 2019
2. TEST TO CODE MAPPING
CODE COVERAGE BY AUTOMATED
AND MANUAL TESTS – EASY!
We made it,
but open-sourced
and free
Dmitriy Gumeniuk
QA Fest
Kyiv, September 2019
ASK QUESTIONS
3. TEST TO CODE MAPPING
CODE COVERAGE BY AUTOMATED
AND MANUAL TESTS – EASY!
We made it,
but open-sourced
and free
Dmitriy Gumeniuk
QA Fest
Kyiv, September 2019
17. Test-to-Code mapping benefits:
Minimize regression testing scope
by proposing to run tests, mapped with modified methods
Reduce duplicating automated tests
by identifying equal code triggering with equal entry parameters
18. Test-to-Code mapping benefits:
Minimize regression testing scope
by proposing to run tests, mapped with modified methods
Reduce manual tests
by identifying equal tests In automated and manual scopes
Reduce duplicating automated tests
by identifying equal code triggering with equal entry parameters
19. Test-to-Code mapping benefits:
Minimize regression testing scope
by proposing to run tests, mapped with modified methods
Reduce manual tests
by identifying equal tests In automated and manual scopes
Collect production usage
to compare with tested areas
Reduce duplicating automated tests
by identifying equal code triggering with equal entry parameters
21. Drill4J: Test to Code Coverage
Drill4J is the tool for real-time application profiling that
doesn’t affect codebase. Provide ability to make white box
functional testing, via access to access to application
instructions and memory.
TEST TO CODE PLUGIN CAPABILILTIES
• Measure code coverage via any types of tests (manual,
automated, integration, performance, etc.)
• Build Code to Test mapping shows which test case
trigger which methods in app code, and vice versa.
• Suggest test cases to trigger, related to code change
• Track manual testing coverage
• Shows uncovered methods by regression
• Shows new methods uncovered by tests for code
change, highlighting it as potential risk
• Track production usage paths vs testing paths
21
25. Drill4J there is thanks to:
Viktor Bellavin
Denis Moiseev
Pavel Iarov Aleksandr Krupenin
Kristina Smirnova Natalia IarchukOlga Danilova
Igor Kuzminykh
26. Plugins
A N D T H E F U T U R E ?
New plugins fit perfectly in architecture.