19. Can you trust 100% coverage?
Code coverage can only show what is not tested.
For interpreted languages 100% code coverage is
kind of like full compilation.
39. Equivalent mutations
// Original
int i = 0;
while (i != 10) {
doSomething();
i += 1;
}
// Mutant
int i = 0;
while (i < 10) {
doSomething();
i += 1;
}
42. Let’s say we have codebase with:
● 300 classes
● around 10 tests per class
● 1 test runs around 1ms
● total test suite runtime is about 3s
Is it really slow?
Let’s do 10 mutations per class
● We get 3000 (300 * 10) mutations
● runtime with all mutations is
150 minutes (3s * 3000)
43. Speeding it up
Run only tests that cover the mutation
● 300 classes
● 10 tests per class
● 10 mutations per class
● 1ms test runtime
● total mutation runtime
10 * 10 * 1 * 300 = 30s
45. Usage scenarios
● Continuous integration
● TDD with mutation testing only on new
changes
● Add mutation testing to your legacy
project, but do not fail a build -
produce warning report
46. Tools
● Ruby - Mutant
● Java - PIT
● And many tools for other languages
47. Summary
● Code coverage highlights code that is
definitely not tested
● Mutation testing highlights code that
definitely is tested
● Given non equivalent mutations, good test
suite should work the same as a hash
function
48. About us
Vaidas Pilkauskas
@liucijus
● Vilnius JUG co-
founder
● Vilnius Scala leader
● Coderetreat facilitator
● Mountain bicycle
rider
● Snowboarder
Tadas Ščerbinskas
@tadassce
● VilniusRB co-
organizer
● RubyConfLT co-
organizer
● RailsGirls Vilnius &
Berlin coach
● Various board sports’
enthusiast
49. Credits
A lot of presentation content is based on
work by these guys
● Markus Schirp - author of Mutant
● Henry Coles - author of PIT
● Filip Van Laenen - working on a book