This document provides an overview of various tools that can help when writing tests, including factories, spies, and mocks. It discusses how each tool can be used to help with different parts of testing like test setup (preparation phase), checking internal behavior, and simulating dependencies. Factory Boy is presented as a tool for generating complex test data structures simply. KGB is shown as a spying library for checking method calls. Mocks from the mock library are demonstrated for simulating object behavior. Hamcrest matchers are described as reusable conditions for assertions. Custom matchers can also be written with matchmaker to dry up assertions.
5. We write tests to save money
We tell the computer how to do [tedious]
testing for us, faster and cheaper
Writing tests == automation
6. SuT: System-under-Test*
Your “system” as a black box:
I am system
!
(with a spec,
if you’re lucky)
in
out
in: data, stimuli
out: data, observable behaviour
7. SuT: System-under-Test
Your “system” as a white/gray box:
I am system
!
(and you can see
what’s inside me)
in
out
…BTW I rely
on a bunch of
external
stuff
8. SuT: System-under-Test
Your “system” as a white/gray box:
zin
out
…BTW I rely
on a bunch of
external
stuff
Explicit/Injected
dependencies
Implicit/Hardcoded
dependencies
9. Anatomy of [manual] testing
Take your code up to the point
you want to test
Run the specific feature you are
testing
Verify that it worked
10. Anatomy of a test case
def test_something_works():
!
prepare
<blank line>
exercise
<blank line>
verify
aka “Arrange, Act, Assert”
11. Anatomy of a test case
def test_something_works():
!
prepare
<blank line>
exercise
<blank line>
verify
Get your code to a
known state:
Generate test data
Navigate
Isolate and monitor:
Set up mocks
Set up spies
12. Anatomy of a test case
def test_something_works():
!
prepare
<blank line>
exercise
<blank line>
verify
Call your code:
result = something(data)
13. Anatomy of a test case
def test_something_works():
!
prepare
<blank line>
exercise
<blank line>
verify
Validate results:
Assertions on results
Check observed
behaviour:
Check reports from
your mocks and spies
15. Time for the actual talk…
!
Let’s look at tools that
can help us with the hard
bits
16. Tools for test setup
(I find the preparation phase to be the hardest bit)
[Complex] test data: factories
Test objects with behaviour:
mocks
Instrument internals: spies
17. Factories
Goal: make it easy to generate
complex test data structures
!
Tool of choice: factory boy*
* I’ve tried others, but I prefer this one
18. Factories: use cases
Create test data with simple
statements
Let the factory fill [irrelevant] details
Black/white box testing
Explicit dependencies
19. Factories: simplicity
Example: we need a django model
instance for our test.
It has lots of mandatory fields…
..but in this test we only care about
“title”
20. Factories: simplicity
Not this:
publisher = Publisher.objects.create(name=‘Old Books’)
book = Book.objects.create(title=‘Tirant Lo Blanc’,
author=‘Joanot Martorell’,
date=1490,
publisher=publisher)
But this:
book = BookFactory(title=‘Tirant Lo Blanc’)
21. Factory Boy in action
import factory
from books.models import Book, Publisher
!
class PublisherFactory(factory.DjangoModelFactory):
FACTORY_FOR = Publisher
name = ‘Test Publisher’
city = ‘Barcelona’
!
class BookFactory(factory.DjangoModelFactory):
FACTORY_FOR = Book
title = ‘Test Book’
author = ‘Some Random Bloke’
year = 2015
publisher = factory.SubFactory(PublisherFactory)
22. Maintainability FTW!
When you maintain large tests suites, you
want to maximise reuse [& DRYness]
Defaults and rules for building your objects
live in a central place - easy to adapt
E.g. adding a mandatory field is no longer a pain
Not tied to your test framework
23. Factory Boy: niceties
Use a sequence for unique values:
name = factory.Sequence(
lambda num: 'Name {}’.format(num)
)
!
Lazy attributes to populate “late”:
slug = factory.LazyAttribute(
lambda obj: slugify(obj.name)
)
25. Spies
Goal: check if something
happened inside your code
!
Tool of choice: kgb*
* I don’t know others in Python, used Jasmine (JS)
26. Spies: use cases
When it’s hard to have externally
observable behaviour
It’s a bit like adding monitoring to your tests
“Blackbox” testing (with some inside
knowledge)
Implicit dependencies
27. Spies: how to
You know a little what your system
does under the hood
You “spy” on a method that should be
called (the spy is a wrapper that “calls
through”)
Your spy reports on how that method
was called
28. KGB in action
from unittest import TestCase
from kgb import spy_on
!
def add_three(number):
return number + 3
!
def do_stuff(number):
return add_three(number + 1)
!
class SpyOnTest(TestCase):
def test_spy_on_add_three(self):
with spy_on(add_three) as spy:
result = do_stuff(15)
self.assertEqual(spy.last_call.args, (16,))
self.assertTrue(spy.called_with(16))
29. KGB extras
You can replace the spied on
method and make it do nothing
or something else
!
with spy_on(SomeClass.add_stuff,
call_fake=add_two):
30. Mocks
Goal: make it easy to simulate
behaviour of a dependency
!
Tool of choice: mock*
* There are others, I haven’t tried them
31. Mocks: use cases
Your SuT has explicit callback
dependencies (objects it calls)
You want to feed valid objects and
inspect what your system did to them
Simulate hard to reproduce
conditions (e.g. exceptions)
32. Mocks vs Factories
Factories generate “real
production objects”
Mocks generate fake objects
(that you can throw anything at)
33. mock in action
>>> from mock import Mock
>>> user = Mock(username='Fred')
>>> user.username
'Fred'
>>> user.save(force=True)
<Mock name='mock.save()' id='4492633872'>
>>> args, kwargs = user.save.call_args_list[0]
>>> kwargs
{'force': True}
35. mock extras: “patch”
Patch existing code and replace
it with a mock for the duration of
a test
Similar use cases to spies [but
without “call through”]
36. Mock: “patch” use cases
Your SuT has hardcoded
dependencies but you want to
test it in isolation
You want to accelerate your
tests [by bypassing expensive calls]
37. patch in action
from mock import patch
!
class MockPatchTest(TestCase):
@patch('test_sample.add_stuff')
def test_do_stuff_calls_add(self, add_stuff):
add_stuff.return_value = ‘whatever'
!
result = do_stuff(123)
!
add_stuff.assert_called_once_with(123)
self.assertEqual(result, 'whatever')
38. Tools for test validation
Built-in assert_ functions from
your test tool (nose, unittest)
assert statement (if you use py.test
it will give useful reporting)
Matchers (hamcrest)
40. Matchers: use cases
You want to check complex or
custom conditions in a DRY
way
Matchers can be composed - no
need for “combinatory” assertions
or assertTrue(<complex expression>)
41. hamcrest highlights
A single assertion: assert_that
Many matchers out of the box (plus you can
write your own)
Useful reporting on mismatches (no more
“False is not True” errors)
Composite matchers:
all_of, any_of, not_
42. hamcrest in action
def test_any_of(self):
result = random.choice(range(6))
assert_that(result, any_of(1, 2, 3, 4, 5))
!
!
!
AssertionError:
Expected: (<1> or <2> or <3> or <4> or <5>)
but: was <0>
43. hamcrest in action
def test_complex_matcher(self):
user = UserFactory()
assert_that(
user.email,
all_of(
not_none(),
string_contains_in_order('@', '.'),
not_(contains_string('u'))
)
)
!
AssertionError:
Expected: (not None and a string containing '@', '.' in order and
not a string containing 'u')
but: not a string containing 'u' was 'clangworth@schuster.biz'
44. Custom matchers
You can write your own
matchers
The syntax is a bit verbose, so I wrote
matchmaker to make it easier
47. Custom matchers in use
def test_custom_matcher(self):
user = UserFactory()
assert_that(user.age, is_even())
!
AssertionError:
Expected: An even number
but: was <19>
48. More custom matchers
@matcher
def ends_like(item, data, length):
"String whose last {1} chars match those for '{0}'"
return item.endswith(data[-length:])
!
def test_custom_matcher(self):
user1, user2 = UserFactory(), UserFactory()
assert_that(
user.email,
ends_like(user2.email, 4),
)
!
AssertionError:
Expected: String whose last 4 chars match those for
'vwalter@yahoo.com'
but: was 'brett.witting@bergnaum.biz'
