A talk given at BarCampJozi on October 12th 2008.

1. Beyond Testing:
Specs and Behaviour
Driven Development
rabble - evan henshaw plath
blog - anarchogeek.com
work - entp.com
explicar quien soy

2. First Tests
First before we can talk about beyond testing we should talk about testing itself.

3. QA Testing
The first thing you think of when you hear testing is Quality Assurance, the QA test. A person
sits and clicks on all the links and buttons. You know everything work if you don’t have a fatal
crash.

4. It’s very useful to know when there are huge problems, but it’s an issue for those other
people. But we aren’t other people, we’re programming, and for us the QA test is just extra
work we don’t like. A huge list of bugs.

5. But there’s another kind of testing, the type which is in code. All the types of testing in code
is to verify the functionality of an application automatically.

6. Regression Testing
But there’s another kind of testing, the type which is in code. All the types of testing in code
is to verify the functionality of an application automatically.

7. Writing code to verify code. It’s recursive. You’ve got your application code, then tests to
verify the application, then tests of the testing library, etc...
From the perspective of the programmer, regression testing is much more useful. It’s the
process by which we can design the implementation of an application.

8. Unit Testing
The most common type of regression testing is the great UNIT TEST. Normally when you hear
people talk about testing beyond QA they’re talking about unit tests.

9. What’s a unit? It’s the smallest piece of functionality of your application. It could be a method,
or a class. Or perhaps a unit is a single line of code.

10. Testing for
Application Design
One thing which is distinctly different about regression testing vs QA testing is that
regression testing is a technique for application design. Not designing the user interface but
designing the code itself.

11. With testing we can come to a better understanding of how the parts of our application work.
Once we know the parts we can use that to design the whole that emerges.

12. TDD - Test Driven
Development
What is this TDD thing? It’s a philosophy of how we design code. Not the graphic design or
interaction design of the interface, but the code, methods, classes, api, and the like. It’s
important to know before we get in to TDD what it’s reacting to. The idea of the waterfall.

13. The waterfall method. In reality it’s the method which is still taught in universities, and which
is certified by the ISO. It’s the core design philosophy of ENTERPRISE software.
The idea is we have a project, we do a planning process, create a map, a specification of how
all of the classes and objects are going to work, design the whole API on paper. Then with
this sacred document we can start to code based on gods plan.

14. The picture before is of Iguazú, a falls at the border of Argentina, Brazil, and Paraguay. But
it’s not fair to use it. Because in reality even the waterfall method is about iteration. So
perhaps it looks a little more like this. Each version’s release is followed by another planning
stage. The idea is to make them short, but by short they mean a year or two.

15. But the process of software engineering is distinctly different from building a bridge. Every
application is different. There are only a limited number of kinds of bridges. Sure there are
differences, but they are variations on a theme. With software, by definition we are building
something new every time. Because if it wasn’t new, we’d just reuse the old library or
applicaiton.

16. Software is more like a puzzle. Something complicated, where we can never really know all
the details. It’s also a continuous process, you’re not ever done. It’s telling that we say a
software project is dead when it stops getting modified.

17. It’s perhaps better to think of software development as a river. The river is something which
is alive. Sometimes it’s calm and slow moving, other times there are whitewater rapids, but it
always is changing. Eventually the river reaches the ocean and dies. Living software is in a
constant state of change. The architecture which was appropriate for one stage of an
application’s life some times does not serve the next.

18. TDD - Test Driven
Development
So with this we return to TDD. The concept is that we do regression tests, tests in code itself,
as a way of helping us do software development.

19. TFD - Test First Development
before or after?
When we talk about TDD a related concept often arises. When do we write the tests. Before,
after, while coding, or while debugging? The reality is that it’s not that important the exact
order of things. What is important is that testing is integrated in to the task of programming
and not separated out. You can’t say, ok now we’re going to do 2 weeks of only writing tests.

20. Ok, so i’ve been saying that TDD is a better way of designing software. Now let’s get on to
the details. What exactly does a unit test look like?

21. An Example

22. the code the test
class Sum class SumTest < Test::Unit::TestCase
def sum a, b def test_sum
a+b s = Sum.new
end assert_equal 42, s.sum(30,12)
end assert_not_equal 5, s.sum(2,2)
end
end
Here’s a really simple example. Yes it’s stupid, yes it’s simple math. But remember unit tests
are about testing all the really small parts. So ALL examples are likely to be stupid simple.

23. the test
rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb
Loaded suite ejemplo_unit_test
Started
.
Finished in 0.000345 seconds.
1 tests, 2 assertions, 0 failures, 0 errors
rabble@blackbook-4 ~/code $
Now we simple run the unit test script and we can confirm that the test passes.

24. the code the test
class Sumate class SumateTest < Test::Unit::TestCase
def sum a, b def test_sum
a*b s = Sumate.new
end assert_equal 42, s.sum(30,12)
end assert_not_equal 5, s.sum(2,2)
end
end
Now we go back to the sum method and we modify it. Really we’re attempting to refactor,
where we change the implementation but NOT the functionality.

25. the test
rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb
Loaded suite ejemplo_unit_test
Started
F
Finished in 0.006554 seconds.
1) Failure:
test_sum(SumateTest) [ejemplo_unit_test.rb:13]:
<42> expected but was <360>.
1 tests, 2 assertions, 1 failures, 0 errors
We re-run the test and now we can we have a failure. We know it failed, but it’s not so easy to
see exactly what the problem is.

26. assert
assert
The fundamental building block of test::unit is the assert.

27. assert_equal expected, result
assert_equal for example, you provide what you expect to have and the actual result returned

28. assert is verification
What asserts do is verify code. And that is a good thing. We can know that there is a problem.

29. With this we can know if the system we are building is working or not. Unfortunately the
failure messages don’t explain clearly what the problem is.

30. the test
rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb
Loaded suite ejemplo_unit_test
Started
F
Finished in 0.006554 seconds.
1) Failure:
test_sum(SumTest) [ejemplo_unit_test.rb:13]:
<42> expected but was <360>.
1 tests, 2 assertions, 1 failures, 0 errors
With error messages like these it’s much harder to understand the problem. Really if your
whole library is a single method it’s pretty easy to figure out, but when you’ve got many
thousands of methods and tests, this kind of error is useless. What are the implications of
failing to get 42 when we sum something?

31. Perhaps we’re testing the wrong things. We’re too close to the code. We need to test
functionality and NOT the implementation.

32. The rhetoric around testing the unit has lead us astray. With the unit test all of our focus is
on the small parts. But the units themselves change, and when they do we are forced to
change the tests as well. When we have to update things two places that increases the
workload and we can become confused about whether the problem is in the code or the test.
We need to focus on what’s important, the functionality, the behaviour of each part of our
application.

33. BDD: Behaviour Driven
Developement
And with this, 33 slides in to the presentation, we finally arrive at BDD, Behaviour Driven
Development.

34. the same difference
What’s BDD? Well the idea of TDD is good, but the language, the terms used, and it’s libraries
are problematic. It encourages you to focus on the wrong things.

35. should vs assert
In BDD we use the method ‘should’ to explain how we want the application to behave, rather
than verifying the particular implementation.

36. in Test::Unit
assert_equal 42, s.sum(30,12)
assert_not_equal 5, s.sum(2,2)
in RSpec
s.sum(30,12).should.be 42
s.sum(2,2).should.not.be 5
Here’s an example of the same thing in test::unit and rspec. There’s not much difference in
what it does. But moving from a test to a spec, represents a different way of thinking about
the problem.

37. in Test::Unit
def test_sum
s = Sum.new
assert_equal 42, s.sum(30,12)
assert_not_equal 5, s.sum(2,2)
end
in RSpec
it “should add the numbers” do
s = Sum.new
s.sum(30,12).should.be 42
s.sum(2,2).should.not.be 5
end
la diferencia es poco, pero con diferente forma de pensar. desde test a spec.

38. the spec
describe Sum do
it quot;should add the numbersquot; do
s = Sum.new
s.sum(30,12).should == 42
s.sum(2,2).should.not == 5
end
end
And here we have the whole spec, you can see it does the same thing with a different
approach.

39. the spec
rabble@blackbook-4 ~/code $ ruby ejemplo_rpsec.rb
F
1) 'Sum should add the numbers' FAILED
expected: 42,
got: 360 (using ==)
ejemplo_rpsec.rb:10:
Finished in 0.007507 seconds
1 example, 1 failure
Now when we run the spec we can get a much better idea of what’s wrong when there is a
failure.

40. the test
rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb
Loaded suite ejemplo_unit_test
Started
F
Finished in 0.006554 seconds.
1) Failure:
test_sum(SumTest) [ejemplo_unit_test.rb:13]:
<42> expected but was <360>.
1 tests, 2 assertions, 1 failures, 0 errors
Remember back to the test.

41. the spec
rabble@blackbook-4 ~/code $ ruby ejemplo_rpsec.rb
F
1) 'Sum should add the numbers' FAILED
expected: 42,
got: 360 (using ==)
ejemplo_rpsec.rb:10:
Finished in 0.007507 seconds
1 example, 1 failure
What’s the difference really? First it’s clearer, but more importantly it represents another way
of thinking about tests.

42. What about
real world examples?
This is nice, but what about when applications get bigger, how do we translate a series of
specs in to a narrative about how the application works.

43. examples
DirectMessagesController handling GET /
direct_messages/1
- should be successful
- should change the status from unread
- should render show template
- should assign the found direct_message for the
view
- should not show a user's message to another
user
- should redirect if they try to view a deleted
message
Now i’m going to go in to some more flushed out examples i stole from one of my work
projects.

44. examples
describe quot;handling GET /direct_messages/1quot; do
define_models :direct_messages_controller
before do
login_as(:default)
end
def do_get
get :show, :id => direct_messages(:default).id
end
it quot;should be successfulquot; do
do_get
response.should be_success
end
it quot;should change the status from unreadquot; do
direct_messages(:default).unread?.should be_true
do_get
direct_messages(:default).reload.unread?.should be_false
end
it quot;should render show templatequot; do
do_get
response.should render_template('show')
end
it quot;should assign the found direct_message for the viewquot; do
do_get
assigns[:direct_message].should == direct_messages(:default)
Those end
english language specs of how the get method on the view action of the
direct_messages controller should work is generated by this code.
As you can see we login the user before each spec, and we also have created a little helper
method to call the action.
Each spec is very short.

45. examples
MembershipsController as a group administrator
removing someone from a group
- redirects to quot;group_path(@group)quot;
- assigns @group
- assigns @membership
- assigns flash[:notice]
- removes the membership
Here’s an example of the spec descriptions for a controller

46. examples
describe quot;as a group administratorquot; do
before do
login_as :jack
@group = groups(:default)
@membership = memberships(:jane)
end
describe quot;removing someone from a groupquot; do
define_models :memberships
act! { delete :destroy, :group_id => @group.permalink, :id => @membership.id }
it_redirects_to { group_path(@group) }
it_assigns :group, :membership, :flash => {:notice => :not_nil }
it quot;removes the membershipquot; do
act!
users(:jane).memberships.for(@group).should be_nil
end
end
end
Here are the associated spec code for the controller. Again you can see we’ve built up a
small dsl, domain specific language, around which we can keep our specs meaningful and
very short.

47. examples
A user who is a moderator
- should not be able to do admin functions
- should be able to do moderator functions
- should be able to do member functions
- should be able to do nonmember functions
Here’s another example, describing a user model’s permissions

48. describe quot;who is a moderatorquot; do
examples
define_models :permissions
before do
@user = users(:mod)
@group = groups(:default)
end
it quot;should not be able to do admin functionsquot; do
@user.can?(ACTIONS[:admin], @group).should be_false
end
[:moderator, :member, :nonmember].each do |role|
it quot;should be able to do #{role} functionsquot; do
@user.can?(ACTIONS[role], @group).should be_true
end
end
end
And we have the code which impliments it.

49. narratives & stories
So we’ve described the idea of a narratives, the individual statements, combined in to
description of the behaviour of each part of the application. Then you can combine those
narratives in to a larger story.

50. St

51. narrative
as a <role>
i want to <activity>
to do <a task>
as a user
i want to publish a comment
to directly participate in the forum
When we are making these spec style tests, we’re creating sentences which describe the
functionality, we can use them to describe the use of the application. We already know the
parts, it grows up from within the code as we write the parts. We can then string the specs
together in to a use story executed in code.

52. example of a story
Story quot;View Home Pagequot;, %{
As a user
I want to view my homepage
To get an overview of the status of the system
}, :type => RailsStory do
Scenario quot;Publisher without vídeosquot; do
Given quot;una empresa se llamaquot;, quot;Sin Vídeosquot; do |nombre|
@empresa = Empresa.create! :nombre => nombre
end
And quot;un usario se llamaquot;, quot;novideosquot; do |login|
@user = create_user login
end
And quot;el usuario es dequot;, quot;empresaquot;, quot;sin vídeosquot; do |klass, company_name|
@user.update_attribute :empresa, klass.classify.constantize.find_by_name(empresa_nombre)
end
And quot;logged in asquot;, quot;novideosquot; do |login|
post quot;/sessionsquot;, :login => login, :password => quot;testquot;
end
Here’s an example of stories written out in to code.. not entirely valid code.

53. an example of a story
Scenario quot;Basic userquot; do
Given quot;A created userquot;
And quot;two existing videosquot;
When quot;visiting /videosquot;
Then quot;both videos should be shownquot;
end

54. other things
There are some other rspec / bdd related issues which i’ve skipped over and not covered.

55. mocks y stubs
Rspec has a mock library included and you can also use external ones. A Mock or Stub object
replaces a real part of your system to make atomic testing simpler and faster.

56. shoulda
BDD over Test::Unit
RSpec is perhaps to magical. The most popular alternative is to use Shoulda which lets you
use an spec syntax.

57. matchy
BDD over Test::Unit
Jeremy McAnally who i’m working with has an alternative implementation which he’s been
working on.

58. Beyond Testing:
specs and Behaviour Driven
Developement
RSpec - www.rspec.info
BDD - www.behaviour-driven.org
Thank you very much.

59. Creative Commons
Photos Used
• http://flickr.com/photos/foreversouls/4254487/
• http://flickr.com/photos/ejpphoto/2314610838/
• http://flickr.com/photos/mrpunto/114862457/
• http://flickr.com/photos/orvaratli/2713730155/
• http://flickr.com/photos/horizon/287190887/
• http://flickr.com/photos/freewine/478332550/
• http://flickr.com/photos/patrick_q/98179665/
• http://flickr.com/photos/gadl/284995199/
• http://flickr.com/photos/darko_pevec/2300487155
• http://flickr.com/photos/atouchofcolor/376242632/
• http://flickr.com/photos/bcnbits/363695635/
• http://flickr.com/photos/thomashawk/340185708/
• http://flickr.com/photos/thomashawk/268524287/