Specification by example with specflow 110217

Speciﬁcation by example
with SpecFlow

Marcus Hammarberg
Avega Group
www.marcusoft.net
söndag den 20 februari 2011 (v.)

http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257


Executable specifications - that verifies that the system is behaving as expected.

You want it - i can give it to you!

This presentation is about Specification by example with SpecFlow.

As a last teaser, a quote by Dan North.
Hows that’s for something to live up to? :)

http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257 http://www.unit5.org/debordc/MPj04071740000[1].jpg






“It’s not a silver bullet –
but it seems a good start”

Dan North

http://www.stochasticgeometry.ie/wp-content/uploads/2009/10/snakeoil.jpg?w=257 http://www.unit5.org/debordc/MPj04071740000[1].jpg







My name is Marcus Hammarberg


I work for Avega Group as an agile coach and consultant


I have one wife


two hobbies


and three kids


I have been working as a programmer


since 1998 - mainly with Microsoft products


During the last 6 years I have been more and more interested in how you can work effective
together


with agile methods - such as scrum

Who are you?


BDD


What’s this BDD thing?

Wasn’t it TDD? Or was it DDD? or xDD?

Here’s a short background

TDD


TDD (Test driven development) is a way to write code where you start from a test describing
the functionality you’re about to write. The test drives the production code from the
developer.

There’s a lot of pros from that way of working
- you think about the behaviour of the code before you write it
- you can take small steps through complex problems
- you only write the code that is really needed
- the tests describes the functionality of the system

- and, as a funny side-effect, you get tests for all the code you’re writing.

But there’s some problems as well:
- where to start
- where to stop
- you easily end up writing test that describes HOW the production code is implemented
- TDD is lowlevel inside-out. You start at a single method or class. How did you get there?

Behaviour
Driven
Design


All these problems can easily get to you when you start TDD’ing.

This was addressed in a blogpost by Dan North (http://blog.dannorth.net/introducing-bdd/)
in which he coined the term BDD - Behavior Driven Design.

With BDD the behavior of the feature we’re about to implement is the focus.

Also the language we’re using is very important in BDD. The whole idea is to use a language
that everyone concerned can read and understand the langauge. So that we can talk about
WHAT the feature does - separated from HOW.

Outside-in
development

http://www.adaptivepath.com/blog/wp-content/uploads/2009/02/oi_model_inward.jpg


On of the problems of TDD, I took up, was that you didn’t know where to start. Or stop. That
has to do with you working inside-out.

BDD gives you the possibility to work from the outside-in.

You start with failing acceptance test that runs end-to-end. When that is implemented - this
feature is done!

This is called ATDD by some - Acceptance test driven development

What’s all these DD’s?


This is my humble understanding. Today
- TDD - helps you write better code by writing a test before the production code

- ATDD - start from a failing end-to-end acceptance test. Jenitta Andrea says this:

ATDD is to describe a functional requirements of a story as concrete examples or
expectations before development is started

- STDD - Story test driven development - don’t know much about this. Seems to be very like
ATDD

BDD - focus on the behavior of the feature. Liz Keogh :

BDD’s focus is to discover the things we didn’t know, especially in the contexts that the
scenarios or examples is run in

Speciﬁcation by example - a way to specify something by giving concrete examples on how
they should work.


• TDD





ATDD



they should work.


• TDD
• ATDD





ATDD



they should work.


• TDD
• ATDD
• STDD





ATDD



they should work.


• TDD
• ATDD
• STDD
• BDD





ATDD



they should work.


• TDD
• ATDD
• STDD
• BDD
• Speciﬁcation by example





ATDD



they should work.


Or you can use this great intro and skip the buzz-words altogether. From Gojko Adzic

This is a great sum up that I like by Janet Gregory:

Where we choose to call it BDD or ATDD or Speciﬁcation by Example, we want the same result
– a shared common understanding of what is to be built to try to build the ‘thing’ right the
ﬁrst time.

We know it never will be, but the less rework, the better.

Whether we choose to call it BDD or ATDD or
Specification by Example, we want the same
result – a shared common understanding
of what is to be built
to try to build the ‘thing’ right the first time.

We know it never will be, but the less rework,
the better.
Janet Gregory


Or you can use this great intro and skip the buzz-words altogether. From Gojko Adzic

This is a great sum up that I like by Janet Gregory:

Where we choose to call it BDD or ATDD or Specification by Example, we want the same result
– a shared common understanding of what is to be built to try to build the ‘thing’ right the
first time.

We know it never will be, but the less rework, the better.

Theory


Some theory on the background of Specﬁcation by example

User stories


User stories is a Extreme Programming way to keep requirement small and manageable.

It’s requirements in a sentence. Often using a template like this:

In order to [beneﬁt]
As a [role]
I want to [feature]

That ﬁrst part is seldom done in any use cases that i’ve read. And that’s the whole reason for
the function to exist.

But this is, even though it’s short, still to wide and open for interpretations. Therefore user
stories are often called “a placeholder for a further conversation”

User stories

In order to [benefit]
As a [role]
I want to [feature]


User stories is a Extreme Programming way to keep requirement small and manageable.

It’s requirements in a sentence. Often using a template like this:

In order to [beneﬁt]
As a [role]
I want to [feature]

That ﬁrst part is seldom done in any use cases that i’ve read. And that’s the whole reason for
the function to exist.

But this is, even though it’s short, still to wide and open for interpretations. Therefore user
stories are often called “a placeholder for a further conversation”


And that discussion should lead to ...

In order to better understand each other one powerful technique is to give concrete example.

Just imaging how hard it would be to learn how to tie a Windsor-tie with just the theoretical
text...

It’s much easier with a couple of pictures to guide us


So a good way to get this in a user story is to give a couple of examples on how it could be
played out.

They could be written with the template; Given / When / Then.

Given [initial context]
When [event]
Then [an outcome]


played out.


Scenario 1: Account is in credit
Given the account is in credit
And the card is valid
And the dispenser contains cash
When the customer requests cash
Then ensure the account is debited
And ensure cash is dispensed
And ensure the card is returned


played out.


Scenario 2: Account is overdrawn
past the overdraft limit
Given the account is overdrawn
Then ensure a rejection message is
displayed
And ensure cash is not dispensed

played out.



This looks a lot like a test-case, right?
Or maybe a detailed requirements document?
And wouldn’t that be nice to code after?

There’s other pro’s too:

- this foster the use of the same language for requirements, test and even maybe code

- we can easily discuss the scenarios, add and remove stuff as we learn more. Before the
system is implemented

- it’s easy to see which functionality that is included (in a release for example)

- and ﬁnally it’s VAD separated from HOW

If you now only didn’t have to run the test cases ...

Scenario 1: Account is in credit
Given the account is in credit
And the dispenser contains cash
Then ensure the account is debited
And ensure cash is dispensed


This looks a lot like a test-case, right?
Or maybe a detailed requirements document?
And wouldn’t that be nice to code after?

There’s other pro’s too:

- this foster the use of the same language for requirements, test and even maybe code

- we can easily discuss the scenarios, add and remove stuff as we learn more. Before the
system is implemented

- it’s easy to see which functionality that is included (in a release for example)

- and ﬁnally it’s VAD separated from HOW

If you now only didn’t have to run the test cases ...

Wrap-up

Executable speciﬁcations
that veriﬁes that the system behaves as
expected,
readable by all in the team, and around
it


This was the promise in the outset - is it kept?

It’s not in the tool

http://www.barbellsandbacon.com/wp-content/uploads/2010/02/mlss_mario-hammer.jpg


It’s very important to remember that the main beneﬁt is not in the tool (which we will talk
about from now on).

The beneﬁt is that you get together and write down your common, shared knowledge in a
well known format. In one place.

That’s why anyone would dare call this a Silver Bullet

Tools


When we have these scenarios in place it would be wonderful if we could use some kind of
tool to verify that the system actually behaves as speciﬁed.

And of course you could - this presentation would not be very meaningful otherwise.

This was the idea of Cucumber that ﬁrst was created in the Ruby-community. Cucumber can
actually execute scenarios written in the Given/When/Then-syntax. This syntax is called
Gherkin.

For .NET there’s a lot of BDD frameworks that emulates Cucumber, but in my opinion the best
one is SpecFlow. You can think of SpecFlow as a port of Cucumber for .NET.

Tools

http://buyorganicvegetable.com/images/Cucumber.jpg


When we have these scenarios in place it would be wonderful if we could use some kind of
tool to verify that the system actually behaves as speciﬁed.

And of course you could - this presentation would not be very meaningful otherwise.

This was the idea of Cucumber that ﬁrst was created in the Ruby-community. Cucumber can
actually execute scenarios written in the Given/When/Then-syntax. This syntax is called
Gherkin.

For .NET there’s a lot of BDD frameworks that emulates Cucumber, but in my opinion the best
one is SpecFlow. You can think of SpecFlow as a port of Cucumber for .NET.

How does this work?


But how can text be executable - it’s so open, you can write almost anything?

It’s pretty simple really:
- SpecFlow creates an automated test for each scenario, with the test runner of your choice
- Each row in the scenario (called a step) will then call a method, that you supply, called step
definition. The method defines what the step means; How do you define “Given I am on the
Activity list”

It might be easier with an example (nudge, nudge :)):

- For the scenario above - SpecFlow will create a test called NavigateToCreateNewActivity
- in that test SpecFlow will look and call methods with attributes as the steps, shown in the
example

How does this work?

[Given("I am on the Activity list")]
public void ThisNameIsNotImportant()

[When("I click the Create New link")]
public void SpecFlowSimplyUsesTheAttribute()

[Then("I should be on the Create Activity page")]
public void AndCouldntCareLessOfYourMethodName()


But how can text be executable - it’s so open, you can write almost anything?

It’s pretty simple really:
- SpecFlow creates an automated test for each scenario, with the test runner of your choice
- Each row in the scenario (called a step) will then call a method, that you supply, called step
definition. The method defines what the step means; How do you define “Given I am on the
Activity list”

It might be easier with an example (nudge, nudge :)):

- For the scenario above - SpecFlow will create a test called NavigateToCreateNewActivity
- in that test SpecFlow will look and call methods with attributes as the steps, shown in the
example

No really,
how does it WORK?

http://www.specflow.org/getfile/a67f8885-4900-4803-a972-5f3c57364097/artifacts.aspx


Here’s the technical part:
1 - You write your scenarios in a .feature-file. Every time it’s saved SpecFlow will generate a
test-class, that contains a test for each scenario

2 - The generated code (that you never care about that code, or rather don’t touch it) it’s just
there to glue together your scenario with your step definitions.

3 - When the tests are run by the test runner (just as ordinary unit tests), it will call you step
definitions in the order they are written in the scenario

4 - And your step definitions then call your application or domain.

Step definitions

http://www.treehugger.com/green%20footsteps%20generic.jpg


So the trick lies in the step definitions.

They are realizing the “promise” of the steps in the scenario.

So when we write a step definition for “When I click the Create New Link” we need to find that
link on the web page (I guess) and then click it.

Step definitions

Scenario: Navigate to Create new Activity
Given I am on the Activity list
When I click the Create New link
Then I should be on the Create Activity page

http://www.treehugger.com/green%20footsteps%20generic.jpg


So the trick lies in the step definitions.

They are realizing the “promise” of the steps in the scenario.

So when we write a step definition for “When I click the Create New Link” we need to find that
link on the web page (I guess) and then click it.

Automation

http://www.cultofmac.com/wordpress/wp-content/uploads/2010/09/automator-icon.png


That’s called automation. To write a program that automate the running of another program.

There’s a lot of framework that assists in this. But that’s not the purpose of this presentation.

I would recommend Watin (http://watin.sourceforge.net/) for web applications and White
(http://white.codeplex.com/) for Windows based applications.

Gherkin


For a Swede Cucumber and Gherkins are more or less the same...

But apparently that’s not true - these are Gherkins.

And Gherkin is also the name of the language that scenarios are written in, invented by Alsak
Hellesoy.

We will now examine it a bit closer.


The language is not big. This is the whole language.

And that means it can be translated. To Swedish. Arabic. Bulgarian.

And some more ... dubious languages too.

but it gives us possibilities to write our scenarios in a suitable language.

Feature

http://www.lushtshirts.co.uk/images/products/bug-feature.jpg


Features and scenarios are written in .feature. It contains an feature description and one or
more scenarios.

It’s a plain text ﬁle and it checked in with and versioned together with the code of the project.

The feature is the description of the functionality. It’s plain text and doesn’t affect how the
scenario is executed or interpreted. It’s very important though - in order to understand the
scenarios.

Every feature has a title and is written after the keyword Feature:

Scenario

http://images2.fanpop.com/images/photos/7000000/Friends-33-friends-7046166-1280-1024.jpg


A scenario also has a title and one or more steps

You can use the Friends-rule to set a good name of the scenario. You know, the episodes of
Friends was called “The one where Ross and Rachel became a couple”, about half of the
episodes was called that if I remember correctly :)

GIVEN
WHEN
THEN

The scenarios follows the template Given / When / Then with the following meaning:
- Given - in this step you setup the system in the desired state.
- When - this is the actual action the scenario is about
- Then - asserts that the right things happend

The scenario is executed top down. And the Given / When / Then keyword can be repeated
several times in the scenario, if you like. Not recommended but possible.

And speaking of not recommended - you can write the Given / When / Thens in any order
you want. But then you lose any pros of the structure in the ﬁrst place.

Scenario: Simple GWT
Given the intial state of the application
is Running
When I ask what the application state is
Then I should see Running as the answer





But? And?


There’s two more keywords AND and BUT to connect several steps with each other.

AND and BUT refers back to the word above it. They are used to get a better ﬂow in the
language.

Try not do use AND/BUT in the When-step. Let that be a single action.

Expressions


Sometime the steps are very similar to other existing steps.

And it would be horrible if we needed to write every step definition explicit.

Therefore Regular expressions can be used to have some part of the steps as variables.

That will be converted into parameters to our step definition methods. Please note that the
order of the regular expressions are important.

If you, like me, don’t like RegExps you’ll be happy to known that these two will get you very
far.

Also, since SpecFlow 1.4, you get intellisense for the step definitions you already written

Expressions
When I write 'My test string' in the
textbox 'Test textbox'






far.


Expressions

[When(@"I write '(.*)' in the textbox '(.*)'")]
public void WriteInTextBox(string textToWrite,
string wellKnownName)






far.


Expressions

[When(@"I write '(.*)' in the textbox '(.*)'")]
public void WriteInTextBox(string textToWrite,
string wellKnownName)

(.*) - en sträng => string
(d+) - ett heltal => integer






far.


Background


With the keyword background you can run some steps before each scenario in the feature.

It’s just like a ordinary scenario, it’s just run before the scenarios.

<CLICK>
In this example we set up the system under test in a common state for all the scenarios

Hooks


From Cucumber SpecFlow borrows the concept of hooks.

Hooks are events that is ﬁred in the following order.

And you can “hook in” code here to do useful things:

- Restore a database

- Dump a log on error

- Init variables before steps

- etc

Tags


Hooks are cool but they become really powerful combined with Tags.

Tags are simple small markers that can be written in your feature.

Only one tag is supported by SpecFlow out of the box; @ignore that simply ignores the
scenario. Great for stuff that isn’t ready yet.

A tag becomes a test category and can be used to categorize your tests (acceptance test,
dailes, slow running etc)

But you can use tag to overload the hooks. So that a certain hook only is run when the
scenario or feature has a certain tag.

@ignore








You can use table to manage multiple data. The data will be passed to you step deﬁnition as a
SpecFlow Table that has values for columns and rows. From that table you can read a data.

[Given("I have the following persons")]
public void IHaveTheFollowingPersons
(Table personsTable)


You can use table to manage multiple data. The data will be passed to you step deﬁnition as a
SpecFlow Table that has values for columns and rows. From that table you can read a data.

SpecFlow.Assist


Working with tables can be quite messy. But in this namespace (SpecFlow.Assist) there’s
some helpers to help us get around that.

Check these examples out:

- https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/
05TablesAndAssist

- https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/
06CompareToAssist

Scenario outline


By using a scenario outline you could reuse the same scenario with several different data.

In this way you can easily write several variants (permutations) of test data for the same
scenario.

This example will create 5 test that performs the calculation. How this is done depends on the
testrunner you’re using; NUnit creates TestCases that sends data to the unit test while MsTest
creates several tests.

Langauge


You can change the language in your feature by using the #language : sv (for Swedish).

But probably you want to do that for all your features, in the .config-file of the specification
project.

ScenarioContext.Current


ScenarioContext is a small dictionary that is used to send data between your steps. It’s much
better than to use private variables in your step classes since that hinders step reuse etc.

So better is to group your step deﬁnitions per logical unit; controller, page, function

See https://github.com/marcushammarberg/ProgressiveNetDemos/tree/master/Specs/
031ScenarioContext

AvegaCoach Activity Log


I have developed a small example site that shows how these techniques can be used.

It’s a small site to keep track of the activities that persons in my group, AvegaCoach,
undertake.

This is the domain model and I’ve scaffolded the whole site.

I have written two sets of speciﬁcations
- against the controller level (https://github.com/marcushammarberg/ProgressiveNetDemos/
tree/master/SpecsAgainstController)
- against the web pages (https://github.com/marcushammarberg/ProgressiveNetDemos/
tree/master/SpecsAgainstGUI)

to show how they differ.

On with
the code!

http://burningbird.net/photos/thumb-victwait.jpg


On to the code!

Learn more


Learn more
• http://blog.dannorth.net/introducing-bdd/


Learn more
• http://www.specﬂow.org/


Learn more
• https://github.com/techtalk/SpecFlow


Learn more
• http://github.com/techtalk/SpecFlow-Examples


Learn more
• http://github.com/aslakhellesoy/cucumber/wiki


Learn more
• http://www.marcusoft.net/search/label/SpecFlow


Learn more
• http://www.marcusoft.net/search/label/SpecFlow
• https://github.com/marcushammarberg/
ProgressiveNetDemos


And even more


www.marcusoft.net
marcus.hammarberg@avegagroup.se

Thanks for listening


The end

Specification by example with specflow 110217

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von Marcus Hammarberg

Mehr von Marcus Hammarberg (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Specification by example with specflow 110217