Behavior Driven Development using Cucumber JVM

1. BDD using Cucumber-
JVM
Vijay Ramaswamy
Test Architect & Consultant

2. Introduction to BDD (~5 mins)
 BDD stands for Behavior Driven Development
 An agile software development process founded by Dan North
 Evolved from Test Driven Development (TDD):
 Often misunderstood as a testing methodology, TDD is actually a software
development process
 BDD focuses on getting the words right:
 “What is the intended behavior”, NOT “What should I test?”
 Attempts to prevent the misconceptions that arise while using TDD
 Can be applied at different levels – the overall system, a specific class, etc.
2

3. BDD for unit tests (~10 mins)
 This is where the BDD movement started from
 Extends the basic principles of TDD (refer to image on the right)
 Test names should be sentences, which avoid using the word
“test”, preferring words such as “should” instead:
 E.g.: emptyStackShouldThrowExceptionOnPop() instead of
testEmptyStackPop()
 Test methods usually implement the test logic directly, without
delegating to a totally different layer of the test framework
 Requires the usage of test doubles (mocks, stubs, etc.) to isolate
the code under test
3

4. BDD for unit tests (~10 mins)
 Other related terminologies:
 Specification Driven Development
 Key benefit:
 Provides all the benefits of TDD, with an added degree of clarity
 It is easy to implement BDD at this level using existing tools such
as JUnit, Hamcrest matchers, AssertJ, etc.
 Apart from this, specialized tools are also available. A few
Java/Groovy tools in this space: Spock, JDave, EasyB
4

5. BDD for acceptance tests (~10 mins)
 Acceptance criteria are defined using natural, English-like language (Domain
Specific Language - DSL):
 Think from the end user’s point of view
 Gherkin (Given-When-Then) is one popular language used in this space, although it
is not a mandatory choice
 Acceptance criteria should be directly automatable, using any appropriate
tool of choice
 Acceptance criteria are usually decoupled from their implementation:
 E.g.: Acceptance criteria using Gherkin, implementation using Java
5

6. BDD for acceptance tests (~10 mins)
 Acceptance criteria should be collaboratively defined by the entire team (not
just QA!) -> This is sometimes known as the 3 amigos principle (3 amigos =
Dev, BA and QA)
 Other related terminologies:
 Acceptance Test Driven Development (ATDD), Specification by Example, Domain
Driven Design
 Key benefit:
 Promotes a shared understanding of user stories within the team, and improves
overall communication
 It is difficult to implement BDD at this level without specialized tooling. Some
of the Java tools in this space: Cucumber-JVM, JBehave, Concordion
6

7. Outside-in BDD (~5 mins)
 It is possible and encouraged to practice BDD at both levels
mentioned in the last couple of slides:
 Start with BDD at the acceptance test layer
 Proceed to expand the acceptance criteria into lower level specifications
as appropriate
 Focus on getting the specification tests to pass first, followed by getting
the acceptance tests to pass
 This is known as outside-in BDD
7

8. Cucumber: An introduction
 A tool designed to enable BDD for acceptance tests:
 Originally written in Ruby, then ported to Java
 The key components of Cucumber are:
 Feature files:
 Used to define user stories and corresponding tests
 Glue code:
 Used to implement (automate) tests defined within feature files
 Glue code is further classified into:
 Step definitions
 Hooks
8

9. Cucumber: Feature files (~15 mins)
 Text files that use Gherkin to define the user stories (features) and
corresponding acceptance criteria (scenarios)
 Acceptance criteria should be in the form of concrete examples:
 Include actual test data within the test steps as relevant
 Feature files promote well defined traceability between requirements and
tests, and are meant to act as the “source of truth” for the entire team
9

10. Cucumber: Feature files (~15 mins)
 Gherkin syntax:
 Given, When, Then, And, But
 Feature, Scenario, Background, Scenario Outline & Examples
 Step arguments:
 Enable passing in more complex test data as part of a test step
 Options available:
 Doc strings
 Data tables
 Tags:
 User-defined strings prefixed with “@” -> similar to Java annotations
 The goal is to enable grouping of features/scenarios as required
 Comments:
 Use “#” to specify comments if required
10

11. Cucumber: Glue code (~15 mins)
 Step definitions (stepdefs):
 Code implementation for various steps used within the scenarios:
 One Java method corresponding to each unique test step
 Use annotations (such as @Given, @When, etc.) and regular expressions to match
step definition code to the corresponding steps
 You can use Lambda expressions if using Java 8 or higher
 Use method arguments to read test data specified within the test steps
 Use assertions to validate conditions specified within the test steps
 It is possible to auto-generate stepdef skeleton code from feature files:
 Simply execute your newly created feature, and Cucumber will automatically print out
code stubs corresponding to all unimplemented test steps
 WebDriver can be used to implement stepdefs for web based applications
11

12. Cucumber: Glue code (contd.)
 Hooks:
 Scenario hooks:
 Code that runs before or after each scenario gets executed (similar to @Before and @After in
TestNG)
 Can be used for global variable initialization, test harness setup, etc.
 Note that hooks are invisible to people reading your feature files; consider using a background
instead of a hook if it makes sense
 Tagged hooks:
 Hooks that will be executed only for specific tags
 “Global” hooks:
 “Global” hooks are those which would run only once per execution (similar to @BeforeClass and
@BeforeSuite in TestNG)
 These are NOT supported in Cucumber!
 If you do need this, you may need to implement some kind of programmatic work-around
12

13. Cucumber: Test execution (~15 mins)
 Cucumber provides multiple test runners that you can use:
 JUnit runner
 TestNG runner
 Command line runner
 All test runners have a common set of configuration options. A few prominent
options:
 Path to the feature files:
 A list of feature files, or a list of directories containing feature files
 If left unspecified, Cucumber searches within the same package as the test runner
13

14. Cucumber: Test execution (~15 mins)
 All test runners have a common set of configuration options. A few prominent
options (contd.):
 Path to the glue code:
 A list of packages on the classpath
 If left unspecified, Cucumber searches within the same package as the test runner
 A set of tags to be executed
 Plugins to be used for formatting your test reports (refer next slide for more
details)
 “Strict” execution:
 Set to true if you want Cucumber to fail scenarios with unimplemented / pending step
definitions
 Default is “false”
14

15. Cucumber: Reporting (~10 mins)
 Cucumber enables generation of test execution reports by specifying a list of
report formatting plugins while configuring your test runner
 Various formatting plugins are available in-built, such as:
 Pretty
 JUnit
 HTML
 JSON
 Usage
15

16. Cucumber: Reporting (~10 mins)
 It is easy to create your own formatting plugins as well:
 This has led to the development of various custom plugins, many of which are even
more user-friendly and visually appealing than the built-in ones
 It is possible to include your own custom content into a report:
 The scenario.write() API:
 Embed text into a report
 The scenario.embed() API:
 Embed images or even videos into a report
16

17. Cucumber: Best practices (~5 mins)
 Avoid using Cucumber as a standalone automation framework -> remember
that it is primarily designed as a BDD tool!
 Write scenarios at the business logic level -> avoid implementation details:
 Test implementations may change, but the tests themselves should not!
 Keep scenarios independent of each other as far as possible
 Organize features neatly into Epics or Themes as appropriate, so that they are
easy to search when required
 Reuse stepdefs to the maximum extent possible; avoid duplication of steps
while writing scenarios:
 Use Eclipse autocomplete
 Evolve a common DSL for the entire team
17

18. Cucumber: Best practices (~5 mins)
 Keep stepdef methods unique:
 It is a common mistake to repeat stepdefs in multiple Java files, leading to
ambiguous matches during execution
 Leverage backgrounds wisely to keep your scenarios more crisp and readable
 Avoid scenario outlines unless really necessary – most times, a single set of
test data should be sufficient to test a given scenario
 Use tags to organize your scenarios and features into groups as appropriate
 Use dependency injection to share state between step definitions as required
 Feature files should be the single source of truth for the entire team:
 If using a system such as JIRA, explore options to integrate
18

19. Reference material
 Behavior Driven Development references:
 http://behaviourdriven.org/
 https://dannorth.net/introducing-bdd/
 Demo websites for practice:
 http://automationpractice.com
 http://newtours.demoaut.com/
 http://demoqa.com/
 http://www.way2automation.com
19

20. Recap
 An overview of BDD:
 Introduction to BDD
 BDD for unit tests
 BDD for acceptance tests
 Outside-in BDD
 An overview of Cucumber:
 Introduction to Cucumber
 Feature files
 Glue code
 Test execution
 Reporting
 Best practices
20

21. THANK YOU!
Vijay Ramaswamy
Test Architect & Consultant
21