Acceptance Test Driven Development, or ATDD, is a very effective technique for driving and guiding development, that helps enhancing communication between developers and other project stakeholders, document your product, and report on project progress. Learn how to use automated web tests as a communication, reporting and product documentation tool. Automated web tests can be used to generate living, illustrated documentation about how the system is used. Organize your automated web tests more effectively, so that they can be used to report on test coverage in terms the business can understand. And discover a new open source library designed to helps developers and testers write better automated acceptance tests for web applications using WebDriver/Selenium 2.

1. Completing the circle
Automated web tests as a team communication tool
John Ferguson Smart

2. So who is this guy, anyway?
Consulta
nt
Trainer
Mentor
Author
Speaker
Coder
John Fer
guson S
mar t

3. ATDD
or Specification by example
The story of your app

4. User stories
As a job seeker
I want to find jobs in relevant categories
So that I can find a suitable job
Features/Epics

5. User stories
As a job seeker
I want to find jobs in relevant categories
So that I can find a suitable job
Acceptance criteria
☑
The
job
seeker
can
see
available
categories
on
the
home
page
☑
The
job
seeker
can
look
for
jobs
in
a
given
category
☑
The
job
seeker
can
see
what
category
a
job
belongs
to

6. User stories
As a job seeker
I want to find jobs in relevant categories
So that I can find a suitable job
Acceptance criteria
☑
The
job
seeker
can
see
available
categories
on
the
home
page
☑
The
job
seeker
can
look
for
jobs
in
a
given
category
☑
The
job
seeker
can
see
what
category
a
job
belongs
to
scenario "A job seeker can see the available job categories on the home page",
{
when "the job seeker is looking for a job",
then "the job seeker can see all the available job categories"
}
Automated acceptance test

7. scenario "A job seeker can see the available job categories on the home page",
{
when "the job seeker is looking for a job",
then "the job seeker can see all the available job categories"
}
Automated acceptance test
Implemented development tests Implemented acceptance tests

8. Not All Web Tests are Acceptance Tests
Technical tests are for Acceptance tests are
the dev team for everyone else

9. Not All Web Tests are Acceptance Tests
Technical tests talk to Acceptance tests talk
the dev team to everyone else

10. The art of sustainable web tests
or how not to have web tests like this

11. So what are our goals?

12. Good web tests should
speak everybody’s language

13. Good web tests should
hide unnecessary details

14. Good web tests should
use reusable components

15. Good web tests should
be low maintenance

16. The Three Ways of Automated Web Testing
Record/Replay
Scripting
Page Objects

17. Record-replay automated tests
Promise Reality

18. Record-replay automated tests

19. Script-based automated tests
Selenium RC
HTMLUnit
JWebUnit
Canoe Webtest
Watir

20. Script-based automated tests
Selenium RC
HTMLUnit
JWebUnit
Canoe Webtest
Watir

21. How about Page Objects?
Reusable
Low
maintenance
Hide unnecessary detail
2

22. A sample Page Object
A web page

23. A sample Page Object
A Page Object
FindAJobPage
lookForJobsWithKeywords(values : String)
getJobTitles() : List<String>

24. A sample Page Object
public class FindAJobPage extends PageObject {
An implemented
WebElement keywords;
WebElement searchButton;
Page Object
public FindAJobPage(WebDriver driver) {
super(driver);
}
public void lookForJobsWithKeywords(String values) {
typeInto(keywords, values);
searchButton.click();
}
public List<String> getJobTitles() {
List<WebElement> tabs = getDriver()
.findElements(By.xpath("//div[@id='jobs']//a"));
return extract(tabs, on(WebElement.class).getText());
}
}

25. A sample Page Object
public class WhenSearchingForAJob {
@Test
public void searching_for_a_job_should_display_matching_jobs() {
FindAJobPage page = new FindAJobPage();
page.open("http://localhost:9000");
page.lookForJobsWithKeywords("Java");
assertThat(page.getJobTitles(), hasItem("Java Developer"));
}
}
A test using this
Page Object

26. Sustainable web tests
Are we there yet?

27. Acceptance Tests
The high-level view
So
where
are
we
at?

28. Page Objects
Page
Objects
rock!
Implementation focus

29. How do we bridge the gap?

30. How do we bridge the gap?
Test steps

31. scenario "A job seeker can see the available job categories on the home page",
{
when "the job seeker is looking for a job",
then "the job seeker can see all the available job categories"
}
Automated
scenario "The user can see the available job categories on the home page",
{
when "the job seeker is looking for a job",
{
job_seeker.open_jobs_page()
}
then "the job seeker can see all the available job categories",
{
job_seeker.should_see_job_categories "Java Developers", "Groovy Developers"
}
}
Implemented
JobSeekerSteps
JobSeekerSteps
JobSeekerSteps
open_jobs_page()
open_jobs_page()
open_jobs_page()
should_see_job_categories(String...
categories)
should_see_job_categories(String...
categories)
... should_see_job_categories(String...
categories)
...
...
Step libraries

32. scenario "The user can see the available job categories on the home page",
{
when "the job seeker is looking for a job",
{
job_seeker.open_jobs_page()
}
then "the job seeker can see all the available job categories",
{
job_seeker.should_see_job_categories "Java Developers", "Groovy Developers"
}
}
Implemented Tests
JobSeekerSteps
JobSeekerSteps
JobSeekerSteps
open_jobs_page()
open_jobs_page()
open_jobs_page()
should_see_job_categories(String...
categories)
should_see_job_categories(String...
categories)
... should_see_job_categories(String...
categories)
...
...
Step libraries
Page Objects

33. Test steps
help organize your tests

34. Test steps
are a communication tool

35. Test steps
are reusable building blocks

36. Test steps
help estimate progress

37. And so we built a tool...

38. Webdriver/Selenium 2 extension
Organize tests, stories and features
Record/report test execution
Measure functional coverage

40. Thucydides in action A simple demo app

41. Defining your acceptance tests
scenario "A
job seeker
{ can see the
available j
ob categori
when "the j es on the h
ob seeker i ome page",
then "the j s looking f
ob seeker c o r a j o b ",
} an see all
the availab
le job cate
gories"
High level requ
irements...
scenario "The administrator adds a new category to the system",
{
given "a new category needs to be added to the system",
when "the administrator adds a new category",
then "the system should confirm that the category has been created",
and "the new category should be visible to job seekers",
}
{
scenario "The admini
strator deletes a ca
tegory from the syst
em",
...defined in business terms
given "a category ne
eds to be deleted",
when "the administra
tor deletes a catego
then "the system will ry",
confirm that the cate
and "the deleted cate gory has been delete
gory should no longer d",
} be visible to job se
eker s",
focus on business value

42. Organizing your requirements
Features public class Application {
@Feature
public class ManageCompanies {
public class AddNewCompany {}
public class DeleteCompany {}
public class ListCompanies {}
}
@Feature
public class ManageCategories {
public class AddNewCategory {}
public class ListCategories {}
public class DeleteCategory {}
}
@Feature Stories
public class BrowseJobs {
public class UserLookForJobs {}
public class UserBrowsesJobTabs {}
}
}

43. Implementing your acceptance tests
using "thucydides" We are testing this story
thucydides.uses_steps_from AdministratorSteps
thucydides.uses_steps_from JobSeekerSteps
thucydides.tests_story AddNewCategory
An acceptance criteria
scenario "The administrator adds a new category to the system",
{
given "a new category needs to be added to the system",
{
Narrative style
administrator.logs_in_to_admin_page_if_first_time()
administrator.opens_categories_list()
} Step through an
when "the administrator adds a new category", example
{
administrator.selects_add_category()
administrator.adds_new_category("Scala Developers","SCALA")
}
then "the system should confirm that the category has been created",
{
administrator.should_see_confirmation_message "The Category has been created"
}
and "the new category should be visible to job seekers",
{ Still high-level
job_seeker.opens_jobs_page()
job_seeker.should_see_job_category "Scala Developers"
}
}

44. Some folks prefer JUnit...
@RunWith(ThucydidesRunner.class)
@Story(AddNewCategory.class) Thucydides handles the
public class AddCategoryStory {
web driver instances
@Managed
public WebDriver webdriver;
@ManagedPages(defaultUrl = "http://localhost:9000")
public Pages pages;
@Steps
public AdministratorSteps administrator;
@Steps
Using the same steps
public JobSeekerSteps job_seeker;
@Test
public void administrator_adds_a_new_category_to_the_system() {
administrator.logs_in_to_admin_page_if_first_time();
administrator.opens_categories_list();
administrator.selects_add_category();
administrator.adds_new_category("Java Developers","JAVA");
administrator.should_see_confirmation_message("The Category has been created");
job_seeker.opens_job_page();
job_seeker.should_see_job_category("Java Developers");
} Tests can be pending
@Pending @Test
public void administrator_adds_an_existing_category_to_the_system() {}
}

45. Defining your test steps
public class AdministratorSteps extends ScenarioSteps { A step library
@Step
public void opens_categories_list() {
AdminHomePage page = getPages().get(AdminHomePage.class);
page.open();
page.selectObjectType("Categories");
} High level steps...
@Step
public void selects_add_category() {
CategoriesPage categoriesPage = getPages().get(CategoriesPage.class);
categoriesPage.selectAddCategory();
}
@Step
public void adds_new_category(String label, String code) {
EditCategoryPage newCategoryPage = getPages().get(EditCategoryPage.class);
newCategoryPage.saveNewCategory(label, code);
}
...implemented
@Step
public void should_see_confirmation_message(String message) {
with Page Objects
AdminPage page = getPages().get(AdminPage.class);
page.shouldContainConfirmationMessage(message);
}
@StepGroup ...or with other steps
public void deletes_category(String name) {
opens_categories_list();
displays_category_details_for(name);
deletes_category();
}
}

46. Defining your page objects
public class EditCategoryPage extends PageObject {
@FindBy(id="object_label")
WebElement label; Provides some useful
@FindBy(id="object_code")
utility methods...
WebElement code;
@FindBy(name="_save")
WebElement saveButton;
public EditCategoryPage(WebDriver driver) {
super(driver);
}
public void saveNewCategory(String labelValue, String codeValue) {
typeInto(label, labelValue);
typeInto(code, codeValue);
saveButton.click();
}
} but otherwise a normal
WebDriver Page Object

47. Data-driven testing
Test data
categories.csv
public class DataDrivenCategorySteps extends ScenarioSteps {
Test steps
public DataDrivenCategorySteps(Pages pages) {
super(pages);
}
private String name;
private String code;
@Steps
public AdminSteps adminSteps;
public void setCode(String code) {...}
public void setName(String name) {...}
@Step
public void add_a_category() {
adminSteps.add_category(name, code);
}
}

48. Data-driven testing
Test data
categories.csv
public class DataDrivenCategorySteps extends ScenarioSteps {
Test steps
public DataDrivenCategorySteps(Pages pages) {
super(pages);
}
private String name;
private String code;
@Steps
public AdminSteps adminSteps;
@Steps
public void setCode(String code) {...}
public DataDrivenCategorySteps categorySteps;
public void setName(String name) {...}
@Step @Test
public void add_a_category() {
Call this step for
public void adding_multiple_categories() throws IOException {
adminSteps.add_category(name, code);
steps.login_to_admin_page_if_first_time();
}
}
steps.open_categories_list(); each row
withTestDataFrom("categories.csv").run(categorySteps).add_a_category();
}

49. Now run your tests

50. Displaying the results in easyb

51. Displaying the results in easyb

52. Thucydides reports
Browse the features

53. Browse the stories
Browse the stories

54. Browse the stories
Browse the test scenarios

55. Illustrating the test paths
A test
scenario
Steps
Test
scenarios

56. Illustrating the test paths
Test
scenario
Steps
Test
scenarios
Screenshots

57. Functional coverage
Features

58. Functional coverage
User
stories

59. Functional coverage
Passing tests
Pending tests

60. Progress

61. Functional coverage
Test
scenarios

62. Functional coverage

63. References
easyb.org
github.com/thucydides-webtests
wakaleo.com/thucydides

64. The Circle is Complete
Automated web tests as a team communication tool
John
Ferguson
Smart
Email:
john.smart@wakaleo.com
Web:
hCp://www.wakaleo.com
TwiCer:
wakaleo