Presentation delivered by Craig Smith, Adrian Smith with Dallas Thorneycroft at STANZ 2011 in Melbourne, Australia. When Suncorp started down the path of rolling out its agile program over four years ago, it was viewed by many internally and the industry with much scepticism and angst, yet now it is approaching mainstream adoption in the industry. One of the key challenges of becoming agile was improving our approach to testing and quality. In this talk we will talk about why we had to change, why we had to improve the “speed to cool” in relation to testing, our challenges and approach and our blueprint for the “future tester” at Suncorp. Like our agile journey, our vision for testing has been regarded an ambitious, so join us to hear why we believe raising the profile, empowerment and skillset of testing is critical to our (and your) future success.

1. The Future
Tester at
Suncorp: Craig Smith
Adrian Smith
A Journey of with
Building Quality Dallas Thorneycroft
In Through Agile

2. Welcome…
Image: © Walt Disney Pictures / Pixar http://st-listas.20minutos.es/images/2008-07/26818/342659_640px.jpg?1216524692 C A

3. Welcome…
Not
Related…
Image: © Universal Pictures http://cf1.imgobject.com/backdrops/8af/4bc92693017a3c57fe00f8af/twins-original.jpg C

4. First, let’s go
back in time
Image: © Universal Pictures http://www.picalls.com/redimensionador/1920x1440/1920x1440.php?src=/1920x1440/./data/media/4/Back_to_the_Future.jpg&w=1024&h=768&zc=0 C

5. Waterfall
…1970
Image: http://images.jbrains.ca/theory_of_bdd/royce_diagram.jpg C

6. Read
past
page 2!
Image: http://blogs.kcls.org/librarytalk/vintage%20boy%20reading.jpg C

7. 1980’s…
1990’s…
Image: http://spaghettitesting.files.wordpress.com/2009/04/fad.png?w=396&h=296
C

8. One TPS
report
too
many…
Image: http://spaghettitesting.files.wordpress.com/2009/04/fad.png?w=396&h=296 C

9. 2001
XP
C

10. XP:
Extreme
Programming
Image: Beck, Kent – Extreme Programming Explained: Embrace Change C

11. Scrum
XP:
Extreme
Programming
Image: http://upload.wikimedia.org/wikipedia/commons/b/bd/Scrum-1.JPG A

12. Scrum
Image: http://www.mountaingoatsoftware.com/system/asset/file/17/ScrumLargeLabelled.png A

13. Agile:
An
Umbrella
Term
Image: http://www.flickr.com/photos/trektrack/2389559700 A

14. Then…
in
Suncorp
land
Image: David Jackmanson on Flickr http://www.flickr.com/photos/djackmanson/350383099/sizes/z/in/photostream/ A

15. A leader
with a
vision
Image: © SlatteryIT C

16. Speed
to cool
Image: Pineapples101 (Flickr) http://www.flickr.com/photos/pineapples101/4292778673/in/set-72157623131356129/ C

17. Thumbs
up…
Cool!
Image: http://brainsyndicate.files.wordpress.com/2011/05/tumblr_lkp6t8505s1qe4wjro1_400.jpg?w=300&h=355 C

18. Choke...
Image:http://blog.vidyagamez.com/wp-content/uploads/2010/06/tumbleweed.jpg C

19. CSI:
Software
Image: © Jerry Bruckheimer Television http://images2.fanpop.com/images/photos/4700000/CSI-Las-Vegas-9x18-Mascara-csi-4772098-2000-1333.jpg C

20. Agile &
Testing
Image: http://3.bp.blogspot.com/_ZdyzysU1QPo/TA0kjLMP43I/AAAAAAAABPk/5ckzpw6OVmk/s1600/Peas+and+Carrots.JPG C

21. To Do Analysis Develop Test Done
We
reflect
work
like
this...
C

22. To Do
Analysis
Develop
Test
Done
Which is
really
working
like this...
C

23. Remind you
of
something?
Image: Craig Smith C

24. Testers
in an Analyst /
agile
team Customer
Project
Elaboration /
Specification
Acceptance
Criteria
Manager
High
Performing
AGILE
Developer Automation
Tester
C

25. Testers in
an agile
team Analysts need more
testing involvement
Analyst
Testers need more
technical involvement Tester
Developer Developers need
more testing involvement
More Less
Requirements
Functional /
Exploratory
Technical Technical
Automated
Customer
Specialist
Unit Test
Manual
Design
Collab.
Code
Tests
Test
Test
C

26. Everyone
likes to be
seen to
value
quality
Image: © Reveille Productions http://blogs.ubc.ca/kdergous/files/2010/10/body-language-the-office.jpg C

27. Quality is
everyone’s
responsibility
Image: © AP http://i.telegraph.co.uk/multimedia/archive/01893/schumaker-dummies_1893704i.jpg C

28. The
Journey
Image: http://www.handheldgpsinformation.com/wp-content/uploads/2010/06/iStock_000001211274XSmall.jpg C

29. Number 1
Problem
Image: http://www.mintvinyl1972.cc.cc/img/product/2009/200912/20091204/279589_0_Journey_Greatest_Hits_Lp_Cbs_4.jpg C

30. Slow
Expensive
Image: http://s1.hubimg.com/u/781000_f520.jpg

31. Fighting
Fires
C
Image: http://rpmedia.ask.com/ts?u=/wikipedia/commons/thumb/b/b2/US_Navy_080730-N-5277R-003_A_Commander%2C_Naval_Forces_Japan_firefighter_douses_a_fire_on_a_dummy_aircraft_during

32. Problem &
Proposed
Solution
A

33. Building
Quality
In
Image: http://www.cupojoy.com/images/image/Quality%20Inn%20Exterior.jpg A

34. Training & Processes & Roles & Tools & Comms &
Coaching Practices Responsibility Infrastructure Change
Agile Create central Define minimum Preferred tools Tester
assessment tool repository for skill set vs technology Survey
for software QA artefacts matrix
quality and Create software
Invigorate
testing maturity Create central / QA hiring guide
Investigate testing
agreed QA opportunities to community
Up-skill existing strategy Define skills for simplify test
Agile coaches in Agile team data and Internal /
testing practices Create standard members in environments External
templates relation to QA
Comms
Update courses Create ready-to- Strategy
and training Create glossary Review position go test tool
guides descriptions and packages
Create BVC career roadmap
template for testing roles
Create Agile Create working
quality/testing example of
champions Create Agile Define skills for
acceptance
software QA speciality testing
driven
one-pagers roles
Assist in-flight development
projects
Streams... C A

35. Processes
& Practices
Image: http://artegami.com/wp-content/uploads/2007/08/toysrus_karate_kid.jpg A

36. Success
sliders
Cost Scope Quality Time
Fixed
1 X
2 X
3 X
Flexible 4 X
A

37. Cost Scope Quality Time
Your Fixed
1 X
Turn! 2 X
3 X
Flexible 4 X
Image: http://4.bp.blogspot.com/-B0hIC1RPJpA/TZjq1IcJF3I/AAAAAAAAGdE/wlJdjuLCWoE/s1600/audience.jpg A

38. Quality
almost
always at
number 3
Cost Scope Quality Time
Fixed
1 X
2 X
3 X
Flexible 4 X
A

39. What does quality mean
Defining 1 Quality
Advocates
to the different roles in
the team?
Quality
2 Quality
Taxonomy
What are quality
attributes?
Quality How do we know what
3 Prioritisation
quality attributes to
include?
What are risks with the
4 Quality
Tradeoff Risks
quality attributes we are
trading off?
5 Quality
Measurement
How do we test and
measure quality?
6 Success
Sliders
How does quality relate
to the sliders?
7 Quality
Definition
What does quality
mean?
8 Next Steps How do we apply
quality to our work
C

40. Quality
Definition
C

41. Shared
understanding
of quality
Image: © 2008 Melissa McKenney http://www.kernel.org/pub/linux/kernel/people/paulmck/Confessions/Elephant_Team_03.jpg C

42. Simple
QA
Test
Strategy
& Plans
A

43. Reporting
PROJECT TESTING
9 25
7 5
8
New Risks & Issues Raised
Number of Risks & Issues
6 7 20
Number of Defects
4
Number of Tests
5 6
15
3 5
4
4
3 10
2 3
2 2 5
1
1 1
0 0 0 0
1 2 3 4 5 6 7 8
HEALTH 1 2 3 4 5 6 7 8 TEST
Iteration Iteration
COVERAGE
OVERALL
DEVELOPMENT USER
5700 1800 9 $30
5600 1600 8
5500 $25
1400 7
Number of Features
Lines of Test Code
5400
Business Value
Lines of Code
1200 6 $20
5300
1000 5
5200 $15
800 4
5100
600 3 $10
5000
4900 400 2
$5
4800 200 1
4700 0 0 $-
MAINTAINABILITY 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 PERFORMANCE
Iteration Iteration
A

44. Tools &
Infrastructure
Image: http://www.datenform.de/blog/uploaded_images/tools-709754.jpg A

45. ATDD /
Specification ATDD Lifecycle
by example
Feature
Discuss
Discuss feature
requirements
Identify SME BA Test Dev
Identify new features BA Test Dev
or requirements
Refactor
Distill
Distill into stories Stories
TDD with acceptance
Cycle criteria
Demo Green Red
Demonstrate feature
and verify using
exploratory testing
Software Develop
SME BA Test Dev
Develop the code and
instrument tests
Test Dev
A

46. ATDD /
Specification Living Documentation
by example
C

47. Preferred
Tools Best
(Specification
by Example)
Better
(ATDD / BDD)
Good
(Automation)
Traditional / Isolated
(Manual Tester Focussed)
C

48. Training &
Coaching
Image: http://www.travelsmartsunshinecoast.com.au/image/image_gallery?uuid=cb59e826-c472-427d-bc6c-e6186ef30a5e&groupId=849750&t=1296775495173 C

49. Training
C

50. Testing
Champions
Image: http://niche-desktop-testing.s3.amazonaws.com/wp-content/uploads/2010/11/WINNER-Secondary-Live-Action.jpg C

51. In-flight
Project
Assistance
Image: © Little Britain Productions / BBC Productions http://www.latimes.com/media/photo/2011-06/62506401.jpg C

52. Quality
Maturity
Image: © Gracie Films / 20th Century Fox Television http://www.widewallpapers.net/mod/cartoons/simpsons/simpsons-1920x1200-021.jpg A

53. Agile
Quality
Practices
A

54. Team
Self-
Assessment
A

55. Quality
Requirements
A

56. Quality
Development
A

57. Quality
Testing
A

58. Quality
Team
Image: http://concordion.org/image/example/online-shop/AcceptanceTest.png A

59. Your
Turn!
Image: http://4.bp.blogspot.com/-B0hIC1RPJpA/TZjq1IcJF3I/AAAAAAAAGdE/wlJdjuLCWoE/s1600/audience.jpg C A

60.  Test artifacts are treated with the same importance as code and are
4 properly maintained and refactored
 The relevant ____ility and security tests form part of the test strategy
 Non-functional requirements are agreed and included in the test
strategy
3  Test planning is based on risk (technical complexity and business
impact)
 Test strategy is agreed and understood by the team and includes
2 targets (coverage, defects, …)
 Test strategy is documented and visible as a BVC used by the team
 An agreed approach to testing that is aligned to a definition of quality
1  Tests are considered a valuable asset that can be reused as features
are progressively implemented
 No agreed approach to testing Test
-1  Tests are not considered a valuable asset Planning
C A

61. ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: THE FUTURE OF LAW ENF
Roles &
RCEMENT ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: THE FUTURE O
F LAW ENFORCEMENT ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: TH
Responsibilities
Image: © Orion Pictures http://sadair.com/wp-content/uploads/2010/07/Robocop_Final012-1024x768.jpg C

62. Collaboration: Estimating:
Recruitment & Development:
I work with analysts and SMEs to I am involved in developing
I am attracted to the organisation
ensure that testable acceptance estimates for projects at a story
because testing is a cool career path
criteria are created for all stories level, for tools and infrastructure
that offers heaps of opportunities and a
and also for deployment/release
way of continually developing my skills
activities
Automation:
I work with developers to
automate tests that provide Reporting:
business value and identify I provide metrics that give
system risks insight into project health and
system quality
Strategy & Planning:
I am involved in the project at all Agile:
stages to ensure that testing I understand Agile project delivery
provides the greatest value and and the differences between testing
quality objectives are achieved in the different phases of an phase
Qualifications & Training:
Tools & Techniques: I have recognised qualifications in
I use both manual and testing and continually update and
automated techniques using the maintain my skills
preferred testing tools based on
the situation
Architecture:
I have an understanding of the system
architecture and can create tests that
Community & Teams: verify individual components and the
I am part of a community of Development:
system as a whole
testers that are embedded I take an interest in
within teams but share development practices and
common values monitor code quality metrics
Future tester
C A

63. Transition
Plan
New Graduates,
Recruits, Consultants &
Partners
Current Test Analysts New Software Test
Assessment
Engineer Role
Capable + Current Skills + Desire
Up
Capable + Desire Skill
Not Capable + Desire
BA
Not Capable + No Desire
SME
Other
**Assessment = Technical Test + Interview**
C A

64. Communications
& Change
Image: http://dallasmorningviewsblog.dallasnews.com/megaphone-kid-cropped.jpg C

65. Internal / External
Communications
Strategy
Image: C

66. Invigorate
Testing
Community
Image: http://2.bp.blogspot.com/_rtqJH_Y4QDw/TTYsl0ieGbI/AAAAAAAABDQ/cMRXApOgV9U/s1600/brent_judd-791018.JPG C

67. Case Study

68. Regroup
Image: http://www.philparkertraining.com/data/uploads/team%20discussion.jpg

69. MAF

70. Story +
Acceptance Test
Criteria Scenarios
Test Case
Capture
MAF for Selenium
MainFrame for Web
Automation Executable Continuous
Process - TAF Specification Integration

71. Wrap
Up
Image: © Chuck Lorre Productions http://24.media.tumblr.com/tumblr_ksvfroko3J1qa88nco1_500.jpg C

72. Agile
testing is
a journey
Image: http://goldeneaglesofamerica.com/wp-content/uploads/2011/04/yellow-brick-2.jpg A

73. Measure,
Improve and
keep improving
your approach
Image: © Talent Associates http://confoundedinterest.files.wordpress.com/2011/06/maxwellsmart_missed_it_by_that_much.jpg?w=600&h=423 C

74. Testing skills
are still really,
really,
important
C
Image: © Jerry Bruckheimer Television http://photos.telestrekoza.com/var/resizes/Current_shows/CSI/Stills/CSI-Trilogy-Crossover/Miami_8x07/CSI-Miami-Episode-8-07-Bone-Voyage-Promotional-Phot

75. Are you
hunting
down best
practices?
Image: © King Features Syndicate / Lee Falk http://www.comicwallpapers.co.uk/images/main/Miscellaneous/Other/1600900/PhantomGen_1010.jpg A

76. Testers need
to be viewed
as super
heroes
Image: © DC Comics http://collider.com/wp-content/uploads/superman_alex_ross2.jpg A

77. Don’t forget
to use
(or improve)
our quality
templates
C

78. Testing
needs to
be cool
Image: © Miller-Milkis Productions http://imagetime.co.gp/key/henry%20winkler%20ron%20howard# C

79. Craig Smith
http://www.suncorp.com.au Questions
http://www.agileacademy.com.au
http://www.craigsmith.id.au ???
craig.smith@suncorp.com.au
@smithcdau
Dr. Adrian Smith
http://www.ennova.com.au
http://www.agileengineeringdesign.com
adrian.smith@ennova.com.au
@adrianlsmith

80. Suncorp is one of Australia and New Zealand's largest diversified financial services
providers, supplying banking, insurance and wealth management products to around
9 million customers through well-established and recognised brands such as AAMI,
Australian Pensioners Insurance Agency, Shannons, Vero, Asteron and Tyndall, as
well as Suncorp and GIO. Today, Suncorp is Australia's fifth largest bank and second
largest domestic general insurance group, with over 16,000 staff. Suncorp has
representation in 450 offices, branches and agencies throughout Australia and New
Zealand. (http://www.suncorp.com.au)
Ennova is an technology company that has focused on providing the engineering and
construction industries with innovative products and services that improve project
execution and operational efficiency. Ennova base their the project delivery approach
on combination of Lean and Agile principles and practices. This provides clients with
a high level of transparency and real value for money. Consulting engagements are
undertaken using a pragmatic and common sense approach that is aimed at
increasing the client's internal capability and building business agility.
(http://www.ennova.com.au)
The Agile Academy is the next wave in Agile learning and collaboration. Designed for
IT professionals, the Agile Academy is a knowledge hub that promotes Agile related
learning, knowledge sharing and capability development across the IT industry. We
offer an integrated curriculum of leading edge courses across the entire solution
development life cycle, covering all roles at varying levels.
(http://www.agileacademy.com.au)