1.
WHAT DOES IT MEANTO BE
ATEST ENGINEER?
Andrii Dzynia
http://bit.ly/1GYquoR

2.
MY BACKGROUND
• 8 years in software engineering
• started as a test engineer in a team of 20 testers
• currently working on improving product quality
and development productivity at Spotify
https://www.linkedin.com/in/andriidzynia@adzynia

3.
SATURDAY EVENING
https://twitter.com/mfloryan/status/655433887015948288

4.
LET ME SHOWYOUTHE PROCESS
video: https://www.dropbox.com/s/un7m49kdgeuzsbz/test-session-
andrii.mov?dl=0

5.
FOUND BUGS

6.
MY COLLEAGUE,
FRONT-END DEVELOPER.

7.
WHAT DID I LEARN?
• Software always has bugs. If
you have not found any, does
not disprove their existence
• Developers do not spend
much time looking for edge
cases, testers do

8.
WHAT I WANTEDYOUTO
LEARN?
Testing is not a set of artefacts.

9.
TESTING IS A PROCESS.
@adzynia

10.
TESTING PROCESS

11.
GUESS WHAT BUG MARCIN
WAS REFERRINGTO?

12.
MORE LESSONS LEARNED
• People are different. It is hard to predict every
single combination when user can get stuck
• Think slower, sometimes

13.
WHAT MAKESYOU ATEST
ENGINEER?
Critical thinking
Fault models
Patterns recognition
@adzynia

14.
CRITICALTHINKING
https://www.criticalthinking.org/ctmodel/logic-model1.htm

15.
FAULT MODELS
“From the model, the designer or user can predict
the consequences of a particular fault.” - Wikipedia
video: https://www.youtube.com/watch?v=rEumMkAhwzA

16.
PATTERNS RECOGNITION
https://www.nasa.gov

17.
BASED ON EXPERIENCE BRAIN RECOGNISES
SEQUENCES WHICH LEADSTO UNDESIRED
OUTCOME.
NOTE: BE CAREFUL WITH FAULTY ASSUMPTIONS.
FACE ONTHE MOON SURFACE IS PAREIDOLIA.
“Pareidolia is a psychological phenomenon when the mind perceives a
familiar pattern of something where none actually exists.” - Wikipedia

18.
LOOK FOR PATTERNS WITH
HIGH FAILURE RISK.
video: https://www.youtube.com/watch?v=qoQssWPRNR0

19.
BRINGINGVIDEOTO SPOTIFY
USERS
http://techcrunch.com/2015/05/20/spotify-introduces-video-clips-
podcasts-and-activity-based-playlists/

20.
TEAM SETUP

21.
COMMUNICATION CAN BE A
PROBLEM
http://dilbert.com/strip/1989-05-31

22.
SOLUTION WAS
Create a #slack channel
Start asking video integration questions
Rise bugs early and facilitate conversations

23.
DID WE PREDICTED ALL DEFECTS BEFORETESTING?
SURELY, NOT.
Data collected by Stefan Rungardt

24.
BUT HOW DIDTHAT HELP?
Integration ended up to be much easier
Testing started earlier
# of production issues was low

25.
WASTHAT EASY?
Not really.You have
to ask questions
people do not want
to hear
http://media2.giphy.com/media/wrTHBbla7suPu/giphy.gif

26.
Tester: Shall video be playing when I lock a screen?
Programmer: I do not know.
Tester: who can know?
Programmer: Product manager?
Product manager: I do not know.
Tester: who can know?
Product manager: let’s take a look howYoutube does it.
After a while we figured why video has to be paused
when you lock a screen. Licensing constraints.

27.
DEFECT PREVENTION
VS
DEFECT DETECTION
“Bugs are so 90s”

28.
LEAN MANUFACTURING
https://en.wikipedia.org/wiki/Lean_manufacturing

29.
KEEPYOU WORK ENVIRONMENT
IN A GOOD SHAPE
• engineering quality
• product quality
• strive for improvements

30.
ENGINEERING QUALITY
• Start testing as early as possible
• Build safety nets
• Fix bugs as fast as they occur
• Conduct root-cause analysis and take actions
http://www.slideshare.net/AndrewDzynia/quality-built-in

31.
PRODUCT QUALITY
• Product success metrics
Time to first video frame
Number of users streaming video content
Average viewing time, etc.
• Number of production defects

32.
SPOTIFYTEST CERTIFIED MODEL
Inspired by Google
https://mike-bland.com/2011/10/18/test-certified.html

33.
TAKE AWAYS
• Testing is a process, not a set of artefacts
• Cost of defect can be lowered.Think about that
• Find ways to influence quality on organization
level. It is hard, but possible, even for a test
engineer

34.
THANKYOU!
@adzynia
andrii@adzynia.com