The Producer-QA relationship in a Free 2 Play mobile games studio. Slides from the Presentation at the Game QA and Localisation Conference in London 26/6/2014 by Thomas Hulvershorn / Operations Manager and Olly Hart / QA Manager from Outplay Entertainment (Outplay.com) Topics: - Working with QA from Production POV -- Producer role, Trust and Sign off responsibility -- Testing the right thing - Finding your successor: -- Internal Recruitment vs. Appointing a successor -- The Interview Process -- The Aftermath

3. “Capable of great and terrible
things”
“leads the Decepticons”
“knows rights from wrong”
“protect the things he cares
about”
Thomas Hulvershorn
Operations Manager
Olly Hart
QA Manager

4. …getting it right
Thomas Hulvershorn
Olly Hart
Outplay Entertainment

5. Established 2011 in Dundee
Outplay Entertainment is a
fast growing, venture-backed
Developer and Publisher of
social and mobile Games.

6. ‘Classic Producer’
> Deadline >> Quality
> QA as excuse
> Quality afterthought
‘Classic QA’
> Non committal
> Over cautious
> All bugs are ‘A’ bugs

7. The Game has changed!

8. Sample Group A Sample Group B Control Group
Project StakeholdersReal Time Analytics
ETL
Layer
Analysts
Tracker
Store Data
Analytics Events
Game
on Store
Download Deploy new
Version
F2P
Data Feedback
Driven
Deployment
Game Life cycle
Player
Launch Amazon Cloud
Game SettingsOutplay Server
ModifySettings
SetupExperiments
Game
Settings
Producer
Feature Backlog
(Jira)
Targets
Designer
Game
in Vault
Product
Team
New Version
Sign off and Store
Submission
Game
Development
Client
Update

9. #1 Priority is Player Satisfaction
“If your game sucks, stability won’t
save you”
> Customer Support support
> Soft Launches

10. Impact on
Managing Quality

11. “The Fear” &
& Know your Coder

14. QA responsible for Customer
Support
> Makes sense – we know the projects
> Creates a direct line between problem and
solution
Use Crash Logs and User Feedback

15. Test is committed to the Project
Producer is overall responsible
Live issues are Team issues

16. New Feature has terrible Bug
Classic Reaction
QA

17. Our take on this:
Why did the team miss this?
What will the team do better next time?

19. Production at Outplay
> Quality && Deadline
> QA right hand ‘man’
> Quality from the start
QA at Outplay
> provides insights
> is Embedded
> tests the right thing

21. > Three options
> External Recruitment
> Internal Appointment
> Internal Interviews
For many people this is a SCARY thought!
> Common Objections
> “We know who should get the job” -> Speed
> “No one is capable” -> Innovation
> “Team will be disgruntled” -> Acceptance
> “Losers work less after” -> Motivation

22. Speed
Cost
Innovation
Acceptance
Risk
Motivation
~
~
~
~
External
Recruitment
Internal
Appointment
Internal
Interviews

23. Test
Management
People
Management
Team
Management + Personality
+ Games
+ Experience
+ Delegation
+ Stress Resistance

24. > Make it comparable!
> Same questions for all candidates
> Pair up with non QA Manager (e.g. Producer)
> Make it count!
> Practical Interview with Exercises
> Real life problems
> Test all areas with examples

25. Postkorbübung
Priorities
Delegation
Pressure
Information
Communication
Results form the basis for the Interview

26. > Situation
> You are the QA Team lead.
It is Tuesday 09:30 and you have a full
day meeting starting at 10:00.
> You have not been in the office
yesterday and your Inbox contains 25
mails.
You got 25 minutes to read and sort out
your mails.

27. There are 3 tester in your team
Adam Supertester
He has just started to conduct a test on the Kindle
Fire
Bernd Normaltester
He has nothing to do at the moment and is looking for work
when he comes in at 10:00.
Callum Problemtester
He started a Smoke Test on the iPad3 handset the day before
at lunchtime, but hasn't finished it. After that he is
supposed to do a full test on the iPhone 4s.

28. Example for a mail
From
When
Subject
Content
Actions

29. The Inbox
Spam
Connections
Underperformance
Misleading
Different Time zones
Delegation

30. Example for a mail with actions

31. > Interview Structure
> Introduction to process // 0.25h
> “Postkorbübung” // 0.5h
> Review of results // 0.5h
> Role Play Staff Feedback // 0.5h
> Traditional Interview // 1.5h
>Team Management
>Testing Know How
>CV and experience
> Total time: ~3 hours per candidate

32. > Process has been regarded as fair
> We made the right choice
> No grumblings in the team
> Clear goals for the non successful
candidates
> Signal to the team there is room
and a fair chance for progression

33. Thanks for reading!
Any Questions?
@DerHUL
@ILOVETHISOZZ