This is from Nate's Meaningful Play presentation on the B|P player experience research on Spore for EA. More info here: http://boltpeters.com/blog/?p=95

1. RESEARCHING
A NEW APPROACH TO
PLAYER TESTING

2. • “Impact of
digital
technology on
society” at
UCSD
• Co-founded
B|P in 2001
• 7 people. UX
research and
Design
• San Francisco
• 2,272
Participants.
188 Research
Studies.
photo by andy price visualpause.com

4. Why we care about games

7. 2007 was
like this
shit.

9. Who cares?

10. We started the spore research with
one question.

11. Why are video games designed
in this environment…

13. but researched here?

15. no idea.

16. “Focus groups were invented to
focus the research, not be the
research”
- Marty Neumeier | “The Brand Gap”

17. So what can we do
different?

18. “Guys... I put my pants on
just like the rest of you - one
leg at a time. Except, once
my pants are on, I make gold
records.”

19. #1 native environment is critical…
But timing is everything

20. One year.
59 Participants
393 Hours of Gameplay.
3 Hour Sessions with 2 Nights Each

22. who cares about fun?

23. who doesn’t.

27. DISCUSSION

28. here arefive
things we changed.

29. 1. simulated native environment.
2. multi-user voice chat
(AKA “talking” and Think Aloud Protocol)
3. touch screens.
4. metrics
(PENS model)
5. delivery

30. Simulated Native Environment

31. Live observation room

32. Multi-user Voice Chat

34. Gamers love TeamSpeak and
keyboard shortcuts,
researchers can too.

35. Talking to six people is crazy

36. 3
1 5
6
4
2
7

37. #3. Touch screens
Game Screen
(w/ webcam)
Survey
Touchscreen

38. #4. The PENS
Model
Player Experience of Need Satisfaction

39. Intrinsic motivational needs?
Doing things for their
inherent satisfactions (e.g.
games, sports). Play
through punishment and
boredom.

40. PENS measures three specific motivational needs:

42. what sucked

43. •Bordering on chaos at first
•Restricted by game genre
•Lots of technical problems
•Maintaining think aloud protocol
•Difficult to respond to questions
•Recordings take weeks to render

44. welcome, now play like

45. (according to our precise time limits)
Event Time
Welcome / Consent 4:30pm
Stage 1 start 4:45pm
Survey 1 5:15pm
Survey 2 5:45 pm
Dinner 6:00 pm
Stage 2 start 6:15 pm
Survey 3 6:45 pm
Survey 4 7:15 pm
Review 7:20 pm
3 Hours

46. We Learned… Not Much

47. But then, something
changed.

48. No more timing structure.
Game Level Test Phase Time Action
NDA check,
equipment check,
Paul intro, think
Welcome / Consent At 0:0 aloud
Stage X Begin Phase At :5
Smiley Survey 1 At :15
Smiley Survey 2 At 1:00
Announce: Mouse
over + Press A to go
End&Survey At 1:30 to Stage Y
Stage Y At 1:40
Smiley Survey 1 At 1:50
Smiley Survey 2 At 2:25
End&Survey At 2:45
Review Survey At 2:55
3 Hours

49. …fun went way up
Stage Y

51. #5. Delivery

54. Technology
•Video Recording
•ZDSoft Game Recorder
•Live Observation
•Six Webcams using VNC
•Long VGA cables
•Audio
•Teamspeak (free)
•ELO Systems Touch Screens

55. $79

56. free

57. $39

58. free

59. Summary
1. Native environment is critical, but timing is
everything
2. You can do this for dirt cheap
3. Touch screens are fancy
4. The Dow Jones is scary

60. the future is remote game research

62. Nov 7th, in San Francisco
userresearchfriday.com

63. Questions
1. You can do this for dirt cheap
2. Go to San Francisco for URF
boltpeters.com nate@boltpeters.com