Games User Research is for Game Design!
â˘
12 gefällt mirâ˘2,073 views
Sarah Romoslawski and Marina Kobayashi presented this talk at the 1st Game Design Conference in San Francisco on September 17th, 2012. The talk defines Games User Research (GUR), and includes two perspectives on practicing GUR at different companies, using GUR with different platforms, genres, and with different target audiences.
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (12)
Ăhnlich wie Games User Research is for Game Design!
Ăhnlich wie Games User Research is for Game Design! (20)
KĂźrzlich hochgeladen
KĂźrzlich hochgeladen (20)
Games User Research is for Game Design!
- 1. Games User Research is for Game Design! Marina Kobayashi, Electronic Arts Sarah Romoslawski, Sifteo
- 2. Who are we? Sarah Romoslawski Marina Kobayashi Game Designer Lead Games User Researcher
- 3. Who are we? Sarah Romoslawski Marina Kobayashi
- 4. Overview ď´Intros ď´What is games user research(GUR)? ď´How does GUR turn game design into GREAT games ď´Applying games user research to game design ď´Platforms ď´Genres ď´Target users (age, lifestyle) ď´Our conclusions ď´Q&A
- 5. From two perspectives: vs. What a mess!
- 6. Who are you? It is ok to raise your hand more than once! ⢠Game Designer ⢠Artist ⢠UX/UI Designer ⢠Programmer ⢠Producer ⢠Wearer of many hats ⢠Iâm just a unicorn
- 7. There is always a player in a game. Games are made for people by people.
- 8. How do games become great games? Answer: by doing Games User Research!
- 9. Games user research is a feedback loop for game designers to understand how users experience their games.
- 10. GUR + Game Designer = Collab Genius Lather Rinse Repeat
- 11. When Should I do GUR? Concept Early Full Alpha Post Phase Production Production Phase Release Collaging/ White Box Vertical Slice Level Tests E3/XBL Demo Full Play Benchmark DLC Tests Tests MP Tests Tests Through Tests Tests Tests All stages of the dev cycle!
- 13. Combat Mechanics Test Test EARLY and OFTEN: White Box Testing
- 16. Full Playthrough Test Average Player Experience (Via Survey Stops) 10.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 Fun Difficulty Frustration Start of End of Game Game
- 17. What do we learn about? FUN Impact Difficulty factor Ramping Usability in context slate of Puzzle of the new platform Content Early Character and Story Development
- 19. Platforms, Genres, Gamers and GUR at EA Game Lab âWhy do we fall sir? So we might learn to pick ourselves up. â - Alfred, Batman Begins
- 21. All Platforms Have Affordances
- 22. All Platforms Have Affordances The quality of the device that lets you perform an action
- 23. All Platforms Have Constraints
- 24. All Platforms Have Constraints Small screens, dual screens, touch screens
- 25. All Platforms Have Standards
- 26. All Platforms Have Standards
- 27. Testing Different Platforms at EA Concrete Examples: Kinect Voice for Mass Effect 3 and EA Sports Active: NFL Training Camp
- 28. GUR on Kinect Voice in Mass Effect 3 + 1:1 think-aloud (command aloud) + exploration & experimentation + usability
- 29. GUR on Kinect Voice for Mass Effect 3 Kinect voice â natural language
- 30. GUR on Kinect Voice in Mass Effect 3 What would you say to control the game?
- 31. Video clips
- 32. Player 1 Ok, Mordin Iâll see what I can do⌠ââŚâ Kill kill kill⌠Die die dieâŚ
- 33. Player 2 âXbox Inventory?â Can she âLiara?â hear me? Hmm⌠I forgot the names of the powers âLiara, Lift Orb?â
- 34. GUR on Kinect Voice in Mass Effect 3 What would you say to control the game? Expected Commands Usability of Voice Commands
- 35. GUR on Sports Active Games
- 36. âI have to wear what?â Usability Ergonomics
- 37. So how do I use this thing?
- 38. So how do I use this thing? 8 Tutorials Opinion 8 8 7 7 7 7 7 6 6 6 6 5 5 5 4 4 4 3 3 3 2 2 2 2 2 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 Liked tutorial video Tutorial video needs improvement Players commented that they would like to see There were two mentions that holding the more of a step-by-step breakdown of the exercises. Wii-motes felt awkward, that they were not sure One player also mentioned not knowing they were how to hold them or that the exercise required supposed to observe the trainer doing the exercises. using them at all.
- 39. Howâd you do?
- 40. GUR and Genre at EA Concrete Examples: different methods for different games
- 41. Why we used different methods for BF3 Multiplayer vs. Warp?
- 42. Why we did different methods for BF3 Multiplayer vs. Warp? First-Person Shooter Puzzle Platformer
- 43. GUR for the First-Person Shooter + 1:1 think-aloud usability tests for Single Player and Multiplayer + Dyad usability tests for Co-op + Multiplayer playtests (over 32 players in-game simultaneously) + Full playthrough tests + PPT screen walkthroughs + Flash Prototype tests
- 45. Some observations are best made with live gameplay and think-aloud.
- 47. GUR for the Puzzle Platformer + 1:1 think-aloud usability + Full playthrough
- 49. GUR for Puzzle Platformers Warp
- 50. GUR for Puzzle Platformers Warp
- 51. GUR for Puzzle Platformers Warp
- 52. GUR for Puzzle Platformers Warp
- 53. GUR for Puzzle Platformers Warp
- 54. GUR for Puzzle Platformers Warp
- 55. GUR for RPGs How to test a game that has 160 hours of content? + 1:1 think-aloud usability + Extended Playthrough + Combat Encounter Tests + Diary Studies
- 56. Dragon Age II
- 57. Dragon Age II
- 58. Dragon Age II
- 59. Dragon Age II
- 60. GUR for Facebook RPGs How to test a game that might be played in 5-10 minute increments? + 1:1 think-aloud usability + Diary Studies
- 63. Dragon Age Legends Dragon Age Legends
- 64. Dragon Age Legends Dragon Age Legends
- 65. Diary Study
- 66. GUR on target audience ~OR~ Figuring out who the right gamers are and how to get them in the door.
- 68. But who is Mr. Right - for this playtest?
- 69. Does Mr. Right check off all these boxes?
- 70. Wait youâre giving out incentives?
- 73. Why getting the right players matters?
- 74. Franchise Fans vs. Newcomers Shooting Target Sliding Shooting Shooting Weak Time #Deaths 1 (max # cycles ) (cyc) Spots (min) New Players Pair 1 8 4 4cyc. 3rd cyc. 3 New Players Pair 2 10 2 5cyc. 1st cyc. 6 Fan Pair 1 10 3 3cyc. 1st cyc. <1 Fan Pair 2 5 1 4cyc. 1st cyc. <1 Average 8.25 2.5 max 4cyc.
- 75. Why getting the right players matters? 4-player co-op campaign
- 76. Why getting the right players matters? When testing a 4-player co-op campaign: âBe the best you can be!â âI will pwn you!â
- 77. Why getting the right players matters? Melee DPS Tank Healer / Tech Ranged DPS 4-player co-op campaign
- 78. Why getting the right players matters? Melee DPS Tank Healer / Tech Ranged DPS âI really like the idea behind the mag-shield. I usually play tanks/defenders aside from assault classes and this âI really enjoyed allowed me to police the playing Izzy. I front lines while my frequently choose teammates flanked and healers in MMO's or sniped from cover. Definitely group/co-op games fits my play style.â and thought she was a great character to start off with.â
- 80. Wait, What is Sifteo?
- 81. Platforms, Game Design, and GUR at Sifteo (or in other words) âneighborâ isnât a verb
- 82. The Need for the Sifteo PlayLab Original Sifteo Cubes Ok so now letâs find some people to playâŚ
- 83. The Need for the Sifteo PlayLab Less Off Site to More On Site Testing! Ok so now letâs find some people to playâŚ
- 84. Design in Parallel Wait, you want me to drive on that?
- 85. Creating Standard Visual Language Neighbor isnât a verb
- 86. What we do at the PlayLab: Measure Fun through Playtesting
- 87. How does Sifteo Measure Fun? Fun = Learning (Koster, A Theory of Fun) tinkering FUN of MASTERING new skill Frustration, the game doesnât teach me!
- 88. All Platforms Have Affordances The quality of the device that lets you perform an action
- 89. Affordances â Spatial reasoning and Cube Buddies Sifteo Cubes AFFORD for spatial reasoning puzzles, but spatial skills are honed throughout life.
- 90. Game Play
- 92. Really Mixed Up Faces
- 93. Almost Done, But Eyes Wrong
- 94. The player would get to here and STOP
- 95. What Skill Were We Forgetting to teach?
- 96. We didnât teach an eye for an eye!
- 97. Solution: Teach How to Swap Upside Down Eyes
- 98. All of the Above -- Sifteo Example Dealing with platform CONSTRAINTS, AFFORDANCES and STANDARDS on Word Caravan
- 100. WC Game Play
- 101. Why WC? ⢠Must be fun for 7-12, the platform requires this as a STANDARD for all games
- 102. Why WC? ⢠Must be fun for 7-12, the platform requires this as a STANDARD for all games ⢠Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform
- 103. Why WC? ⢠Must be fun for 7-12, the platform requires this as a STANDARD for all games ⢠Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform ⢠Shuffling letters to make words exploits the AFFORDANCES of the platform
- 104. Why WC? ⢠Must be fun for 7-12, the platform requires this as a STANDARD for all games ⢠Must improve our original word game, Word Play, yet still be playable on 3 or more cubes, a design CONSTRAINT of the platform ⢠Shuffling letters to make words exploits the AFFORDANCES of the platform ⢠ALL OF THE ABOVE due to the platform resulted in the creation of an appropriate hint mechanism imited inputs and screen space
- 105. Pre-existing Models of Word Games
- 106. Problem(s) ⢠Game gets hard too quickly ⢠Players get stuck too early in the game ⢠Players naturally want to apply models of other word games â˘BUT we didnât want to make it too easy for the older age groups Everyone I know has a big butâŚ
- 107. Too much frustration, too little fun FUN of MASTERING tinkering new skill Frustration, the game doesnât teach me!
- 108. Will pattern recognition help? â˘Block, lock â˘Lock, unlock â˘Date, update â˘Make remake â˘Act, fact, facts, factor, artifact
- 109. We simplified the puzzles BAT FIX WORD
- 110. What do kids do when they canât read a word?
- 111. Solution: Hint Mechanism Have the game be the parent! SCAFFOLD the player.
- 112. Iterate, test, iterate, test, iterate, test ⌠⢠Earned hint, tracked with icon ⢠Automatic hint ⢠Increase the size of the hint icon ⢠Delayed hint Older version, Not noticeable
- 113. Hypotheses vs. Actual âŚwell I guess not this timeâŚ
- 114. Final Implementation â Infinite Delayed Hints, Large Icons
- 115. Developer On Site! Children often do not report seeing animated elements that appear on one or more of the screens, Even if the elements are high contrast and cover 1/64 of the total screen area. Increasing the size to 1/16 total screen area seemed to solve this problem. Chris March, Lead on Word Caravan
- 116. GUR and Genre at Sifteo Designing task oriented 1:1 sessions for the Sandwich Kingdom RPG
- 117. Sandwich Kingdom RPG
- 118. Task Oriented, 1 or 1:2, Think Aloud
- 119. Designing a Script: Quest or Task Focused I am⌠vs. Pearl is⌠Yes, yes with help, or no
- 120. PLAY THE BUILD
- 121. WHERE DID I GET CONFUSED?
- 122. Early Feature Test: Dangerous Pits
- 123. Early Feature Test: Dangerous Pits
- 124. Feature Test: Dangerous Pits No NPCs No Final Art
- 125. So we use similar methods over all games
- 126. But we ask different questions in the end
- 127. Acquiring Actionable Data from Kids ~OR~ when the chocolate covered pretzels are more interesting than your game, you might have a design problem
- 128. Informed Consent, NDA and Photo Release
- 130. Facilitate Play We want you to help us find problems! We want you to tinker!
- 131. Facilitate Communication Teach each other how to play.
- 132. Play with Parent
- 133. Everyone wants to feel smart, even kids!
- 134. Sifteo: Longitudinal Studies Concrete Example -- Collecting Data With Kids on the ~Entire Puzzle Game~
- 135. No Stats Saved on HW at this Point. ď ⢠So how do we know how long it takes? ⢠How many tries to solve the puzzle? ⢠Where is the player successful? ⢠Hrmmm⌠⢠We can do this!
- 136. Solution: Pizza and Games â˘ITâS SUMMER! â˘2-3 x 3hr sessions â˘Lunch included (pizza) â˘Gift card for the parent â˘Certificate of completion â˘Did I mention these were 3 - 4 hr sessions with kids 14 and under?
- 137. Testing Chroma Splash Puzzle Mode
- 138. Game Play
- 139. Goal: Puzzle Difficulty Ramp Up
- 140. A Chroma Splash Puzzle
- 145. What about adults? Sometimes you run out of kids⌠BUT you still need data!
- 146. Kids and Adults are Different My biggest surprise was how reticent older adults seemed to be to experiment with the game. That resulted in certain adults taking far longer to complete puzzles than kids did. Scott Kim, Chroma Splash
- 147. Actionable Data can be Tricky wanted a Some testers perfect solver that would fail the user in any situation where they could not proceed. This seemed dangerous and very difficult, so I did not attempt it. Eric Liao, Chroma Splash
- 148. Key Takeaways
- 149. GUR is your hydrant to feedbackâŚenjoy it!
- 150. Thank you! Marina Kobayashi Sarah Romoslawski mkobayashi@ea.com sarah@sifteo.com
Hinweis der Redaktion
- ----- Meeting Notes (9/15/12 12:58) -----introduce names, say names, and say "today we are going to talk to you about WHY and HOW games user research is for game desing
- Sarah RomoslawskiSifteoGame Designer (Producer) (Playtesting)MFA from ASUâs arts media and engineering program, 5+ years game and experience design for learning.Marina KobayashiElectronic ArtsSr. Games User Research @ EA Game LabMasters in Human-Computer Interaction (CMU HCII), expert with 7+ years experience in usability and user research.----- Meeting Notes (9/15/12 12:58) -----My name is Sarah, I am a game designer at Sifteo where I also help lead playtesting for all dev teams, I have over 5 years experience doing game design and experiential design for learning, a game I really like is TWEWY
- Sarah RomoslawskiSifteoGame Designer (Producer) (Playtesting)MFA from ASUâs arts media and engineering program, 5+ years game and experience design for learning.Marina KobayashiElectronic ArtsSr. Games User Research @ EA Game LabMasters in Human-Computer Interaction (CMU HCII), expert with 7+ years experience in usability and user research.----- Meeting Notes (9/15/12 12:58) -----My name is Sarah, I am a game designer at Sifteo where I also help lead playtesting for all dev teams, I have over 5 years experience doing game design and experiential design for learning, a game I really like is TWEWY
- ----- Meeting Notes (9/15/12 12:58) -----we will tell you a little bit about ourselves, we will define games user research, we will explain how GUR helps turn games into great games, and we will provide concrete examples of how GUR helps game design in the context of target user, platform and genre, then we will wrap it up and open the session for Q and A
- We will provide you with an understanding that you can do GUR on any budget. The types and amount of GUR you can do obviously scales with the amount of support you have and the scope of the game, but every game can be playtested and should be by your target audience.----- Meeting Notes (9/15/12 12:58) -----but Marina, doesn't GUR cost lots of money? I am just a team of one!Marina: yes, it does in some cases, but by providing examples from Sifteo and EA we hope to show that GUR can be scalable depending on your budget, the size of your game, everyone can do GUR
- Through careful planning, development, and analysis.Different types of research informs the development and strategy. Market researchGames user researchTechnical researchGames user research is a feedback loop for game designers to understand how users experience their games.
- Design IntentHypotheses about how users will behave or respondExperiment design âhow are we going to observe no one doing that thing you want them to do?âReporting test results quickly and effectively â because its all for you!
- Test early and often â the lab is a safe place to failIterative testing throughout the game development lifecycle â see if design changes improve the experienceTesting different facets of the game.----- Meeting Notes (9/15/12 12:58) -----its important to test at all stages of the dev cycle
- Test EARLY and OFTENZoom in to core features and mechanicsThis true is because the basic mechanics will be used as building blocks in the game systems which will rely on these mechanics being solid. An example from Danteâs Inferno is the melee combat mechanic of using a scythe for short and long range attacks against mobs of enemies.
- Which we are able to capture during a full playthrough. We do this through survey metrics collected at stops at regular intervals and/or after specific events. Usually these pop up when thereâs a lull in the gameplay
- EXP CHART. Zoom out to full game experience, understanding how long it takes to play the game, where are players having the most fun and most frustrationLooking at these metrics holistically we can see how the experience unfolds over the course of the entire game. This helps the dev team visualize the experience and alerts them to any times the player may be significantly derailed. However, if you test whitebox levels and mechanics earlier in the process there should be fewer of these as they should have already been identified. ----- Meeting Notes (9/15/12 12:58) -----to understand the full game experience, how long it takes to play, where users are getting stuck
- Really weâre not evil â we care about our consumers!
- Video clip 1: radio silenceJust let me shoot the enemies!
- WHAT GAME IS THIS lol
- Our research looked at what players expected to be able to say to control the game, what kinds of actions and controls would be available via this new platform.Then the question was how usable and accessible were the voice commands? Were players able to be as effective with the new controls? Could they get the characters to do what they wanted in close to the same number of attempts as they would with the controller? How would the players feel about the experience of using voice commands?
- Player 1 tyler: ââŚâPlayer 2 chris: âliara?â Oh it fires off where Iâm looking? Yeah thatâs what its doing.
- Player 1 tyler: ââŚâPlayer 2 chris: âliara?â Oh it fires off where Iâm looking? Yeah thatâs what its doing.
- Player 1 tyler: ââŚâPlayer 2 chris: âliara?â Oh it fires off where Iâm looking? Yeah thatâs what its doing.
- Our research looked at what players expected to be able to say to control the game, what kinds of actions and controls would be available via this new platform.Then the question was how usable and accessible were the voice commands? Were players able to be as effective with the new controls? Could they get the characters to do what they wanted in close to the same number of attempts as they would with the controller? How would the players feel about the experience of using voice commands?
- Design intent: create a game that encourages players to do physical activity with fitness in mind.Measure the gamers exercises and movement and provide feedback and performance metrics.GUR seeks to address whether the UX is doing this, and where it is failing. When we test a user experience â this must encompass more than just the software â it also takes into account the context, hardware and platform, any related systems that the game will interact within. In Sports active â the game relies on sensors to capture the playerâs movement and behavior. The test may look at whether the player understands how to use the hardware effectively, whether they are physically able to, and the emotional experience â do they enjoy the activity? If there are breakdowns in any of those 3 areas, identifying them is key â knowing what the problems are means they can be addressed or at least the team is aware of them. Testing this meant having real gamers come to the lab and workout! Boy did we get sweaty. As the moderator â I wore workout clothes too â to help make the participants feel comfortable participating in an exercise test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- We dedicated a lot of resources to testing Battlefield 3 â doing all these methods to get feedback on the game throughout the development lifecycle.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- BF3 Military FPSRealism and fast reaction times are critical â twitch gamingHighest priority gameplay mode is the online multiplayer â interruptions and distractions need to be minimized, so we do group playtest to have them be in as close to the end-user experience as possible. Collecting feedback in surveys immediately after the match â we know that there will be some data we donât capture because it is retrospective and we are not probing real-time, but the benefit is that itâs a more ideal gameplay experience. Warp - Puzzle Platformer game â its played single player, so in some ways reaction times are not quite as urgent, the game can be more forgiving of having the player think-aloud. For puzzle games our goal is to understand how players are solving the puzzles â their cognition! This doesnât mean we didnât DO 1:1 think-aloud usability tests for BF3⌠we did! But it depended on what the goal was â getting feedback on the UX/menus/HUD etc we did 1:1 think-aloud usability and maximized efficiency â having 1 consumer participant recruited at a time, and having internal players act as âserver fodderâ. Also minimal time spent in-game â maximizing tie spent on the focus areas for the test.
- What other methods are there?Weâre looking at
- What other methods are there?Diary study
- What other methods are there?Diary study
- What other methods are there?Diary study
- What other methods are there?Diary study
- What other methods are there?Diary study
- What other methods are there?Diary study
- What other methods are there?Diary study
- Gamers lie to get into studies and get the incentivesCareful screening is important for making sure gamers have the experience we are looking for
- Gamers lie to get into studies and get the incentivesCareful screening is important for making sure gamers have the experience we are looking for
- Gamers lie to get into studies and get the incentivesCareful screening is important for making sure gamers have the experience we are looking for
- Gamers lie to get into studies and get the incentivesCareful screening is important for making sure gamers have the experience we are looking for
- Gamers lie to get into studies and get the incentivesCareful screening is important for making sure gamers have the experience we are looking for
- Tell me why you like Skyrim â well I like the characters and the graphics. Oh which characters did you like? Ummm the king? I canât remember his name right now, but I thought he was cool. Ok, how about the graphics? What did you like about them?They were cool â they looked good. I liked the style.How about the gameplay? Yeah it was cool. I liked it.
- So why does getting the right players matter?When weâre looking for players to test a co-op campaign
- So why does getting the right players matter?When weâre looking for players to test a co-op campaign
- Because weâre looking for qualified responses.
- Targeted to ages 7-12, the sifteo cube gaming platform is hands on. Games are played by manipulating a set of cubes, putting them together (we call that neighboring), shaking, tilting, flipping â Since the cubes are a spatial and tangible interface, the platform really lends itself to puzzle games including word games, logic games, and maze adventure games. The system has audio as well that comes out of what we call the master cube. So now that you have a better understanding of the platform, we can dig a little deeper into what playtesting this platform is like.
- Finally, the platform AND the games were being built at the same time. As our LEAD QA put it, it is like buiding an overpass and driving on it at the same time.
- So what do we do at the PlayLab?
- Cube buddies is a puzzle game. There are characters in the game and you engage with their faces by putting them back together after they have been scrambled. Faces are put back together by neighboring cubes and thus allowing the game to swap parts. So basically, if the blue/green guy and the orange guy were just neighbored, there eyes would have just swapped.
- Learned through playtesting
- Lots of things are wrong here, especially the eyes. Zorgâs eyes are on Suliâs head and Suliâs eyes are on Zorgâs head.
- The player would start by putting the cubes together to swap the eyes.
- Kids understood that the eyes were upside down, but making the leap to swap the same type of part was HUGE for kids. Why werenât we teaching this? Kids could obviously tell that the eyes were wrong, they just had no idea what to do with it.
- We had to single out this puzzle solving skill. This was tough to pinpoint just in level design but easy to pinpoint while playtesting. And playtesting resulted in an entire puzzle level around the core skill â swapping eyes.
- Tilt the cubes to rearrange letters only on the cube, move the cubes around and neighbor to spell a target word
- Why present word caravan in the context of platform?
- Through playtesting we realized that word games are extra tricky. Players came to the game with models of other word games in their mind. They wanted to swipe the cubes like they were an iPad and they wanted to be able to swap out the letters, which they couldnât. We had to find a way to teach OUR model of a word game.
- SO how do we teach them our model and our rules, when they have something engrained in their mind of how word games work, AND how do we do that in a way that made the game too easy for older kids.
- It helps a little, but if you are not an avid reader, you wont even know what patterns to look for, 7-9year olds may only just be learning compound words, phonemic patterns, prefixes and suffixes
- This certainly helped in early levels of the game, but we were still finding that kids got stuck in later levels
- We worked with our designers, creating that feedback loop â to carefully to make hypothesis on hinting, and initially, we still werenât getting results
- Developer watched each iteration Turn around within a day Sometimes reprogrammed hint mechanism on the fly after watching kids play, seeing first hand that the kids did not notice the icons encouraged the developer to take action on the critical items
- Sandwich Kingdom is a
- For this game, it was really important to work with the game designer to come up with a script of specific tasks we wanted to the player to do.
- Are the users having fun playing as a girl character? Do they feel emersed in the world? What causes them not to be? Will the platform be a distraction?
- But you have to start somewhere, the best way to start writing a script for your playtest is to play that version of the build
- And figure out where you yourself are getting confused⌠this works great in early stages of the game, later one, you might know too much, so put pressure on the developer to discuss what the critical paths are to ensure the player makes it through their quest
- One feature that is a great example are the dangerous pits. In the final implementation of the game, which you see here, we have a lot of modalities teaching us about the pits, we have an NPC telling us, and art that we will stumble upon if we have pearl walk through the temple a little further
- So the purpose of the pit is that if you walk over it, it sends you back to the beginning, but early in the gameâŚ
- We didnât have NPCs to tell the player to be aware and the art for the pit was just some hash marks. Testing something like this early, even before the feature was done, allowed us to understand if the purpose feature was clear to the player or if it was just annoying. By testing this we learned that the player enjoyed avoiding the pits and saw it as a challenging mechanic rather than just a frustrating obstacle, so we continued with the feature and built it out.
- And depending on your genre, you will need to ask different questions.
- First off, the boring stuff â forms forms formsâŚ
- Make sure kids know you arenât testing them, you are testing the game⌠you are only there to observe. Let them know that
- Having siblings play together naturally is a great way to learn if your game is usable or not. It is especially helpful if you can see one child teach the other child how to play. If they are explaining a feature that doesnât even exist or if they have trouble explaining a feature, then you know you may not be teaching core skills of the game properly
- Its tough to ask kids âwas this hardâ
- But how do we know how the entire game works???
- We were developing the hw and the sw at the same time. Saving game metrics wasnât a sensible priority for the team so we had to find another way to get quantitative data
- Adults can be easier to come by, but you have to keep in mind that adults play differently than kids, so use your target audience as much as possible but recognize what value you can get from having a different set of eyes
- Longitudinal testing allows you to see major issues, but sometimes it just isnât possible to fix them
- - Doing Games User Research is better than nothing! Get your feedback loop on!Think about what your design is about â what do you want the player to experience?Work closely with the Games User Researcher or persons working on the test to make sure they understand what feedback you need â what your priorities are? What information are you lacking?Games user research can help you improve the Game Designthe platform experience, Experience when used by your target audience, and collecting the most actionable feedback â regardless of development phase, gameplay, genreâŚ
- - Doing Games User Research is better than nothing! Get your feedback loop on!Think about what your design is about â what do you want the player to experience?Work closely with the Games User Researcher or persons working on the test to make sure they understand what feedback you need â what your priorities are? What information are you lacking?Games user research can help you improve the Game Designthe platform experience, Experience when used by your target audience, and collecting the most actionable feedback â regardless of development phase, gameplay, genreâŚ
- - Doing Games User Research is better than nothing! Get your feedback loop on!Think about what your design is about â what do you want the player to experience?Work closely with the Games User Researcher or persons working on the test to make sure they understand what feedback you need â what your priorities are? What information are you lacking?Games user research can help you improve the Game Designthe platform experience, Experience when used by your target audience, and collecting the most actionable feedback â regardless of development phase, gameplay, genreâŚ