Presentation held at 6th Qualinet General Assembly to introduce the current status of research related to Games with a Purpose, with notes regarding the QoE in Gwaps
2. ABOUT MYSELF
LUCA GALLI
Ph.D. Student - Politecnico di Milano
http://www.lucagalli.me
lgalli@elet.polimi.it
2
• RESEARCH BACKGROUND AND INTERESTS
• Robotics & AI
• Game Design
• Crowdsourcing and Human Computation
6th Qualinet General Meeting
4. AGENDA
• Play vs Games
• Pointers to Game Design
• Introduction to Games with a
Purpose
• Gameplay validation through
playtesting
• A Classic Example: Big Rigs
6th Qualinet General Meeting
4
7. WHAT IS PLAY?
6th Qualinet General Meeting
7
Autonomy: play is a voluntary activity.
Safety: during play there are radically reduced serious
consequences in what we do
Exploration: the possibility to experiment and try out new
things
Mastery: the will to improve one’s own skills
8. GAMES VS PLAY
6th Qualinet General Meeting
8
Games are distinguished from play
– Play is free-form
– Games are rule-based
A game is a closed, formal system that
· Engages players in structured conflict and
· Resolves its uncertainty in an unequal
outcome.
Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric approach
to creating innovative games, 2008
13. 6th Qualinet General Meeting
13
Turn based boardgame vs Real time action shooter
Handmade physical board vs Personal Computer
Public domain rules vs Copyrighted
16. OBJECTIVES: EXAMPLES
6th Qualinet General Meeting
16
Solution: solve a problem or puzzle before (or
more accurately) than the competition or
following certain constraints
Connect Four, Milton
Bradley, 1974
Professor Layton and the
Miracle Mask, Level-5, 2011
18. 3 – PROCEDURES: EXAMPLE
6th Qualinet General Meeting
18
Starting action: Choose a player
to go first. Each player chooses a color:
red or yellow.
Progression of action: On each
turn, a player drops one colored
checker down any of the slots in
the top of the grid.
Resolving actions: The play alternates
until one of the players gets four
checkers of one color in a row. The row
can be horizontal, vertical, or diagonal.
Connect Four
19. 4 - RULES
19
• Define Objects
• Restrict Actions
• Determine Effects (ECA rules)
Chess: A player cannot move her king into check.
Poker: A straight is five consecutively ranked
cards; a straight flush is five consecutively ranked
cards of the same suit.
WarCraft II: To create knight units, a player must have
upgraded to a keep and built a stable.
6th Qualinet General Meeting
20. 5 – RESOURCES: EXAMPLE
20
Card Hunter, Blue Manchu Pty Ltd, TBR
Time
Points
Actions
6th Qualinet General Meeting
22. GAMES WITH A PURPOSE
6th Qualinet General Meeting
22
Games with a Purpose (GWAPs) are digital games that generate
useful data as a by-product of play. [vA06,LvA09]
The design of a GWAP requires to create a game so that its
structure encourages computation, correctness of the output and
players retention.
26. CASE STUDY: FASHION TREND MINING
6th Qualinet General Meeting
26
Problem statement: segment fashion images for mining trends based
on visual features of garments (e.g. color and texture)
Use case: identifying trends in collections of images of people
and garments
Applications: retrieving similar garments, inspect clothing trends
in image collections, analyzing trends change in the years
Color
descriptors
Texture
descriptors
coarse
(sub-)image
similarity
27. THE FASHION TREND MINING PIPELINE
6th Qualinet General Meeting
27
Male, 24 Female, 22Female?, ??
28. REQUIREMENTS SPECIFICATION
6th Qualinet General Meeting
28
Task Description: Recognize if a particular
garment is present within a picture or define a
new one and outline its contours.
Input Objects: A fashion image, an optional
tag defining the garment to identify.
Aggregation Strategy: assign a value of 0 to
each pixel outside the contour and 1 to each
pixel contained within the contour, sum all the
contribution and apply a threshold based on the
number of players.
Output Data: For each submitted task the game
has to provide the contour of the garment within
the image (Polyline) and a tag defining the
garment that has been segmented
29. Solution Mechanics:
Pattern Recognition
Established genre:
Draw and Guess
Inversion Problem Mechanic
PLAYERS:Number >=2
Multilateral Competition
Two different roles:
Sketcher: has to draw the contours
of the stated garment
Guesser: has to guess the garment
drawn by the sketcher
SKETCHNESS
29
6th Qualinet General Meeting
30. PLAYER ROLE: SKETCHER
30
● The only player to see the
low confidence image
● “May” be asked to provide a
tag for the image
● Is asked to draw the contour
of the object for which the
tag is provided within the
allotted time
● Goal of the Sketcher is to
let the other players guess
the tag within a time slot
without providing
any other hints than the
contour
Low
Confidence
Image
Tag of the
target
object
Contour
provided
by the user
6th Qualinet General Meeting
31. PLAYER ROLE: GUESSER
31
● Any other player in the game
● His/Her goal is to guess the
object for which the Sketcher
has provided the contour
● Not allowed to draw on the
whiteboard, just to type
in the chat box the probable
answer as fast as possible
● Scoring:
● Sketcher: 10 pts + 1 for each guesser
● Guesser: 10 pts to the first, then
decreasing down to five
Contour of the garment
provided by the Sketcher
6th Qualinet General Meeting
34. TASK TO GAME
MECHANICS MATCHING
34
Does a game with game
mechanics similar to the task
exist?
If so, integrate the task within
the existing game
If not, a custom game with
custom game mechanics has to
be implemented...
6th Qualinet General Meeting
35. HOW TO EVALUATE A GAME?
6th Qualinet General Meeting
35
The Goal is a fun game…
… but also to solve a task!
Game designs are hypotheses
Playtests are experiments
Evaluate designs off playtest results
Repeat
37. HOW TO IMPROVE GAME
DESIGN
6th Qualinet General Meeting
37
We want to make informed decisions:
• Get data early, get data often
• Iterate constantly
• We don’t know what’s best (players do)
• Create a feedback loop between design and
playtest
41. 6th Qualinet General Meeting
41
TRADITIONAL METHODS
ISSUES
Artificial gameplay sessions
– Many potential biases
– Distorted data (interpreted behavior)
– Lack of empiricism
– Missing elements of objectivity
– Sometimes difficult to establish emotions,
baselines, and independence
44. 6th Qualinet General Meeting
44
TECHNICAL APPROACHES
PHYSIOLOGICAL MEASUREMENTS
Heartrate
Skin
Conductance Level
Eye Tracking
Face Recording
EEG
45. 6th Qualinet General Meeting
45
A CLASSIC EXAMPLE:
Just how bad is Big Rigs:
Over the Road Racing?
It's as bad as your
mind will allow you to
comprehend.
http://www.youtube.com/watch?v=7f3HDsg
LV68
46. 6th Qualinet General Meeting
46
CONCLUSIONS
•Play is a universal need
•Games and videogames are entertaining and
controlled means to satisfy it
•We can exploit players to accomplish astonishing
results...
•...if we design a good and tailored gaming experience
•Technological improvements cannot fix a gameplay
mined by bad design choices
47. THE CUBRIK PROJECT
47
● CUbRIK is a research project
financed by the European Union
● Goals:
● Advance the architecture of
multimedia search
● Exploit the human
contribution in multimedia
search
● Use open-source
components provided by
the community
● Start up a search business
ecosystem
● http://www.cubrikproject.eu/
6th Qualinet General Meeting
Hinweis der Redaktion
Norbert Rosing German Photographer
the polar bear returned every night that week to play with the huskys
“Adult play”
Mancala is not one game, but a family of pit-and-pebble games. Play involves scooping up pebbles from a pit and sowing the pebbles, one at a time, into the other pits. These games were probably created in Africa hundreds (if not thousands) of years ago.A "standard" Mancala board is composed of six pits on each side of the board, and a larger scoring pit on each side. Two players sit across from each other over the board. The large scoring pit to each player's right is "her" scoring pit.
One is turn based boardgame, the other is a real time action shooterOne requires a personal computer the other requires a handmade board and some stonesOne is copyrighted and the other has public domain rulesBut...Spettro e caratteristiche differenti, ma di fondo hanno parecchi elementi in comune, quelli elencati dopo, che permettono di rappresentare un gioco.
Number of playrsRoles of the playersInteractions among players
Objectives: specific goals for the player to accomplish
Procedures: actions or methods of play allowed by the rules, they guide player behaviors and interaction with the game.E.G. Walk, run, swim, shot, pick up power ups.Who does what, where and how. Usually they are defined as:Starting actionProgression of ActionSpecial actionResolving action
Rules: describe what the objects in the game consists of and clarifies what happens in various situations that might arise, limit players behavior and proscribe reactive events. The rules may be respected by an implicit agreement of the player or by the underlying code of the game in case of digital games.
http://www.cardhunter.com/
Outcome: the outcome of the game (if present) is uncertain, since it is not possible to predict the results ahead.If an outcome can be obtained it has to be quantifiable with respect to the defined goals.It is different than the objective since all the player can reach the same objective
Games with a purpose exploit gamers’ time to perform useful tasks where human judgment is necessaryCombinatorial optimization tasks judging the quality of motion, a sound, or an imageutilizing human interaction with the physical environment to solve real world problems
Il messaggio è che, nonostante sia solo un gioco, i giocatori sono stati in grado di organizzarsi in maniera complessa e coordinata in gruppi numerosi per poter raggiungere l'obiettivo di coprire una zona grande come l'australia.
Before even thinking about a gwap, it is necessary to define a task. Without a task to be solved, a Gwap would just be a normal game, we want to solve a problem and we will show how it can be done. The requirement specification phase involves the collection of the information necessary for the definition of a task, a unit of work performed by human worker in the process of solving computational problems that cannot be resolved by AI. Cropping the silhouette of the models in a picture, recognizing and identifying the people contained in a set of images, collecting labeled data for training are examples of what a task involving user of a GWAP may look like.So far the GWAPS that have been proposed have never had a complex flow control (fresh results or improvements over the previous results of other players) and the tasks to be performed are considered already as monolithic microtasks.
“Typical” playtest Watch people play the game Observe their gameplay/behavior Simulate at‐home experiencePresence of observers can bias results– Salient event can slant interpretation– Behavior requires interpretation
People describe their actions as they play Unprompted and uncorrectedInterfere with gameplay/create an artificialexperience/distracting– Inaccurate and biased
Record of gameplay behaviors Deaths, level times, friendly fire, … Objective measurements Aggregate perspective Quantify behavior Opportunity for analyses T‐tests Regressions
Objective notions of player behavior+ See global trends+ Readily enables comparisons, baselineestablishment, and metric creation+ Track changes over time– Averages hide extreme examples– Miss nuance (lacking context)– Requires rigor– Can see ‘illusory’ patterns
More objective measurements of player state+ Quantifiable emotional response+ Analysis/comparison metrics– Expensive– Intrusive– Artificial experience– Requires experimental control