2013, 08 July
Part 2 of the tutorial illustrated at ICWE 2013, by Luca Galli (Politecnico di Milano)
Crowdsourcing and human computation are novel disciplines that enable the design of computation processes that include humans as actors for task execution. In such a context, Games With a Purpose are an effective mean to channel, in a constructive manner, the human brainpower required to perform tasks that computers are unable to perform, through computer games. This tutorial introduces the core research questions in human computation, with a specific focus on the techniques required to manage structured and unstructured data. The second half of the tutorial delves into the field of game design for serious task, with an emphasis on games for human computation purposes. Our goal is to provide participants with a wide, yet complete overview of the research landscape; we aim at giving practitioners a solid understanding of the best practices in designing and running human computation tasks, while providing academics with solid references and, possibly, promising ideas for their future research activities.
2. ABOUT THE TUTORIAL
• Crowdsourcing, Human Computation, and GWAPs are hot topics
• “Human Computation” => more than 3000 papers
• 400 in 2013
• “Crowd Sourcing” => more than 36000 papers
• 4800 in 2013
• “Games With A Purpose” => more than 1400 papers
• 162 in 2013
• This short tutorial is necessarily shallow, but
• Concrete Examples
• Lot of references and links
• An outlook on the future
• Slides and additional materials available
• http://hcgwap.blogspot.com
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
2
3. ABOUT THE SPEAKERS
ALESSANDRO BOZZON
Assistant Professor - TU Delft
http://www.alessandrobozzon.com
a.bozzon@tudelft.nl
LUCA GALLI
Ph.D. Student - Politecnico di Milano
http://www.lucagalli.me
lgalli@elet.polimi.it
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
3
• RESEARCH BACKGROUND AND INTERESTS
• Web Data Management
• Crowdsourcing and Human Computation
• Game Design
• Web Engineering and Model Driven Development
4. AGENDA
4
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
5. AGENDA
• PART 1 => CrowdSourcing and Human Computation
• Introduction
• Design of Human Computation Tasks
• Frameworks For/With Human Computation
• The Future of Human Computation
• PART 2 => Games With a Purpose
• Play vs Games
• Introduction to Game Design
• Games with a Purpose Design Guidelines
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
5
6. PART 2
AN INTRODUCTION TO
GAMES WITH A PURPOSE
6
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
7. 7
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
Something really bad is
going to happen…
8. 8
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
Or not?
9. 9
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
MAGIC CIRCLE
“All play moves and has its being within a playground marked off beforehand materially or
ideally, deliberately or as a matter of course… The arena, the card-table, the magic circle,
the temple, the stage, the screen, the tennis court, the court of justice, etc., are all in form
and function play-grounds, i.e., forbidden spots, isolated, hedged round, hallowed, within
which special rules obtain. All are temporary worlds within the ordinary world, dedicated
to the performance of an act apart.”
Johan Huizinga, “Homo Ludens”, (1938-1950)
10. WHAT IS PLAY?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
10
Autonomy:
play
is
a
voluntary
ac2vity.
Safety:
during
play
there
are
radically
reduced
serious
consequences
in
what
we
do
A<unement:
being
interested
in
the
same
things
temporarily
while
playing
together
Explora2on:
the
possibility
to
experiment
and
try
out
new
things
Mastery:
the
will
to
improve
one’s
own
skills
11. WHICH ARE THE
BENEFITS?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
11
“Play
is
an
enjoyable
ac/vity
that
allows
us
to
train
and
test
our
skills
in
a
safe
environment.”
Fosters
crea2vity,
flexibility,
and
learning
An2dote
to
loneliness,
isola2on,
anxiety,
and
depression
Teaches
perseverance
Hints
on
how
to
cooperate
with
others
Increases
energy
and
prevents
burnout
Stuart Brown, “Play: How it Shapes the Brain,
Opens the Imagination, and Invigorates the
Soul”, 2009.
12. GAMES VS PLAY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
12
Games
are
dis/nguished
from
play
–
Play
is
free-‐form
–
Games
are
rule-‐based
A
game
is
a
closed,
formal
system
that
·∙
Engages
players
in
structured
conflict
and
·∙
Resolves
its
uncertainty
in
an
unequal
outcome.
Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric approach
to creating innovative games, 2008
13. CLASSIC GAME DEFINITION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
13
1. Rules
•
Games
are
rule-‐based.
2.
Variable,
quan/fiable
outcome
•
Games
have
variable,
quan/fiable
outcomes.
3.
Valoriza/on
of
outcome
•
The
different
poten/al
outcomes
of
the
game
are
assigned
different
values,
some
posi/ve
and
some
nega/ve
14. CLASSIC GAME DEFINITION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
14
4.
Player
effort
•
The
player
exerts
effort
in
order
to
influence
the
outcome
(games
are
challenging).
5.
Player
aVached
to
outcome
•
The
player
is
emo/onally
aVached
to
the
outcome
of
the
game
in
the
sense
that
a
player
will
be
winner
and
“happy”
in
case
of
a
posi/ve
outcome,
but
a
loser
and
“unhappy”
in
case
of
a
nega/ve
outcome.
6.
Nego/able
consequences
•
The
same
game
[set
of
rules]
can
be
played
with
or
without
real-‐life
consequences
Juul,J.“Half-Real:VideoGamesBetweenReal
RulesandFictionalWorlds”,2005
15. INCENTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
15
Money, Passion, Ease of Participation,
Altruism, Appeal to one’s own
knowledge, Reputation, Fun
Organisciak, Piotr. Why Bother?: Examining the Motivations of Users
in Large-scale Crowd-powered Online Initiatives. Diss. University of
Alberta, 2010.
16. INCENTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
16
Intrinsic
mo2va2on
is
defined
as
doing
an
ac2vity
for
its
inherent
sa2sfac2ons
rather
than
for
some
separable
consequence.
Extrinsic
mo2va2on
is
defined
as
doing
an
ac2vity
to
a<ain
some
separable
outcome
Deci, Edward L., and Richard M. Ryan. Self‐Determination. ,1985.
17. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
17
18. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
18
19. GAMES WITH A PURPOSE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
19
Games
with
a
Purpose
(GWAPs)
are
digital
games
that
generate
useful
data
as
a
by-‐product
of
play.
[vA06,LvA09]
The
design
of
a
GWAP
requires
to
create
a
game
so
that
its
structure
encourages
computa2on,
correctness
of
the
output
and
players
reten2on.
20. WHAT THEY ARE USED FOR
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
20
Krause,M.&Smeddinck,J.(2011b).HumanComputation
Games:ASurvey.In:Proceedingsofthe19thEuropean
SignalProcessingConference(EUSIPCO-2011).
Intui2ve
Decisions
Aesthe2c
Judgment
Contextual
Reasoning
Embodiment
Issues
21. ARE THEY WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
21
ESP Game
Purposes:
Let players determine the
contents of images by
submitting meaningful labels
they can agree on.
Results:
A total of 13,630 people
played the game during the
test phase, generating
1,271,451 labels for
293,760 different images.
Luis von Ahn and Laura Dabbish. 2004.
Labeling images with a computer game.
22. ARE THEY WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
22
Foldit
Purposes:
The objective of the game is
to fold the structure of
selected proteins to the best
of the player's ability, using
various tools provided within
the game to discover new
structural configurations.
Results:
In 2011, players of Foldit helped to decipher the crystal structure of
an AIDS-causing monkey virus. While the puzzle was available to play
for a period of three weeks, players produced an accurate 3D model of
the enzyme in just ten days. The problem of how to configure the
structure of the enzyme had stumped scientists for 15 years.
Khatib, F.; Dimaio, F.; Cooper, et al. (2011). "Crystal structure of a monomeric
retroviral protease solved by protein folding game players"
23. GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
23
“The
use
of
game
design
techniques
and
game
mechanics
to
enhance
non-‐game
contexts”
S. Deterding, M. Sicart, L. Nacke, K. O’Hara, and D. Dixon,
“Gamification. Using game-design elements in non-gaming contexts”
Actually
ques/onable,
we
will
see
why
Sebastian Deterding Miguel Sicart
24. WHAT IS USED FOR
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
24
25. IS IT WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
25
Samsung Nation
Purposes:
Samsung Nation, a social
loyalty program that lets
users earn badges for
activities as writing reviews
and watching videos and
compete for rewards.
Results:
500% increase in
customers product reviews
66% increase in site
visitors
30% increase in comments
reduced marketing costs
reduced product support
costs
26. IS IT WORKING?
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
26
MTV Italia MyChart
Purposes:
User generated video chart
based on various “game
dynamics” like avatars,
points and leaderboards to
drive users from Facebook
to TV and to loyalize existing
TV audience.
Results:
More than 500.000 votes in
less then three months
More than 150.000 videos
viewed
27. COMMONALITIES AND
DIFFERENCES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
27
One
or
more
objec2ves
or
tasks
that
users
have
to
accomplish.
A
gaming
experience,
defining
challenges
to
overcome
and
rewards
for
their
solu2on.
One
or
more
players,
the
users
of
the
applica2on,
who
are
profiled
and
monitored
in
their
ac2vi2es.
But...
28. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
28
INTRODUCTION TO
GAME DESIGN
29. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
29
Mancala, 7th
century AD
30. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
30
Unreal
Tournament 3,
Epic Games,
2007
31. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
31
Turn
based
boardgame
vs
Real
/me
ac/on
shooter
Handmade
physical
board
vs
Personal
Computer
Public
domain
rules
vs
Copyrighted
32. 1 - PLAYERS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
32
Number
of
players
Roles
of
the
players
Interac2ons
among
players
and
the
game:
Single
Player/Mul2
Single
Unilateral/Mul2lateral
Compe22on
Coopera2ve
Team
compe22on
33. 2 - OBJECTIVES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
33
Objec2ves:
specific
goals
for
the
player
to
accomplish
Capture:
capture
or
destroy
a
resource
Chase:
catch
an
opponent
or
elude
one
Race:
reach
a
goal
before
the
other
players
Alignment:
arrange
game
objects
in
a
spa/al
or
conceptual
configura/on
RescueEscape:
bring
a
unit
to
safety
Construc2on:
build,
maintain,
manage
objects
Explora2on:
explore
the
environment
where
the
game
take
place
Solu2on:
solve
a
problem
or
puzzle
before
(or
more
accurately)
the
compe//on
or
certain
constraints
Outwit:
gain
and
use
knowledge
to
defeat
other
players
34. 2 – OBJECTIVES: EXAMPLES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
34
Alignment:
arrange
game
objects
in
a
spa2al
or
conceptual
configura2on
Tetris, Alexey Pajitnov, 1984 Puzzle Bobble, Taito
Corporation, 1994
35. 2 – OBJECTIVES: EXAMPLES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
35
Solu2on:
solve
a
problem
or
puzzle
before
(or
more
accurately)
than
the
compe22on
or
following
certain
constraints
Connect Four, Milton Bradley,
1974
Professor Layton and the
Miracle Mask, Level-5, 2011
36. 3 - PROCEDURES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
36
Procedures:
the
methods
of
play
and
the
ac/ons
that
players
can
take
to
achieve
the
game
objec/ves.
They
are
used
to
define
who
does
what,
where
and
how.
Typically
a
game
is
composed
of:
Star%ng
ac%on:
How
to
put
a
game
into
play.
Progression
of
ac%on:
Ongoing
procedures
a_er
the
star/ng
ac/on.
Special
ac%ons:
Available
condi/onal
to
other
elements
or
game
state.
Resolving
ac%ons:
Bring
gameplay
to
an
end.
37. 3 – PROCEDURES: EXAMPLE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
37
Star%ng
ac%on:
Choose
a
player
to
go
first.
Each
player
chooses
a
color:
red
or
black.
Progression
of
ac%on:
On
each
turn,
a
player
drops
one
colored
checker
down
any
of
the
slots
in
the
top
of
the
grid.
Resolving
ac%ons:
The
play
alternates
un/l
one
of
the
players
gets
four
checkers
of
one
color
in
a
row.
The
row
can
be
horizontal,
ver/cal,
or
diagonal.
Connect Four
38. 4 - RULES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
38
•
Define
Objects
•
Restrict
Ac/ons
•
Determine
Effects
(ECA
rules)
Chess:
A
player
cannot
move
her
king
into
check.
Poker:
A
straight
is
five
consecu8vely
ranked
cards;
a
straight
flush
is
five
consecu8vely
ranked
cards
of
the
same
suit.
WarCrad
II:
To
create
knight
units,
a
player
must
have
upgraded
to
a
keep
and
built
a
stable.
39. 5 - RESOURCES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
39
Resources:
valuable
objects
that
can
help
the
players
to
achieve
their
goals
but
are
scarce
in
the
system.
Lives:
number
of
“trials”
that
can
be
aVempted
to
reach
a
goal
Units:
mul/ple
objects
that
has
to
be
managed
instead
of
lives
Health:
represent
the
status
of
loss
or
near
loss
of
lives
and
units
Points:
numerical
value
that
represents
a
measure
of
the
skill
and
progression
of
a
player
Ac2ons:
number
of
possible
dis/nct
choices
that
a
player
can
make
within
a
defined
/meframe
Power-‐ups:
object
that
gives
a
boost
to
the
player
Items:
used
by
the
player
to
accomplish
an
objec/ve,
made
scarce
by
the
system
Turns:
the
number
of
game
phases
within
which
a
player
must
accomplish
the
objec/ves
Time:
restricts
player
ac/ons
or
phases
in
periods
of
/me.
40. 5 – RESOURCES: EXAMPLE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
40
Card Hunter, Blue Manchu Pty Ltd, TBR
Time
Points
Ac/ons
41. 6 - CONFLICTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
41
Conflicts:
emerges
from
the
players
trying
to
accomplish
the
goals
of
the
game
within
its
rules
and
boundaries,
since
procedures
and
rules
tend
to
deter
players
from
accomplishing
goals
directly
or
make
players
work
against
each
other.
The
most
common
conflicts
are
generated
by:
Obstacles,
objects
or
rules
that
limit
the
freedom
of
the
players
Opponents,
since
they
are
usually
trying
to
achieve
an
objec/ve
faster
than
us
Meaningful
Choices,
players
have
to
make
choices
that
will
influence
the
outcome
of
the
game
42. 7 - BOUNDARIES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
42
Boundaries are what
separate the game from
everything that is not the
game and defines the
physical or virtual
scopes in which the
game is performed.
Example: Football would
not be the same game if
the boundaries of the
football field were not
defined.
43. 8 - OUTCOME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
43
Outcome:
the
outcome
of
the
game
(if
present)
is
uncertain,
since
it
is
not
possible
to
predict
the
results
ahead.
If
an
outcome
can
be
obtained
it
has
to
be
quan/fiable
with
respect
to
the
defined
goals.
It
is
different
than
the
objec/ve
since
all
the
player
can
reach
the
same
objec/ve.
44. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
44
Derived by modeling the guidelines and best practices found in:
Fullerton, T.; Swain, C. & Hoffman, S.
Game Design Workshop: A playcentric
approach to creating innovative games,
2008
Crowford, C. The Art of Computer
Game Design, 1984
GAME DEVELOPMENT PROCESS
45. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
45
46. GWAPS AS SERIOUS GAMES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
46
BenSawyer,PeterSmith:SeriousGames
Taxonomy,2008
47. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
47
Katharina Siorpaes and Martin Hepp. 2008. OntoGame:
weaving the semantic web by online games.
48. GAMES WITH A PURPOSE:
CURRENT ISSUES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
48
● “Ad-‐Hoc”
Design
● Engagement
and
reten/on
mechanisms.
● Mapping
from
task
to
game
mechanics
● Valida/on
techniques
and
strategies
E.G.
Real
user
comments
about
OntoGalaxy
49. ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
49
1)
PURPOSE
2)
TASK
DEFINITION
3)
GAME
MECHANICS
DESIGN
4)
INSTANCE
MATCHING
5)
OUTPUT
VALIDATION
6)
PERFORMANCE
EVALUATION
7)
ENGAGEMENT
SRATEGY
GWAPS DIMENSIONS
51. REQUIREMENTS
SPECIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
51
uTask
Task
Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
52. INPUT OBJECTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
52
Adapted from“Mathematical Modeling of social games, IrwinKing, SIAG2009”
An input object is an object with data t23e and a set of
finite a67ibutes denoted as
tex9, images, audio seg=ents, video seg=ents, other
unst7@ct@red data, st7@ct@red data
where the data t23e is the media t23e presented by
Each a67ibute has a relationship to a set of
values .
Each is an object with its own data t23e that represent the
metadata of
53. DEFINING
OPERATIONS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
53
uTask
Task
Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
54. OPERATION TYPES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
54
They
fall
in
two
broad
categories:
Genera/ve
tasks
Decision
tasks
A
possible
(non-‐exhaus/ve)
list
of
human
computa/on
tasks
may
include:
•
Object
Recogni/on/Iden/fica/on/Detec/on
•
Sor/ng
(Clustering/Ordering)
•
Natural
Language
Processing
•
State
Space
Explora/on
•
Content
Genera/on/Submission
•
User
preference/opinion
elicita/on
55. ENSURE OUTPUT QUALITY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
55
uTask
Task
Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
56. Twofold
purpose:
ensure
output
correctness
and
counter
player
collusion.
OUTPUT VALIDATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
56
Player
Tes2ng:
compare
the
output
provided
by
the
user
with
known
correct
outputs
Repe22on:collect
more
than
one
solu/on
for
the
same
input
object,
then
aggregate
the
solu/ons
Taboo
Output:
limit
the
possible
solu/ons
that
can
be
submiVed
Random
Pairing:
pairing
players
at
random
57. TASK EXECUTION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
57
uTask
Task
Description
Input Objects
Output Objects
Execution Interface
Operations Output Validation
58. TASK TO GAME
MECHANICS MATCHING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
58
Does
a
game
with
game
mechanics
similar
to
the
task
exists?
If
so,
integrate
the
task
within
the
game
If
not,
custom
game
mechanics
has
to
be
implemented.
Clear
and
Transparent
game
mechanics
59. SOLUTION MECHANICS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
59
Pa<ern
Recogni2on
Wikipedia
stands
to
knowledge
as
BoardGameGeek
stands
to
games.
Benng/Wagering
Line-‐Drawing
Tile-‐Placement
Pa<ern
Building
Memory
Hand
Management
Ac2on
Programming
Auc2on/Bidding
Partnership
Possible,
meaningful
game
mechanics:
Take
Inspira/on!
60. TILE PLACEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
60
Tile Placement games feature placing a piece to
score points, based on adjacent pieces or pieces in
the same group/cluster, taking into consideration
non-spatial properties like color, "feature
completion", cluster size etc.
61. PATTERN RECOGNITION APPLIED TO
GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
61
Kawrykow A, Roumanis G, Kam A, Kwak D, Leung C, et al. (2012) Phylo: A Citizen
Science Approach for Improving Multiple Sequence Alignment
Purpose:
Used
to
arrange
the
sequences
of
D.N.A,
R.N.A
or
proteins
to
iden/fy
regions
of
similarity
Game
Mechanic:
align
the
sequence
contained
in
each
row
in
order
to
obtain
the
greatest
number
of
columns
with
matching
colors.
62. LINE DRAWING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
62
Games that make use of this mechanics
involve drawing drawing of lines in one
way or another.
63. LINE DRAWING
APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
63
Purpose:
Segment
images
Game
Mechanic:
draw
the
shapes
of
objects
in
a
provided
image
in
order
to
make
the
other
players
guess
the
underlying
object.
64. MEMORY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
64
Games that use the Memory mechanic
require players to recall previous game
events or information in order to reach
an objective.
65. MEMORY APPLIED TO
GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
65
Purpose:
Find
similar
Images
Game
Mechanic:
Exploit
the
visual
memory
of
players
to
find
images
which
are
similar.
66. PARTNERSHIP
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
66
Games
with
partnerships
offer
players
a
set
of
rules
for
alliances
and
teams.
Partners
are
o_en
able
to
win
as
a
team,
or
penal/es
are
enforced
for
not
respec/ng
alliances.
67. PARTNERSHIP
APPLIED TO GWAPS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
67
Purpose:
Collect
“common-‐sense”
fact
for
specific
words.
Game
Mechanic:
provide
hints
to
the
partner
in
order
to
let
him
guess
the
secret
word
68. GWAPS VALIDATION
MECHANICS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
68
Bad
solu/ons
due
to:
incompetence
mischief
behavior
plain
mistakes
Find
or
adapt
game
mechanics
in
order
to
validate
the
results.
69. GWAP FORMAL MODEL
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
69
Based on Mathematical Modeling of social games, Irwin King, SIAG2009
is the set of uTask that are associated with the
problem, with the uTask defined as we have
previously seen.
is the answer domain; it contains the set of
possible cer9ain answers fi (based on the input
D) if the out3ut is contained within a defined
domain, or empt2 if the answers are provided by
the players
is a f@nction that deter=ines if a par9icular
out3ut is a valid solution to the problem
is the GWAP Domain, that maps the input to
the out3ut of a GWAP
70. MULTIPLAYER: INPUT AGREEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
70
Same
or
different
input
Based
on
each
other’s
hints,
player
must
agree
if
they
are
dealing
with
the
same
input
Given
a
correct
answer
set
for
the
problem
players
have
a
set
of
poten/al
outputs
which
cardinality
influence
the
probability
of
agreeing
upon
the
input
Results Submission:
One Trial
Suggested for:
Subjective Information
Same
or
different
input,
same
roles
71. INPUT AGREEMENT: TAG A TUNE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
71
Same
Input:
Audio
file
Two
different
shared
sets
of
descrip/ons,
one
for
each
player
Result:
same
input
or
different
input
Results
Submission:
One
Trial
72. MULTIPLAYER: OUTPUT AGREEMENT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
72
Same
Input
Agreeing
on
common
output
without
communica/on
Same
Input,
Same
Roles.
Given
a
correct
answer
set
for
the
problem,
players
have
a
set
of
poten/al
outputs
the
probability
to
obtain
an
agreement
depends
on
the
shared
output
Results
Submission:
Mul/ple
Trials
Suggested
for:
Objec/ve
Informa/on
73. OUTPUT AGREEMENT: ESP GAME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
73
Same
Input:
Image
List
of
possible
guesses,
hidden
to
the
other
players
Result:
Meaningful tag
for the provided
image
Results
Submission:
Mul/ple
Trials
74. MULTIPLAYER: INVERSION PROBLEM
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
74
One
players
knows
the
whole
problem
and
gives
hints
to
the
other
players.
The
others
try
to
unveil
the
secrets
based
on
the
hints.
Input
to
just
one
player
the
output
of
one
player
is
the
input
for
the
others
The
correct
answer
set
for
the
problem
is
unknown
The
probability
of
guessing
depends
on
the
cardinality
of
the
hints
Result:
User
generated
content,
solu/ons
for
complex
data
structures
Results
Submission:
Mul/ple
Trials
75. INVERSION PROBLEM: IMAGE
LABELING GAME
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
75
Describer:
Image
Seekers:
Textual
description
given by the
describer
Results
Submission:
Multiple trials
Results:
Meaningful
descriptions
for the image
76. INSUFFICIENT PLAYERS:
PRERECORDED GAMES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
76
Fake players based on the logs of previous matches
Pros: The gwap can be played anytime
Cons:
The actions in the game has to be modeled and
stored
Inversion problem games can be difficult to simulate
Requires a bootstrapping phase to acquire the initial
traces
77. SINGLE PLAYER: ALGORITHMIC
EVALUATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
77
Automa/c
algorithm
able
to
par2ally
assess
the
quality
of
the
provided
results.
For
each
instance
of
the
game
the
input
is
provided
to
just
one
player
The
correct
answer
set
for
the
problem
is
defined
The
probability
to
obtain
meaningful
results
is
applica/on
specific,based
on
the
algorithm
that
is
used
to
make
the
evalua/on
78. ALGORITHMIC EVALUATION: TYPE
ATTACK
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
78
Jovian,
L.T.;
Amprimo,
O.,
"OCR
Correc8on
via
Human
Computa8onal
Game,"
System
Sciences
(HICSS),
2011
44th
Hawaii
Interna8onal
Conference
on
,
vol.,
no.,
pp.1,10,
4-‐7
Jan.
2011
Improving
the
results
of
OCR
in
digitalizing
ar/cles
old
newspapers.
Human
players
transcribe
snippet
of
text
and
their
performance
is
measured
by
comparing
the
number
of
similar
words
in
the
player’s
output
and
the
text
from
the
library’s
OCR
transcrip/on.
79. HYBRID
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
79
Combines
the
previous
approaches
Input
data
based
on
previous
results
or
user
generated
content
Inputs
to
one
or
more
players
with
same
or
different
roles
Answer
set
depending
on
the
nature
of
the
problem
or
provided
by
a
player
The
probability
of
obtaining
a
solu/on
depends
on
the
combina/on
of
the
strategies
80. HYBRID: SKETCHNESS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
80
Input:
One
image
to
a
player
Hints
to
the
others
Segments
traced
by
one
player,
possible
tags
submiVed
by
the
others
Result:
Meaningful
tag
+
segmenta/on
Results
Submission:
Mul/ple
Trials
81. INSTANCE MATCHING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
81
How to assign tasks (instances) to players?
• Random
Strategy:
assign
a
random
task
Pros:
Easy
to
implement
Cons:
Does
not
take
into
considera/on
player
skills
• Past
History:
assign
a
task
based
on
past
performances
Pros:
Can
improve
the
quality
Cons:
Keeping
track
of
gaming
history
Performance
measures
must
be
defined
• Selec2ve
Assignment:
assign
a
specific
task
based
on
skills
and
past
performances
Pros:
Can
improve
the
quality
Cons:
Same
as
Past
History
Needs
profiling
of
the
player
to
assess
skills
82. PERFORMANCE EVALUTION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
82
Luis von Ahn and Laura Dabbish. 2008. Designing games with a
purpose. Commun. ACM 51, 8
Throughput = average number of problem instances
solved per human hour
ALP = average (across all people who play the game)
overall amount of time the game will be played by an
individual player
Expected contribution = throughput multiplied by
ALP.
83. ENGAGEMENT STRATEGY
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
83
Core activity must be already engaging for the user.
Gamification involves defining clear and definitive objectives
that last even past the gaming experience, and are either set
by, or negotiated between the user and the game along with a
progressive path of short and intermediate goals leading to it.
Onboarding
Retention
Virality
Social Dimensions
Additional Mechanics
84. POINTS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
84
Points
or
Player
Scores
are
a
numerical
value
that
represents
a
measure
of
the
skill
of
a
player.
•
Immediate
and
las/ng
feedback
•
External
display
of
progression
•
May
determine
the
win
state
• Connec/on
between
progress
in
the
game
and
rewards
Galli, L., Fraternali, P. “Achievement
Systems Explained“ SGSC2012,
Singapore
Werbach, K. & Hunter, D.
For the Win: How Game Thinking
Can Revolutionize Your Business
Wharton Digital Press, 2012
85. LEADERBOARDS
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
85
A
Leaderboard
is
an
ordered
list
of
players
based
on
the
scores
they
have
obtained
in
a
specific
game
or
system.
•
Relates
the
performance
of
a
player
to
the
others
•
Fosters
compe//on
and
par/cipa/on
•
Risky:
May
be
demo/va/ng.
Galli, L., Fraternali, P. “Achievement
Systems Explained“ SGSC2012,
Singapore
Werbach, K. & Hunter, D.
For the Win: How Game Thinking
Can Revolutionize Your Business
Wharton Digital Press, 2012
86. ACHIEVEMENTS AND BADGES
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
86
An
Achievement
is
a
set
of
tasks,
defined
by
a
designer,
for
the
player
to
fulfill
so
to
achieve
a
milestone
and
track
the
progress
in
a
system.
A
Badge
is
an
ar/fact
associated
to
the
comple/on
of
an
achievement
and
given
to
a
player
a_er
its
comple/on,
or,
in
gaming
terms,
a_er
“unlocking
the
achievement”.
•
Define
goals
•
Onboarding
tool
•
Visual
markers
for
reputa/on,
•
Provide
las/ng
rewards
Galli, L., Fraternali, P. “Achievement
Systems Explained“ SGSC2012,
Singapore
87. THIS IS JUST A GLIMPSE OF
WHAT GAMIFICATION IS...
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
87
88. EXAMPLE: ZAMZEE GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
88
Personalization Rewards
Challenges
Social Status
89. CASE STUDY: FASHION TREND MINING
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
89
Problem statement: segment fashion images for mining trends based
on visual features of garments (e.g. color and texture)
Use case: identifying trends in collections of images of people
and garments
Applications: retrieving similar garments, inspect clothing trends
in image collections, analyzing trends change in the years
Color
descriptors
Texture
descriptors
coarse
(sub-)image
similarity
90. THE FASHION TREND MINING PIPELINE
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
90
Male, 24 Female, 22Female?, ??
91. REQUIREMENTS SPECIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
91
Task Description: Recognize if a particular
garment is present within a picture or define a
new one and outline its contours.
Input Objects: A fashion image, an optional
tag defining the garment to identify.
Aggregation Strategy: assign a value of 0 to
each pixel outside the contour and 1 to each
pixel contained within the contour, sum all the
contribution and apply a threshold based on the
number of players.
Output Data: For each submitted task the game
has to provide the contour of the garment within
the image (Polyline) and a tag defining the
garment that has been segmented
92. GWAP DESIGN
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
92
Solution Mechanics:
Pattern Recognition
Established genre:
Draw and Guess
Inversion Problem Mechanic
PLAYERS:Number >=2
Multilateral Competition
Two different roles:
Sketcher: has to draw the contours
of the stated garment
Guesser: has to guess the garment
drawn by the sketcher
93. PLAYER ROLE: SKETCHER
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
93
● The only player to see the
low confidence image
● “May” be asked to provide a
tag for the image
● Is asked to draw the contour
of the object for which the
tag is provided within the
allotted time
● Goal of the Sketcher is to
let the other players guess
the tag within a time slot
without providing
any other hints than the
contour
94. PLAYER ROLE: GUESSER
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
94
● Any other player in the game
● His/Her goal is to guess the
object for which the Sketcher
has provided the contour
● Not allowed to draw on the
whiteboard, just to type
in the chat box the probable
answer as fast as possible
● Scoring:
● Sketcher: 10 pts + 1 for each guesser
● Guesser: 10 pts to the first, then
decreasing down to five
96. FRAMEWORKS: GAMIFICATION
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
96
Google Play Game
Services
Mozilla Open Badges
User Infuser
97. THE CUBRIK PROJECT
ICWE 2013 - An Introduction To Human Computation and Games With a Purpose
97
● CUbRIK is a research project
financed by the European Union
● Goals:
● Advance the architecture of
multimedia search
● Exploit the human
contribution in multimedia
search
● Use open-source
components provided by
the community
● Start up a search business
ecosystem
● http://www.cubrikproject.eu/