Pattern-Based Specification of Crowdsourcing Applications
1. Pattern-Based Specification of
Crowdsourcing Applications
Alessandro Bozzon (TU Delft)
Marco Brambilla (Politecnico di Milano)
Stefano Ceri (Politecnico di Milano)
Andrea Mauri (Politecnico di Milano)
Riccardo Volonterio (Politecnico di Milano)
2. Crowdsourcing and Human Computation
It works like magic!
Endless Applications
Endless Success Stories
2008 Olympics Opening Ceremony
3. Actually…
Often a Try&HopeError process
Task Design Matters
Crowd can be unreliable
($) Incentives Matter
Quality Control Matters
Platform of Execution Matters
19. Crowd Task
[T operation types]
(intra-task patterns)
Object Type
block size
min #obj
(cons)
input buffer
batch flow (on closed task)
stream flow (on closed object)
MicroTask
[MT operation types]
r
data manipulator
20. Case Study: Movie Scene Analysis
Scenario 1: Scene Positioning
Spoiler Alert!
Order Scenes
Scene in Beg/Mid/End
Scenario 2: Actor Identification
Find Actor
Validate Actor
22. Position Scenes
[Classify]
(Static Agreement@3)
MicroTask [Classify]
Scene
block 1
min 1
Spoiler Scenes
[Like]
(Static Agreement@3)
MicroTask [Like]
Scene
block 1
min 1
5
Order Scenes
[Order] (SortByLiking)
MicroTask [Like]
Scene
block 2
min 2
[Class=E]
[Class=B OR M]
Example of Scenario 1 Model