Workshop "Weaving Relations of Trust in Crowd Work: Transparency and Reputation across Platforms" co-located with ACM Web Science Conference 2016. http://trustincrowdwork.west.uni-koblenz.de/

1. Cristina Sarasua, University of Koblenz-Landau
M. Six Silberman, IG Metall
Kristy Milland, Turkernation
Gianluca Demartini, University of Sheffield

2. Welcome!
Benvenuto!
¡Bienvenidos
Willkommen!

3. Workshop Web Site
http://trustincrowdwork.west.uni-koblenz.de/
Workshop Twitter Hashtag
#trustincrowdwork
Workshop Twitter Account
https://twitter.com/trustincw
Conference Twitter Hashtag
#websci16

4. Crowd Work

5. Reputation
Turkopticon
https://turkopticon.ucsd.edu/
Turkernation
http://turkernation.com/

6. Still, so much to improve!

7. Goals for today
● analyze different aspects of trust in online crowd work
● discuss ways of boosting transparency and managing reputation
● learn about problems, solutions and wishes related to trust in crowd
work
● understand the three perspectives:
○ Crowd workers
○ Requesters
○ Crowdsourcing Platforms

8. Topics

9. Measuring trust
● Information needs of crowd workers, requesters and platforms
● Relevant data sources to foster trust among workers and requesters
● Extraction of explicit and implicit information from crowdsourcing data
sources
● Subjective and behavioral assessments of trust (i.e., measures)

10. Building and reinforcing trust
● Relation between transparency, reputation and trust
● Trust in traditional workplace vs. the crowd “workplace”: similarities
and differences
● Reputation in crowd work vs. reputation in other online systems (e.g.,
Airbnb)
● Monitoring reputation dynamics (e.g., preventing gaming of reputation
mechanisms)
● New automatic and hybrid methods to build reputation out of available
and reusable online data
● Privacy concerns: What information may be public and which may be
private? Who decides?
● Boosting direct communication to clarify and enhance reputation
● Incentives for opening crowdsourcing platform data
● Infrastructures for achieving transparency
● Socioeconomic consequences of crowd work reputation in digital
labor platforms

11. Making use of reputational and other trust-related data
● Methods for multi-criteria work and people matchmaking
● Reputation-based crowd recruitment
● Cross-platform user authentication and data provenance verification
methods
● Ethical aspects
● New interaction patterns (worker-worker, requester-requester, worker-
requester, worker-platform owner and requester-platform owner
interaction)
● Reputation management in new models for platform governance (e.g.,
cooperative platforms)
● Trust by design in new crowdsourcing platforms

12. Agenda
Time Session
9:00 - 9:15 Hello! session
9:15 - 10:30 Invited talk
10:30 - 10:45 Coffee break
10:45- 11:30 Paper presentations (part I - studies on
workers)
11:30-12:00 Crowd Statement Marathon
12:00-13:00 Lunch
13:00-14:00 Paper presentations (part II - platforms)
14:00-15:00 Panel & discussions
15:00 -15:15 Coffee break
15:15 - 16:00 Crowdsourced reviews & awards
16:00 -16:30 Closing session

13. Introductions

14. Logistics
● WiFi
○ Name: Courtyard_LOBBY
○ Password: no pssw
● Permission to publish (beginning of paper presentation sessions)
○ PDF files of papers at our Web site
○ Slides at our Web site
○ Fotos at our Web site / Twitter
○ Videos at our Web site

15. Thank you