The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design

The Many Faces of Reputation:

Towards a discipline of Web 2.0
reputation system design

Prof. Chris Dellarocas
Robert H. Smith School of Business
University of Maryland
cdell@umd.edu

Copyright © C. Dellarocas 2008-09

Suddenly everything Web 2.0 became so sexy…


1

Except, perhaps, reputation systems…

RepuNomics

How Online Reputation
Enables Markets
Sustains Communities
This
and Creates
New Business
book
Opportunities
does not
exist


But reputation systems are everywhere

eCommerce
eBay
Amazon
Epinions
Web 2.0
Yelp
Slashdot
Crowdsourcing
Yahoo! Answers
iStockPhoto
Gaming
Xbox Live


2

Reputation Systems
are the Unsung Heroes
of the Web


Outline of this talk

Reputation Systems Serve Business Objectives

Four areas where practice runs ahead of
research

Our challenge: Developing a rigorous discipline
of real-life reputation system design


3

Some Definitions

Reputation = a summary of one’s past actions
defined within the context of a specific community

Reputation system = a system that mediates
and automates the process of assessing one’s
reputation
Keeps track of a user’s actions
Aggregates and displays summary statistics


Bird’s eye view of a reputation system

(Inter)
actions Inputs

Aggregation

Outputs
Users Artifacts

COMMUNITY


4

Reputation Systems
serve a variety of
Business Objectives


Why do communities need reputation
systems?
Trust
Encourage “good” and discourage “bad” behavior
Quality
Provide incentives for quality contributions
Recognize best contributors
Matching and Filtering
Assist users in finding suitable partners
Reduce information overload
Participation and loyalty
Give user reasons to join and stay in your community


5



6



7


Four examples

Trust Quality Filtering Loyalty

eBay +++ + + ++
Amazon ++ +++ + ++
Yelp ++ +++ +++ +++
Yahoo!Answers ++ ++ + +++


8

The design space of
Reputation Systems
is very rich


Four dimensions of reputation system design
where practice runs ahead of research

What inputs should be solicited
1.
What outputs should be presented
2.
How transparent should the rules be
3.
How should reputation evolve over time
4.


9

What inputs should be solicited?

What internal actions to keep track of
What external feedback to solicit
Ratings of artifacts vs. ratings of people



10

Determining inputs

Perform an audit of your system
List all actions a user can take
Assess how each action relates to each of the
four classes of objectives
Track/report actions that:
you want to encourage/discourage
provide the most information related to one or more of
your key objectives


A story of a site that got it wrong:
Consumating.com


11

An interesting question:
Should you allow ratings of users….

… separately from the artifacts they create?

Pluses:
Makes a person feel like a person… which might
increase site loyalty
Minuses:
Can distract from quality of one’s work
Might encourage personal attacks



12


1.
How outputs should be presented
2.
3.
4.


What outputs should be presented?

Simple statistics
Star ratings
Score
Named levels
Achievement badges
Social network


13


Simple Statistics


14

Statistics

Score

Social
Network


15



16



17


To what extent should the system allow
comparison across users?

Display of relative performance statistics
Top N lists
“Leaderboards”


18



19



20



21

To compare or not to compare?

Pluses
Increases incentives to contribute
Enhances filtering role of reputation
Minuses
Instills a culture of competition
Obsession with rankings might lead to manipulative,
disruptive behavior
Low ranked users might be discouraged and exit


A cautionary tale

Digg.com


22

How you display reputation information
affects the spirit of your community

COMPETITIVE
CORDIAL

Named
Ranking
Statistical Levels Point-based score
Leaderboards
evidence Badges

Reputation system design must be consistent
with the community’s overall culture


Some ideas

Implement multiple ways of ranking users
No single measure that people obsess on
Allows users with different qualities to feel good
Particularly relevant in systems where matching and
filtering is an important objective


23



1.
2.
3.
4.


24

How transparent should be the aggregation
rules?

Transparency breeds trust

… but also facilitates gaming



25



26



1.
2.
3.
4.


27

How should reputation evolve over time?

Accumulate and grow
Decay and refresh

Tradeoffs
Accumulation breeds complacency
Accumulation discourages new entrants
BUT
Accumulation promotes loyalty and lock-in


Accumulation

Decay and Refresh Copyright © C. Dellarocas 2008-09

28

A case study:
Yelp



29



30


The challenge:
Mapping Business Objectives
to Design


31

Observation #1: Everything should be driven
from a clear understanding of objectives
Trust building/Community Policing
Incentives for Quality
Matching and Filtering
Site Loyalty

It is important for the designer to be very clear with
respect to how these goals are prioritized
It is important to understand how every aspect of design
affects each of these goals


Observation #2: The design of a reputation
system can affect a community’s culture

COMPETITIVE
CORDIAL

Named
Ranking
Statistical Levels Point-based score
Leaderboards
evidence Badges

Reputation system design must be consistent
with the community’s overall culture


32

Observation #3: Intangible aspects of
reputation matter

Users care about their reputation beyond the
tangible benefits it confers to them
Reputation as pure status
Status is zero-sum
Users who are not recognized might get upset
So, introduction of a reputation system might make
some users worse off and cause them to leave


Observation #4: A reputation system can be
a site’s competitive weapon

A way to lock-in users to our site

Lock-in properties of reputation system can
sometimes be in conflict with its other objectives

Understanding the design of competitors’
reputation mechanisms matters


33

In summary

Much of what we know about reputation system
design in Web 2.0 communities is still anecdotal

We need research-driven guidelines on how to
design such systems

A lot of open areas where theoretical and
experimental work needs to be done


In the meantime…
Reputation systems designers will be the poor
lonesome cowboys of the Web
Not necessarily such a bad place to be (hey, it can
get you to places like Gargonza)


34

The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design

Ähnlich wie The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design (20)

Mehr von harrikipper

Mehr von harrikipper (6)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design