This is an overview of the 3-year research works done at the Augmented Social Cognition research group at PARC. See blog at: http://asc-parc.blogspot.com

1. Ed
H.
Chi,
Area
Manager
Peter
Pirolli,
Lichan
Hong,
Bongwon
Suh,
Gregorio
Convertino,
Les
Nelson,
Rowan
Nairn
Augmented
Social
Cognition
Area
Palo
Alto
Research
Center
Interns:
Sanjay
Kairam,
Jilin
Chen,
Michael
Bernstein
Alumni:
Raluca
Budiu,
Bryan
Pendleton,
Niki
Kittur,
Todd
Mytkowicz,
Terrell
Russell,
Brynn
Evans,
Bryan
Chan,
KMRC
students
2009-05-01 Ed H. Chi ASC Overview 1
Image from: http://www.flickr.com/photos/ourcommon/480538715/

2. 14 years of work in foraging and sensemaking
 Information
Scent
– WUFIS
/
IUNIS
(Basic
scent
modeling
algorithms)
[CHI2000,2001]
– Bloodhound
(Simulation
of
web
navigation)
[CHI2003]
– LumberJack
(Log
analysis
of
user
needs)
[CHI2002]
 Information
Foraging
– ScentTrails
[TOCHI2003]
– ScentIndex
[CHI2004]
– ScentHighlight
[IUI2005]
– Visual
foraging
of
highlighted
text
[HCII]
 Sensemaking
– Visualization
of
Web
Ecologies
[CHI98]
– Visualization
Spreadsheets
[Infovis97,
Infovis99]
2009-05-01 Ed H. Chi ASC Overview 2

3. Wikipedia is the best thing ever. Anyone in the world can write
anything they want about any subject, so you know you’re getting the
best possible information.”
– Steve Carell, The Office
2009-05-01 Ed H. Chi ASC Overview 3

4. 2009-05-01 Ed H. Chi ASC Overview 4

5.  Groups
utilize
systems
to
make
sense
and
share
complex
topics
and
materials.
 Wikipedia
(social
status)
 Slashdot
(karma
points)
 WikiHow.com
 Lostpedia.com
2009-05-01 Ed H. Chi ASC Overview 5

6.  Systems
that
evolve
structures
that
can
be
used
to
organize
information.
 Del.icio.us
 Flickr
 YouTube
 Friendster
2009-05-01 Ed H. Chi ASC Overview 6

7.  Counting
votes
– A
way
to
increase
signal-­‐to-­‐noise
ratio
– Information
faddishness
 Examples:
– Digg.com
– Most
bookmarked
items
on
del.icio.us
– Estimating
the
weight
of
an
ox
or
temperature
of
a
room
– The
true
value
of
a
stock
– PageRank
or
Hub
/
Authority
algorithms
2009-05-01 Ed H. Chi ASC Overview 7

8. Voting systems Col. Information Collaborative
Structures Co-Creation
Digg.com eHow.com
IBM dogear Wikipedia
PageRank
Del.icio.us Flickr Slashdot Naver
Heavier
collaboration
2009-05-01 Ed H. Chi ASC Overview 8

9. Voting systems Col. Information Collaborative
Structures Co-Creation
Digg.com
Understanding of eHow.com
Understanding of info Understanding of
micro-economics and social networks
IBM dogear Wikipedia
conflicts and
PageRank coordination
• of foraging [PARC] Del.icio.us Flickr
• Tag network analysis [PARC, Slashdot Naver
Golder, Yahoo] • Wikipedia coordination
• Personal vs. group costs [PARC]
[Huberman, Adamic] • Structural holes (info brokerage) Heavier
• Invisible Colleges [Sandstrom]
• Wisdom of Crowd [Burt] collaboration effects [Pirolli]
• Interference
[Surowieki] • Network constraints and • Co-laboratories [Olson and
• Information cascades structure [various] Olson]
• Community networks / Col.
[Anderson and Holt] • Semantic of semiotic structures /
Problem solving [Carroll]
words [IR, LSA]
2009-05-01 Ed H. Chi ASC Overview 9

10.  Cognition:
the
ability
to
remember,
think,
and
reason;
the
faculty
of
knowing.
 Social
Cognition:
the
ability
of
a
group
to
remember,
think,
and
reason;
the
construction
of
knowledge
structures
by
a
group.
– (not
quite
the
same
as
in
the
branch
of
psychology
that
studies
the
cognitive
processes
involved
in
social
interaction,
though
included)
 Augmented
Social
Cognition:
Supported
by
systems,
the
enhancement
of
the
ability
of
a
group
to
remember,
think,
and
reason;
the
system-­‐supported
construction
of
knowledge
structures
by
a
group.
Citation:
Chi,
IEEE
Computer,
Sept
2008
2009-05-01 Ed H. Chi ASC Overview 10

11. Characteriza*on
Models
Evalua*ons
Prototypes
2009-05-01 Ed H. Chi ASC Overview 11

12. Characteriza*on
Models
Evalua*ons
Prototypes
2009-05-01 Ed H. Chi ASC Overview 12

13. 100%
95% Maintenance
90%
Percentage of total edits
Other
85%
80%
User Talk
75%
User
70%
Article Talk
65%
Article
60%
2001 2002 2003 2004 2005 2006
2009-05-01 Ed H. Chi ASC Overview 13

14.  Conflict
is
growing
at
the
global
level,
and
we
have
some
idea
about
where
it
is.
 But
what
defines
conflict
inside
Wikipedia?
 Build
a
characterization
model
of
article
conflict
– Identify
metrics
relevant
to
conflict
– Automatically
identify
high-­‐conflict
articles
2009-05-01 Ed H. Chi ASC Overview 14

15.  Controversial”
tag
 Use
#
revisions
tagged
controversial
2009-05-01 Ed H. Chi ASC Overview 15

16.  Possible
metrics
for
identifying
conflict
in
articles
Metric type Page Type
Revisions (#) Article, talk, article/talk
Page length Article, talk, article/talk
Unique editors Article, talk, article/talk
Unique editors / revisions Article, talk
Links from other articles Article, talk
Links to other articles Article, talk
Anonymous edits (#, %) Article, talk
Administrator edits (#, %) Article, talk
Minor edits (#, %) Article, talk
Reverts (#, by unique
Article
editors)
2009-05-01 Ed H. Chi ASC Overview 16

17.  5x
cross-­‐validation,
R2
=
0.897
10000
9000
Actual controversial revisions
8000
7000
6000
5000
4000
3000
2000
1000
0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
2009-05-01 Predicted controversial revisions
Ed H. Chi ASC Overview 17

18.  5x
cross-­‐validation,
R2
=
0.897
10000
9000
Actual controversial revisions
8000
7000
6000
5000
4000
3000
2000
1000
0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
Predicted controversial revisions
2009-05-01 Ed H. Chi ASC Overview 18

19.  Highly weighted features of conflict model:
 Revisions
(talk)
 Minor
edits
(talk)
 Unique
editors
(talk)
 Revisions
(article)
 Unique
editors
(article)
 Anonymous
edits
(talk)
 Anonymous
edits
(article)
2009-05-01 Ed H. Chi ASC Overview 19

20.  Revert:
Undoing
one
or
more
edits
– The
page
being
restored
to
a
version
that
existed
sometime
previously.
– Often
used
to
fight
vandalism
 Revert
ratio
as
resistance
metric
– #
of
reverted
edits
/
#
of
total
edit
– This
analysis
excludes
vandalism
to
model
“resistance”

21.  Research
Goal
– How
can
we
identify
point
of
views
between
users?
– Group
people
share
a
common
point
of
view
 Using
revert
as
proxy
for
disagreement
between
users
– Revert
edits:
3,711,638
6.3
%
of
total
edits
– Due
to
vandalism:
577,643
0.99%
of
total
edits
(15.6%
of
reverts)
 Force
directed
layout
– Node:
user,
Edge:
revert
relationship
2009-05-01 Ed H. Chi ASC Overview 21

22. Group D
Group A
Group B
Group C
Number of users in user group A B C Total
Users with Korean point of view 10 6 0 16
Users with Japanese point of view 1 8 7 16
Neutral or Unidentified 7 3 6 17
2009-05-01 Ed H. Chi ASC Overview 22

23. Anonymous (vandals/
spammers)
Sympathetic to husband
Mediators
Sympathetic to parents
2009-05-01 Ed H. Chi ASC Overview 23

24. Monthly Ratio of Reverted Edits

25. Characteriza*on
Models
Evalua*ons
Prototypes
2009-05-01 Ed H. Chi ASC Overview 25

26. Encoding
Retrieval
“video
people
talks
technology”
h:p://www.ted.com/index.php/speakers
h:p://edge.org
“science
research
cogni*on”
26
2009-05-01 Ed H. Chi ASC Overview 26

27. Concepts
Topics
Users
Documents
Noise
Tags
Decoding
Encoding
T1…Tn
2009-05-01 Ed H. Chi ASC Overview 27

28. 2009-05-01 Ed H. Chi ASC Overview 28

29. 2009-05-01 Ed H. Chi ASC Overview 29

30. 2009-05-01 Ed H. Chi ASC Overview 30

31. Source: Hypertext 2008 study on del.icio.us (Chi & Mytkowicz)
2009-05-01 Ed H. Chi ASC Overview 31

32. Bongwon
Suh,
Gregorio
Convertino,
Ed
H.
Chi,
Peter
Pirolli
Bongwon Suh, Gregorio Convertino, Ed H. Chi, Peter Pirolli. The
Singularity is Not Near: Slowing Growth of Wikipedia. In Proc. of
WikiSym 2009. Oct, 2009. Florida, USA
2009-05-01 Ed H. Chi ASC Overview 32

33. Monthly Edits

34. Monthly Active Editors

35.  Edits
beget
edits
– more
number
of
previous
edits,
more
number
of
new
edits
Growth rate depends on
current population size N and
r = growth rate of the population
N(t) = N 0 ⋅ e rt
dN
= r⋅ N
dt
Growth rate Current
of population €
population
€

36.  Ecological
population
growth
model
– r,
growth
rate
of
the
population
– K,
carrying
capacity
(due
to
resource
limitation)
4000000
3500000
K
3000000
dN N Population
2500000
= r ⋅ N ⋅ (1− ) 2000000
dt K 1500000
1000000
500000
0
2000 2002 2004 2006 2008 2010
Year

37.  Follows
a
logistic
growth
curve
New Article
http://en.wikipedia.org/wiki/Wikipedia:Modelling_Wikipedia’s_growth

38.  Carrying
Capacity
as
a
function
of
time.
K(t)
Population
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Year

39. Characteriza*on
Models
Evalua*ons
Prototypes
2009-05-01 Ed H. Chi ASC Overview 39

40. Create
a
Living
Laboratory
as
a
platform
to
develop,
test,
and
market
innovations
[Chi,
HCIC
workshop
2009,
HCII
2009,
IEEE
Computer
Sep/2008]
2009-05-01 Ed H. Chi ASC Overview 40

41. Joint
work
with
Bongwon
Suh,
Aniket
Kittur,
Bryan
Pendleton
Bongwon
Suh,
Ed
H.
Chi,
Aniket
Kittur,
Bryan
A.
Pendleton.
Lifting
the
Veil:
Improving
Accountability
and
Social
Transparency
in
Wikipedia
with
WikiDashboard.
In
Proceedings
of
the
ACM
Conference
on
Human-­‐factors
in
Computing
Systems
(CHI2008).
ACM
Press,
2008.
Florence,
Italy.
2009-05-01 Ed H. Chi ASC Overview 41

42.  Social
translucent
for
effective
communication
and
collaboration
[Erickson
and
Kellogg
2002]
– Make
socially
significant
information
visible
and
salient
– Support
awareness
of
the
rules
and
constraints
– Accountability
for
actions
 Wikis
can
be
a
prime
candidate
– Every
edit
is
logged
and
retrievable
– WikiScanner.com:
analyze
anonymous
IP
edits
– WikiRage.com:
top
edits
2009-05-01 Ed H. Chi ASC Overview 42

43. 2009-05-01 Ed H. Chi ASC Overview 43

44. 2009-05-01 Ed H. Chi ASC Overview 44

45. 2009-05-01 Ed H. Chi ASC Overview 45

46.  Surfacing
hidden
social
context
to
users
 For
readers
– Any
incidents
in
the
past
e.g.
A
sudden
burst
of
edits?
– Who
are
the
top
editors?
– What
is
their
motivation
/
point
of
views
/
expertise
/
topics
of
interest?
– Help
them
judging
the
quality/trustworthiness/usefulness
of
an
article.
 For
writers
– Measure
expertise
/
contribution
/
reputation
– Motivate
them
to
be
more
active
/
responsible
(?)
2009-05-01 Ed H. Chi ASC Overview 46

47.  3
x
2
x
2
design
Controversial Uncontroversial
Visualization Abortion Volcano
High quality
• High stability George Bush Shark
• Low stability
• Baseline
(none) Pro-life feminism Disk
defragmenter Low quality
Scientology and
celebrities Beeswax

50.  Users
recruited
via
Amazon’s
Mechanical
Turk
– 253
participants
– 673
ratings
– 7
cents
per
rating
– Kittur,
Chi,
&
Suh,
CHI
2008:
Crowdsourcing
user
studies
 To
ensure
salience
and
valid
answers,
participants
answered:
– In
what
time
period
was
this
article
the
least
stable?
– How
stable
has
this
article
been
for
the
last
month?
– Who
was
the
last
editor?
– How
trustworthy
do
you
consider
the
above
editor?

51. 1. Significant
effect
of
visualization
– High
>
low,
p
<
.001
2. Both
positive
and
negative
effects
– High
>
baseline,
p
<
.001
– Low
>
baseline,
p
<
.01
3. No
effect
of
article
uncertainty
– No
interaction
of
visualization
with
either
quality
or
controversy
– Robust
across
conditions

52. Joint
work
with
Rowan
Nairn,
Lawrence
Lee
Kammerer,
Y.,
Nairn,
R.,
Pirolli,
P.,
and
Chi,
E.
H.
2009.
Signpost
from
the
masses:
learning
effects
in
an
exploratory
social
tag
search
browser.
In
Proceedings
of
the
27th
international
Conference
on
Human
Factors
in
Computing
Systems
(Boston,
MA,
USA,
April
04
-­‐
09,
2009).
CHI
'09.
ACM,
New
York,
NY,
625-­‐634.
2009-05-01 Ed H. Chi ASC Overview 52

53.  Help
understand
the
importance
of:
– social
cues
and
information
exchanges
– vocabulary
problems
– distribution
and
organization
2009-05-01 Ed H. Chi ASC Overview 53

54. 3 kinds of search
59% 28% 13%
informational navigational transactional
You roughly know what you want You know what you want and where it is You know what you want to do
but don’t know how to find it
Difficult for existing search engines Existing search engines are OK
Opportunity
2009-05-01 Ed H. Chi ASC Overview 54

55. Social Tagging Creates Noise
• Synonyms
• Misspellings
• Morphologies
People use different tag
words to express similar
concepts.
2009-05-01 Ed H. Chi ASC Overview 55

56. 2009-05-01 Ed H. Chi ASC Overview 56

57. Semantic Similarity Graph
Web
Tools
Reference
Guide
Howto
Tutorial
Tips
Help
Tip Tutorials
Tricks
2009-05-01 Ed H. Chi ASC Overview 57

58. Tags URLs
P(URL|Tag)
P(Tag|URL)
 Spreading
Activation
in
a
bi-­‐graph
 Computation
over
a
very
large
data
set
– 150
Million+
bookmarks
2009-05-01 Ed H. Chi ASC Overview 58

59. Database Lucene
• Delicious • P(URL|Tag) • Serve up search
• Ma.gnolia • P(Tag|URL) results
• Tuples of • Pre-computed
• Other social cues bookmarks • Bayesian Network patterns in a fast • Well defined APIs
• [User, URL, Tags, Inference index
Time]
Crawling MapReduce Web Server
Web
Server
UI Search
Frontend Results
• MapReduce:
months
of
computa*on
to
a
single
day
• Development
of
novel
scoring
func*on
2009-05-01 Ed H. Chi ASC Overview 59

60.  Exploratory
interface
users:
– performed
more
queries,
– took
more
time,
– wrote
better
summaries
(in
2/3
domains),
– generated
more
relevant
keywords
(in
2/3
domains),
and
– had
a
higher
cognitive
load.
 Suggestive
of
deeper
engagement
and
better
learning.
 Some
evidence
of
scaffolding
for
novices
in
the
keyword
generation
and
summarization
tasks.
2009-05-01 Ed H. Chi ASC Overview 60

61. Joint
work
with
Lichan
Hong,
Raluca
Budiu,
Les
Nelson,
Peter
Pirolli
Lichan
Hong,
Ed
H.
Chi,
Raluca
Budiu,
Peter
Pirolli,
and
Les
Nelson.
SparTag.us:
A
Low
Cost
Tagging
System
for
Foraging
of
Web
Content.
In
Proceedings
of
the
Advanced
Visual
Interface
(AVI2008),
(to
appear).
ACM
Press,
2008.
2009-05-01 Ed H. Chi ASC Overview 61

62.  Interaction
costs
# People willing to produce for “free”
determine
number
of
people
who
participate
 Surplus
of
attention
&
motivation
at
small
transaction
costs
 Therefore…
 Important
to
keep
interaction
costs
low
Cost of participation
2009-05-01 Ed H. Chi ASC Overview 62

63.  In situ tagging while reading
– No new window
– Clicking vs typing
 Tagging + highlighting
2009-05-01 Ed H. Chi ASC Overview 63

64.  Intuition:
sub-­‐doc
nuggets
useful
– Entities,
facts,
concepts,
paragraphs
 Annotations
attached
to
paragraphs
 Portable
across
pages
and
other
contents
(e.g.
Word
documents)
– Dynamic
pages
– Duplicate
content
2009-05-01 Ed H. Chi ASC Overview 64

65. 2009-05-01 Ed H. Chi ASC Overview 65

66. 2009-05-01 Ed H. Chi ASC Overview 66

67. 2009-05-01 Ed H. Chi ASC Overview 67

68. 2009-05-01 Ed H. Chi ASC Overview 68

69. N=18
SparTag.us + Friend superior to both individual conditions
No difference between the two controls
SparTag.us
With A
Friend (SF)
SF group,
M=0.46, SD=0.22
SO group,
Without M=0.13, SD=0.32
SparTag.us
WS group,
(WS)
M=0.27, SD=0.23
SparTag.us
Only (SO)
[Nelson et al., CHI2009]
2009-05-01 Ed H. Chi ASC Overview 69

70. Social Transparency create
trust and attribution:
• Increase participation via
attribution
Collective Intelligence
• Increase credibility and trust
with community feedback
TagSearch: Mining social • Reduce wiki risks
data for automatic data
clustering and organization:
• Better organization via user-
assigned tags
Higher Productivity via • Better UI for browsing
Collective Intelligence interesting contents sharing
Generic benefits:
• Recommendation instead of • Greater trust
just search • Better decision-making
Intelligence that emerges • Useful sharing of info
from the collaboration and • Auto-organization thru
search
social data
competition of many
individuals
foraging
Foundation:
SparTag.us: sharing of
• Understanding of human
interesting contents:
cognition and behavior • A notebook that automatically
• Data mining of social data organizes your reading
• Modeling of consensus- • Social sharing of important
and interesting tidbits
driven decision-making
• Viral sharing of highlighted
and tagged paragraphs
2008-10-28 Ed H. Chi ASC Overview 70

71.  ASC is creating a plug-and-play platform to enable a number of
applications in support of the Open Web Applications
Social Data Mining Platform Recommendations
App Connectors
App Connectors Pattern Operators, e.g., Tag
Normalization, LDA Clustering, Topic Identification
Combine with
Summarization, Voting other applications
App Connectors Techniques… to create full
Expertise Identification products
App Connectors Hadoop MapReduce, Pig,
MySQL, Django, Java
Extracts data in the form of
tuples from applications, e.g. Dashboard
(user, tag, URL)
…
(user, activity, object)
Core Advantage

72.  Crowdsourcing
[collaborative
co-­‐creation]
– Is
there
a
wisdom
of
the
crowd
in
Wikipedia?
– How
does
conflict
drive
content
creation?
 Collective
Intelligence
[folksonomy]
– Are
social
tags
collectively
gathered
useful
for
organization
of
a
large
document
collection?
 Collective
Averaging
[social
attention]
– Does
voting
systems
identify
the
best
quality
and
most
interesting
information
for
that
community?
 Participation
Architecture
[interaction]
– Does
lowering
the
interaction
cost
barrier
increase
participation
productively?
 Expertise
finding
[social
networking]
– Does
getting
experts
through
social
network
gets
you
to
better
quality
information
sooner?
2009-05-01 Ed H. Chi ASC Overview 72

73. 2009-05-01 Ed H. Chi ASC Overview 73

74.  Research
Vision:
Understand
how
social
computing
systems
can
enhance
the
ability
of
a
group
of
people
to
remember,
think,
and
reason.
 Living
Laboratory:
Create
applications
that
harness
collective
intelligence
to
improve
knowledge
capture,
transfer,
and
discovery.
http://asc-­‐parc.blogspot.com
http://www.edchi.net
echi@parc.com
2009-05-01 Ed H. Chi ASC Overview 74
Image from: http://www.flickr.com/photos/ourcommon/480538715/