We desribe Return On Contribution (ROC), a social metric for social software. ROC can be used to characterize social software at the level of (a) an application, (b) types of contributions, (c) particular contributions, and (d) particular contributors (where permitted by privacy rules). Our work also highlights the importance of "lurkers" or "non-public participants" in social software. ROC can be applied across diverse types of social software and forms of participation.

1. Return On Contribution (ROC):
A Metric for Enterprise Social Software
Michael Muller, Jill Freyne*, Casey Dugan,
David R Millen, & Jennifer Thom-Santelli
IBM Research & IBM Center for Social Software
Cambridge MA USA
*Jill Freyne is now at Tasmanian ICT Center, CSIRO, Australia
1

2. Agenda
• How to measure the benefits of social software in
organizations?
• Return On Contribution (ROC)
• Applying ROC to
– Enterprise social software applications
– Types of resources in social software applications
– Points of articulation in social software applications
– Individual users (with a few notes on privacy)
• Conclusion and Next Steps
2

3. Benefits of Social Software
• Informal arguments are known
– Better knowledge-sharing
– Better personal effectiveness
– Improved ability to manage one’s reputation
– In organizations, better satisfaction and retention
• There are few strong studies to support those claims
• Management’s desire: Return On Investment
– Has been shown for niche social software applications
• Customer-support operations
• Customer communities
– Has been shown for advertising opportunities
• Social networking sites
3

4. Return On … What?
• Return On Investment
– ROI = Benefit / Cost € / € (unitless economic ratio)
– We hope for ROI >> 1.0
• Social software benefits – and even costs – are
difficult to measure
– Is the purpose of social software to increase productivity?
– How do you calculate ROI of a telephone? an IM product?
– How do you calculate ROI of a relationship?
4

5. Return On Contribution (ROC)
• A social ratio
– ROI = € / €
– ROI = Benefit / Cost
– ROC = Beneficiaries / Contributors (unitless social ratio)
= Consumers / Producers
• Return On Contribution
– A measure of social effectiveness – do more people benefit
(or consume) than contribute (or produce)?
– Rational Choice theory (Pirolli, 2007)
• Over time, people’s work-oriented decisions are beneficial to them
• Measure those choices and summarize them as a metric
5

6. ROC for Two Enterprise Services
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewa
– Overall usage – Overall usagell”
– Common goods – Common goods
• Bookmarks • Photos
• Tags • Lists (HiveFives)
• Events
• (Person-summaries) • (Person-summaries)
6

7. ROC for Two Enterprise Services
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags • Lists (HiveFives)
• Events
• (Person-summaries) • (Person-summaries)
7

8. ROC for Two Enterprise Services
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags • Lists (HiveFives)
• Events
• (Person-summaries) • (Person-summaries)
8

9. ROC for Two Enterprise Services
• Dogear • Beehive
9

10. ROC for Two Enterprise Services
• Dogear • Beehive
10

11. ROC for Types of Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags • Lists (HiveFives)
• Events
• (Person-summaries) • (Person-summaries)
11

12. ROC for Types of Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
Monthly Social-Networking ROC for three media types
behind the firewall” 20
18
behind the firewall”
Photo
List
– Overall usage 16
14
– Overall usage
Event
12
– Common goods – Common goods
ROCC
10
Data range of Figure 1
8
• Bookmarks 6
• Photos
• Tags 4
2
• Lists (HiveFives)
0
Jun-07
•
Jul-07
Events
Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08
Date
• (Person-summaries) • (Person-summaries)
12

13. ROC for Specific Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags – specific tags • Lists (HiveFives)
• Events
• (Person-summaries) • (Person-summaries)
13

14. ROC for Specific Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags – specific tags Tagging• for audiences (Thom-Santelli
Lists (HiveFives)
et al., 2008)
• Events
• Publishers
• (Person-summaries) • (Person-summaries)
• Evangelists
14

15. ROC for Specific Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags – specific tags • Lists (HiveFives)
– Publisher: podcast tag “Tag-City” • Events
Tag ROC = 7.41 readers/contributor – Personal ROC = 63.00
• (Person-summaries)
– Evangelist: tag “web2.0”
Tag ROC = 1.95 readers/contributor – Personal ROC = 1245.00
• (Person-summaries)
15

16. ROC for Specific Contributions
• Dogear • Beehive
– “Social bookmarking – “Social networking
behind the firewall” behind the firewall”
– Overall usage – Overall usage
– Common goods – Common goods
• Bookmarks • Photos
• Tags – specific tags • Lists (HiveFives)
– Publisher: podcast tag “Tag-City” • Events
Tag ROC = 7.41 readers/contributor – Personal ROC = 63.00
• (Person-summaries)
– Evangelist: tag “web2.0”
Tag ROC = 1.95 readers/contributor – Personal ROC = 1245.00
• (Person-summaries)
16

17. ROC for Other Social Applications
Service Beneficiaries Contributors ROC C
Dogear 10896 4213 2.59
Beehive 21453 8397 2.55
Wiki server 238838 36377 6.57
Discussion server 150000 23000 6.52
Person-tagging 20973 3102 6.76
File-sharing 68762 11276 6.19
17

18. Implications for Design or Potential Use
• Track the development of organizational value of an application
over time
– Does it increase? Does it stabilize?
• Compare the organizational value of different types of
contributions over time
• Compare the organizational value of specific contribution
instances
• Assist the development of individual contributors, especially in
assigned roles such as “evangelist” or “publisher”, by providing
private views of her/his personal ROC
• Monitor, on an anonymous basis, the development of social
capital through aggregate, summary ROC measures across all
beneficiaries and contributors
18

19. Unanswered Questions about ROC
• Are there characteristic “signature” ROC values for
different types of applications?
• How to determine “stabilization” of ROC over time?
• What should the “target” ROC be for a discussion
forum?
– ROC >> 6.0 for some applications looked very nice
– However, ROC=1.0 suggests full democratic participation
– When are different values of ROC desirable?
• What should the “target” ROC be for a type of object,
or a particular object (e.g., a tag)?
• Can ROC help to show the value of “lurkers”? When is
it permissible (under privacy rules) to study “lurking”?
19

20. Summary of Contributions
• Lurkers as non-public participants (Nonnecke and
Preece, 2001) and as altruists (Takahashi et al., 2003)
– Employees in some jobs are “paid to lurk”
– Lurkers’ “consumption” of shared objects is a test of the
organizational value of those objects
• ROC provides a “social value” metric for
– Social software applications (Dogear, Beehive)
– Types of contributions (Photos, Lists, Events in Beehive)
– Specific contributions (Tags in Dogear)
– (where permitted) specific contributors (Taggers in Dogear)
• ROC can help organizations and researchers to assess
and study the value of social media
20

21. Thank you!
Slides may be found on slideshare.net
michael_muller@us.ibm.com
21

22. ROC for Two Enterprise Services
• Dogear • Beehive
22

23. ROC for Two Enterprise Services
• Dogear • Beehive
Two ways to think about benefit
• All users (including contributors)
ROC C = AllUsers/Contributors
• Lurkers only
ROC L = Lurkers/Contributors
23

24. ROC C and ROC L
• Dogear • Beehive
Two ways to think about benefit
• All users (including contributors)
ROC C = AllUsers/Contributors
• Lurkers only
ROC L = Lurkers/Contributors
24

25. ROC for Two Enterprise Services
• Dogear • Beehive
25