Updated version of a talk first given at the e-Metrics Marketing and Optimization Summit, Washington, D.C., October 5, 2010. This version is for the J. Boye conference, Philadelphia, PA, 5-4-2011. The talk spells out the lofty goals of the Smithsonian Institution and the Smithsonian Commons project, and then describes some of the unconventional measurement methodologies we're using to figure out what to do and how to do it. J. Boye Conference page: http://jboye.com/conferences/philadelphia11/program/speakers/michael-edson/#presentation

1. Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
Jedi Mind Tricks for
Optimizing
Lofty Goals
andMeasuring

2. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
@mpedson

3. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
@mpedson
slideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com

4. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
@mpedson
slideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com
Get the full story here

5. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
@mpedson
slideshare.net/edsonm
smithsonian-webstrategy.wikispaces.com
…and here

6. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
The Smithsonian Commons
www.si.edu/commons

7. Preamble
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson
He’s not a
spokesman!

8. First, we need to tell the story…

9. Smithsonian Strategic Plan
Smithsonian Secretary G. Wayne Clough
http://www.si.edu/about/
Four Grand Challenges

10. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/

11. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/

12. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/

13. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
http://www.si.edu/about/

14. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
Strategy
Schmatergy! We’ve
seen strategy
before!

15. Smithsonian Strategic Plan
1. Unlocking the Mysteries of
the Universe
2. Understanding and
Sustaining a Biodiverse
Planet
3. Valuing World Cultures
4. Understanding the
American Experience
Four Grand Challenges
No! This is about
solving big hairy
problems— “work
that matters”
(via @timoreilly)
http://radar.oreilly.com/2009/01/work-on-stuff-that-
matters-fir.html

16. c

17. c
This is big, audacious,
important stuff.
This is work!
This is relevance earned
through a job well done.

18. First order questions

19. First order questions
• Where is this work going to take place?
• What kind of organization, infrastructure,
platforms will be needed to support it?
• What is the organizational change model?
How will change happen?
• Who will be the innovators? The connectors?
The drivers of change?
• …and how do you measure it?

20. Web and
New Media
Strategy

21. Web and
New Media
Strategy
http://smithsonian-
webstrategy.wikispaces.com
More in Fast, Open, and Transparent: developing
the Smithsonian’s Web and New Media Strategy
http://www.slideshare.net/edsonm/michael-edson-the-smithsonian-
web-and-new-media-strategy-what-it-is-how-we-made-it-and-why-it-
makes-a-difference-3656578
And http://www.archimuse.com/mw2010/papers/edson/edson.html

22. http://smithsonian-webstrategy.wikispaces.com/

23. http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital
experience, a new learning model that helps
people with their "lifelong learning journeys,"
and the creation of a Smithsonian Commons—
a new part of our digital presence dedicated to
stimulating learning, creation, and innovation
through open access to Smithsonian research,
collections and communities.

24. http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital
experience, a new learning model that helps
people with their "lifelong learning journeys,"
and the creation of a Smithsonian Commons—
a new part of our digital presence dedicated to
stimulating learning, creation, and innovation
through open access to Smithsonian research,
collections and communities.

25. http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital
experience, a new learning model that helps
people with their "lifelong learning journeys,"
and the creation of a Smithsonian Commons—
a new part of our digital presence dedicated to
stimulating learning, creation, and innovation
through open access to Smithsonian research,
collections and communities.

26. http://smithsonian-webstrategy.wikispaces.com/
The strategy talks about an updated digital
experience, a new learning model that helps
people with their "lifelong learning journeys,"
and the creation of a Smithsonian Commons—
a new part of our digital presence dedicated to
stimulating learning, creation, and innovation
through open access to Smithsonian research,
collections and communities.
Old Learning Model
New Learning Model

27. http://smithsonian-webstrategy.wikispaces.com/
Balancing autonomy and control within the
Smithsonian.rt of our digital presence
dedicated to stimulating learning, creation,
and innovation through open access to
Smithsonian research, collections and
communities.

28. http://smithsonian-webstrategy.wikispaces.com/
Balancing autonomy and control within the
Smithsonian.rt of our digital presence
dedicated to stimulating learning, creation,
and innovation through open access to
Smithsonian research, collections and
communities.
Innovation at the Edges
---
A commons in the middle

29. http://smithsonian-webstrategy.wikispaces.com/
…and the creation of a Smithsonian
Commons—a new part of our digital presence
dedicated to stimulating learning, creation,
and innovation through open access to
Smithsonian research, collections and
communities.

30. http://smithsonian-webstrategy.wikispaces.com/
…and the creation of a Smithsonian
Commons—a new part of our digital presence
dedicated to stimulating learning, creation,
and innovation through open access to
Smithsonian research, collections and
communities.

31. What is a Commons?

32. What is a Commons?
A set of resources maintained in the public
sphere for the use and benefit of everyone

33. What is a Commons?

34. What is a Commons?
The Anti-Commons…

35. What is a Commons?
An organized workshop where raw materials can be
found and assembled into new things.

36. What is a Commons?
Imagining a Smithsonian Commons
• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version
Museum Commons: A professional interaction
http://www.slideshare.net/edsonm/museum-commons-a-professional-
interaction-museums-and-the-web-2010-michael-edson-and-rich-cherry
What is a Commons?

37. What is a Commons?
Imagining a Smithsonian Commons
• http://www.slideshare.net/edsonm/cil-2009-michael-edson-text-version
Museum Commons: A professional interaction
http://www.slideshare.net/edsonm/museum-commons-a-professional-
interaction-museums-and-the-web-2010-michael-edson-and-rich-cherry
What is a Commons?
Also…
A 15-minute talk at the Walker Art Center
Text/footnotes: http://www.slideshare.net/edsonm/m-4402558
Video (starts at minute 12):
http://channel.walkerart.org/play/opening-the-field/
Updated 6/21/2010

38. So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you talk about
these things in
measurable business
terms?

39. So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you use
“information” to
persuade, guide,
inform?

40. So now we’re at a moment…
The Grand Challenges
The Smithsonian Commons
How do you measure
things that are… difficult
to measure?

41. Jedi Mind Tricks for
Optimizing
Lofty Goals
andMeasuring

42. Jedi Mind Trick #1
Show your users to management

43. Relevance
http://www.youtube.com/watch?v=N5x4Sga0d1s
Q: Have you ever visited a Smithsonian Web site?

44. Jedi Mind Trick #2
Use Organic Search and Alexa

45. Relevance
Unexpected Rivals in Google Search
Google Images
Wikipedia
Ocean.com
Discoveryeducation.com
NASA
Enchantedlearning.com

46. Relevance
Unexpected Rivals in Reach
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com
Google Images
Wikipedia
Ocean.com
…so much more reach than SI
that we don’t even show up on the graph…

47. Relevance
Unexpected Rivals in Reach (July – Sept, 2009)
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com

48. Relevance
Unexpected Rivals in Reach (July – Sept, 2009)
Enchantedlearning.com
si.edu
discoveryeducation.com
ocean.com
Enchantedlearning.com
is a two person team,
with more online reach
than the world’s largest
museum and research
complex!

49. Relevance
Traffic Trending Down
si.edu – 4% reach
Wikipedia.org + 8% reach
MoMA.org + 12% reach
npr.org + 20% reach

50. Jedi Mind Trick #3
Humble yourself with
brandtags.net

51. Relevance
Brand Identity
Brandtags.net
We are the 560th of 928 brands

52. Jedi Mind Trick #4
Highlight your competitors

53. Relevance
We’re competing with… everybody!

54. Relevance
We’re competing with… everybody!

55. Relevance
We’re competing with… everybody!

56. Jedi Mind Trick #5
Use 3rd party research

57. Relevance
The Demographic Tsunami
0
10
20
30
40
50
60
70
80
90
Ages 12-
17
Ages 18-
29
Ages 30-
38
Ages 39-
48
Ages 49-
60
Ages 61-
69
Ages 70+
Percentage
Online Content Creation by Age
Internet users
November 2007 data: Pew Internet and American Life Project

58. Relevance
The Demographic Tsunami
“Everything we hear from people we interview is
that today’s consumers draw no distinctions
between an organization’s Web site and their
traditional bricks-and-mortar presence: both
must be excellent for either to be excellent.”
Lee Rainie
Pew Internet and American Life Project

59. Jedi Mind Trick #6
Demonstrable feedback from users

62. 1,144 positive votes/comments on prototype
(out of 1,171 total votes)

63. 1,144 positive votes/comments on prototype
(out of 1,171 total votes)
It’s just Google
Spreadsheet hooked up
to a form.

66. Jedi Mind Trick #7
Show validation from VIP’s

71. Jedi Mind Trick #8
Measure end-user satisfaction

75. Jedi Mind Trick #9
Bake measurement into the
platform from the start

76. Jedi Mind Trick #9
Bake measurement into the
platform from the start
Smithsonian Office of the CIO
is starting to study the
as-is measurement/analysis
program

77. YES
NO
17%
83%
77
Survey Highlights
Which web analytics tools are you using?
What are the some
of the most
valuable behaviors
a user can perform
on your site?
10%
29%
61%
Are there web-user activities NOT being measured
that you WANT to measure?

78. http://smithsonian-webstrategy.wikispaces.com/
The importance of
measurement/analysis
and UX is laced
throughout the strategy

79. Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter

80. Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
Measurement and
analysis resources in the
project plan from the
very start

81. Project Charter (draft)
http://smithsonian-webstrategy.wikispaces.com/Smithsonian+Commons+Project+Charter
Measurement and
analysis resources in the
project plan from the
very start
Eight measurable goals
described

82. Project Charter (draft)
This is what the charter says…

83. Project Charter (draft)
3. Measurable Project Goals
The goal of the Smithsonian Commons project is
to stimulate learning, creation, and innovation
through open access to Smithsonian research,
collections and communities.

84. Project Charter (draft)
3. Measurable Project Goals
Precise, measurable goals are a hallmark of
successful project governance, but these kinds
of outcomes are notoriously hard to define and
measure in practice: seeking to define them too
precisely or in too nuanced a way early in the
project can be as counterproductive as not
measuring them at all.

85. Project Charter (draft)
3. Measurable Project Goals
Precise, measurable goals are a hallmark of
successful project governance, but these kinds
of outcomes are notoriously hard to define and
measure in practice: seeking to define them too
precisely or in too nuanced a way early in the
project can be as counterproductive as not
measuring them at all.

86. Project Charter (draft)
3. Measurable Project Goals
Therefore, the project will initially focus on
three measurements: the amount of
information in the Smithsonian Commons, the
amount of activity on the Smithsonian
Commons, and the satisfaction of people who
use the Smithsonian Commons.

87. Project Charter (draft)
3. Measurable Project Goals
Therefore, the project will initially focus on
three measurements: the amount of
information in the Smithsonian Commons, the
amount of activity on the Smithsonian
Commons, and the satisfaction of people who
use the Smithsonian Commons.

88. Project Charter (draft)
3. Measurable Project Goals
Therefore, the project will initially focus on
three measurements: the amount of
information in the Smithsonian Commons, the
amount of activity on the Smithsonian
Commons, and the satisfaction of people who
use the Smithsonian Commons.

89. Project Charter (draft)
3. Measurable Project Goals
Therefore, the project will initially focus on
three measurements: the amount of
information in the Smithsonian Commons, the
amount of activity on the Smithsonian
Commons, and the satisfaction of people who
use the Smithsonian Commons.

90. Project Charter (draft)
The specific, measurable 5-year goals
1) 100 million items in the commons…

91. Project Charter (draft)
The specific, measurable 5-year goals
1) 100 million items in the commons…
2) 100 million transactions a year…

92. Project Charter (draft)
The specific, measurable 5-year goals
1) 100 million items in the commons…
2) 100 million transactions a year…
3) High end-user satisfaction, both internally
and internally, as measured through the Foresee
Results measurement tool or equivalent

93. Project Charter (draft)
The specific, measurable 5-year goals
1) 100 million items in the commons…
2) 100 million transactions a year…
3) High end-user satisfaction, both internally
and internally, as measured through the Foresee
Results measurement tool or equivalent
“Bits in: bits out”
(via Brewster Kahle)

94. Project Charter (draft)
Five additional measurable goals guide the
project. The Smithsonian Commons will,
4) Increase reuse of Smithsonian resources
5) Be family-safe and school-safe
6) Demonstrate new revenue models
7) Drive traffic to unit websites
8) Help eliminate cost and work at the unit level

95. Jedi Mind Trick #10
Measurement is only one piece of
the equation

96. Jedi Mind Trick #10
Measurement is only one piece of
the equation
It’s about
understanding and
improving user
experience

97. Jedi Mind Trick #10
Measurement is only one piece of
the equation
You need a team

98. Smithsonian Commons Prototype
So let’s see what that’s like!

100. Thanks!!!
Michael Edson
Director, Web and New Media Strategy
Smithsonian Institution, Office of the CIO
edsonm@si.edu | @mpedson