Presenter: Tammi Marcoullier, Lead Strategist & Program Manager, GSA, Challenge.gov. PPT, 755 KB, 10 slides, January 2013

1. Metrics & Measuring for Success
Tammi Marcoullier
Lead Strategist & Program Manager
January 2013
Federal Challenge & Prize Competitions

2. 5/16/2014Office of Citizen Services and Innovative Technologies 2
“From about 100,000 to 50,000 years
ago, Homo sapiens looked pretty much as we
do today, and had ample brainpower…
Suddenly came an explosion of innovation—
finely hewn knives, carved figurines, hearths—
roughly coinciding with the arrival of language.
Which came first is a topic of debate, but
without question this was humankind’s great
“Aha!” moment, unleashing a wave of change
that led to Homo sapiens quickly becoming the
most widely dispersed species on earth.”
From Smithsonian Magazine, December 2012
Innovation

3. Why Reporting Metrics & Measures Matter
1. Storytelling lies at the heart of innovation. Fleeting
insights by one can be conveyed to another.
2. Cultural or technical innovation shares traits with
biological evolution: Good insights provide a
platform for ever-more-advanced ones.
~ from “What Technology Wants,” by Kevin Kelly
5/16/2014Office of Citizen Services and Innovative Technologies 3

4. Problem definition sets the framework for
what you’ll measure: People are
passionate about memes & missions –
provide an anchor point with meaning.
Parameters for success are the roadmap
for defining winning opportunities:
Contestants may innovate, design and
deliver beyond your expectations.
Share implementation and use plans with
contestants: This can motivate contestants
to enter and help stimulate a market or drive
innovation beyond government.
Share metrics at key intervals: Adjust in
process and report findings post-competition.
Mission Alignment
Agency
Challenge
Competition
Clearly Define the Problem
Focus on a problem that aligns with your
agency mission.
Articulate What it Takes to Win
Be brief, be clear, write in plain language.
Plan for Implementation
Outline options and get buy-in for what will be
done with the winning solutions.
Measure Short- & Long-Term Impact
List what you’ll track and measure during and
after the competition.

5.  Number of contestants
 Number of qualified entries
 Calculate internal cost savings
 Calculate contestant hours on
project
 Can solution be implemented?
 Did you engage entrepreneurs
and small businesses?
 Number of experienced
solvers.
 Number of new solvers.
 Measure interactions in
development communities
(github, data.gov, developer
boards, etc.)
 Tech list +
 Did solvers ask for
additional materials/data
sets to do more?
 Can you use more than one
solution or a combination?
 Did solution bring more
than what competition
required?
 Is there demand for
solution beyond agency &
in public?
 Solvers beyond “the usual
subjects?” Define what
new communities were
tapped.
 Accelerated time to
solution
Technology
Solution
InnovationAwareness
 Number of followers
 Number contestants/ entries
 Press attention
 Reach new audience or
demographic
 Increase in traffic to your
agency website
 Increase in transactions or use
of agency materials/data/etc.
 Measure social media
amplification
 Did audience, public come to
agency for 2nd or 3rd level
interaction?
 Impact 1 year later – where is
solution, has traffic increased
or dropped, etc.
Measurements & Indicators

6. 5/16/2014Office of Citizen Services and Innovative Technologies 6
Accelerate time to solution with
crowdsourcing
100 contestants x 10 hours per week x 6 weeks =
6,000 hours invested in solution
Equal to 150 weeks of 1 FTE in agency
or
Contracted talent at $TK x number of hours

7. Measurements on Challenge.gov
• Number of followers
• Number of entries
• Public voting
• Number of discussion topics &
participants
• Pilot Option: Google analytics to get
detail about your challenge.gov page
metrics.
U.S. General Services Administration
Office of Citizen Services and Innovative Technologies
7

8. When you measure & tell your story, consider:
• Internal change: people, participation, or processes
• Building sustainable and long-term communities
• Demand for agency data or programs continues
• Developing new partnerships
• New ideas & innovation integrated in agency
5/16/2014Office of Citizen Services and Innovative Technologies 8

9. When you measure & tell your story, consider:
• Repeat competitors and new competitors
• Recognizing evangelists in general public and business
• No solution found? See upside in cost & time savings
• Building on successful challenge to launch multi-agency
initiatives, Grand Challenges, etc.
• Assuring that competitions are integrated in agency
procurement toolkit
5/16/2014Office of Citizen Services and Innovative Technologies 9

10. Tammi.Marcoullier@gsa.gov and Karen.Trebon@gsa.gov
@ChallengeGov | facebook.com/ChallengeGov