SpigitEngage is a Crowd innovation Software as a Service technology platform used by large organizations including many of the Fortune 100 to tap into their "crowd" of employees, customers and partners to generate a pipeline of innovation for the business. Predictions is a new module that allows you to use the crowd to predict the value of ideas - how much revenue will it generate, how much will it cost, what's the likely time to market. Using a diverse crowd of people in your business typically yields better results than asking just the business analysts or project managers in your business. This slideshare discusses the Predictions functionality and the science behind it.
1. v
Welcome to the Innovation Café
Today’s Presenters: Milind Pansare, Kate Bennet and Dr. Anna Gordon, Mindjet
2. Milind Pansare, Vice President Marketing
Milind is Vice President of Marketing at Mindjet. He has held product engineering and marketing leadership roles for over 25
years at Silicon Valley startups and large tech companies such as Sun Microsystems (now part of Oracle), HP and Saba. Milind
is a product expert on social collaboration and innovation management technologies. He has served as an advisor to several
startups and startup incubators and holds a degree in Computer Science.
Kate Bennet, Director of Product
Kate Bennet is responsible for the design and delivery of Mindjet’s enterprise platform SpigitEngage and for researching and
scoping the next generation of crowdsourced innovation. Kate has worked in the innovation space in both the public and private
sectors, and has experience implementing, advising on and supplying innovation management software. Prior to Mindjet Kate
was Innovation Manager for the UK government and a Director and Trustee at Warwick University.
Dr. Anna Gordon, Data Scientist
Anna Gordon analyzes data across all departments – working on a variety of projects from measuring and optimizing marketing
and sales efforts to developing algorithms and data-driven features for SpigitEngage. Anna’s team creates approaches to study
and adapt innovation networks to generate the best outcomes through modeling and measuring engagement, understanding
how cooperation evolves in the network, and how trust behaviors affect information flows.
About our speakers
3. MINDJET CONFIDENTIAL
ABOUT MINDJET
Mindjet enables organizations to manage a pipeline of ideas
to drive new business strategies, product development,
operational efficiencies, customer insight and employee
engagement.
3
4. MINDJET CONFIDENTIAL
REDUCE COSTS INCREASE PRODUCTIVITY ACCELERATE GROWTH
DRIVING BUSINESS IMPACT
Proven & Quantifiable Impact on ROI
4
5. MINDJET CONFIDENTIAL
CROWD INNOVATION IS FOR EVERY PART OF THE BUSINESS
5
INNOVATION CLOUD
Strategy
Business Model, Revenue
Streams
Marketing
Consumer Insights, Brand
Engagement
Product Development
New Products, Time to Market
HR
Employee Engagement,
Innovation Culture
Market Innovation
Customer Insights
Offer Innovation
Products & Services
Process Innovation
Cost Reduction
Employee Engagement
Culture of Innovation
6. MINDJET CONFIDENTIAL
INNOVATION IS A CONTINUOUS PROCESS
Innovation must be managed just like any other important
process. An accelerated pipeline of the best, actionable ideas
that are vetted and ready for development and
implementation.
Idea Submission Idea Vetting Idea Feasibility Idea Implementation
6
7. MINDJET CONFIDENTIAL
UNIQUE ADVANTAGE OF MINDJET OUR KEY DIFFERENTIATORS
SINGLE PLATFORM AND BEST PRACTICES
Crowd Science + Behavioral Dynamics + Predictive Analytics
Manage
(Select & Align)
Outcomes
(Execute & Repeat)
Engage
(Source & Refine)
7
8. MINDJET CONFIDENTIAL
UNIQUE ADVANTAGE OF MINDJET
• Purpose-driven: Drive real outcomes – new products & services,
customer experience, operational efficiency & agility
• Social: Game mechanics for viral adoption & ongoing participation
• Mobile: Any device, anytime, anywhere, anyone
• Fun: Friendly, intuitive, user experience
• Culture: Make innovation a daily habit for the entire enterprise
Engage
Crowd Dynamics
8
9. CREATE A CULTURE OF INNOVATION – SUSTAINED ENGAGEMENT
Anywhere, anytime,
any device
Engage at scale, ensure
diversity of opinions and
aggregation of
knowledge
Solve specific
organizational
challenges on a
timeline
10. MINDJET CONFIDENTIAL
UNIQUE ADVANTAGE OF MINDJET
• Crowd science: Industry-leading algorithms to find the best ideas
• Velocity: Speed decision-making & vet ideas faster with
Automated Graduation & unique Pairwise Voting
• Optimization: Pattern recognition & machine learning for
continuously improvement based on past results
• Unbiased: Eliminates mob behavior to get true outcomes:
prevents gaming
• Nurture: Continuously surface & refine ideas for ongoing
innovation
Manage
Crowd Science
10
11. FIND BEST IDEAS– CROWDSCIENCE
Define idea stages and automated
graduation criteria unique for your
organization. Use the crowd to do
heavy lifting in initial stages. Apply
built-in workflow engine for expert
evaluations and vetting in later
stages.
Ensure all ideas are
carefully considered and
the best ideas are
surfaced.
12. MINDJET CONFIDENTIAL
UNIQUE ADVANTAGE OF MINDJET
• Repeatable: Institutionalized innovation process; proactively manage
and measure a pipeline
• Predictable: Validate the cost, feasibility, time, connection to strategy,
value of ideas with unique crowd Predictions and expert evaluations
• Measureable: Data visualizations and Analysis for measurement of
benchmarks and effectiveness
Outcomes
Crowd Analytics
12
13. DRIVE BEST OUTCOMES– REPEATABILITY & PREDICTABILITY
Create expert workflows
matched to your
business process
Measure innovation program
health with the help of business
analytics, dashboards and
reports
Quantify your innovation pipeline
with crowd predictions of cost,
time and revenue.
14. SpigitEngage Predictions
A new premium module for SpigitEngage that enables customers to:
Leverage the crowd to quantify the value of ideas in the innovation pipeline.
15. Predictions: Business Benefits
BUSINESS BENEFITS
Use the crowd to forecast revenue, time to market
and implementation costs.
Make better decisions with crowd predicted
quantitative data on each idea, in addition to experts.
Predict the ROI of the Innovation Program.
INNOVATION PROGRAM CHALLENGES
“Good” ideas, but lack of quantitative
metrics.
Too many ideas, not enough experts to
analyze business value.
Hard to measure and predict success of
the program.
17. Sociological validation of the wisdom of the crowd
1987, Prof. Jack Treynor, University of Southern California:
850 jelly beans, 56 students. Group estimate was only 2.5% off.
2007, Michael Mauboussin, Columbia Business School:
1,116 jelly beans, 73 students. Average guess was only 3% off.
The collective knowledge of the crowd creates
better estimates.
Financial analysts have long studied the wisdom of the crowd and use this for quantitative models to drive
Investment Strategies:
18. There’s a long history of leveraging crowds to predict results
1968- Locating a lost submarine 1998- Who Wants To Be a Millionaire
Experts guess correctly 65% of the time.
Audience guesses correctly 91% of the time.
20 miles search area. Diverse experts submitted
guesses. Collective estimate was only 220yds off.
Crowd is historically more accurate than
experts with an error of less than 1.64%.
Election results
20. Essential qualities for accurate crowd prediction
Diversity of opinion – to get different information.
Independence from one another – so one strong leader doesn’t dominate.
Decentralization – so that errors are balanced by the others.
A method for aggregating opinions – so that all opinions are included in the
decisions.
1
2
3
4
33. Using Predictions for automated graduation
Best practice:
Use Predictions as a
graduation criterion.
“Top x” in Best Overall move
to the next stage.
34. Setting up Predictions
Configured for each challenge:
1. The roles that are able to cast prediction
votes.
2. The stage that an idea must be in to
have Predictions cast on it.
3. The Prediction questions that will be
shown.
4. The currency that will be used for the
Revenue and Cost questions.
35. Recap
Predictions is a time-boxed phase used within a challenge.
Diverse crowds make forecasts on revenue, cost and/ or time.
Results highlight low-hanging fruit vs the more disruptive investments.
Internationalized module – available with a wide choice of currencies and in 11
different languages.
1
2
3
4
37. Under the hood
Sociological experiments validate the necessity of Crowd Dynamics in the platform to
ensure unbiased predictions.
In order to make good decisions, Crowd Analytics in the platform enable you to
visualize the data meaningfully.
In addition to these, ensuring the validity of the data requires advanced patented
algorithms – Crowd Science.
38. Under the hood
The Science of Convergence
Much like Maps on your phone that leverage multiple geo-location signals.
Predictions creates convergence through many data points and use of crowd science algorithms.
39. Under the hood
When a large, diverse crowd is asked a numerical
question, their answers take the shape of a
probability distribution, with the majority of answers
centered around the middle.
In the Predictions algorithm, we draw random
samples from a distribution like the “bell curve”, and
update it as we receive information, or votes, from
the crowd.
If the majority of the crowd votes up, we move the
distribution to the right, if the crowd votes inward
toward a single point, we shrink the distribution, until
we hone in on the crowd’s estimate.
A Bell Curve: Common User Voting Behavior
52. Under the hood
A measurement of certainty based on
the extent to which the crowd agrees
is factored into the final computation.
High agreement happens when the
majority of the crowd vote in the same
direction, so the distribution is able to
shrink around the crowd’s estimate.
Low agreement occurs when people
vote in different, disagreeing
directions, so the distribution can
never converge.
Black = low agreement
Red = high agreement
Possible Distributions of Users’ Votes
53. Under the hood
To compare ideas with varying results and calculate the “best overall” ranking, we normalize
by using an equation approximately equal to
Each variable is weighted by it’s agreement score, so a high agreement score will leave the
final numbers as they are, whereas a low agreement score will pull down revenue, and
increase cost and time.
revenue-cost
time
.
54. Summary: Predictions
Real-world business application requires:
Crowd Dynamics to ensure unbiased outcomes;
Crowd Science to deliver patented algorithms for accurate data;
Crowd Analytics to meaningfully visualize big data and make better decisions.
Predictions is a new premium module for SpigitEngage that enables customers to quantify the innovation
pipeline.
Familiarize yourself with a tleast 1 story in each category by reading the use case in the Customer Case study book prior to delivering this presentation.
The Innovation Cloud will transform all aspects of your business – Strategy, Marketing, Product and People with the outcomes listed on the right hand side.
Much like CRM helps you manage your sales process, or HCM helps you manage your people processes, innovation should be managed with the same discipline in your business, to create a pipeline of innovation.
Mindjet is the Innovation Cloud company. The Mindjet Innovation Cloud is a new way to transform and manage innovation as an essential driver of value, competitiveness and outperformance across the company by delivering an accelerated pipeline of the best, most actionable ideas.
Why is Mindjet different?
Mindjet’s Innovation Cloud is built around contemporary technology that leverages game mechanics to ENGAGE people, inside and outside the company to MANAGE the process of finding the best IDEAS- leveraging deep computational algorithms to discover the best ideas, and then vet and operationalize a continuous pipeline and process to turn ideas into OUTCOMES – in order to apply innovation across the organization as a new source of value, competitiveness, operational efficiency and out-performance.
How do we ENGAGE?
Mindjet Innovation platform drives new levels of engagement
The Mindjet Innovation Cloud was built to be apply innovation to real business issues including cost reduction, growth strategy, productivity and competitive concerns. The friendly, intuitive and highly relevant user experience enables anyone to participate from any device – anytime, anywhere. Challenge-based “game mechanics” that include reputation, recognition, rewards, notifications, leaderboards and social features, are designed to deliver the true engagement that makes innovation a daily habit.
Now a quick high level view of the capabilities that SpigitEngage provides in 3 slides:
First, SpigitEngage enables you to create sustained engagement.
Many of our customers recognize the need to change, the need to create a culture of innovation.
SpigitEngage provides them the capability to create crowd participation at scale
This is what will ultimately ensure a company has a large pipeline of ideas
Unlike first gen social software that enables just discussions and conversations, SpigiEngage enables employees to participate in the process of innovating – products and processes – in their organization that will create meaningful business outcomes. This “Closing of the loop” of participation with visible purposeful business outcomes is what leads to sustained levels of engagement.
Besides features like activity streams, trending ideas and other “social” features, SpigitEngage has the unique concept of Challenges.
Challenges are questions posed by business stakeholders to the “crowd” that licit participation to solve specific business problems, on a timeline.
Throughout the product, subtle but effiective social and game mechanics contribute to increased participation – reputation score, rewards and recognition, a reddit-like trending idea algorithm, idea trading, virtual currency, a store to redeem rewards…
And of course, access to challenges and the ability to contribute an idea, comment on one, vote etc can be done from any device at any time. (iOS and Android)
How does Mindjet surface the best IDEAS?
Innovation with Mindjet Innovation Cloud means the ability to find and surface ideas in the most appropriate way via a flexible, open platform.
Mindjet’s patented crowd science finds the best ideas by applying the deepest algorithms in the industry to ensure that the right ideas are surfaced quickly
Exclusive Mindjet features such as Automated Graduation, Pair-Wise Voting and Workflow for Pattern recognition underscore Mindjet’s scientific approach to making sure the best ideas are surfaced and vetted.
An impartial, data-driven approach obtains true outcomes by aggregating and analyzing data for the best results, while preventing the “idea gaming” and “herd behaviors” that can skew results.
In a future release, Mindjet is pushing the boundaries of innovation science –
enabling the continual nurturing and improvement of the best ideas to drive a new level of the best outcome.
Engagement needs to be followed by the creation of business value, i.e realy, implementable product ideas
This is where SpigitEngage can truly help organizations scalably separate idea signal from idea noise.
A couple quick examples:
Pairwise voting is a unique algorithm that provides a pair of ideas (Kind of like a hot or not?). Once you vote, it presents another pair and so on in rapid-fire succession.
On the surface, this is fun and game like, so people enjoy it.
But what’s happening beneath the hood is that the algorithms are ensuring that over time each idea is getting viewed equally by the crowd, so that good ideas don’t die on the vine, solving a major shortcoming of up/down voting mechanisms in social software.
Idea stages can be defined by customers that are unique to their process, and graduation parameters that define when an idea can move from, say, a submitted idea, to one under review for implemntation, can be set. These can be set as “crowd parameters” based on number of votes, views, discussions etc. or they can be combined with a built-in workflow engine to bring in experts who evaluate it objectively of are required to fill out specific templates such as business plans, legal review etc. This allows you to leverage the crowd to do the heavy lifting of spotting ideas in early phases of a challenge, and use the limited resources of your experts in later stages when the filed has been winnowed so to speak. As you might imagine, if you’re Citi and you get over 10,000 ideas at that scale from your employees on how to improve the private banking experience, (this is a true example, BTW) you need an efficient capability to surface the best ones to implement into projects.
These deep algorithms in crowd science , are unique technology differentiators of SpigitEngage.
How does Mindjet deliver Outcomes?
Mindjet Innovation Cloud create an institutionalized, repeatable innovation process. Mindjet enables every company to be in the innovation business, proactively managing a pipeline of ideas that deliver business value, while conforming to specific business processes that incorporate the right people, the right roles and the right challenge.
Crowd science validates specific ideas against key metrics, testing for connection to strategy, appropriate cost, feasibility and time to implement.
Executive reporting includes visual and social analytics, dashboards for benchmarks, effectiveness and measurement; Mindjet also enables executives and sponsors to monitor “project health,” gaining organizational insight and greater understanding about the success of a given idea.
Most of our customers need the capability to be able to support their innovation processes at scale, so that predictable outcomes can be derived across every department, every business unit, every geography.
The ability to set up an innovation program with specific templates (An Idea that’s worthy of submission can mean something different to the engineering dept at Daimler or Rolls Royce for example, where they may need something far more structured with say engineering details, than say retail employees at Estee Lauder)
The ability to setup workflows (so that the right expert can review, add, complement, improve, at the right time)
The ability to setup templates for bsuiness plans, legal review, design document…
Reporting (standard reports on engagement, participation, login) and custom reports scheduled to run at certain intervals on any aspect in the system using the embedded reporting engine, and multiple widgets to create sophisticated dashboards.
A dynamic visualization of the entire innovation pipeline, with ideas at every stage….\
To put Predictions in context it’s useful to start with a little history on the wisdom of crowds.
You will probably be familiar with the jelly bean experiment: if you fill a jar with jelly beans and ask a group of people to estimate the number of beans in the jar, the average of the group’s estimates will be very close to the actual number.
The first recorded instance of this was in 1987, when Jack Treynor asked 56 students to submit estimates on the number of beans in a jar. The group estimate was only 2.5% off of the actual number.
This has been repeated many times since. For example, in 2007 Michael Mauboussin ran the experiment with 73 students. Despite an average error of 700, the average guess was only 3% off.
-----------------------------------------------------------------------------------------------------------------------
*Jack Treynor is President of Treynor Capital Management, Inc. He was the editor of the Financial Analysts Journal for many years and is the author of more than ninety articles that have appeared in numerous financial publications. Treynor is a Distinguished Fellow for the Institute for Quantitative Research in Finance (the Q group). He serves on the advisory boards of the Financial Analysts Journal and the Journal of Investment Management, and has taught courses at Columbia University and USC. Treynor has served as a general partner, trustee, and director of seventy investment companies.
*Michael J. Mauboussin is a Managing Director and Head of Global Financial Strategies at Credit Suisse. He has also served as Chief Investment Strategist at Legg Mason Capital Management. He is the author of three books, including More Than You Know: Finding Financial Wisdom in Unconventional Places, named in the The 100 Best Business Books of All Time by 800-CEO-Read. Michael has been an adjunct professor of finance at Columbia Business School since 1993, and received the Dean's Award for Teaching Excellence in 2009.
There are other examples of this throughout the last 100 years.
For example, in 1906 Francis Galton was visiting a livestock fair and observed people guessing the weight of an ox. He averaged the 800 guesses and found that they were within 0.8% of the actual weight- closer than the individual cattle experts.
In 1968 a submarine called the Scorpion went missing. The search area was 20 miles wide and 1000s of miles deep. Chief Scientist John Craven came up with scenarios and asked a group of diverse experts to submit guesses on those scenarios. When the Navy found the submarine 5 months later it was only 220 yards from Craven’s estimate.
Final example- Who Wants To Be A Millionaire. Participants have a few help options. The “experts” on the phone are correct 65% of the time. The audience- who are not experts- are correct 91% of the time.
If you’re interested in reading more on this then I recommend the book “The Wisdom of Crowds.”
We can see that there’s something very interesting going on here when your crowd has particular qualities.
Can see in this graph that predictions are very close to the actual result over a number of years of presidential elections. The average prediction market error is only 1.49%.
Studies such as these present very strong evidence of crowds prevailing- if four criteria are in place: Diversity (can’t just use experts), Independence, Decentralization, and having a Method of aggregating opinions.
A “Good Crowd” has these characteristics and will result in more accurate predictions.
The Crowd Dynamics in SpigitEngage help correct for the sociological conditions inherent in unmanaged crowd behavior
Once the predictions phase ends, the system generated visual analytics that are available to all participants.
Note: The bubble graph will not be displayed on older browsers such as IE7 and IE8.
Note: The bubble graph will not be displayed on older browsers such as IE7 and IE8.
The larger the crowd, the more accurate the result.
Predictions works best with 50+ users.
Predictions uses the diversity in perspectives to increase accuracy.
Just as your maps app locates you on your phone, first by getting cell towers, next signals from known WiFi spots, and so on until the blue dot gets tighter, so also we use convergence algorithms based on similar principles, but using data from the crowd of people rather than cell towers.
Low agreement could still be of value to the decision maker!