Brian Fitzgerald came to Knewton with 15 years of experience developing digital media consumer products and services. Most recently, he served as Vice President, Education for Audible.com (an Amazon.com company), where, among other initiatives, he created AudibleKids and VangoNotes. Prior to joining Audible, Fitzgerald worked at Yahoo as an original member of the Yahoo Mail team. He spent 8 years at Yahoo in product management roles in Silicon Valley and Europe, building and managing more than 50 Yahoo products such as Yahoo Auctions, Yahoo Messenger, and Yahoo Small Business. He served on the Board of Directors of the Just Think Foundation. He received an M.A. in Instructional Technology and Media from Columbia University’s Teachers College and a B.S. in Entrepreneurship from Babson College. Knewton (www.knewton.com) is the world’s leading adaptive learning company. Knewton’s cloud-based platform uses concept-level data to create uniquely personalized learning plans, allowing educators to tailor their content to the exact needs of individual students. The company was named a 2011 Technology Pioneer by the World Economic Forum at Davos. Educational technology has recently become an extremely hot area. Though in the early days, Knewton was perceived as building a better mousetrap for test prep and facing off against companies like Kaplan (Knewton CEO Jose Ferreira is an alum), the underlying strategy is far more ambitious. Knewton’s team is an A-list of hardcore techies building a scalable platform, using data science to disrupt an enormous industry, and hoping to bring wide-scale improvement to how anyone in the world learns just about anything. Join us at ProductCampNYC 2011 for Brian’s fascinating talk about the challenges and opportunities facing this exciting venture-funded company and its product team.

1. Product management…
an evolution.
Brian Fitzgerald
VP, Product Development
September 17, 2011
Product Camp NYC, 2011
My last 16 years!
Learning
!"
Paying my bills
Product Camp NYC, 2011

2. I like to make things!
#"
Product Camp NYC, 2011
Established 2008
Product Camp NYC, 2011

3. Long on vision!
To be the data platform
powering the world’s
education.
$"
Product Camp NYC, 2011
Decent sized industry!
%"
Product Camp NYC, 2011

4. Knewton: B.P.M.
(before product management)
Product Camp NYC, 2011
The founder
• Former Kaplan exec
• “Change agent” for the Kaplan test prep
product
• Brut-force adaptive learning - early 90’s
• Slightly rebellious…“Broke the code” for a
certain standardized testing question type,
forcing ETS to pull revise their tests;
earned the name “the antichrist.”
• Knewton was percolating
in his head for ~15 years…
&"
Product Camp NYC, 2011

5. A legend among legends!
Meeting #1…
Meeting #2…
'"
Product Camp NYC, 2011
A team who just might be able to pull it off!
• Young
• Exceptionally bright
• Hungry
• Tireless
()"
Product Camp NYC, 2011

6. Start up style
• Product was purely CEO’s vision
• A willing team building on demand
• Real time cycles
• No direct customer input/revisions for v1
(("
Product Camp NYC, 2011
The build goal
• v1 of the adaptive learning platform
• Live streaming classroom
• On demand student experience
• Develop all of the content
• A content management system
• Pre-market
• Launch a GMAT course
• …in 6 months!
(!"
Product Camp NYC, 2011

7. Knewton: A.P.M.
(after product management)
Product Camp NYC, 2011
My arrival
• April, 2009
• 1st product person
• 740 feature ideas in the hopper
(*"
Product Camp NYC, 2011

8. So!now what?
• CEO’s perception: “move faster, do more
things!”
• CTO’s desire: “need a buffer, let’s move slower
so I can scale this”
• New head of research: “we’re almost at rung
#1 on a very tall adaptive learning ladder”
• Academic team: “finally, we can fix course
development tools and broken student features”
• Me: “focus, brian, focus…”
($"
Product Camp NYC, 2011
A healthy tension
The
disruptive
visionary
Balance
The
Synthesis
The academic
consumer
app guy
Decisiveness pragmatist
The product
minded techy
(%"
Product Camp NYC, 2011

9. Align on goals
• Collective view of the landscape
• Agreement on the business strategy?
• Validate with customers
• What are we committing to for resources?
• What is core to our business?
(+"
Product Camp NYC, 2011
Business drivers
• Acquire new customers
• Generate meaningful data
• Engage existing customers
• Gain operational efficiency
(&"
Product Camp NYC, 2011

10. Introducing tools
• Long term roadmap
• Focus projects
• Roadmap review
• Lightweight
requirements/user
story templates,
backlogs,
retrospectives, 5-
whys
('"
Product Camp NYC, 2011
1 Pager
• 1 page pitch (“vision
and scope”)
• Anyone can do one
• Aligning with focus
projects for a better
chance at daylight
• ROI/metric driven is
essential for making
the business case
• Feedback is promised
!)"
Product Camp NYC, 2011

11. A Well-rounded product team
• Uncompromising on hiring
• Varied skillsets; cross-training
• Data driven
• Consistent coaching
• Ideas from everywhere
• Product managers spread the
culture throughout the org
!("
Product Camp NYC, 2011
UX!it matters (a lot)
!!"
Product Camp NYC, 2011

12. Building Product Culture
• Product before revenue
• Customer centric
• Creating space vs. churning features
• Iterative process – customer feedback
• Sketchstorming
• What is the 1 feature customers will
remember?
!#"
Product Camp NYC, 2011
Hack days
!*"
Product Camp NYC, 2011

13. A word about transparency!
• What are we working on, and why?
• What are we choosing not to pursue?
• Pre-release sharing
• Exposing decisions (teachable moments)
• Acknowledgement of success
• Brown bags
!$"
Product Camp NYC, 2011
Scaling Challenges
• Parallel tracking
• Product groups
• Staffing
• Knowledge transfer
!%"
Product Camp NYC, 2011

14. The F word (focus)
• It’s painful
• It’s especially hard once you’ve built a
participatory culture
• But, it matters more than anything
• Card sorting
!+"
Product Camp NYC, 2011
Product stages
Product Camp NYC, 2011

15. Pivot: Test Prep
• Proof of concept
• Direct to consumer
• Immediate revenue generation
!'"
Product Camp NYC, 2011
Pivot: Higher Education
• Math Readiness, Blended Learning
• Big data
• Institutional customers
• Use of API
#)"
Product Camp NYC, 2011

16. Pivot: Open Platform
!"#$%&"'%()%*+%&,%' -$-)"+.%'/%-*&+&0' -&-/1"+,!' -)+'
• Self Paced • Atomic-concept- • What students • RESTful
Environment based learning know interface
• Game-like flow objects • How students • Course creation
dynamics • # of practice learn best • Personalized
• Social learning items • Content efficacy recommendation
• Personalized • Difficulty level • Cohort analysis s
UX • Format & media • Analytics
• Time of day
• Bite size/
Frequency
• Retention
#("
Product Camp NYC, 2011
Which brings us to today…
Product Camp NYC, 2011

17. Knewton platform transforms education
Adaptive learning, adaptive tutoring, adaptive engagement unique lifelong learning profiles
Individual profile data Network Effect
2345673'
*4879943:;<73'
1000s data/day !=>649' Combined power of
millions of users
Learning path optimization Evolutionary algorithm
?@43')A;B7C9'
,-./0123/"45.6"78,9":;<5-=".;420>"?/@1A?/0</".;;@?".;"30/2./"B/0?;-2@5C/>"@/20-5-="
/DB/05/-3/?"
33
Product Camp NYC, 2011
Knewton Adaptation /"#$%&'()$001
/30032'56
• Knewton Knowledge Graph
• Proficiency Estimation /3003142-
/3003:5;
/"#$%&'()$002
• Activity Selection
/300375/
/30038
• Machine learning, data mining /30039
• Delivering optimized learning
content
!"#$%&'()$*+,-.
!3*3152
!3*3:
!3*3;57
Product Camp NYC, 2011

18. Knewton Knowledge Graph
!
Product Camp NYC, 2011
Big data meets the world’s biggest data industry
E-/4.;-".;>2J"
/75F94:GH9'>6;D4>':;6;' K?/0?L"65=6/?."?.2M/?">2.2"
())?"A"(9)))?";1">2.2"B/0"
LM>'7N':;6;'@7G36>'@4C'H>4C'@4C':;=' G?/0"B/0">2J"
E-/4.;-"5-"!)(#"
N2?32>/"/O/3.?P"/236"
+>7A;64:':;6;I'@77C'87CC4A;<73>I'AGJA4'6;KKG3K' >2.G:"5-?.2-.@J"B0;>G3/?"
(9)))?":;0/">2.2"Q>G/".;"
3;00/@2H;-?9"65/020365/?RS"
,::/-?/"34657CD'4E486>'
E-/4.;-"5?"FG5@>5-=".;420>"=/-/02H-="())9)))I""
:/2-5-=1G@">2.2"B;5-.?"B/0">2J"
#%"
Product Camp NYC, 2011

19. What does it mean for students?
#+"
Product Camp NYC, 2011
My most important lessons
• Refuse to start running without goals; alignment may
bring conflict, but don’t avoid it
• Transparency takes time, but matters – trains the org
• Process should be like pajamas – on, but not tight
• Committing to a customer-driven, iterative process is
not at odds with innovating
• Expect that building product for scale will lead you down
a different path
• Don’t underestimate people’s fear of shipping – but
make them do it
• Focus is really, really hard when you work with creative,
convincing people
#&"
Product Camp NYC, 2011

20. thank you.
Brian Fitzgerald
brian@knewton.com
19 Union Square West
New York, NY 10003
212-KNEWTON
Product Camp NYC, 2011