As learning and development professionals, we have the power to change people’s lives. Each day our efforts drive toward developing our organizations’ people to perform to their full potential. Given this high calling, paired with current trends in business and industry, thoughtful and consistent measurement is a necessity. Effectively incorporating analytics into our decision-making processes not only increases our credibility with senior leaders but also enables us to continuously improve our efforts to develop our organization’s human capital. This insight is not new. L&D professionals have attempted to find the “right” way to measure their contributions for several decades. However, limited resources, lack of buy-in and uncertainty about the best way to start have prevented many organizations from acting on their best measurement intentions. Through the exploration of JetBlue’s learning analytics evolution, this webinar will explore how organizations can overcome these challenges via the use of a few key ingredients. Including these ingredients can help to maximize the value of limited resources, while failing to consider them can lead to wasted time and money. We will discuss how over the past 10 years, JetBlue’s learning analytics team has gone from a team of order-taking analysts working with limited “smile-sheet” data to a team of internal consultants and capability-builders utilizing multiple sources of data and advanced analyses, driving toward predictive and prescriptive analytics. Finally, we will discuss how organizations at any point on their learning analytics journey can use similar key ingredients to accelerate their progress toward effective data-driven decision-making. Key takeaways: How to use targeted involvement to generate buy-in and drive action from data. How to build the capability and culture necessary for effective data-driven decision-making. How to make due with limited resources and get the most use out of minimal data. How to accelerate your organization’s learning analytics strategies to drive better business decision-making.

1. #CLOwebinar
Sponsored by
The presentation will begin at the top of the hour.
A dial in number will not be provided.
Listen to today’s webinar using your computer’s
speakers or headphones.
Welcome to the webinar!

2. #CLOwebinar
Sponsored by
Speaker: Teri Schmidt
Manager-Assessment, Measurement, & Evaluation (AME)
JetBlue Airways
Moderator: Sarah Sipek
Associate Editor
Chief Learning Officer magazine
The Evolution of Learning Analytics at JetBlue:
Key Ingredients for Business Impact

3. #CLOwebinar
Sponsored by
Tools You Can Use
Audio Control
– A dial in number will not be provided.
– Adjust the volume by sliding the
indicator in the Media Player box to the
right.
– Also check your computer’s volume for
external speakers or headsets.

4. #CLOwebinar
Sponsored by
Tools You Can Use
Speaker Bios
Resources
Media Player
Q&A
Slides
Twitter
Help
Post Event
EvaluationShare This
Group Chat
HRCI
CLO Webinars

5. #CLOwebinar
Sponsored by
Tools You Can Use
Q&A and Live Group Chat
– Enter your first name and initial
and join the live discussion with
other webinar attendees
– Enter technical or content-
related questions into the Q&A
box
– Type your question in the space
at the bottom.
– Click “Submit.”
Q&A
Group Chat

6. #CLOwebinar
Sponsored by
Tools You Can Use
Twitter
- Click “Post” in the Twitter widget.
#CLOwebinar
@CLOmedia

7. #CLOwebinar
Sponsored by
1. May I receive a copy of the slides?
YES! Click on the resource list located on the top left portion of your
screen.
2. May I review the webinar recording at a later date?
YES! You may log in again using today’s link to review the presentation
on-demand.
3. Is this webinar HRCI certified?
YES! The HRCI certification code will appear in the box to the right of the
slides after the required watch-time has elapsed.
Frequently Asked Questions

8. #CLOwebinar
Sponsored by
Sarah Sipek
Associate Editor
Chief Learning Officer magazine
The Evolution of Learning Analytics at JetBlue: Key
Ingredients for Business Impact

9. #CLOwebinar
Sponsored by
Teri Schmidt
Manager-Assessment, Measurement, &
Evaluation (AME)
JetBlue Airways
The Evolution of Learning Analytics at JetBlue: Key
Ingredients for Business Impact

10. The Evolution of Learning
Analytics at JetBlue:
Key Ingredients for Business Impact
TERI SCHMIDT | MANAGER, AME

11. 11
Agenda
§ Data Invasion
§ In Your Organization
§ In Human Resources
§ About JetBlue
§ Key Ingredients and JetBlue's Journey
ü Common Metrics
ü Collaborative Data Reviews
ü Capability Building
§ Q &A

12. 12
The White House Names Dr. DJ Patil as the
First U.S. Chief Data Scientist
Big data: The next frontier for competition

13. 13
POLL
Select an option below to fill in the blank.
§ The focus on using data for decision-making in my
organization had ________________ in the past five
years.
a. Decreased significantly
b. Decreased slightly
c. Remained steady
d. Increased slightly
e. Increased significantly

14. 14

15. 15
“It’s just so important. Data today,
information today, is so prevalent. How
do we use it to improve the customer
experience and also to improve our
operational performance…”

16. ABOUT US

17. 17
POLL
What is JetBlue’s vision?
A. Get you where you need to be
B. Give ordinary people the opportunity to fly
C. Inspire humanity
D. Make flying fun

18. SECTION BREAK
30 million+ Customers
16,000+ Crewmembers
87 cities
825 daily flights
6 Focus Cities
3 types of aircraft:
- A321, A320, E190
3 Support Centers:
- Long Island City
- Orlando
- Salt Lake City
INSPIRE HUMANITY

19. 19
Preparing Crewmembers for exceptional career success at JetBlue

20. 20
Learning Shared Services
Admin/Logistics, AME, Learning Design, Learning Systems, Lodge, Uniforms
EVP Operations
College
of
Flight
College
of
Tech
Ops
SVP Safety, Security, & Training
Learning
Systems
College
of
Inflight
College
of
Sys Ops
Learning
Design
College
of
Airports
College
of
Customer
Support
Learning
Support
(Admin &
Logistics,
Uniforms,
Lodge)
Learning
Analytics
&
Strategic
Evaluation
Director
Technical Training
Director
Inflight & Sys Ops Training
Director
Customer Service Training
Manager
AME

21. AME’s
Contribu/on
Assessment, Measurement, and
Evaluation (AME)
Supporting leaders in making evidence-based decisions

22. 22
22
POLL
Who handles learning analytics in your
organizations?
A. No one
B. Courseware developers and/or instructors
C. A dedicated team or individual
D. Someone outside of the training department
E. Other

23. KEY INGREDIENTS

24. 24
Key Ingredients
1
2
3
Common Metrics
Collaborative Data
Reviews
Capability Building

25. 25
2005
2006
2007
2009
• Standard Level 1
• New technology
• Workload increases
• Pair of Hands to Consulting
• “Franchising Model”
• Centers of Excellence
• AME
• Defined influential partners
• MTM
• College-specific
evaluation strategies
• Business Alignment
Process
2010
2008
• Data Into
Action
Process
• AME
Roundtable
• College-
specific
annual
reviews
2011
• Mid-year
action reviews
• Data
visualization
• Crewmember
Experience &
Impact Team
formed • AME
Strategy
• Executive
Reporting
2012
• Strategic
Evaluation
• PMO
• Evaluate
Evaluation
2013
AME History
2014
• Learning
Analytics
• Strategic
Evaluation &
Education
1
3
2
3
+

26. 26
#1 Common Metrics

27. JBU Data Type Use
Recommendations
Supported
Decision(s)
- Learning experience improvements (ICE)
- Targeted curriculum improvements
- Policy, process, and/or procedure changes
- Training time re-distribution
- Assessment (e.g., test, checklist.) changes
- Revisions to objectives to better align with operation
- Work environment changes (tools,
expectations & feedback, leadership
support, rewards &recognition)
- Additions/
revisions to JBU
catalog of offerings
When to Use - Lower cost data
collection is needed
- As a data source
for cause analysis
into performance
and result data
- When the investment of resources required to collect, analyze, and
report on the data is justified (by criticality, visibility, and resources
investment)
- Monitoring:
Continuous
- Detailed analysis:
When justified (by
criticality, visibility,
and resources
investment)

28. 28
Data Inventory

29. 29
Research and Analysis Plans (RAPs)
Decision/Question Analysis Frequency/Date Courses Sources
Ongoing Monitoring
How satisfied are participants in JBU’s
training programs?
JBU Customer
Satisfaction Trend
Analysis
Monthly All that are in
MTM
MTM
What courses have the greatest
opportunities for improvement? What
courses are most successful? What
factors contribute?
JBU Customer
Satisfaction Trend
Analysis and comment
analysis
Monthly; Faculty
Meetings; Quarterly;
Annually
All that are in
MTM
MTM
Context-driven analyses
What impact does changing an individual
assessment to Train-to-Proficiency have
on crew assessment performance?
Chi-sq: pass rates pre &
post TTP
Q1 2015 Inflight
Qualification
Training
Inflight AQP
database
What are the drivers of attrition? Attrition by course,
instructor, day of class;
chi-sq: comparison of
attrition pre/post
revisions to assessments
Q1 2015 College of
Airports
Instructor-led
training
courses
PPDMS,
Attrition report,
Portal

30. 30
Strategic Evaluation

31. 31
#2: Collaborative Data Reviews
“Feedback (data) itself at best can only initiate
change; it cannot bring it about. People in groups
making use of feedback can create change.”
-Nadler, D. A. (1977). Feedback and organizational development: Using data-based methods. p. 144.

32. Participants
Department
Heads
Managers
AME
PRIORITIZE COMPARE EXPLAIN ACT
Monitoring of
college-
specific data
reports and
dashboards
Quarterly
College Data
Meetings
Quarterly JBU
Data
Actioning
Team
Quarterly
Department
Head Data
Meetings JBU Annual
Review
Meetings• Managing by
exception
• Deeper dives
as necessary
• Review of JBU
metrics
• Discussion to
action critical
risks and
successes
JBU’s Data Into Action Process
• Review of JBU
metrics and insights
gained
• Actions generated
and support needed
for action items
• AME shares insight
into the past
quarter’s data
• College leadership
and analysts share
insights, actions
taken, and potential
upcoming data-
influencers

33. 33
Data Review Meeting Agenda

34. 34
#3: Capability Building
“Organizations that want employees to
be more data oriented in their thinking
and decision making must train them to
know when to draw on data and how to
frame questions, build hypotheses,
conduct experiments, and interpret
results.”
Marchand, Donald A., & Peppard, Joe (2013). Why IT Fumbles Analytics. Harvard Business Review.
Retrieved from https://hbr.org/2013/01/why-it-fumbles-analytics

35. • Identifying & involving stakeholders
• Defining expectations
• Defining roles & responsibilities
• Managing risk (including potential
biases)
Manage
Stakeholder
Involvement
• Identifying research
questions/decisions
• Identifying measures
• Choosing methodologies
• Developing tools
• Administering data collection
tools for maximum participation
• Developing themes
• Conducting effective data
review sessions
• Developing actionable data
visualizations
• Defining the gap between
current & desired results
• Determining root causes
• Selecting appropriate
solutions
• Evaluating solutions
Plan for Data
Collection
Collect Data
Analyze,
Interpret, &
Communicate
Data
Use Data to
Improve
Performance
Intro
Continue

36. THANK YOU
Teri Schmidt
Teri.schmidt@jetblue.com

37. #CLOwebinar
Sponsored by
Please complete the
webinar evaluation.

38. #CLOwebinar
Sponsored by
Join our next Spotlight webinar!
Learning’s in BLOOM – How On-Demand
Video Learning’s Transforming Blue Cross
Blue Shield of North Carolina
Tuesday, April 28, 2015
Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register for all upcoming Talent Management
Webinars at www.clomedia.com/webinars
OR click the icon on the widget bar!