How do you know if your learning program is really working? Proper course tracking and sharing of learning data can help organizations develop best practices for how organizations measure the impact of learning. When historically, tools and systems haven’t made it easy to access and correlate data in order to measure, finding the right combination of strategy and technology can help optimize learning results to increase performance and impact business outcomes across the entire organization. In this webinar you will learn about: Best practices for measuring and optimizing learning programs. Learning tools that improve workflow efficiency. Expanding L&D value across the enterprise to drive results.

1. #CLOwebinar
The presentation will begin at the top of the hour.
A dial in number will not be provided.
Listen to today’s webinar using your computer’s
speakers or headphones.
Welcome to the webinar!

2. #CLOwebinar
Tools You Can Use
Audio Control
– A dial in number will not be provided.
– The audio will stream through your
headphones or computer speakers.
– Also check your computer’s volume for
external speakers or headsets.

3. #CLOwebinar
Questions and Handouts
You can submit questions by clicking on
this icon here.
Download a PDF of the slides here.

4. #CLOwebinar
1. May I receive a copy of the slides?
YES! Click on the handouts located on the right portion of
your screen.
2. May I review the webinar recording at a later date?
YES! You may log in again using today’s link to review the
presentation on-demand.
3. Is this webinar HRCI or SHRM certified?
YES! The HRCI and SHRM certification codes will be sent in
the follow up email.
Frequently Asked Questions

5. 2019 PeopleFluent
Strategy + Technology:
A Winning Combination for Effective Learning
Nancy Dietz, MA.Ed., CPLP
Customer Success Manager
Sam Howes
Director, Mid Market Sales

6. • Describe Effective Learning
• How do we provide
effective Learning
• How do we capture data?
• What do we do with data?
Agenda

7. https://www.strategyex.com/about-strategy-execution/press-releases/2010/organizations-
fail-to-tie-learning-to-business-impact
https://info.udemy.com/rs/273-CKQ-053/images/State_of_the_ROI_of_Learning_report.pdf?
mkt_tok=eyJpIjoiWlRVNFpUaGhNV0k0TkRFdyIsInQiOiJxMHBXZ2V2UU9WQVdMYkVMKzV0UUd1TkwwcldMbE5jT
3doa1NuSTNKNEc0c0xZMkE0ckFPTU9sN3EwK2ZlQ3IrbHdUM2tFa2RYZERYQWtoR004U3Q5T3dtekVPQnpWUTd6
UE80QU5ySzUzXC9ZQVN2VXl1ZDRXVmNYTjkxT3lMeisifQ%3D%3D
Strategy + Technology
Effective Learning
• 2011 global survey:
• 50.7% did not measure business impact
• 49.3% did measure for business impact
• For the ones that measure
• Used anecdotal evidence
• No specific methodology
• 2018 Udemy for Business, “State of
the ROI of Learning Report”
• 44% rated 5 out of 5 (Very Well)
• 31% rated 4 out of 5
• 16% rated 3 out of 5
• 5% rated 2 out of 5
• 5% rated 1 out of 5 (Not Very Well)
• For the ones that measure
• Used Kirkpatrick Level 1

8. Strategy + Technology

9. • Kirkpatrick Partners: Modern application of the Kirkpatrick Model
• ATD’s Certified Professional in Learning and Performance (CPLP) certification
model includes Evaluating Learning Impact
• xAPI standards
• Analytic Software & Data Scientist
• Executive support to identify and apply information to improved business
success
Effective Learning: 2019
Strategy + Technology

10. Understand Business
Strategy
Understand
Business Goals
or Drivers
Reduce Expenses
Generate
Revenue
Build Employee
Engagement
Understand how Learning
will be linked and its
context
STRATEGY
How do we provide
effective learning?

11. Where do
we begin?
DESIGN WITH BUSINESS
GOAL/DRIVER
TRACK & EVALUATE
LEARNING:
FORMAL AND
INFORMAL
ANALYZE & SHARE
RESULTS
IDENTIFY & IMPLEMENT
INTERVENTION
REVIEW BUSINESS
GOAL/DRIVER
Strategy + Technology

12. Design: Popular Models
• ADDIE or ISD
• Agile Design
• Design Thinking
• System Thinking
• X Problem
Images from: http://www.nwlink.com/~donclark/design/design_models.html
Strategy + Technology

13. https://www.chieflearningofficer.com/2019/09/23/so-you-want-to-measure-impact-now-what/
Measurement Map
Strategy + Technology

14. Time and Resource Considerations
Strategy + Technology
● How critical is the initiative to the success of the organization?
● How frequently will the content, system, or process change?
● Are there legal concerns about measuring the impact of the training (compliance issues)?
● Are you measuring confidence, competence or both?
● Do you have the infrastructure to provide the necessary data for measurement analysis?
● What do you plan to do with the information? Are you REALLY going to use it?
● Will your organization's culture embrace the idea of any kind of measurement?
● What accountability is in place to ensure measurement and transfer of learning takes place?
ATD article, https://www.td.org/newsletters/atd-links/ten-ways-to-measure-learning-impact

15. Data
• Track Learning
• Formal and Informal Tools
Information
• Analyze
• “Make the data talk”
Knowledge
• Use Information
• Identify & Implement an intervention
Capture and Using Data
Strategy + Technology

16. COURSE
EVALUATIONS
360 FEEDBACK ASSESSMENTS OBSERVATIONS EXTERNAL
REPORTS
PROGRESS
REPORTS
Strategy + Technology
TECHNOLOGY
Track Learning: Formal Tools

17. Visual Learning Journey Gap Analysis for Current and Future Careers
Tracking Learning: Informal Tools
Strategy + Technology

18. Course Progress
Tracking Learning: Informal Tools
Strategy + Technology

19. Strategy + Technology
Track
Learning:
Identify Gaps
Practice Workshops
and Skill Checks
Observation
Video scenarios

20. Strategy + Technology
Track
Learning:
Support
Materials
• Video on completing the annual performance form
• Success? Forms are completed accurately on first attempt.
• Reduced support from HR answering questions via phone or email
• Operating equipment
• Onboarding – set up standing desk; dependent tasks (sequencing)
• AR/VR are expensive, when instant mobile responsive video could
be used (manufacturing)
• Learners are videotaped, then upload as User Generated Content –
reviewable by Subject Matter Experts.
On demand “how to” guides, videos
• Can be done during performance appraisals
• Can be done as part of a course or post-work
Learners pledge to achieve X
• Review progress by manager or mentor
• Initiate required alterations
• Use LMS to initiate this follow-up review time – enroll in f/u class
(ILT or VILT) or just a task for learner and manager to meet
Follow-up in 30, 60, 90 days plus 6, 9, 12 months

21. Strategy + Technology
Track
Learning:
xAPI
Video Interaction
User Generated Content

22. • Aggregate
• Categorize
• Pivot Tables
• Graphical Displays
• Analytics
• Compare data
• Identify Trends
• Traditional embedded charts
(depending on your technology)
• Full package to generate more
complex comparisons
Strategy + Technology
Analyze and Share Results

23. Standard
Strategy + Technology

24. Strategy + Technology
Graphical Displays

25. Strategy + Technology
Share Results

26. Identify and Implement the Intervention
Strategy + Technology
Edit Learning Design
for activities or length
Create job aids
Modify policies and/or
procedures.
Modify workflows
Provide positive
reinforcement to encourage
progress

27. Service
• Increased Net
Promoter Scores
• HCAAPS scores
Safety
Reduced lost time
accidents
Production
Increased
throughput
Culture
Increased
requests for
learning
Whatever your
business goal
may be…
Strategy + Technology
Review Business Goal

28. Strategy + Technology
Recap
• Understand Business Industry
• Understand Business Goals
• Design as a Business Partner
• Incorporate measurement milestones
• Capture data
• Analyze & Share Information
• Review Business Goal for intervention
effectiveness
• Course Evaluations
• 360 Feedback
• Observations
• Assessments
• xAPI
• Excel with Pivot Tables
• Analytic software
Strategy Technology

29. Strategy + Technology
Measuring the business impact of learning:
What we’ve learned
December 12, 2019 | 2:00pm ET
TOMORROW – Register now!

30. For more information:
www.peoplefluent.com

31. #CLOwebinar
Register for the next webinar!
Measuring the Business Impact of
Learning: What We’ve Learned
Thursday, December 12, 2019
Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register for all upcoming Chief Learning Officer Webinars at
clomedia.com/webinars