Training requires time, energy and money. Therefore an organization needs to know whether the investment made in training is being effectively and is worth the effort. As a part of lesson “Effective Hr training and Development Strategy”, brought to you by Welingkar’s Distance Learning Division.
For more such innovative content on management studies, join WeSchool PGDM-DLP Program: http://bit.ly/effectivehrtrainingslides
Join us on Facebook: http://www.facebook.com/welearnindia
Read our latest blog at: http://welearnindia.wordpress.com
Subscribe to our Slideshare Channel: http://www.slideshare.net/welingkarDLP
2. Welingkar’s Distance Learning Division
Introduction
Training requires time , energy and money. Therefore an
organization needs to know whether the investment
made in training is being effectively and is worth the
effort. Top management is concerned with evaluation as
a process by which the effectiveness of the
organizations programs and operating procedures are
demonstrated. Supervisors are concerned more with
the “results” of training as compared by changes in
workers on the job performance.
Contd…
We Learn – A Continuous Learning Forum
3. Welingkar’s Distance Learning Division
Evaluation is a planned process which provides specific
information about a selected session, program for the
purpose of determining value and or decision making.
Related to training evaluation is concerned with
providing information on the effectiveness of the
training activity to decision-makers who will make
decisions based on the information. There are various
models that have been developed to describe the role
of evaluation in the training process. It is important that
evaluation be a planned or systematic effort that is
built-in from the start of the training process.
We Learn – A Continuous Learning Forum
4. Welingkar’s Distance Learning Division
Ascertaining Reactions Of the Participants
As the programme gets over, it is important to ascertain the
participants views and observations about its various
components both in terms of the contents and the training
process. The evaluation can either focus on a session-by-
session review or on the programme as a whole. The formal
format that the trainers choose to adopt for eliciting end-
term appraisal of the participants’ ranges from the relatively
simple “highlight the best and worst aspects” to a detailed
response through a properly devised questionnaire or
instrument.
We Learn – A Continuous Learning Forum
5. Welingkar’s Distance Learning Division
Areas for Ascertaining Reactions of the Participants
An important decision that the training team takes prior
to the programme is about the specific elements it
would like to include in the evaluation exercise.
Course Objective
• How far the programme objectives are realistic.
• Clarity
• Individual understanding and response
• How far the training objectives are realized.
Contd.
We Learn – A Continuous Learning Forum
6. Welingkar’s Distance Learning Division
Trainers Performance
• Effectiveness of the presentations.
• Skill of the trainers to use the training methods
• Trainers expertise in responding to the questions of the
participants.
• Whether the trainers attitude was supportive and
helpful.
• Their relationship with the training group.
Training Methods
• Their appropriateness tot the programme and the
modules.
• The extent to which they facilitated learning. Contd.
We Learn – A Continuous Learning Forum
7. Welingkar’s Distance Learning Division
Training Group
• Its size and composition
• Selection procedure
• Level of the group and quality of its participation.
Time Schedule
• Duration of the programme
• Allocation of time to each module
• Was the daily schedule hectic.
• Sequencing the topics
• General flow and momentum of the programme
Contd.
We Learn – A Continuous Learning Forum
8. Welingkar’s Distance Learning Division
Training Facilities
• Suitability of the sitting arrangement
• General ambience
• Appropriateness and quality of the equipment used
• General learning environment at the venue.
Physical Arrangements
• Appropriateness of the accommodation
• Arrangement for food etc.
Training Support Materials
• Usefulness and quality of the materials
• Their timely distribution. Contd.
We Learn – A Continuous Learning Forum
9. Welingkar’s Distance Learning Division
Sources for Ascertaining Participants’ Reactions
• Structured Questionnaire
• Tests
• Open Forum
• Personal interviews
• Program committee meetings for ongoing evaluation
of the contents and process of the programme.
• Observations of the trainers
• Comments of the participation
Contd.
We Learn – A Continuous Learning Forum
10. Welingkar’s Distance Learning Division
Purposes and Uses of Evaluation
The definition of evaluation implies 2 purposes:
1. Making decisions about improvements to be
made in the training program itself.
2. Making decisions about the value of the
training program in terms of whether to
continue to conduct the program.
Contd.
We Learn – A Continuous Learning Forum
11. Welingkar’s Distance Learning Division
Specifically, evaluation can be used:
a) To determine whether the training program is accomplishing
its assigned objectives and if they were the “right”
objectives.
b) To Identify the strengths and weaknesses of training
activities.
c) To determine the cost/Benefit Ratio of the Training Program.
d) To Establish a Data base which Organization Leaders can use
to Demonstrate the Productivity and Efficiency of their
operational Procedures.
e) To Establish a Data Base which can assist Organization
Managers in Making Decision.
We Learn – A Continuous Learning Forum
12. Welingkar’s Distance Learning Division
Evaluation of Training
After the training program has been completed and the
current training record filled in, the performance of
trainees can be evaluated and the attention of the
training staff drawn towards specific action for
improvement. Thus evaluation of training should help
to:
• Isolate areas of difficulty and suggest strategies for
overcoming them.
• Modify unrealistic training targets in the light of
actual group performance Contd.
We Learn – A Continuous Learning Forum
13. Welingkar’s Distance Learning Division
• Determine whether motivation of trainees is of the
required level e.g. financial incentives for greater
output during training.
• Compare initial selection to actual performance
rating
• Highlight causes of absenteeism and labour turnover
during training
• Evaluate and modify instruction for future training
programme
• Formalized management training programme have a
vital role to play in the development of management
skills. Contd.
We Learn – A Continuous Learning Forum
14. Welingkar’s Distance Learning Division
Evaluation can be four-pronged attempt.
1. Evaluation of reaction
2. Evaluation learning
3. Evaluation behavior
4. Evaluation results
Implementation of Kirkpatrick-Phillips Model
Level 1 in which evaluation is used by most organizations through
questionnaires
Level 2 evaluations can be accomplished by written or oral tests
Level 3 is Application
Level 4 is tying training to organizational impact
Level 5 A new approach, measuring returns on investment
We Learn – A Continuous Learning Forum
15. Welingkar’s Distance Learning Division
Measuring Training Effectiveness and Impact
I. Prior to training
• The number of people that say they need it during the needs
assessment process.
• The number of people that sign up for it.
II. At the end of training
• The number of people that attend the session.
• The number of people that paid and attended.
• Customer satisfaction at the end of training
• A measurable change in knowledge
• Ability to solve a “mock” problem at the end of training
• Willingness to try to use the skill at the end of training. Contd.
We Learn – A Continuous Learning Forum
16. Welingkar’s Distance Learning Division
III. Delayed Impact (non-job)
• Customer satisfaction at X weeks after the end of
training
• Customer satisfaction at X weeks after the training when
customers know the actual costs of training
• Retention of knowledge at X weeks after the end of
training.
• Ability to solve a “mock” problem at X weeks after the
end of training.
• Willingness to try the skill at X weeks after the training.
• The three systems followed are: 1. 360 degree
2. Performance DNA system 3. Janus performance
Management System
We Learn – A Continuous Learning Forum
17. Welingkar’s Distance Learning Division
IV. On the job behavior change
• Trained individuals that self-report that they
changed their behavior , used the skill on the
job after the training.
• Trained individuals who’s managers report
that they changed their behavior or used the
skill after training.
• Trained individuals that actually are observed
to change their behavior or use the skill after
the training.
Contd.
We Learn – A Continuous Learning Forum
18. Welingkar’s Distance Learning Division
V. On the job performance change
• Trained individuals who self-report that their actual
job performance changed as a result of their
changed behaviour, skill.
• Trained individuals who’s manager’s report that their
actual job performance changed as a result of their
changed behaviour/ skill.
• Trained individuals whose manager’s report that
their job performance changed either through
improved performance appraisal scores or specific
notations about the training on the performance
appraisal forms. Contd.
We Learn – A Continuous Learning Forum
19. Welingkar’s Distance Learning Division
• Trained individuals that have observable /
measurable improvement in their actual job
performance as a result of their changed
behaviour.
• The performance of employees that are managed
by individuals that went through the training.
• Departmental performance in departments with
X % of employees that went through training ROI
of return on training dollar spent.
Contd.
We Learn – A Continuous Learning Forum
20. Welingkar’s Distance Learning Division
(a) Other measures
• CEO / top management knowledge of/ approval of/
or satisfaction with the training program.
• Rank of training seminar in forced ranking by
managers of what factors contributed most to
productivity/ profitability improvement.
• Number of referrals to the training by those who
have previously attended the training
• Additional number of people who were trained by
those who have previously attended the training.
• Popularity of the program compared to others.
Contd.
We Learn – A Continuous Learning Forum
21. Welingkar’s Distance Learning Division
(b) What to evaluate
One way to approach the issue of what to evaluate is to
identify what kind of information is needed. Dr.
Kirtpatrick developed four levels of evaluation. Reaction
1. Learning
2. Behaviour
3. Results
1. Reaction
Reaction is what a trainee thought of a particular
program. There are some guidelines for determining
participant reaction. Contd.
We Learn – A Continuous Learning Forum
22. Welingkar’s Distance Learning Division
1. Design a questionnaire based on information
obtained during the need assessment phase.
2. Design the instrument so that the response can be
tabulated and quantified.
3. To obtain more honest opinions provide for the
anonymity of the participants.
4. Provide space for opinions about items that are not
covered in the questionnaire.
5. Pretest the questionnaire n a sample of participants
to determine its completeness.
Contd.
We Learn – A Continuous Learning Forum
23. Welingkar’s Distance Learning Division
2. Learning
The trainer is concerned with measuring the
learning of principles, facts, techniques and
attitudes that were specified as training objectives.
There are many different measures of learning
performance including paper-and-pencil test,
learning curves etc. The objectives determine the
choice of the most appropriate measure. There are
several guideposts used in establishing a procedure
for measuring the amount of learning that takes
place. Contd.
We Learn – A Continuous Learning Forum
24. Welingkar’s Distance Learning Division
1. The learning of each participant should be measured
so that quantitative results can be determined.
2. A before-and-after approach should be use so that
learning can be related to the program.
3. Learning should be measured in an objective basis.
4. A group not receiving training should be compared
with the group receiving training.
5. Where possible the evaluation results should be
analyzed statistically.
Where principles and facts are taught rather than
skills, it is more difficult to evaluate learning. Contd.
We Learn – A Continuous Learning Forum
25. Welingkar’s Distance Learning Division
3. Behaviour
The term is used in reference to the measurement of
job performance. There are several guideposts in
evaluating training programs in terms of behaviour
changes.
1. A systematic appraisal should be made on-the-job
performance on a before-and-after basis.
2. The appraisal of performance should be made by
one or more of the following groups:
a. The person receiving the training. b. The persons
supervisor. c. The persons subordinates.
d. The persons peers. Contd.
We Learn – A Continuous Learning Forum
26. Welingkar’s Distance Learning Division
3. A statistical analysis should be made to compare
the performance before and after and to relate
changes to the training program.
4. The post training appraisal should be made three
months or more after the training so that
trainees have an opportunity to put into practice
what they have learnt.
5. A group not receiving the training should be used
for comparison.
Contd.
We Learn – A Continuous Learning Forum
27. Welingkar’s Distance Learning Division
4. Results
Evaluations in this level are used to relate the results
of the training program to organizational objectives.
Some of the results that could be examined are
costs, turnover, grievances and morale. Where the
objectives of the training program are closely tied to
specific organizational objectives effort should be
made to show a link between the training and the
changes called for in the organization objective.
We Learn – A Continuous Learning Forum
28. Welingkar’s Distance Learning Division
The process of Evaluation---Measuring Change
One way of visualizing a process of evaluation is to set the
four levels of evaluation against a backdrop of a training
program. Evaluation should start before the training
program is designed. The next step is to plan the evaluation,
how is the data to be gathered. Who will perform the
evaluation? What techniques will be used? When will the
evaluation be conducted? The evaluation design needs to be
used on the objectives of the program and the criteria to be
measured. Once the plan is completed and the training
program begins, the data collection phase is initiated.
We Learn – A Continuous Learning Forum
29. Welingkar’s Distance Learning Division
Validity and Reliability
Validity is a concept that means the degree to which an
evaluation technique or instrument measures what it
was intended to measure. Its a measure of accuracy.
Reliability is the degree to which evaluation techniques
and instruments measure a given characteristic
consistently. In order to get a long term view of a
training programs effectiveness, an instrument must
measure reliably for each session of the training
program.
We Learn – A Continuous Learning Forum
30. Welingkar’s Distance Learning Division
Types of Evaluation Techniques and Instruments
The process of Training Evaluation
A training programme, like design, development, and
manufacture of a product passes through several
stages.
1. Pre-Training Evaluation
This is prior to the course and should cover an analyses
of the expectations of the trainee and his superior
officer. The existing level of knowledge and skills are
reassessed. It is carried out through discussions,
workshops etc. Contd.
We Learn – A Continuous Learning Forum
31. Welingkar’s Distance Learning Division
2. Input and Delivery Evaluation
This is done concurrent to the training. It involves long
term management programme. It is done individually or
in group. Each topic, module is evaluated in terms of its
content, presentation, relevance and applicability. Using
a questionnaire, soon after the course, invariably on the
last day the participants reaction is obtained which
generally elicits information about the course inputs
and the impressions about the course.
Contd.
We Learn – A Continuous Learning Forum
32. Welingkar’s Distance Learning Division
3. Post Training Evaluation
The objectives of training is to enhance individual
effectiveness which helps in improving organizational
performance. Thus the process of post-evaluation must
have reference to job improvement plan. This cannot be
done when the person is training , he needs to go back
to his job and demonstrate his newly acquired
knowledge and skills. After a lapse of 6-12 months the
results obtained are used for evaluating the applicability
of training.
We Learn – A Continuous Learning Forum
33. Welingkar’s Distance Learning Division
Evaluation Of The Training Institutes
It is believed that organization climate is known to
contribute or hinder the transfers of learning to the job, the
training centre itself is expected to contribute towards
acquiring the required knowledge and skills by the trainees.
Its climate, facilities faculty resources, library are some
factors which would differentiate one centre from the other.
The Study
To study the effectiveness of the training of the in-house
training Institutes in public enterprises detailed
investigations using structured, protested questionnaires are
undertaken. Contd.
We Learn – A Continuous Learning Forum
34. Welingkar’s Distance Learning Division
Three sets of questionnaires:
1. For the trainees
2. To the superior officers of trainees and
3. To the training centers are prepared and
administered.
These questionnaires contain information such as
relevance of the course inputs, extent of its fulfillment,
perceptions of organizations climate for effective
transfers of learning to the job, possible hindrances,
positive as well as negative factors for training in the
organization. The variables identified as relevant are
1. organizational variable. 2. Behavioral variables
We Learn – A Continuous Learning Forum
35. Welingkar’s Distance Learning Division
Transfer of Learning
Individual Gains
To evaluate the effect of training on the individual after
6-12 months, the training persons stated the significant
areas where the individual development had taken
place. They were: 1. State of art knowledge on the
subject2. Development of the analytical skills. 3.
Refined interpersonal skills. 4. Innovation/ creativity. 5.
Management of stress.
Contd.
We Learn – A Continuous Learning Forum
36. Welingkar’s Distance Learning Division
Transfer of learning to the organization
To evaluate whether in-house training has resulted in
enhanced organization performance, trainees were
requested to indicate the resultant benefits to public
enterprises. The non-tangible benefits are: 1. Better
management practices. 2. improved problem solving. 3. job
improvement plans. 4. better systems.
There is general evidence that systematic in-house training
has contributed to individual growth in public enterprises,
while the same level of organization improvement has not
taken place because of organization climate, PE systems and
Lack of motivation in PEs
We Learn – A Continuous Learning Forum
37. Welingkar’s Distance Learning Division
Alternate approach in post-course evaluations
New approaches to post-course evaluation are being
explored. One such recent development is monitoring the
progress of the participant through post-course project
work. In this approach participants at the time of
nomination to the course in consultation with their
departmental heads identify a proble, of practical value to
the organization. During the program participants discuss,
reformulate and identify approaches to its solution in the
light of the knowledge and skill inputs in the training
sessions. This way they related the learning to work
situations and made innovating solutions.
We Learn – A Continuous Learning Forum
38. Welingkar’s Distance Learning Division
Evaluation of in-house training institutes
In order to make training a continuous activity
addressing itself to specific training needs of the
organization, public enterprises have established large
number of in-house training institutes/ Centers.
There are 5 factors considered important for evaluating
the institute and methods of assigning point rating to
each.
a) Faculty Resources: The number of qualified faculty
members on full time basis at the centre, years of
experience in training are considered. Contd.
We Learn – A Continuous Learning Forum
39. Welingkar’s Distance Learning Division
b) Infrastructure: This covers all infrastructure facilities
other than library, such as class rooms, syndicate rooms,
hostel, recreation facilities etc.
c) Library: This is the “brain” of any training institution/
Centre. Relevant books, journal, cases, audio, video etc.
d) In-house Research / Consultancy: The in-house
centers suffer from under exposure to research
development and consultancy work. Such
research/consultancy may be both internal or external.
e) Financial Support: The annual budget allocation, unit
cost per training are items under this factor.
We Learn – A Continuous Learning Forum